Developer Quickstart

YumaPro Tools

Refer to the YumaPro User Manual for a complete introduction to YumaPro Tools.

This section focuses on the software development aspects of NETCONF, YANG, and the netconfd-pro server.

YANG provides a completely programmable management API.

• A specific YANG module defines how to do something (e.g, ietf-yang-push.yang) and the server binds the instrumentation to the specified YANG objects.

• Callback functions are used to hook device and data model specific behavior to database objects and RPC operations.

• The yangdump-pro program is used to generate the glue code and instrumentation code stubs

• The developer fills in the YANG instrumentation code.

• The completed code is built and loaded into the server at compile-time, boot-time, or run-time.

What is a Developer Expected to Implement?

A server developer must provide the server instrumentation to perform the management changes to the system requested by a client. The most common YANG statements that need instrumentation:

• Configuration data nodes (use EDIT2 Callback and EDIT3 Callback)

• Operational data nodes (use GET2 Callback)

• RPC operations (use RPC Callbacks)

• YANG actions (use Action Callbacks)

• Notifications (use Notifications send function)

Multi-Threaded or Single-Threaded Mode?

The YumaPro SDK programs can be used with or without Posix threads. Packages that include pthreads in the package name are built using the PTHREADS=1 make flag. This enables multi-threaded operation within the 'netconfd-pro' program and client applications built with the 'yp-client' library.

For all sessions:

• All operations within a single session are serialized, even if multiple messages are received for the session at once.

• The client must maintain the NETCONF session or the operation in progress will be canceled, and any pending messages will be dropped.

• It is possible for one session to have a message ready for processing before another one that started the message first.

For Single-Threaded Mode:

• RPC requests are processed until completion in the order they are ready.

• It is possible for a message in progress to be blocked, waiting for a SIL callback or SIL-SA response.

• YControl sessions (e.g., SIL-SA or DB-API) should have priority over client sessions. This may cause client sessions to be blocked.

For Multi-Threaded Mode:

• This allows each session to operate in parallel (with limitations).

• SIL code must be thread-safe for an unrestricted operation!

• SIL-SA transactions (except EDIT) are not stateful. SIL-SA code should be written so each server request is independent of the previous or next request.

Restrictions For Multi-Threaded Mode

The following restrictions apply for processing of the RPC Operation Interface on a multi-threaded session:

• All Database Operations are serialized.

  • The EDIT callbacks and all In-Transaction Callbacks should be thread-safe.

  • All datastore access is blocked while an EDIT transaction is in progress.

  • Any operation that needs to access a configuration datastore (e.g. "<candidate>" or "<running>") is also serialized.

  • A datastore read operation will cause any edit operations to wait until that read is done.

  • A YANG action will be serialized in the same way as an edit operation.

• Concurrent operation can be suppressed for specific operations using the ywx:exclusive-rpc YANG extension.

• All other RPC operations, including custom operations not provided by 'netconfd-pro', can be invoked concurrently by different sessions.

Database Deployment Variants

There are many ways to configure and build the server for storing and retrieving datastore contents.

• Local internal database: configuration in memory and NV-storage in netconfd-pro owned XML file

• Local internal database + NV-store hook: configuration in memory and NV-storage via callback functions, and transferred to/from the server as an XML file

• Local internal database + no-NV-store: configuration in memory and NV-storage is managed by the yp-system and/or SIL code. The server will not attempt to load or store the configuration to non-volatile storage

• Local internal database + external edits: your external process communicates with the netconfd-pro process to initiate external edits (e.g., initiated from legacy CLI or internal database)

Local internal database

This is the default database configuration and provides complete management of all database storage and management. No extra APIs or CLI parameters are required.

Configuration properties:

• Tree-based data structures in memory for run-time transaction processing

• This is the canonical database

• There is no other database

• The non-volatile load and store is done by netconfd-pro using the --startup filespec to store the configuration

• The file is encoded as an XML instance document containing 1 element named 'data'

• No YANG default leafs are stored in this file

Local internal database + NV-store Hook

This database variant allows the vendor/system code to manage the non-volatile configuration. The system control is constrained to the replacement of the functions that load and store the configuration file.

This variant requires use of the NV-store APIs (described in the Database Transaction Callbacks section of this document).

Configuration properties:

• Tree-based data structures in memory for run-time transaction processing

• This is the canonical database

• There is no other database

• The non-volatile load and store is done by netconfd-pro via user callback functions.

• The NV-store callback functions manage the actual non-volatile representation and storage.

• The NV-Store callback transfers the configuration to/from netconfd-pro as an XML file

• YANG default leafs will be treated as explicitly set if contained in the transfer file

Local internal database + no-NV-store

• This variant allows either a simple deployment that does not support non-volatile storage of configuration or a more flexible system-managed non-volatile storage design, using multiple server APIs.

• This variant requires that the CLI parameter --no-nvstore be set to 'false'.

• The --startup CLI parameter is ignored if --no-nvstore is used.

• Various initialization callback APIs and transaction APIs can be used to load and save the non-volatile storage

Configuration properties:

• Tree-based data structures in memory for run-time transaction processing

• This is the canonical database

• There is no other database

• The non-volatile load and store is done by netconfd-pro via various user callback functions.

• The internal NV-store and NV-store callback functions are not used

• The 'yp-system' library or individual SIL libraries can be used to initialize and fill the empty database with configuration and operational data nodes

• Transaction hooks can be used to save the running configuration at run-time if any client edit transactions are processed by the server

Local internal database + External Edits

This database variant allows an external process to initiate database edits, that are processed by the server as user or system edits.

The DB-API functions described in the next section can be used by the subsystem to edit the server configuration database.

This variant can be used with other variants for processing non-volatile storage. This variant is used to co-exist with a legacy or canonical database owned by the system.

Configuration properties:

• Tree-based data structures in memory for run-time transaction processing

• This may or may not be the canonical database, depending on the non-volatile storage options.

• There may be another database in the system

• The server uses its copy in memory as the complete database

• The DB-API “send_edit” and “send_edit_ex” API functions can be used to transfer configuration from the system to netconfd-pro.

• Data is transferred in XML

• Error responses are sent by the server if an edit is not accepted.

• An error will be returned if a client is already in the process of editing the database

• The db-api-app application is installed in /usr/share/yumapro/src/db-api-app by default, and includes some examples of initializing the DB-API service and sending edits to the server.

Startup Options

There are 3 startup parameter variants, using the YANG 'start' choice statement.

• choice of 3 leafs:

  • --startup

  • --factory-startup [ + --startup-factory-file ]

  • --no-startup

The --startup=filespec CLI parameter can be used to specify the NV-storage file. The default filespec is $HOME/.yumapro/startup-cfg.xml.

The --factory-startup parameter can be used to load an empty configuration instead of the stored configuration. If the default configuration is used (startup-cfg.xml) then it is reset to factory default values.

The --startup-factory-file parameter can be used to load an non-empty configuration instead of the stored configuration. This file is used instead of the default configuration (startup-cfg.xml). It is ignored unless --factory-startup is also used.

The --no-startup parameter can be used to load an empty configuration instead of the stored configuration. The default configuration (startup-cfg.xml) is not affected by this parameter.

Server Source Code Generation

SIL Variants

Server instrumentation libraries can be local or remote:

• SIL: synchronous local server instrumentation library within the netconfd-pro process via callback functions

• SIL-SA: asynchronous remote sub-agent server instrumentation library, within another process, connected via YControl messages

• PY-SIL: asynchronous remote sub-agent Python based server instrumentation library, within another process, connected via YControl messages You can find all documentation related to PY-SIL here YumaPro PY-SIL Manual

Server instrumentation libraries can be for one module or a set of modules (called a bundle):

• module: library is for one module. OK to use when there no "augment" modules needed within the same SIL or SIL-SA library

• bundle: library is for a bundle of modules. OK to use when there is a base module and additional modules that augment the base module.

  • It is strongly recommended that a bundle be used for a base module and the modules that augment that base module.

  • This allows the callback code for augmenting modules to be generated correctly by yangdump-pro.

  • Any modules can be included a bundle, not just a base modules and modules that augment it.

make_sil Scripts

There are 4 scripts that can be used to generate SIL or SIL-SA code stubs.

For PY-SIL stub code generation refer to PY-SIL Code Generation

Script	Purpose
make_sil_dir_pro	Generate C code stubs for a combined or split SIL variant
make_sil_sa_dir	Generate C code stubs for a combined or split SIL-SA variant
make_sil_bundle	Generate C code stubs for a split SIL bundle variant
make_sil_sa_bundle	Generate C code stubs for a split SIL-SA bundle variant

The SIL code for a YANG module can be generated with the make_sil_dir_pro script described in the next section.

A split SIL module for 'foo.yang' would be generated using the following files:

make_sil_dir foo

u_foo.c	User SIL	User-provided server instrumentation code for the 'foo' module.
u_foo.h	User SIL	User-provided external definitions for the 'foo' module. Should not edit!
y_foo.c	YumaPro SIL	YumaPro server glue code for the 'foo' module. Do not edit!
y_foo.h	YumaPro SIL	YumaPro server external definitions for the 'foo' module. Do not edit!

Combined SILs are considered deprecated, but still supported. A combined SIL module for 'foo.yang' would be generated using the following files:

make_sil_dir --no-split foo

foo.c	Combined YumaPro and User SIL	User-provided server instrumentation code for the 'foo' module.
foo.h	Combined YumaPro and User SIL	User-provided external definitions for the 'foo' module. Should not edit!

Note

SIL bundles always use "split mode" code generation.

A SIL bundle named bundle for 'test1.yang' and 'test2.yang' would be generated using the following files:

make_sil_bundle bundle1 test1 test2

y_bundle1.c	Bundle Wrapper	Bundle init and cleanup C code; Do not edit!
y_bundle1.h	Bundle Wrapper	Bundle init and cleanup H code; Do not edit!
u_test1.c	User SIL	User-provided server instrumentation code for the 'test1' module.
u_test1.h	User SIL	User-provided external definitions for the 'test1' module. Should not edit!
y_test1.c	YumaPro SIL	YumaPro server glue code for the 'test1' module. Do not edit!
y_test1.h	YumaPro SIL	YumaPro server external definitions for the 'test1' module. Do not edit!
u_test2.c	User SIL	User-provided server instrumentation code for the 'test2' module.
u_test2.h	User SIL	User-provided external definitions for the 'test2' module. Should not edit!
y_test2.c	YumaPro SIL	YumaPro server glue code for the 'test2' module. Do not edit!
y_test2.h	YumaPro SIL	YumaPro server external definitions for the 'test2' module. Do not edit!

Basic Development Steps

The steps needed to create server instrumentation for use within YumaPro are as follows:

• Create the YANG module data model definition, or use an existing YANG module.

  • Validate the YANG module with the yangdump-pro program and make sure it does not contain any errors. All warnings should also be examined to determine if they indicate data modeling bugs or not.

    Example toaster.yang

• Make sure the $YUMAPRO_HOME environment variable is defined, and pointing to your YumaPro development tree.

• Create a SIL development subtree

  • Generate the directory structure and the Makefile with the make_sil_dir_pro script, installed in the /usr/bin directory. This step will also call yangdump-pro to generate the initial H and C files file the SIL.

    Example:

    mydir> make_sil_dir_pro test
    

• Use your text editor to fill in the device-specific instrumentation for each object, RPC method, and notification. (In this example, edit test/src/u_test.c) Almost all possible NETCONF-specific code is either handled in the central stack, or generated automatically. so this code is responsible for implementing the semantics of the YANG data model.

• Compile the SIL code

  • Use the 'make' command in the SIL 'src' directory. This should generate a library file in the SIL 'lib' directory.

    Example:

    mydir/test/src> make
    

• Install the SIL library so it is available to the netconfd-pro server.

  • Use the 'make install' command in the SIL 'src' directory.

    Example:

    mydir/test/src> sudo make install
    

• Run the netconfd-pro server (or build it again if linking with static libraries)

• Load the new module

  • Be sure to add a 'load' command to the configuration file if the module should be loaded upon each reboot.

    yangcli-pro session:

    user@server1> load test
    

• The netconfd-pro server will load the specified YANG module and the SIL and make it available to all sessions.

Basic Development Steps for PY-SIL

YumaPro PY-SIL Manual

Add a SIL or SIL-SA Library to the Server

After the SIL or SIL-SA code is built and installed, it must be added to the server configuration. The mandatory information includes:

• module or bundle name

• enabled or disabled features

Configure the Library Name

The library name is the bundle name for a bundle and the module name for a module.

The server must be configured somehow to know which modules and bundles to load and support at runtime. The --module and --bundle CLI parameters are used for this purpose. The --loadpath CLI parameter can also be used to select modules to load at boot-time.

Configure the YANG Features

The server must also be configured so any YANG features are configured to match the implementation.

Note

If there are YANG features that are not supported in the SIL or SIL-SA instrumentation, then the server MUST be configured with the proper parameters to match the implementation.

The default server configuration will enable all YANG features. If not all YANG features are supported then the features must be configured.

There are two approaches supported:

• Method 1:

  • Set --feature-enable-default CLI parameter to true

  • Add --feature-disable CLI parameters for each unsupported feature

• Method 2:

  • Set --feature-enable-default CLI parameter to false

  • Add --feature-enable CLI parameters for each supported feature

These parameters are usually stored in the configuration file (default /etc/yumapro/netconfd-pro.conf) but they can also be specified in the command line invoking the server.