Development Environment

This section describes the YumaPro Tools development environment used to produce the Linux binaries.

Programs and Libraries Needed

There are several components used in the YumaPro software development environment:

gcc compiler and linker
ldconfig and install programs
GNU make program
shell program, such as bash
YumaPro development tree: the source tree containing YumaPro code, specified with the $YUMAPRO_HOME environment variable.
SIL development tree: the source tree containing server instrumentation code

The following external program is used by YumaPro, and needs to be pre-installed:

opensshd (needed by netconfd-pro)
- The SSH2 server code does not link with netconfd-pro. Instead, the netconf-subsystem-pro program is invoked, and local connections are made to the netconfd-pro server from this SSH2 subsystem.

The following program is part of YumaPro Tools, and needs to be installed:

netconf-subsystem-pro (needed by netconfd-pro)
- The thin client sub-program that is called by 'sshd' when a new SSH2 connection to the 'netconf' sub-system is attempted.
  - This program will use an AF_LOCAL socket, using a proprietary <ncx-connect> message, to communicate with the netconfd-pro server..
  - After establishing a connection with the netconfd-pro server, this program simply transfers SSH2 NETCONF channel data between sshd and netconfd-pro.

The following program is part of YumaPro Tools, and usually found within the YumaPro development tree:

netconfd-pro
- The NETCONF server that processes all protocol operations.
  - The 'agt_ncxserver' component will listen for <ncx-connect> messages on the designated socket (/tmp/ncxserver.sock). If an invalid message is received, the connection will be dropped. Otherwise, the netconf-subsystem-pro program will begin passing NETCONF channel data to the netconfd-pro server. The first message is expected to be a valid NETCONF <hello> PDU. If

The following external libraries are used by YumaPro, and need to be pre-installed:

ncurses (needed by yangcli-pro)
- character processing library needed by 'libtecla'; used within yangcli-pro
libc (or glibc, needed by all applications)
- unix system library
libssh2 (needed by yangcli-pro)
- SSH2 client library, used by yangcli-pro
libxml2 (needed by all applications)
- xmlTextReader XML parser
- pattern support

SIL and SIL-SA Overview

The SIL and SIL-SA callback functions are normally built as shared libraries that are loaded dynamically and accessed by the server with the 'dlopen' and 'dlsym' library functions.

They can also be linked statically at build time. A SIL-SA library can also be built as a static library and linked to the “sil-sa-app” or “combo-app” programs.

The user callback functions and their source files use the “u_” prefix.
YumaPro callback functions and their files use the “y_” prefix.
SIL callback code is executed in the netconfd-pro process.
SIL-SA callback code is executed in a different (possibly remote) process.

The following YANG statements are supported in SIL and SIL-SA code:

<rpc> statement
<notification> statement
Any data definition statement
<deviation> statement

The following YANG statement is supported in SIL or SIL-SA code if built into a bundle:

<augment> statement for external module

Note

SIL libraries cannot be generated for YANG submodules.
Only full modules are supported in SIL and SIL-SA automatic code generation.

SIL Library Names

SIL libraries must be named with a predictable string in order for the server to find it with minimal configuration.

The name must be “lib” + <library-name> + “.” + <file-ext>

where:

<library-name> is the YANG module name or bundle name
<file-ext> is the shared library file extension (usually “so”)

Example:

test.yang → test.c → libtest.so

SIL-SA Library Names

SIL libraries must be named with a predictable string in order for the server to find it with minimal configuration.

The name must be “lib” + <library-name> + “_sa” + “.” + <file-ext>

where:

<library-name> is the YANG module name or bundle name
<file-ext> is the shared library file extension (usually “so”)

Example:

test.yang → test.c → libtest_sa.so

SIL and SIL-SA Library Location

By default, SIL and SIL-SA binaries are installed in the /usr/lib/yumapro directory (/usr/lib64/yumapro on Fedora x64 platforms).

The $YUMAPRO_RUNPATH environment variable (or --runpath CLI parameter) is used to specify alternate locations for SIL library binary files.

Building SIL or SIL-SA Libraries

The compiler flags passed to “make” are extremely important when building SIL source code. The flags must align with those used to build the main server.

Most importantly, the “yumapro-pthreads” server must use SIL code that is compiled with the 'PTHREADS=1' compiler flag.** This flag is used in some H files and therefore data structures will be misaligned if the same value is not used in both the server and the SIL libraries.

The version string for a PTHREADS server build contains the phrase “THD” at the end. The command “netconfd-pro --version” can be used to print the server version string.

The DEBUG=1 flag can also be used when building SIL code. This will cause the --ggdb compiler flag to be used.

Extra yangdump-pro Parameters

The "make_sil_" scripts will pass extra parameters to the yangdump-pro program to affect code generation.

The following parameters are normally added automatically by these scripts:

--sil-edit2:
- Use the second generation 'EDIT2' functions.
- Now the default so this parameter can be omitted.
--sil-get2
- Use the second generation 'GET2' functions.
- Now the default so this parameter can be omitted.

The following parameters must be used to override these defaults:

--sil-edit1:
- Use the first generation 'EDIT1 functions.
- Now deprecated so do not use.
- Will override the --sil-edit2 default.
--sil-get1:
- Use the first generation 'GET1 functions.
- Now deprecated so do not use.
- Will override the --sil-get2 default.

The following parameter must be used to enable NMDA GET2 function generation:

--sil-nmda
- Use the second generation 'GET2' functions.
- There will be extra GET2 callbacks generated for config=true data nodes.

The following parameter can be used to alter code generation:

--sil-include=filename
- The name of an include file to inject into C files
- An include statement will be generated for each instance of this parameter, in the order these parameters are given.
- Zero or more instances of this parameter are allowed.
- The include statements will be generated after the system <include> statement and general YumaPro include statements, but before the YANG module specific include statements.

If any YANG deviations are used, then these files should be specified in the code generation parameters:

--deviation=module-spec

Warning

If any YANG data types are altered or replaced with deviations then the --deviation parameter must be used or improper code could be generated, possibly causing a server crash.

Refer to the Server SIL and SIL-SA Code Generation section for more details.

Using SIL and SIL-SA libraries with netconfd-pro

Static Configuration

Libraries for a single module are loaded at boot-time into the server with the --module parameter. This parameter applies to both SIL and SIL-SA libraries.

Libraries for a bundle are loaded at boot-time into the server with the --bundle parameter. This parameter applies to both SIL and SIL-SA libraries.

Note

If SIL-SA bundles are used, the server will not know the contents until a subsystem registers for the bundle.
This will prevent client access to all datastores until completed.
See the --wait-datastore-ready parameter for details.

Dynamic Configuration

Libraries for a single module are loaded at run-time into the server with the <load> operation and removed with the <unload> operation.

Libraries for a bundle are loaded at run-time into the server with the <load-bundle> operation and removed with the <unload-bundle> operation.

make_sil_dir_pro

The 'make_sil_dir_pro' script can be used to generate SIL code stub files for a single YANG module, for instrumentation running in the netconfd-pro process.

The script will generate a directory sub-tree and fill in the Makefiles.
The requested C and H file code stub files for a single YANG module will be generated in the "src" directory.
This script uses the yangdump-pro program to generate these files.

Note

In this mode, SIL code will be generated only for the definitions in the target module. External augment statements from the target module (or any other module) will not affect SIL code generation.

make_sil_dir_pro Parameters

make_sil_dir_pro [--no-split | --split] module-spec 0*(extra-parms)

Example:

> make_sil_dir_pro address

--split
- Generate separate user and system code stubs. Now the default so this parameter can be omitted.
--no-split
- Generate combined code stubs. This mode is deprecated.
module-spec
- module name to pick the module revision based on the search path
```
make_sil_dir_pro ietf-ip
```
- module name plus revision data to pick a specific module revision
```
make_sil_dir_pro ietf-ip@2014-06-16
```
extra-parms
- Refer to Extra yangdump-pro Parameters for details

make_sil_bundle

The 'make_sil_bundle' script can be used to generate SIL code stub files for multiple YANG modules.

The parameters are a list of strings.
The first parameter is the bundle name.
Any additional "plain string" parameters are module specifications. These module names should be listed so the base modules are specified first and modules that augment the base modules are listed last.
Strings beginning with -- will be passed to yangdump-pro as parameters
The script will generate a directory sub-tree and fill in the Makefiles and source files.
SIL code will be generated for the definitions in the target module.
External augment statements from all the modules loaded together will affect SIL code generation.

make_sil_bundle Parameters

make_sil_bundle  bundle-name 1*(module-spec) 0*(extra-parms)

Example:

> make_sil_bundle ifbundle ietf-interfaces iana-if-type ietf-ip

In this example, a bundle named 'ifbundle' is created. The same directory structure is used as for a single module. C files are created for the bundle as well as the module files.

bundle-name
- Name of the bundle.
- Must not conflict with any module name.
module-spec
- module name to pick the module revision based on the search path
- module name plus revision data to pick a specific module revision
extra-parms
- Refer to Extra yangdump-pro Parameters for details

make_sil_sa_dir

The 'make_sil_sa_dir' script can be used to generate the C and H file code stubs for a single YANG module.

The script will generate a directory sub-tree and fill in the Makefiles.
The requested C and H file code stub files for a single YANG module will be generated in the "src" directory.
This script uses the yangdump-pro program to generate these files.
The --sil-get1 parameter is not allowed for SIL-SA code

Note

In this mode, SIL code will be generated only for the definitions in the target module. External augment statements from the target module (or any other module) will not affect SIL code generation.

make_sil_sa_dir Parameters

make_sil_sa_dir [--no-split | --split] module-spec 0*(extra-parms)

Example:

> make_sil_sa_dir address

--split
- Generate separate user and system code stubs. Now the default so this parameter can be omitted.
--no-split
- Generate combined code stubs. This mode is deprecated.
module-spec
- module name to pick the module revision based on the search path
- module name plus revision data to pick a specific module revision
extra-parms
- Refer to Extra yangdump-pro Parameters for details

make_sil_sa_bundle

The 'make_sil_sa_bundle' script can be used to generate the C and H file code stubs for a set of YANG modules.

The same YANG usage restrictions for SIL bundles apply to SIL-SA bundles. Refer to the make_sil_bundle section for details.

make_sil_sa_bundle Parameters

make_sil_sa_bundle  bundle-name 1*(module-spec) 0*(extra-parms)

Example:

> make_sil_sa_bundle ifbundle ietf-interfaces iana-if-type ietf-ip

In this example, a bundle named 'ifbundle' is created. The same directory structure is used as for a single module. C files are created for the bundle as well as the module files.

bundle-name
- Name of the bundle.
- Must not conflict with any module name.
module-spec
- module name to pick the module revision based on the search path
- module name plus revision data to pick a specific module revision
extra-parms
- Refer to Extra yangdump-pro Parameters for details

Static SIL-SA Libraries

The SIL-SA library for a module or a bundle can be built as a static library and linked directly into the “sil-sa-app” or “combo-app” programs (or an application based on these programs).

Quick Steps

make_sil_sa_dir or make_sil_sa_bundle
make STATIC=1
register static library in the SIL-SA application

There is no limit to the number of static SIL-SA libraries that can be linked into the program image or registered with the SIL-SA subsystem.

A static SIL-SA library is not used unless it is configured for use in the main server. The code is available at all times (since it is statically linked) and cannot be removed (only disabled).

Enable a static SIL-SA library

Configuration parameter
- --module=<module-name>
- --bundle=<bundle-name>
RPC operation
- <load>
- <load-bundle>

Disable a static SIL-SA library

RPC operation
- <unload>
- <unload-bundle>

Static SIL-SA Library Example

This information is also in the README-STATIC_SILSA.txt file in the root of the source code directory.

IMPORTANT info on Using Static SIL-SA Libraries

Assume a SIL-SA library is setup
```
> make_sil_sa_dir test2
```
This could also be a bundle (make_sil_sa_bundle)
Build a static version of the SIL-SA library
```
> cd test2

> make STATIC=1
```
Copy the static library to a common directory if desired.

This is not required but may make using the STATIC_SILSA variable easier.
```
> cp lib/libtest2.a $HOME/silsa/libtest2_sa.a
```
Use the macro STATIC_SILSA when building the sil-sa-app

Option 1: Change in Makefile:
```
STATIC_SILSA=-L /home/andy/silsa -ltest2_sa
```
Option 2: set from command line:
```
> STATIC_SILSA='-L /home/andy/silsa -ltest2_sa' make
```

Note

The -L part must be first and it must specify where the static libraries are located. One or more -l parts can follow.

The -l parameter does not use the full name. Instead libtest2_sa.a can be specified as -ltest2_sa.

Register the static library in the sil-sa-app or combo-app program

Example from sil-sa-app/main.c:

/* extern definitions for the 3 expected callbacks */
AGT_SIL_LIB_EXTERN(test2)

static status_t static_silsa_init (void)
{
    /* example: module=test2;
     * need to use in Makefile (example)
     * STATIC_SILSA=-L /home/andy/silsa -ltest2_sa
     * The actual library names are not needed in this code
     */
    status_t res =
        agt_sil_lib_register_statlib((const xmlChar *)"test2",
                                     y_test2_init,
                                     y_test2_init2,
                                     y_test2_cleanup);

    return res;
}

The SIL-SA library will not be used unless the module or bundle is loaded.

# <load> at runtime with yangcli-pro
> load test2

# load at boot-time in server
> netconfd-pro module=test2

SIL Makefile

The SIL-SA Makefile is based on the Makefile for YumaPro sources. The automake program is not used at this time. There is no ./configure script to run. There are 2 basic build steps:

make [DEBUG=1] [STATIC=1]
[sudo] make install

The installation step may require root access via the 'sudo' program, depending on the system.

Target Platforms

The following target platforms are supported:

Fedora, Ubuntu, CentOS: These are the default targets and no special command line options are needed.

Build Targets

The following table describes the build targets that are supported from the top-level SIL directory. More targets are available in the 'src' directory.

YumaPro Build Targets

Target	Description
all	Make library binaries
clean	Remove the target library binary files
distclean	Remove all the dependencies and all target files
doc	Make the Doxygen files
install	Install the library components into the system.
superclean	Remove all the dependencies and the library binaries
uninstall	Remove the library components from the system.

Command Line Build Options

There are several command line options that can be used when making the YumaPro source code. These may have an impact on building and linking the SIL source code. The top-level README file in the source code contains the latest definition of the make flags.

Example SIL Makefile

The script make_sil_dir_pro (installed as /usr/bin/make_sil_dir_pro) or make_sil_bundle (installed as /usr/bin/make_sil_bundle) is used to automatically create a SIL work sub-directory tree. Support files are located in the /usr/share/yumapro/util directory.

The Makefiles for SIL and SIL-SA modules are located in the “util” directory

Default SIL Makefile location: /usr/share/yumapro/util

Short Names for SIL and SIL-SA Code Generation

Starting in release 20.10-5 the --short-names parameter is available to control long or short name generation for SIL and SIL-SA code.

long name: contains the module name and the complete path name of the object in the name.
- This generates a completely unique function or identifier name that cannot conflict within the module or with any other module. However, the names can be very long and difficult to read or use in C code.
short name: the module prefix or --force-prefix parameter value is used instead of the module name.
- The object name is used instead of the path component.
- If the name is already used then a suffix such as “_1” will be added to the name so that it is unique.
- A short name is a match if the prefix part and the path part are the same strings.
- A short name is unique within the module or bundle, and should be globally unique if the module prefixes are unique.
- A short name can change if the code is regenerated and the module has changed so that duplicate names have been added.
- E.g., name might change to name_1 and the old name_1 might change to name_2.

Example:

Path: leaf /interfaces/interface/admin-status

GET2 long function:

u_ietf_interfaces_interfaces_interface_admin_status_get

GET2 short function:

u_if_admin_status_get

CLI Parameters For Short Names

There are two yangdump-sdk (or make_sil* script) CLI parameters that control the short name generation

--short-names: Selects short names or long names
--force-prefix: Override the module prefix in short name generation. Ignored if --short-names=false.

Short Name Summary

If the --short-names parameter is set to “true” (the default value) then short names will be used.

The header for each C file will contain the Short Name Mapping used.

SIL Example:

> make_sil_dir_pro ietf-interfaces

Example Short Name Summary from u_ietf-interfaces.c

Short Name Mappings

  Short Name Mappings
    admin_status = /interfaces/interface/admin-status
    admin_status_1 = /interfaces-state/interface/admin-status
    description = /interfaces/interface/description
    discontinuity_time = /interfaces/interface/statistics/discontinuity-time
    discontinuity_time_1 = /interfaces-state/interface/statistics/discontinuity-time
    enabled = /interfaces/interface/enabled
    higher_layer_if = /interfaces/interface/higher-layer-if
    higher_layer_if_1 = /interfaces-state/interface/higher-layer-if
    if_index = /interfaces/interface/if-index
    if_index_1 = /interfaces-state/interface/if-index
    in_broadcast_pkts = /interfaces/interface/statistics/in-broadcast-pkts
    in_broadcast_pkts_1 = /interfaces-state/interface/statistics/in-broadcast-pkts
    in_discards = /interfaces/interface/statistics/in-discards
    in_discards_1 = /interfaces-state/interface/statistics/in-discards
    in_errors = /interfaces/interface/statistics/in-errors
    in_errors_1 = /interfaces-state/interface/statistics/in-errors
    in_multicast_pkts = /interfaces/interface/statistics/in-multicast-pkts
    in_multicast_pkts_1 = /interfaces-state/interface/statistics/in-multicast-pkts
    in_octets = /interfaces/interface/statistics/in-octets
    in_octets_1 = /interfaces-state/interface/statistics/in-octets
    in_unicast_pkts = /interfaces/interface/statistics/in-unicast-pkts
    in_unicast_pkts_1 = /interfaces-state/interface/statistics/in-unicast-pkts
    in_unknown_protos = /interfaces/interface/statistics/in-unknown-protos
    in_unknown_protos_1 = /interfaces-state/interface/statistics/in-unknown-protos
    interface = /interfaces/interface
    interface_1 = /interfaces-state/interface
    interfaces = /interfaces
    interfaces_state = /interfaces-state
    last_change = /interfaces/interface/last-change
    last_change_1 = /interfaces-state/interface/last-change
    link_up_down_trap_enable = /interfaces/interface/link-up-down-trap-enable
    lower_layer_if = /interfaces/interface/lower-layer-if
    lower_layer_if_1 = /interfaces-state/interface/lower-layer-if
    name = /interfaces/interface/name
    name_1 = /interfaces-state/interface/name
    oper_status = /interfaces/interface/oper-status
    oper_status_1 = /interfaces-state/interface/oper-status
    out_broadcast_pkts = /interfaces/interface/statistics/out-broadcast-pkts
    out_broadcast_pkts_1 = /interfaces-state/interface/statistics/out-broadcast-pkts
    out_discards = /interfaces/interface/statistics/out-discards
    out_discards_1 = /interfaces-state/interface/statistics/out-discards
    out_errors = /interfaces/interface/statistics/out-errors
    out_errors_1 = /interfaces-state/interface/statistics/out-errors
    out_multicast_pkts = /interfaces/interface/statistics/out-multicast-pkts
    out_multicast_pkts_1 = /interfaces-state/interface/statistics/out-multicast-pkts
    out_octets = /interfaces/interface/statistics/out-octets
    out_octets_1 = /interfaces-state/interface/statistics/out-octets
    out_unicast_pkts = /interfaces/interface/statistics/out-unicast-pkts
    out_unicast_pkts_1 = /interfaces-state/interface/statistics/out-unicast-pkts
    phys_address = /interfaces/interface/phys-address
    phys_address_1 = /interfaces-state/interface/phys-address
    speed = /interfaces/interface/speed
    speed_1 = /interfaces-state/interface/speed
    statistics = /interfaces/interface/statistics
    statistics_1 = /interfaces-state/interface/statistics
    type = /interfaces/interface/type
    type_1 = /interfaces-state/interface/type

Short Names for Constants

The constants generated in a SIL or SIL-SA H file will be affected by the short-names parameter.

Examples

Module and Revision Constants

Long name constant:

#define y_ietf_interfaces_M_ietf_interfaces (const xmlChar *)"ietf-interfaces"
#define y_ietf_interfaces_R_ietf_interfaces (const xmlChar *)"2018-02-20"

Short name constant:

#define y_if_M_if (const xmlChar *)"ietf-interfaces"
#define y_if_R_if (const xmlChar *)"2018-02-20"

Feature Constants

Long name constant:

#define u_ietf_interfaces_F_arbitrary_names 1

Short name constant:

#define u_if_F_arbitrary_names 1

Data Structure Typedef Constants

Long name constant:

/* leaf-list /interfaces/interface/lower-layer-if */
typedef struct y_ietf_interfaces_T_interfaces_interface_lower_layer_if_ {
    dlq_hdr_t qhdr;
    xmlChar *v_lower_layer_if;
} y_ietf_interfaces_T_interfaces_interface_lower_layer_if;

Short name constant:

typedef struct y_if_T_lower_layer_if_ {
    dlq_hdr_t qhdr;
    xmlChar *v_lower_layer_if;
} y_if_T_lower_layer_if;

Object Name Constant

Long name constant:

#define y_ietf_interfaces_N_admin_status (const xmlChar*)"admin-status"

Short name constant:

#define y_if_N_admin_status (const xmlChar *)"admin-status"

Function and Variable Names

The function and variable names generated in a SIL or SIL-SA H file will be affected by the short-names parameter.

GET2 Example

Long name:

extern status_t u_ietf_interfaces_interfaces_interface_admin_status_get (
    getcb_get2_t *get2cb,
    const xmlChar *k_interfaces_interface_name);

Short name:

extern status_t u_if_admin_status_get (
    getcb_get2_t *get2cb,
    const xmlChar *k_name);

EDIT2 Example

Long name:

extern status_t u_ietf_interfaces_interfaces_interface_edit (
    ses_cb_t *scb,
    rpc_msg_t *msg,
    agt_cbtyp_t cbtyp,
    op_editop_t editop,
    val_value_t *newval,
    val_value_t *curval,
    const xmlChar *k_interfaces_interface_name);

Short name:

extern status_t u_if_interface_edit (
    ses_cb_t *scb,
    rpc_msg_t *msg,
    agt_cbtyp_t cbtyp,
    op_editop_t editop,
    val_value_t *newval,
    val_value_t *curval,
    const xmlChar *k_name);