SIL-SA Subsystem

The SIL-SA subsystem provides sub-agent support for SIL callbacks. It uses messages between the server and subsystem to distribute SIL operations across multiple remote subsystems.

SIL-SA Usage

SIL-SA libraries can only be used within the sil-sa-app or combo-app applications.

Note

• SIL-SA instrumentation will only be supported if used with the unmodified sil-sa-app application. Support issues must be reproducible using this application.

• Custom combo-app applications are not supported.

If SIL-SA libraries are available to the subsystem, and the YANG modules are available to the main server and the subsystem, then the SIL-SA callback registration and message handling will be automatically handled by the subsystem sil_sa and ycontrol libraries.

YANG Modules in the SIL-SA Subsystem

There are several ways that a subsystem can be deployed in the server platform. Any combination of the following is allowed:

• One or more subsystems located on the same host as netconfd-pro

  • Each subsystem is often used with the --library parameter to select SIL-SA libraries from a common library location

• One or more subsystems located on a different host than netconfd-pro

  • Each subsystem is often used with overlapping YANG modules.

  • It is possible for each subsystem to handle different instances in the same YANG list.

The main requirement is that each subsystem register a different subsystem identifier (see --subsys-id for details).

When a subsystem starts up, the server informs the subsystem of all the modules and bundles configured to be loaded into the server. By default a subsystem will attempt to load every module or bundle in this list.

The --library CLI parameter can be used by each subsystem to limit the libraries that are loaded.

Example: 2 Subsystems on the Same Host as netconfd-pro

Assume each subsystem is intended to implement a different bundle.

> # in each of 3 terminal windows
> netconfd-pro --bundle=bundle1 --bundle=bundle2
> sil-sa-app --subsys-id=subsys1 --library=bundle1
> sil-sa-app --subsys-id=subsys2 --library=bundle2

It is possible for multiple subsystems and the main module to register for the same object. Each callback must decide if it should handle the specific instance being requested (E.g. interface “eth0” or “eth3”). Subsystems need to return 'ok' for any edit request it ignores, and return 'no-instance' for any GET2 requests for unsupported instances.

Note

In a system where multiple subsystems can register for the same list object, there are specific behaviors to note when the list lacks defined keys in YANG. In such cases, the server will accept responses from all registered subsystems. Importantly, it will use the response from the last responding subsystem for the <get> response. This "last one wins" approach means that the final subsystem's response takes precedence and is reflected in the <get> response output.

In the registration phase, the main server will send the complete module and bundle parameter list to each subsystem. Each subsystem will load its SIL-SA library code as required for that subsystem.

It is up to the developer to make sure each subsystem registers with the main server with a unique subsys-id string. Only one subsystem can use the default (subsys1), The --subsys-id CLI parameter can be used to set the subsystem identifier.

Example SIL-SA Application

The following file can be found in the /usr/share/yumapro/src/sil-sa-app/src directory.

The sil-sa-app program is a complete application. Only the 'main.c' file is shown here, not the entire program.

/********************************************************************
* FUNCTION main
*
* This is an example SIL-SA service main function.
*
* RETURNS:
*   0 if NO_ERR
*   status code if error connecting or logging into ncxserver
*********************************************************************/
int main (int argc, char **argv)
{
#ifdef MEMORY_DEBUG
    mtrace();
#endif

    char *subsys = NULL;
    char *address = NULL;
    uint16 port = 0;
    uint16 retry_limit = 0;
    boolean if_notif=false;  // hidden parameter
    boolean ycontrol_done = FALSE;

    /* need to check for the subsys-id parm before
     * the system is initialized
     */
    status_t res = get_subsys_parm(argv, &subsys);
    if (res != NO_ERR) {
        print_usage();
    }

    /* 1) setup yumapro messaging service profile */
    if (res == NO_ERR) {
        if (subsys == NULL) {
            res = ycontrol_init(argc, argv,
                                (const xmlChar *)"subsys1");
        } else {
            res = ycontrol_init(argc, argv,
                                (const xmlChar *)subsys);
        }
        ycontrol_done = TRUE;
    }

    /* 2) register services with the control layer */
    if (res == NO_ERR) {
        res = sil_sa_register_service();
    }

#ifdef SIL_SA_APP_STATLIB_TEST
    /* 2B) setup any static SIL-SA libraries */
    if (res == NO_ERR) {
        res = static_silsa_init();
    }
#endif  // SIL_SA_APP_STATLIB_TEST


    /* get the CLI parameters after the system is initialized
     * so library parameter handled correctly
     */
    if (res == NO_ERR) {
        res =  get_cli_parms(argv, &address, &port, &if_notif, &retry_limit);
        if (res != NO_ERR) {
            print_usage();
        }
    }

    /* set the retry limit if provided */
    if ((res == NO_ERR) && (retry_limit > 0)) {
        ycontrol_set_retry_limit(retry_limit);
    }

    /* It is also possible to set the retry_interval but there is
     * no CLI parameter provided for this purposes
     * if (res == NO_ERR) {
     *     ycontrol_set_retry_interval(retry_int_milliseconds);
     * }
     */


    /* 3) do 2nd stage init of the control manager (connect to server) */
    if (res == NO_ERR) {
        if (address) {
            if (port == 0) {
                port = 2023;
            }
            res = ycontrol_init2_ha("server1", address, port);
        } else {
            res = ycontrol_init2();
        }
    }

    useconds_t usleep_val = 10000;  // 10K micro-sec == 1/100 sec
    boolean done = FALSE;

    /* 4) call ycontrol_check_io periodically from the main program
     * control loop
     */
#ifdef SIL_SA_APP_DEBUG
    int id = 0;
#endif  // SIL_SA_APP_DEBUG

#ifdef SIL_SA_APP_NOTIF_TEST
    uint32 loop_cnt = 0;
#endif // SIL_SA_APP_NOTIF_TEST

#ifdef SIL_SA_APP_TERM_MSG_TEST
    uint32 term_msg_loop_cnt = 0;
#endif // SIL_SA_APP_TERM_MSG_TEST

    while (!done && res == NO_ERR) {
#ifdef SIL_SA_APP_DEBUG
        if (LOGDEBUG3) {
            log_debug3("\nsil-sa-app: checking ycontrol IO %d", id++);
        }
#endif  // SIL_SA_APP_DEBUG

        res = ycontrol_check_io();

        if (ycontrol_shutdown_now()) {
            // YControl has received a <shutdown-event>
            // from the server subsystem is no longer active
            // could ignore or shut down YControl IO loop
            done = TRUE;
        }

        // Using sleep to represent other program work; remove for real
        if (!done && res == NO_ERR) {
            (void)usleep(usleep_val);
        }

#ifdef SIL_SA_APP_NOTIF_TEST
        loop_cnt++;
        if (loop_cnt % 50 == 0 && if_notif) {
            sil_sa_notif_test(10, 20, (const xmlChar *)"this is a test");
        }
#endif // SIL_SA_APP_NOTIF_TEST

#ifdef SIL_SA_APP_TERM_MSG_TEST
        term_msg_loop_cnt++;
        if (term_msg_loop_cnt % 100 == 0) {
            sil_sa_term_msg_test(term_msg_loop_cnt);
        }
#endif  // SIL_SA_APP_TERM_MSG_TEST

    }

    /* 5) cleanup the control layer before exit */
    if (ycontrol_done) {
        ycontrol_cleanup();
    }

#ifdef MEMORY_DEBUG
    muntrace();
#endif

    return (int)res;

}  /* main */

SIL-SA API Functions

Most SIL callbacks have corresponding SIL-SA callbacks, which are the same or almost the same as the SIL API functions.

Some-SA APIs are different from SIL code and some callbacks cannot provide the same set of pointers to use during the invocation> For example, Transaction Complete and Start Hook Callbacks cannot provide Transaction Control Block in SIL-SA version of the callback since the Transaction Control Block is not available in the SIL-SA subsystem. It is only part of the netconfd-pro server and only accessible from the SIL code.

However, to supply as much information to the SIL-SA callbacks and provide as much possible the same functionality as in the SIL callbacks there are multiple API that can be used to achieve SIL like functionality in the SIL-SA code:

Session information is not available in the session control block for SIL-SA callbacks. Special APIs have to be used instead.

Function	Description
sil_sa_get_username()	Get the user_id value from the message header
sil_sa_get_client_addr()	Get the client address (client_addr value from the message header)
sil_sa_get_rpc_msg_id()	Get the rpc message id (txid_str) from the message header
sil_sa_set_error_msg()	Set Custom Error Message string in case of error

These APIs are only available in the SIL-SA version of the SIL code and are intended to provide access to specific fields in the client message header.

Client Username

The following APIs provide a way to access the User Identifier and Client Address that are stored in the message header in regular SIL code.

const xmlChar *sil_sa_get_username(void)

Get the user_id value from the message header.

Time-sensitive API. Each message received by a user will cause the saved state to be updated.

Returns:

user_id value or NULL

/* Get the user_id value from the message header */
const xmlChar *user = sil_sa_get_username();

Client Address

const xmlChar *sil_sa_get_client_addr(void)

Get the client address (client_addr value from the message header)

This is the external client address, not the SIL-SA peer address. Time-sensitive API. Each message received by a user will cause the saved state to be updated.

Returns:

client_addr value or NULL

/* Get the client address (client_addr value from the message header) */
const xmlChar *client_addr = sil_sa_get_client_addr();

Message ID

The following API provides a way to access the Message ID (txid_str). This API is only available to EDIT callbacks.

It will be NULL for all the other callbacks.

const xmlChar *sil_sa_get_rpc_msg_id(rpc_msg_t *msg)

Get the rpc transaction id (txid_str) from the rpc_sil_sa_cb.

This function is only available to EDIT callbacks! It is NULL for GET2 callbacks.

Parameters:

msg -- rpc msg to retrive rpc msg id from

Returns:

txid_str value or NULL

/* Get the rpc message id (txid_str) from the rpc_sil_sa_cb */
const xmlChar *msg_id =  sil_sa_get_rpc_msg_id(msg);

Set Error Message

The <error-message> field in the <rpc-error> response can be set from a SIL-SA callback.

The following YANG module will be used for this example:

module error-example {
  namespace "http://netconfcentral.org/ns/error-example";
  prefix "err";

  revision 2021-11-19 {
    description
      "Example module";
  }

  container test-silsa-errors {                 // EDIT2 CB
    presence "Presence";
    leaf test-error {
      type string;
    }
  }
}

Assume the SIL-SA code reports an error in case the 'test-error' leaf creation and when the value of a new leaf is 'force-error'.

In this case there is an API 'sil_sa_set_error_msg' to set up a custom error to be returned to the server. Refer to the SIL-SA Creation of Custom Error Messages And AppInfo section for details on this API function.

status_t
    u_err_test_silsa_errors_edit (
    ses_cb_t *scb,
    rpc_msg_t *msg,
    agt_cbtyp_t cbtyp,
    op_editop_t editop,
    val_value_t *newval,
    val_value_t *curval)
{
    val_value_t *child_val = NULL;
    if (newval) {
        child_val =
            val_find_child(newval,
                           (const xmlChar *)"error-example",
                           (const xmlChar *)"test-error");
    }

    /* Get the rpc message id (txid_str) from the rpc_sil_sa_cb */
    const xmlChar *msg_id = sil_sa_get_rpc_msg_id(msg);

    /* Get the user_id value from the message header */
    const xmlChar *user = sil_sa_get_username();

    /* Get the client address (client_addr value from the message header) */
    const xmlChar *client_addr = sil_sa_get_client_addr();

    if (LOGDEBUG3) {
        log_debug3("\nEnter u_err_test_silsa_errors_edit");
        log_debug3_append("\nmsg_id:%s", msg_id);
        log_debug3_append("\nuser:%s", user);
        log_debug3_append("\nclient_addr:%s", client_addr);
    }

    switch (cbtyp) {
    case AGT_CB_VALIDATE:
        /* description-stmt validation here */
        break;
    case AGT_CB_APPLY:
        /* database manipulation done here */
        break;
    case AGT_CB_COMMIT:
        /* device instrumentation done here */
        switch (editop) {
        case OP_EDITOP_LOAD:
            break;
        case OP_EDITOP_MERGE:
            break;
        case OP_EDITOP_REPLACE:
            break;
        case OP_EDITOP_CREATE:

            if (child_val &&
                !xml_strcmp(VAL_STR(child_val), (const xmlChar *)"force-error")) {

                /* TRIGGER ERROR */
                res = ERR_NCX_OPERATION_NOT_SUPPORTED;

                sil_sa_set_error_msg(msg,
                                     (const xmlChar *)"SOME CUSTOM ERROR MSG");
            }

            break;
        case OP_EDITOP_DELETE:
            break;
        default:
            /* USE SET_ERROR FOR PROGRAMMING BUGS ONLY */
            res = SET_ERROR(ERR_INTERNAL_VAL);
        }
        break;
    case AGT_CB_ROLLBACK:
        /* undo device instrumentation here */
        break;
    default:
        /* USE SET_ERROR FOR PROGRAMMING BUGS ONLY */
        res = SET_ERROR(ERR_INTERNAL_VAL);
    }
    return res;

} /* u_err_test_silsa_errors_edit */

As a result of this operation, the server will produce an error with a custom error message:

{
 "ietf-restconf:errors": {
  "error": [
   {
    "error-type":"application",
    "error-tag":"operation-not-supported",
    "error-app-tag":"no-support",
    "error-path":"/err:test-silsa-errors",
    "error-message":"SOME CUSTOM ERROR MSG",
    "error-info": {
     "error-number":273
    }
   }
  ]
 }
}

SIL-SA Message Format

The SIL-SA service uses several messages to interact with the netconfd-pro server.

These messages are defined in the yumaworks-sil-sa.yang module. The SIL-SA payload is defined as a YANG container that augments the YControl “message-payload” container.

SIL-SA Registration Message Flow

The SIL-SA service needs to initialize with the netconfd-pro server.

1. SIL-SA to Server: config-request The SIL-SA service registers itself with the server and requests its configuration in 1 message.

2. Server to SIL-SA: config-response The server responds to the config request with a list of modules and SIL-SA bundles that need to be loaded

3. SIL-SA to Server: register-request The SIL-SA service registers the SIL-SA callbacks supported by the subsystem.

4. Server to SIL-SA: ok The server responds to the config request with an <ok> or an <error> message

Server Edit Transaction Message Flow

The netconfd-pro server will initiate edit datastore transactions when the subsystem starts or restarts (load-config), or when clients edit the datastore via a network management protocol.

1. Server to SIL-SA: start-transaction The server starts the transaction with the “validate” SIL-SA callback phase.

2. SIL-SA to Server: ok The SIL-SA service responds to the start-transaction request with an <ok> or an <error> message

3. Server to SIL-SA: continue-transaction The server continues the transaction with the “apply” SIL-SA callback phase.

4. SIL-SA to Server: ok The SIL-SA service responds to the continue-transaction request with an <ok> or an <error> message

5. Server to SIL-SA: continue-transaction The server continues the transaction with the “commit” or “rollback” SIL-SA callback phase.

6. SIL-SA to Server: ok The SIL-SA service responds to the continue-transaction request with an <ok> or an <error> message. The SIL-SA service removes any saved state for this transaction.

Exception handling:

validate:

The server may send a “start-transaction” message with the “validate” flag set. The SIL-SA service will know that the transaction is only 1 message and not to expect a “continue-transaction” message from the server.

reverse-edit:

The server may send a “start-transaction” message with the “reverse-edit” flag set. The SIL-SA service will handle all 3 callback phases for the edit at once. Only 1 response message is sent to the server. This mode is used when a different subsystem responded with an error after the subsystem performing the reverse edit has already successfully completed the transaction. A new transaction is sent to the subsystem in this case to undo the edit.

cancel-transaction:

The server may send a “cancel-transaction” message after a transaction has started. This can occur if another SIL-SA subsystem or main server SIL callback returned an error. Any error causes the transaction to be canceled. The SIL-SA service will not expect a “continue-transaction” message from the server.

SIL-SA Register Request Message

The <register-request> is a subsystem request message. The purpose of the message is to register the SIL-SA callback functions for this sub-system.

• Expected Response Message: ok or error

The following diagram illustrates the <register-request> event message:

SIL-SA Start Transaction Message

The <start-transaction> message is a server request message. The purpose of the message is to start an edit transaction which may require the SIL-SA callback functions on the subsystem to be invoked.

This message requests that a new edit transaction be started on the subsystem. Only 1 transaction can be in progress at a time.

If this transaction is for a validate operation then there will not be any followup messages. Otherwise, the subsystem must retain the information in this message until a cancel-transaction message has been received with the same transaction-id value, or a continue-transaction message has been received with the same transaction-id value for the 'rollback' or 'commit' phase.

• Expected Response Message: transaction-response or error

The following diagram illustrates the <start-transaction> message:

SIL-SA Continue Transaction Message

The <continue-transaction> is a server request message. The purpose of the message is to invoke a callback phase for an edit transaction in progress.

• Expected Response Message: transaction-response or error

The following diagram illustrates the <continue-transaction> message:

SIL-SA Transaction Response Message

The <transaction-response> is a subsystem response message. The purpose of the message is:

• Set Hook: return added edits data or status

• Post Set Hook: return added edits data or status

• Transaction Hook: Expected Response Message: ok or error

• If no Hook invoked: Expected Response Message: ok or error

The following diagram illustrates the <transaction-response> event message:

SIL-SA Cancel Transaction Event

The <cancel-transaction> is a server event message. The purpose of the message is to cancel an edit transaction in progress.

• Expected Response Message: none

The following diagram illustrates the <cancel-transaction> message:

SIL-SA Load Event

The <load-event> is a server event message. The purpose of the message is to notify the SIL-SA subsytem that a module or bundle has been loaded or unloaded at run-time. Subsystem will load SIL-SA code and trigger a register event for any SIL calls registered.

• Expected Response Message: none

The following diagram illustrates the <load-event> message:

SIL-SA Bundle Load Event

The <bundle-load-event> is a subsystem event message. The purpose of the message is to notify the server that a SIL-SA bundle has been loaded with a load-event sent from the server. This has caused some modules to be loaded on the subsystem, that need to be reported back to the main server so the datastore validation, agt_state, and other system book-keeping can be done.

• Expected Response Message: none

The following diagram illustrates the <bundle-load-event> message:

SIL-SA Get Request Message

The <get-request> is a server request message. The purpose of the message is to send a composite retrieval request to support NETCONF and RESTCONF get operations.

• Expected Response Message: get-response or error

The following diagram illustrates the <get-request> message:

SIL-SA Get Response Message

The <get-response> is a subsystem response message. The purpose of the message is to send a subsystem generated composite retrieval response to support NETCONF and RESTCONF get operations.

• Expected Response Message: none

The following diagram illustrates the <get-response> message:

SIL-SA Notification Message

The <notification> is a subsystem event message. The purpose of the message is to send a subsystem generated YANG notification event for NETCONF and RESTCONF streams.

• Expected Response Message: none

The following diagram illustrates the <notification> message:

SIL-SA RPC Request Message

The <rpc-request> is a server request message. The purpose of the message is to start a RPC transaction which may require the require the SIL-SA callback functions on the subsystem to be invoked.

• This message requests that a new remote procedure call be validated and invoked on the subsystem.

• If there are input parameters the subsystem must validate them.

• If not valid or if the operation cannot be performed, the subsystem must return an error.

• Expected Response Message: rpc-response

The following diagram illustrates the <rpc-request> event message:

SIL-SA RPC Response Message

The <rpc-response> is a subsystem response message. The purpose of the message is to return RPC data or status.

• Expected Response Message: none

The following diagram illustrates the <rpc-response> message:

SIL-SA Action Request Message

The <action-request> is a server request message. The purpose of the message is to start an ACTION transaction which may require the SIL-SA callback functions on the subsystem to be invoked.

• This message requests that a new action call be validated and invoked on the subsystem.

• If there are input parameters the subsystem must validate them.

• If not valid or if the operation cannot be performed, the subsystem must return an error.

• Expected Response Message: action-response

The following diagram illustrates the <action-request> message:

SIL-SA Action Response Message

The <action-response> is a subsystem response message. The purpose of the message is to return Action data or status.

• Expected Response Message: none

The following diagram illustrates the <action-response> message:

SIL-SA Transaction Start Message

The <trans-start-hook> is a server request message. The purpose of the message is to send a request which may require the SIL-SA callback functions on the subsystem to be invoked.

• This message requests that Transaction Start Hook callbacks should be invoked on the subsystem.

• Expected Response Message: a regular error message or OK.

The following diagram illustrates the <trans-start-hook> event message:

SIL-SA Transaction Complete Event

The <trans-complete-hook> is a server event message. The purpose of the message is to send an event which may require the SIL-SA callback functions on the subsystem to be invoked.

• Expected Response Message: none

The following diagram illustrates the <trans-complete-hook> message:

SIL-SA Hook Get Request Message

The <hook-get-request> is a subsystem request message. The purpose of the message is to start a transaction which may require the server to run 'agt_val_get_data' API and return the result.

• The server will get the data node value based on XPath of object instance.

• This function will return a value only if there is an existing node in the datastore or there is a default for the node.

• Expected Response Message: hook-get-response

The following diagram illustrates the <hook-get-request> message:

SIL-SA Hook Get Response Message

The <hook-get-response> is a server reply message. The purpose of the message is to send the element containing the requested val_value_t based on XPath of the object instance.

• Expected Response Message: None

The following diagram illustrates the <hook-get-response> message:

SIL-SA Stream Callback Event

The <stream-callback-event> is a server event message. The purpose of the message is to invoke an event-stream callback on a SIL-SA platform.

• One event will be generated for each remote subsystem entry found that needs to be invoked. E.g., if 3 modules register but all are mapped to the same stream, then 3 events would be sent to the subsystem.

• Expected Response Message: None

The following diagram illustrates the <stream-callback-event> message:

SIL-SA Commit Completeness Hook Message

The <commit-completeness-hook> is a server request message. The purpose of the message is to start a transaction which may require the SIL-SA callback functions on the subsystem to be invoked.

• This message requests that a Commit Completeness callback should be invoked on the subsystem.

• Expected Response Message: a regular error message or OK.

The following diagram illustrates the <commit-completeness-hook> message: