Database Operations

The server database is designed so that the SIL callback functions do not need to really know which database model is being used by the server (E.g., target is <candidate> vs. <running> configuration).

There are three SIL database edit callback phases:

  1. Validate: Check the parameters no matter what database is the target

  2. Apply: The server will manipulate the database nodes as needed. The SIL usually has nothing to do in this phase unless internal resources need to be reserved.

  3. Commit or Rollback: Depending on the result of the previous phases, either the commit or the rollback callback phase will be invoked, if and when the changes are going to be finalized in the running configuration.

The SIL code is not responsible for maintaining the value tree for any database. This is done by the server.

The SIL database edit callback code is responsible for the following tasks:

  • Perform any data-model specific validation that is not already covered by a machine-readable statement, during the validation phase.

  • Reserve any data-model specific resources for the proposed new configuration content, during the apply phase.

  • Activate any data-model behavior changes based on the new configuration content, during the commit phase.

  • Release any reserved resources that were previously allocated in the apply phase, during the rollback phase.

The YANG "config" statement is used to identify configuration data nodes in NETCONF and RESTCONF.

  • If config='true':

    • the data nodes can be edited, and supports EDIT1 or EDIT2 callbacks

    • if NMDA enabled, the operational data nodes can be retrieved with GET2 callbacks

    • the data can be retrieved by a NETCONF client with these operations:

  • If config='false':

    • the data nodes cannot be edited, and supports GET1 or GET2 callbacks

    • the data can be retrieved by a NETCONF client with these operations:

Example YANG module:

container interfaces {
   list interface {
       key "name";

       leaf name {
           type string;
       }
       leaf-list untagged-ports {
           type string;
       }
       leaf speed {
           type enumeration {
               enum 10m;
               enum 100m;
               enum auto;
           }
       }
       leaf observed-speed {
           config false;
           type uint32;
       }
   }
}

leaf version {
  config false;
  type string;
}

Each callback type has its own callback API function prototype, and each callback has its own register and unregister functions that need to be used. This code is usually generated automatically by yangdump-sdk.

The following database APIs are described in this section:

Edit Callback Overview

Getting the Current Edit Transaction ID

In order to determine if an edit transaction is in progress, an API can be used. If the transaction-id returned by this function is zero, then no transaction is in progress.

ncx_transaction_id_t agt_cfg_txid_in_progress(ncx_cfg_t cfgid)

Return the ID of the current transaction ID in progress.

Parameters:

cfgid -- config ID to check

Returns:

txid of transaction in progress or 0 if none

Example: Get the current transaction ID for the running datastore

ncx_transaction_id_t txid = agt_cfg_txid_in_progress(NCX_CFGID_RUNNING);

Database Edit Validate Callback Phase

../_images/database_validate_phase.png

A SIL database validation phase callback function is responsible for checking all the 'description statement' sort of data model requirements that are not covered by any of the YANG machine-readable statements.

For example, if a 'user name' parameter needed to match an existing user name in /etc/passwd then the SIL validation callback would call the system APIs needed to check if the 'newval' string value matched a valid user name. The server will make sure the user name is well-formed and could be a valid user name.

Database Edit Apply Callback Phase

The callback function for this phase is called when database edits are being applied to the running configuration. The resources needed for the requested operation may be reserved at this time, if needed.

Database Edit Commit Callback Phase

This callback function for this phase is called when database edits are being committed to the running configuration. The SIL callback function is expected to finalize and apply any data-model dependent system behavior at this time.

Database Edit Rollback Callback Phase

This callback function for this phase is called when database edits are being undone, after some apply phase or commit phase callback function returned an error, or a confirmed commit operation timed out.

The SIL callback function is expected to release any resources it allocated during the apply or commit phases. Usually only the commit or the rollback function will be called for a given SIL callback, but it is possible for both to be called. For example, if the 'rollback-on-error' option is in effect, and some SIL commit callback fails after your SIL commit callback succeeds, then your SIL rollback callback may be called as well.

The server supports 2 modes of database editing callbacks. The original mode (EDIT1) is designed to invoke data node callbacks at the leaf level. This means that each altered leaf will cause a separate SIL callback. If no leaf callbacks are present, then the parent node will be invoked multiple times.

The EDIT2 mode is “list-based” or “container-based” instead. The following key aspects define EDIT2 callbacks:

  • In this mode there are no SIL callback functions expected for terminal nodes (leaf, leaf-list, anyxml).

  • The SIL callback for a container or list will be invoked, even if only child terminal nodes are being changed.

  • The parent SIL callback will be invoked only once (per phase) if multiple child nodes are being altered at once

  • The parent node for such an edit is flagged in the “undo” record as a special “edit2_merge”. The edit operation will be OP_EDITOP_MERGE, but the parent node is not being changed

  • The special “edit2_merge” type of edit will have a queue of child_undo records containing info on the child edits. For example, 1 leaf could be created, another leaf modified, and a third leaf deleted, all in the same edit request. The child_undo records provide the edit operation and values being changed.

The 'scb' parameter represents a session control block structure that is defined in the ncx/ses.h. This control block is primarily used for error reporting, as described in the example section later. However, can be used for more advanced actions. It provides access to the session specific information, such as current message input/output encoding, current session ID information, current protocol information, user name information, peer address information, etc. Note, almost all the fields in this structure should NOT be changed and accessed directly. This control block ideally should be used only for getting more information about the current session, not for alteration of any of its fields.

The 'msg' parameter represents the NETCONF Server and Client RPC Request/Reply Message Handler control block that is defined in the netconf/src/ncx/rpc.h. Similarly to SCB, this control block is primarily used for error reporting, as described in the example section later. The fields of this control block should NOT be access and changed for other purposes.

The 'cbtype' parameter represents an enumeration structure of the different server EDIT callback types that is defined in the netconf/src/agt/agt.h. This control block specifies what Phase is in the process, Validation, Apply, or Commit/Rollback, as described in the example section later.

The 'editop' parameter represents an enumeration structure of the NETCONF <edit-config> operation types that is defined in the netconf/src/ncx/op.h. This control block specifies what operation is in the process, merge, replace, delete, or other, as described in the example section later.

The 'newval' and 'curval' parameters represent data nodes that are being edited. The example section demonstrates how they can be utilized.

EDIT1 Callback

First generation edit callbacks, generated by yangdump-sdk using node-based APIs. An edit function is generated for every node, including terminal nodes (leaf, leaf-list, anyxml, anydata). An EDIT2 callback handles all the terminal nodes within the parent function instead of separate functions for each node.

New SIL or SIL-SA code should use EDIT2 callbacks instead.

EDIT1 Callback Function

The same callback template is used for EDIT1 and EDIT2 callbacks:

typedef status_t (*agt_cb_fn_t)(ses_cb_t *scb, rpc_msg_t *msg, agt_cbtyp_t cbtyp, op_editop_t editop, val_value_t *newval, val_value_t *curval)

Callback function for server object handler.

Used to provide a callback sub-mode for a specific named object

Param scb:

session control block making the request

Param msg:

incoming rpc_msg_t in progress

Param cbtyp:

reason for the callback

Param editop:

the parent edit-config operation type, which is also used for all other callbacks that operate on objects

Param newval:

container object holding the proposed changes to apply to the current config, depending on the editop value. Will not be NULL.

Param curval:

current container values from the <running> or <candidate> configuration, if any. Could be NULL for create and other operations.

Return:

status:

EDIT1 Callback Registration and Cleanup

status_t agt_cb_register_callback(const xmlChar *modname, const xmlChar *defpath, const xmlChar *version, agt_cb_fn_t cbfn)

Register an object specific edit callback function use the same fn for all callback phases all phases will be invoked.

Parameters:
  • modname -- module that defines the target object for these callback functions

  • defpath -- Xpath with default (or no) prefixes defining the object that will get the callbacks

  • version --

    exact module revision date expected if condition not met then an error will be logged (TBD: force unload of module!)

    NULL means use any version of the module

  • cbfn -- address of callback function to use for all callback phases

Returns:

status

EDIT1 Callback Registration

The following example shows an EDIT1 callback registration.

#define EXAMPLE_MODNAME (const xmlChar *)"ietf-interfaces-example"
#define EXAMPLE_VERSION (const xmlChar *)"2017-01-01"
#define EXAMPLE_DEFPATH (const xmlChar *)"/if:interfaces/if:interface"

/********************************************************************
* FUNCTION interfaces_init
*
* initialize the server instrumentation library.
* Initialization Phase 1
*
*********************************************************************/
static status_t
     interfaces_init (void)
{
    status_t res =
        agt_cb_register_callback(EXAMPLE_MODNAME,
                                 EXAMPLE_DEFPATH,
                                 EXAMPLE_VERSION,
                                 edit_callback_example);
    return res;
}

EDIT1 Callback Cleanup

Use the 'agt_cb_unregister_callbacks' function to unregister all callbacks for the data node.

EDIT1 Callback Function Example

In this example, the callback code forces Rollback Phase if a new value of an “interface” list is not acceptable. Otherwise an agent can process to the next step and run device instrumentation as required.

A new list validation, in this example, is done during “commit” phase. A new value is already written to the datastore (value is getting written during apply phase) which means the server will have to reverse the edit. The server will automatically delete just created new list element from the datastore and restore the state to the initial state, to the state before the edit.


/********************************************************************
* FUNCTION  edit1_callback_example
*
* Callback function for server object handler
* Used to provide a callback sub-mode for
* a specific named object
*
* Path: /interfaces/interface
* Add object instrumentation in COMMIT phase.
*
********************************************************************/
static status_t
    edit1_callback_example (ses_cb_t *scb,
			        rpc_msg_t *msg,
			        agt_cbtyp_t cbtyp,
			        op_editop_t editop,
			        val_value_t *newval,
			        val_value_t *curval)
{
    status_t res = NO_ERR;
    val_value_t *errorval = (curval) ? curval : newval;
    const xmlChar *errorstr = (errorval) ? NCX_NT_VAL : NCX_NT_NONE;

    /* try to find a key value of the /interfaces list to validate */
    val_value_t *child_val = NULL;
    if (newval) {
        child_val =
            val_find_child(newval,
                           EXAMPLE_MODNAME,
                           (const xmlChar *)"name");

        if (child_val && typ_is_string(VAL_BTYPE(child_val))) {
            log_info("\ncallback for %s editop, test child name=%s",
                op_editop_name(editop), VAL_STR(child_val));
        }
    }

    switch (cbtyp) {
    case AGT_CB_VALIDATE:
        /* description stmt validation here */
        break;
    case AGT_CB_APPLY:
        /* database manipulation done here */
        break;
    case AGT_CB_COMMIT:
        /* device instrumentation done here */
        switch (editop) {
        case OP_EDITOP_LOAD:
            interface_enabled = TRUE;
            break;
        case OP_EDITOP_MERGE:
            break;
        case OP_EDITOP_REPLACE:
            break;
        case OP_EDITOP_CREATE:
            interface_enabled = TRUE;

             /* Force Rollback if the key value is not acceptable */
            if (newval && child_val && typ_is_string(VAL_BTYPE(child_val)) &&
                !xml_strcmp(VAL_STR(child_val), (const xmlChar *)"not-supported"))  {

                log_info("\nKey value is not supported for %s editop, name=%s",
                   op_editop_name(editop), VAL_STR(child_val));

                /* Validation failed if a key value is not supported */
                errorval = child_val;
                res = ERR_NCX_OPERATION_NOT_SUPPORTED;
            } else {
                /* Run device instrumentation here  */
            }
            break;
        case OP_EDITOP_DELETE:
            interface_enabled = FALSE;
            break;
        default:
            res = SET_ERROR(ERR_INTERNAL_VAL);
        }

        break;
    case AGT_CB_ROLLBACK:
        /* undo device instrumentation here */
        break;
    default:
        res = SET_ERROR(ERR_INTERNAL_VAL);
    }

    /* if error: set the res, errorstr, and errorval parms */
    if (res != NO_ERR) {
        agt_record_error(scb,
			     &msg->mhdr,
			     NCX_LAYER_CONTENT,
			     res,
			     NULL,
			     NCX_NT_STRING,
			     errorstr,
			     NCX_NT_VAL,
			     errorval);
    }

    return res;

} /* edit_callback_example */

The following part of the above code example is used to find the “name” key node of the edited “interface” list.

val_value_t *val_find_child(const val_value_t *parent, const xmlChar *modname, const xmlChar *childname)

Find the first instance of the specified child node.

Parameters:
  • parent -- parent complex type to check

  • modname --

    module name; the first match in this module namespace will be returned

    NULL: the first match in any namespace will be returned;

  • childname -- name of child node to find

Returns:

pointer to the child if found or NULL if not found

boolean typ_is_string(ncx_btype_t btyp)

Check if the base type is a simple string (not list)

Parameters:

btyp -- base type enum to check

Returns:

TRUE if base type is textual

FALSE if some other type

val_value_t *child_val = NULL;
if (newval) {
    child_val =
        val_find_child(newval,
                       EXAMPLE_MODNAME,
                       (const xmlChar *)"name");

    if (child_val && typ_is_string(VAL_BTYPE(child_val))) {
        log_info("\ncallback for %s editop, test child name=%s",
            op_editop_name(editop), VAL_STR(child_val));
    }
}

The 'val_find_child' API function finds the specified child node. Alternatively, 'val_find_child_fast', 'val_find_child_obj', 'val_find_child_que' API functions could be used to retrieve the desired value.

The 'typ_is_string' API function checks if the base type is a simple string to use this string later for logging. Alternatively, 'typ_is_enum', 'typ_is_number' API functions could be used to check the desired type.

The 'VAL_BTYPE()' and 'VAL_STR()' macros are used to access val_value_t structure and get the information about its base type and get string value. If the type of the value would be an integer, for example, VAL_INT32, VAL_UINT16, etc. macros could be used to retrieve the actual set value of it.

In the following part of the above code example the previously retrieved key value is validated. If the provided in the <edit-config> key value is not acceptable as specified below, then the '*res' return status will be set to ERR_NCX_OPERATION_NOT_SUPPORTED enumeration value, that would signal to record an error and rollback the <edit-config> operation.

/* Force Rollback if the key value is not acceptable */
if (newval && child_val && typ_is_string(VAL_BTYPE(child_val)) &&
    !xml_strcmp(VAL_STR(child_val), (const xmlChar *)"not-supported"))  {

    log_info("\nKey value is not supported for %s editop, name=%s",
       op_editop_name(editop), VAL_STR(child_val));

    /* Validation failed if a key value is “not-supported” */
    errorval = child_val;
    res = ERR_NCX_OPERATION_NOT_SUPPORTED;
} else {
    /* Run device instrumentation here  */
}

EDIT2 Callback

The same callback template is used for EDIT1 and EDIT callbacks. The difference is how terminal child nodes are handled. Second generation callbacks are generated by yangdump-sdk for container or list-based APIs. An edit function is generated for “parent” list and container nodes only. The terminal child nodes are handled by this callback.

Each nested container or list has its own callback. This is generated in yangdump-sdk or 'make_sil_*' scripts using the --sil-edit2 parameter. The same callback function signature as EDIT1 is used, but the registration function and procedure is different.

The server supports 2 modes of database editing callbacks. The original mode (EDIT1) is designed to invoke data node callbacks at the leaf level. This means that each altered leaf will cause a separate SIL callback. If no leaf callbacks are present, then the parent node will be invoked multiple times.

The EDIT2 mode is “list-based” or “container-based” instead.

  • In this mode there are no SIL callback functions expected for terminal nodes (leaf, leaf-list, anyxml).

  • The SIL callback for a container or list will be invoked, even if only child terminal nodes are being changed.

  • The parent SIL callback will be invoked only once (per phase) if multiple child nodes are being altered at once

  • The parent node for such an edit is flagged in the “undo” record as a special “edit2_merge”. The edit operation will be “OP_EDITOP_MERGE”, but the parent node is not being changed

  • The special “edit2_merge” type of edit will have a queue of child_undo records containing info on the child edits. For example, 1 leaf could be created, another leaf modified, and a third leaf deleted, all in the same edit request. The 'child_undo' records provide the edit operation and values being changed.

EDIT2 Callback Function

The EDIT2 callback uses the same function as the EDIT1 Callback Function.

EDIT2 Callback Initialization and Cleanup

The EDIT2 callback function is hooked into the server with the 'agt_cb_register_edit2_callback' function, described below. The SIL code generated by yangdump-pro uses this function to register a single callback function for all callback phases.

The registration is done during the Initialization Phase 1 before the startup configuration is loaded into the running configuration database and before running configurations are loaded.

status_t agt_cb_register_edit2_callback(const xmlChar *modname, const xmlChar *defpath, const xmlChar *version, agt_cb_fn_t cbfn)

Register an object specific edit2 callback function.

Use the same fn for all callback phases all phases will be invoked

Only Callbacks for containers and lists are allowed in edit2 mode; Top Level Terminal Nodes are NOT SUPPORTED in edit2 mode

Parameters:
  • modname -- module that defines the target object for these callback functions

  • defpath -- Xpath with default (or no) prefixes defining the object that will get the callbacks

  • version --

    exact module revision date expected if condition not met then an error will be logged (TBD: force unload of module!)

    NULL means use any version of the module

  • cbfn -- address of callback function to use for all callback phases

Returns:

status

EDIT2 Callback Registration

static status_t interfaces_init (void)
{
    status_t res =
        agt_cb_register_edit2_callback(EXAMPLE_MODNAME,
                                       EXAMPLE_DEFPATH,
                                       EXAMPLE_VERSION,
                                       edit2_callback_example);
    return res;
}

EDIT2 Callback Cleanup

Use the 'agt_cb_unregister_callbacks' function to unregister all callbacks for the data node.

The unregister function needs to be called just once for a specific object. It will unregister EDIT1, EDIT2, and GET2 callbacks for the object. However, if it is called multiple times for the same object, it will return with NO errors.

All other callbacks that the object may hold should be unregistered separately.

EDIT2 Callback Function Example

The EDIT2 callback template is the same as the EDIT1 callback. The difference is that there is a queue of child edit records that may need to be accessed to reliably process the edit requests.

In the following example, EDIT2 callback code checks each 'child_undo' record in order:



/********************************************************************
* FUNCTION  edit2_callback_example
*
* Callback function for server object handler
* Used to provide a callback sub-mode for
* a specific named object
*
* Path: /interfaces/interface
* Add object instrumentation in COMMIT phase.
*
********************************************************************/
    edit2_callback_example (ses_cb_t *scb,
                    rpc_msg_t *msg,
                    agt_cbtyp_t cbtyp,
                    op_editop_t editop,
                    val_value_t *newval,
                    val_value_t *curval)
{
    status_t res = NO_ERR;
    val_value_t *errorval = (curval) ? curval : newval;

    if (LOGDEBUG) {
        log_debug("\nEnter edit2_callback_example callback for %s phase",
            agt_cbtype_name(cbtyp));
    }

    switch (cbtyp) {
    case AGT_CB_VALIDATE:
        /* description-stmt validation here */
        break;
    case AGT_CB_APPLY:
        /* database manipulation done here */
        break;
    case AGT_CB_COMMIT:
        /* device instrumentation done here */
        switch (editop) {
        case OP_EDITOP_LOAD:
            break;
        case OP_EDITOP_MERGE:
            /* the edit is not really on this node; need to get
             * each child_undo record to get the real edited nodes
             * and the edited operations
             */
            agt_cfg_transaction_t *txcb = RPC_MSG_TXCB(msg);
            agt_cfg_undo_rec_t *child_edit =
                agt_cfg_first_child_edit(txcb, newval, curval);

            while (child_edit) {
                op_editop_t child_editop = OP_EDITOP_NONE;
                val_value_t *child_newval = NULL;
                val_value_t *child_curval = NULL;
                xmlChar *newval_str = NULL;
                xmlChar *curval_str = NULL;

                agt_cfg_child_edit_fields(child_edit,
                                          &child_editop,
                                          &child_newval,
                                          &child_curval);

                if (child_newval) {
                    newval_str = val_make_sprintf_string(child_newval);
                    if (newval_str == NULL) {
                        return ERR_INTERNAL_MEM;
                    }
                }
                if (child_curval) {
                    curval_str = val_make_sprintf_string(child_curval);
                    if (curval_str == NULL) {
                        m__free(newval_str);
                        return ERR_INTERNAL_MEM;
                    }
                }

                log_info("\n        %s: editop=%s newval=%s curval=%s",
                      child_newval ? VAL_NAME(child_newval) : NCX_EL_NULL,
                      child_editop ? op_editop_name(child_editop) : NCX_EL_NONE,
                      child_newval ? newval_str : NCX_EL_NULL,
                      child_curval ? curval_str : NCX_EL_NULL);


		   /* Force Rollback if the child value is not acceptable */
                if (child_newval &&
                    !xml_strcmp(VAL_NAME(child_newval),
                                (const xmlChar *)"untagged-ports") &&
                    !xml_strcmp(VAL_STR(child_newval),
                                (const xmlChar *)"not-supported")) {

                    res = ERR_NCX_OPERATION_NOT_SUPPORTED;
                    m__free(newval_str);
                    m__free(curval_str);
                    break;
                }

                /**** process child edits here ****/


                m__free(newval_str);
                m__free(curval_str);

                child_edit = agt_cfg_next_child_edit(child_edit);
            }

            break;
        case OP_EDITOP_REPLACE:
        case OP_EDITOP_CREATE:
            /* the edit is on this list node and the child editop
             * can be treated the same as the parent (even if different)
             * the val_value_t child nodes can be accessed and there
             * are no child_undo records to use
             */
            val_value_t *child_newval = NULL;
            child_newval =
                val_find_child(newval,
                               EXAMPLE_MODNAME,
                               (const xmlChar *)"untagged-ports");

                val_value_t *leaflist_val = child_newval;
                while (leaflist_val) {

                    /**** process child leaf-list edits here ****/

                    leaflist_val = val_next_child_same(leaflist_val);
                }

                /**** process other child edits here if needed ****/

            break;
        case OP_EDITOP_DELETE:
            break;
        default:
            res = SET_ERROR(ERR_INTERNAL_VAL);
        }
        break;
    case AGT_CB_ROLLBACK:
        /* undo device instrumentation here */
        break;
    default:
        res = SET_ERROR(ERR_INTERNAL_VAL);
    }

    if (res != NO_ERR) {
        agt_record_error(scb,
                         &msg->mhdr,
                         NCX_LAYER_CONTENT,
                         res,
                         NULL,
                         (errorval) ? NCX_NT_VAL : NCX_NT_NONE,
                         errorval,
                         (errorval) ? NCX_NT_VAL : NCX_NT_NONE,
                         errorval);
    }

    return res;

} /* edit2_callback_example */
const xmlChar *agt_cbtype_name(agt_cbtyp_t cbtyp)

Get the string for the server callback phase.

Parameters:

cbtyp -- callback type enum

Returns:

const string for this enum

Accessing Child Node Edits

If the operation is “merge”, the edit is not really on the container or list node. The callback needs to get each child_undo record to get the real edited nodes and the edit operations.

If the parent operation is “merge” and the actual edit is on the node with default value. The edited node has a “default” YANG statement, then the actual child edit operation will always be “merge”. The difference will be in the newval, curval values.

create default node

new non-default value

default value

modify default node

new non-default or set back to default value

old non-default value

delete default node

default value

old non-default value

The following code demonstrates how to loop through the child undo records and retrieve the actual operation and the actual child nodes.

case OP_EDITOP_MERGE:
    agt_cfg_transaction_t *txcb = RPC_MSG_TXCB(msg);
    agt_cfg_undo_rec_t *child_edit =
        agt_cfg_first_child_edit(txcb, newval, curval);

    while (child_edit) {
        op_editop_t child_editop = OP_EDITOP_NONE;
        val_value_t *child_newval = NULL;
        val_value_t *child_curval = NULL;
        xmlChar *newval_str = NULL;
        xmlChar *curval_str = NULL;

        agt_cfg_child_edit_fields(child_edit,
                                  &child_editop,
                                  &child_newval,
                                  &child_curval);

        if (child_newval) {
            newval_str = val_make_sprintf_string(child_newval);
            if (newval_str == NULL) {
                return ERR_INTERNAL_MEM;
            }
        }
        if (child_curval) {
            curval_str = val_make_sprintf_string(child_curval);
            if (curval_str == NULL) {
                m__free(newval_str);
                return ERR_INTERNAL_MEM;
            }
        }

        log_info("\n        %s: editop=%s newval=%s curval=%s",
              child_newval ? VAL_NAME(child_newval) : NCX_EL_NULL,
              child_editop ? op_editop_name(child_editop) : NCX_EL_NONE,
              child_newval ? newval_str : NCX_EL_NULL,
              child_curval ? curval_str : NCX_EL_NULL);


           /* Force Rollback if the child value is not acceptable */
        if (child_newval &&
            !xml_strcmp(VAL_NAME(child_newval),
                        (const xmlChar *)"untagged-ports") &&
            !xml_strcmp(VAL_STR(child_newval),
                        (const xmlChar *)"not-supported")) {

            res = ERR_NCX_OPERATION_NOT_SUPPORTED;
            m__free(newval_str);
            m__free(curval_str);
            break;
        }

        /**** process child edits here ****/


        m__free(newval_str);
        m__free(curval_str);

        child_edit = agt_cfg_next_child_edit(child_edit);
    }
    break;
agt_cfg_undo_rec_t *agt_cfg_first_child_edit(agt_cfg_transaction_t *txcb, val_value_t *newnode, val_value_t *curnode)

Get the first child node edit record for a given transaction.

Parameters:
  • txcb -- transaction control block to clean

  • newnode -- new value node passed to callback

  • curnode -- current value node passed to callback

Returns:

pointer to the undo edit record for the first child node edit

NULL if none found

agt_cfg_undo_rec_t *agt_cfg_next_child_edit(agt_cfg_undo_rec_t *curedit)

Get the next child node edit record for a given transaction.

Parameters:

curedit -- pointer to the current child edit

Returns:

pointer to the undo edit record for the next child node edit

NULL if none found

void agt_cfg_child_edit_fields(agt_cfg_undo_rec_t *child_undo, op_editop_t *editop, val_value_t **newval, val_value_t **curval)

Get the child edit fields from the undo record.

Parameters:
  • child_undo -- child undo record that was returned

  • editop -- [out] address of return editop

  • newval -- [out] address of return new value

  • curval -- [out] address of return current value

The following code illustrates how to loop through the children undo records in order to retrieve the actual operation and the actual edited children:

agt_cfg_undo_rec_t *child_edit =
    agt_cfg_first_child_edit(txcb, newval, curval);

while (child_edit) {
    op_editop_t child_editop = OP_EDITOP_NONE;
    val_value_t *child_newval = NULL;
    val_value_t *child_curval = NULL;

    agt_cfg_child_edit_fields(child_edit,
                              &child_editop,
                              &child_newval,
                              &child_curval);

     /**** process child edits here ****/


    child_edit = agt_cfg_next_child_edit(child_edit);
}

EDIT2 Callback For a Create Operation

val_value_t *val_next_child_same(val_value_t *curchild)

Get the next instance of the corresponding child node.

Parameters:

curchild -- child type to find next instance of

Returns:

pointer to the next child of same type or NULL if none

The following code shows how to access the 'untagged-ports' leaf-list child nodes that are getting created along with their “interface” parent:

case OP_EDITOP_CREATE:
    val_value_t *child_newval = NULL;
    child_newval =
        val_find_child(newval,
                       EXAMPLE_MODNAME,
                       (const xmlChar *)"untagged-ports");

        val_value_t *leaflist_val = child_newval;
        while (leaflist_val) {

            /**** process child leaf-list edits here ****/

            leaflist_val = val_next_child_same(leaflist_val);
        }

        /**** process other child edits here if needed ****/

    break;
  • If an edit operation creates a new or modifies an existing data node, or its children, the EDIT2 callback will be invoked and run.

  • If the edit operation creates a new list entry, the operation will be “create” and any children that are getting created along with its parent should be treated as regular nodes and should be accessed the same way as for EDIT1 callback.

  • If the operation is “merge”, the actual operation and edit is on children nodes, so the child edits should be checked and applied.

SIL-SA EDIT2 Callbacks

The SIL-SA EDIT2 callback usage is the same as the EDIT2 callback except EDIT2 MERGE handling. The difference is only in the children edits access APIs.

There are SIL-SA EDIT2 mode specific high-level transaction access and management utilities in sil-sa/sil_sa.h. These functions access the lower-level functions to provide simpler functions for common transaction management tasks.

The following table highlights available functions and SIL vs SIL-SA difference:

SIL-SA Function

SIL Function

Description

sil_sa_first_child_edit

agt_cfg_first_child_edit

Get the first child node edit record for a given transaction.

sil_sa_next_child_edit

agt_cfg_next_child_edit

Get the next child node edit record for a given transaction.

sil_sa_child_edit_fields

agt_cfg_child_edit_fields

Get the child edit fields from the specified undo record.

sil_sa_child_edit_t *sil_sa_first_child_edit(rpc_msg_t *msg)

Get the first child edit from the transaction control block.

SIL-SA EDIT2 MODE

SIL-SA Analogue agt_cfg_first_child_edit() API. Get the first child node edit record

Parameters:

msg -- rpc msg to use to find keys

Returns:

pointer to the first child edit

sil_sa_child_edit_t *sil_sa_next_child_edit(sil_sa_child_edit_t *curedit)

Get the next child edit from the transaction control block.

SIL-SA EDIT2 MODE

SIL-SA Analogue agt_cfg_next_child_edit() API. Get the next child node edit

Parameters:

curedit -- pointer to the current child edit

Returns:

pointer to next child node edit

NULL if none found

void sil_sa_child_edit_fields(sil_sa_child_edit_t *child_edit, op_editop_t *editop, val_value_t **newval, val_value_t **curval)

Get the child edit fields.

SIL-SA EDIT2 MODE

SIL-SA Analogue agt_cfg_child_edit_fields() API. Get the child edit fields from the child_edit record

Parameters:
  • child_edit -- child_edit record that was returned from the server

  • editop -- [out] address of return editop

  • newval -- [out] address of return new value

  • curval -- [out] address of return current value

SIL-SA EDIT2 Callback Function Example

In the following example, SIL-SA EDIT2 callback code gets each 'child edit' record in order to retrieve the real edited nodes.



/********************************************************************
* FUNCTION  silsa_edit2_callback_example
*
* SIL-SA EDIT2 Callback function for server object handler
* Used to provide a callback sub-mode for
* a specific named object
*
* Path: /interfaces/interface
* Add object instrumentation in COMMIT phase.
*
********************************************************************/
    silsa_edit2_callback_example (ses_cb_t *scb,
                                  rpc_msg_t *msg,
                                  agt_cbtyp_t cbtyp,
                                  op_editop_t editop,
                                  val_value_t *newval,
                                  val_value_t *curval)
{
    status_t res = NO_ERR;
    val_value_t *errorval = (curval) ? curval : newval;

    if (LOGDEBUG) {
        log_debug("\nEnter silsa_edit2_callback_example callback for %s phase",
            agt_cbtype_name(cbtyp));
    }

    switch (cbtyp) {
    case AGT_CB_VALIDATE:
        /* description-stmt validation here */
        break;
    case AGT_CB_APPLY:
        /* database manipulation done here */
        break;
    case AGT_CB_COMMIT:
        /* device instrumentation done here */
        switch (editop) {
        case OP_EDITOP_LOAD:
            break;
        case OP_EDITOP_MERGE:
            /* the edit is not really on this node; need to get
             * each child_undo record to get the real edited nodes
             * and the edited operations
             */
            sil_sa_child_edit_t *child_edit = sil_sa_first_child_edit(msg);
            while (child_edit) {

                op_editop_t child_editop = OP_EDITOP_NONE;
                val_value_t *child_newval = NULL;
                val_value_t *child_curval = NULL;
                xmlChar *newval_str = NULL;
                xmlChar *curval_str = NULL;

                sil_sa_child_edit_fields(child_edit,
                                         &child_editop,
                                         &child_newval,
                                         &child_curval);
                if (child_newval) {
                    newval_str = val_make_sprintf_string(child_newval);
                    if (newval_str == NULL) {
                        return;
                    }
                }
                if (child_curval) {
                    curval_str = val_make_sprintf_string(child_curval);
                    if (curval_str == NULL) {
                        if (newval_str) {
                            m__free(newval_str);
                        }
                        return;
                    }
                }

                log_debug("\n        %s: editop=%s newval=%s curval=%s",
                          child_newval ? VAL_NAME(child_newval) : NCX_EL_NULL,
                          child_editop ? op_editop_name(child_editop) : NCX_EL_NONE,
                          child_newval ? newval_str : NCX_EL_NULL,
                          child_curval ? curval_str : NCX_EL_NULL);

                m__free(newval_str);
                m__free(curval_str);

                child_edit = sil_sa_next_child_edit(child_edit);
            }

            break;
        case OP_EDITOP_REPLACE:
        case OP_EDITOP_CREATE:
            /* the edit is on this list node and the child editop
             * can be treated the same as the parent (even if different)
             * the val_value_t child nodes can be accessed and there
             * are no child_undo records to use
             */
            val_value_t *child_newval = NULL;
            child_newval =
                val_find_child(newval,
                               EXAMPLE_MODNAME,
                               (const xmlChar *)"untagged-ports");

                val_value_t *leaflist_val = child_newval;
                while (leaflist_val) {

                    /**** process child leaf-list edits here ****/

                    leaflist_val = val_next_child_same(leaflist_val);
                }

                /**** process other child edits here if needed ****/

            break;
        case OP_EDITOP_DELETE:
            break;
        default:
            res = SET_ERROR(ERR_INTERNAL_VAL);
        }
        break;
    case AGT_CB_ROLLBACK:
        /* undo device instrumentation here */
        break;
    default:
        res = SET_ERROR(ERR_INTERNAL_VAL);
    }

    if (res != NO_ERR) {
        agt_record_error(scb,
                         &msg->mhdr,
                         NCX_LAYER_CONTENT,
                         res,
                         NULL,
                         (errorval) ? NCX_NT_VAL : NCX_NT_NONE,
                         errorval,
                         (errorval) ? NCX_NT_VAL : NCX_NT_NONE,
                         errorval);
    }

    return res;

} /* silsa_edit2_callback_example */

EDIT2 KnowledgeBase FAQs

Database Template

Every NETCONF database has a common template control block, and common set of access functions.

NETCONF databases are not separate entities like separate SQL databases. Each NETCONF database is conceptually the same database, but in different states:

  • candidate: A complete configuration that may contain changes that have not been applied yet. This is only available if the :candidate capability is advertised by the server. The value tree for this database contains only configuration data nodes.

  • running: The complete current server configuration. This is available on every NETCONF server. The value tree for this database contains configuration data nodes and non-configuration nodes created and maintained by the server. The server will maintain read-only nodes when <edit-config>, <copy-config>, or :ref"<commit> operations are performed on the running configuration. SIL code should not alter the data nodes within a configuration directly. This work is handled by the server. SIL callback code should only alter its own data structures, if needed.

  • startup: A complete configuration that will be used upon the next reboot of the device. This is only available if the :startup capability is advertised by the server. The value tree for this database contains only configuration data nodes.

NETCONF also recognized external files via the <url> parameter, if the :url capability is advertised by the server. These databases will be supported in a future release of the server. The NETCONF standard does not require that these external databases support the same set of protocol operations as the standard databases, listed above. A client application can reliably copy from and to an external database, but editing and filtered retrieval may not be supported.

cfg_template_t Structure

The following typedef is used to define a NETCONF database template:

struct cfg_template_t

struct representing 1 configuration database

Public Members

ncx_cfg_t cfg_id

config ID: Internal configuration ID assigned to this configuration.

cfg_location_t cfg_loc

config location: Enumeration identifying the configuration source location.

cfg_state_t cfg_state

config state: Current internal configuration state.

ncx_transaction_id_t last_txid

last edit transaction ID

ncx_transaction_id_t cur_txid

current edit transaction ID

time_t last_modified

last modified timestamp

xmlChar *name

datastore name string

xmlChar *src_url

source URL: URL for use with 'cfg_loc' to identify the configuration source.

time_t lock_itime

time_t timestamp for max-lock-hold-time enforcement

xmlChar lock_time[TSTAMP_MIN_SIZE]

Date and time string when the configuration was last locked.

xmlChar last_ch_time[TSTAMP_MIN_SIZE]

Date and time string when the configuration was last changed.

uint32 flags

Internal configuration flags.

Do not use directly.

ses_id_t locked_by

Session ID that owns the global configuration lock, if the database is currently locked.

boolean locked_by_lockall

Flag indicating the global lock is from lock-all not lock.

cfg_source_t lock_src
dlq_hdr_t load_errQ

Queue of rpc_err_rec_t structures that represent any <rpc-error> records that were generated when the configuration was loaded, if any.

dlq_hdr_t plockQ
boolean rwlock_initialized

RWLOCK for this config - used by multiple reader/writer threads according to RWLOCK rules: basically, max one writer, multiple readers, ordered by request time.

pthread_rwlock_t rwlock

PTHREADS=1 only: rwlock data.

ses_id_t rw_wrlocked_by

PTHREADS=1 only: Single write lock holder, if any.

ses_id_t rw_rdlocked_by

Most recent read lock holder only, used for debugging if the background thread locks the cfg for fill_candidate_from_running or other reason then the SID will be zero.

boolean rw_wrlocked

PTHREADS=1 only: TRUE if currently write-locked.

boolean rw_rdlocked

PTHREADS=1 only: TRUE if currently read-locked.

boolean wrlock_pending

TRUE if the wrlock is active and set.

val_value_t *root

datastore root value.

btyp == NCX_BT_CONTAINER. The root of the value tree representing the entire database.

boolean fake_candidate

TRUE if this is YANG-PATCH request fake candidate template.

boolean in_cc_rollback

YPW-2036: flag to allow rollback transaction to indicate if the transaction is due to a confirmed-commit timeout Only used in agt.

Set in agt_ncx_cancel_confirmed_commit

boolean defer_load

TRUE if load is deferred due to sil-sa bundles.

cfg_template_t Access Functions

The file ncx/cfg.h contains some high-level database access functions that may be of interest to SIL callback functions for custom RPC operations. All database access details are handled by the server if the database edit callback functions are used (associated with a particular object node supported by the server). The following table highlights the most commonly used functions. Refer to the H file for a complete definition of each API function.

cfg_new_template

Create a new configuration database.

cfg_free_template

Free a configuration database.

cfg_get_state

Get the current internal database state.

cfg_get_config

Get a configuration database template pointer, from a configuration name string.

cfg_get_config_name

Get the config name from its ID.

cfg_get_config_id

Get a configuration database template pointer, from a configuration ID.

cfg_set_target

Set the CFG_FL_TARGET flag in the specified config.

cfg_fill_candidate_from_running

Fill the <candidate> config with the config contents of the <running> config.

cfg_fill_candidate_from_startup

Fill the <candidate> config with the config contents of the <startup> config.

cfg_fill_candidate_from_inline

Fill the candidate database from an internal value tree data structure.

cfg_get_dirty_flag

Returns TRUE if the database has changes in it that have not been saved yet. This applies to the candidate and running databases at this time.

cfg_clear_dirty_flag

Clear the cfg dirty flag upon request.

cfg_clear_running_dirty_flag

Clear the running dirty flag when it is saved to NV-storage or loaded into running from startup.

cfg_clear_candidate_dirty_flag

Clear the candidate dirty flag when it is saved to NV-storage or loaded into running from startup.

cfg_get_dirty_flag

Get the config dirty flag value.

cfg_ok_to_lock

Check if the database could be successfully locked by a specific session.

cfg_ok_to_unlock

Check if the database could be successfully unlocked by a specific session.

cfg_ok_to_read

Check if the database is in a state where read operations are allowed.

cfg_ok_to_write

Check if the database could be successfully written by a specific session. Checks the global configuration lock, if any is set.

cfg_is_global_locked

Returns TRUE if the database is locked right now with a global lock.

cfg_get_global_lock_info

Get some information about the current global lock on the database.

cfg_is_partial_locked

Check if the specified config has any active partial locks.

cfg_add_partial_lock

Add a partial lock the specified config. This will not really have an effect unless the CFG_FL_TARGET flag in the specified config is also set . For global lock only.

cfg_find_partial_lock

Find a partial lock in the specified config.

cfg_first_partial_lock

Get the first partial lock in the specified config.

cfg_next_partial_lock

Get the next partial lock in the specified config.

cfg_delete_partial_lock

Remove a partial lock from the specified config.

cfg_ok_to_partial_lock

Check if the specified config can be locked right now for partial lock only.

cfg_get_root

Get the config root for the specified config.

cfg_lock

Get a global lock on the database.

cfg_unlock

Release the global lock on the database.

cfg_unlock_ex

Release the global lock on the database. Do not always force a remove changes. This is needed for the <unload> operation which locks the datastores while deleting data nodes and schema nodes for the module being unloaded.

cfg_release_locks

Release all locks on all databases.

cfg_release_partial_locks

Release any configuration locks held by the specified session.

cfg_get_lock_list

Get a list of all the locks held by a session.

cfg_update_last_ch_time

Update the last-modified time-stamp.

cfg_update_last_txid

Update the last good transaction ID.

cfg_update_stamps

Update the last-modified and last-txid stamps.

cfg_get_last_ch_time

Get the last-modified time-stamp.

cfg_get_last_txid

Get the last good transaction ID.

cfg_sprintf_etag

Write the Entity Tag for the datastore to the specified buffer.

cfg_get_startup_filespec

Get the filespec string for the XML file to save the running database.

Database Access Utilities

Finding Data Nodes with XPath

The internal XPath API can be used to access configuration data nodes. The data should not be altered. This API should be considered read-only.

Step 1) Create the XPath parser control block

xpath_pcb_t *xpath_new_pcb(const xmlChar *xpathstr, xpath_getvar_fn_t getvar_fn)

malloc a new XPath parser control block

xpathstr is allowed to be NULL, otherwise a strdup will be made and exprstr will be set

Create and initialize an XPath parser control block

Parameters:
  • xpathstr --

    XPath expression string to save (a copy will be made)

    NULL if this step should be skipped

  • getvar_fn --

    callback function to retirieve an XPath variable binding

    NULL if no variables are used

Returns:

pointer to malloced struct, NULL if malloc error

xpath_pcb_t *pcb =
    xpath_new_pcb((const xmlChar *)"/interfaces/interface", NULL);
if (pcb == NULL) {
    return ERR_INTERNAL_MEM;
}

Step 2) Evaluate the XPath expression

xpath_result_t *xpath1_eval_expr(xpath_pcb_t *pcb, val_value_t *val, val_value_t *docroot, boolean logerrors, boolean configonly, status_t *res)

Evaluate an XPath expression use if the prefixes are YANG: must/when.

Evaluate the expression and get the expression nodeset result

Parameters:
  • pcb -- XPath parser control block to use

  • val -- start context node for value of current()

  • docroot -- ptr to cfg->root or top of rpc/rpc-replay/notif tree

  • logerrors --

    TRUE if log_error and ncx_print_errormsg should be used to log XPath errors and warnings

    FALSE if internal error info should be recorded in the

    xpath_result_t struct instead

  • configonly -- config mode

  • res -- [out] address of return status

    • *res is set to the return status

Returns:

malloced result struct with expr result NULL if no result produced (see *res for reason)

  • 'docroot' is usually a datastore root like 'cfg->cfg_root'

  • context node can be an interior node, but can also be the 'docroot'

xpath_result_t *result =
    xpath1_eval_expr(pcb,
                     root,
                     root,
                     FALSE, // logerrors
                     TRUE,  // configonly
                     &res);

Step 3) Iterate through the node-set result

xpath_resnode_t *xpath_get_first_resnode(xpath_result_t *result)

Get the first result in the renodeQ from a result struct.

Parameters:

result -- result struct to check

Returns:

pointer to resnode or NULL if some error

xpath_resnode_t *xpath_get_next_resnode(xpath_resnode_t *resnode)

Get the next result in the renodeQ from a result struct.

Parameters:

resnode -- current result node to get next from

Returns:

pointer to next resnode or NULL if some error

val_value_t *xpath_first_resnode_valptr(xpath_resnode_t *resnode)

Get the first result in the renodeQ from a result struct.

Parameters:

resnode -- result node struct to check

Returns:

pointer to value node resnode or NULL if some error

val_value_t *xpath_next_resnode_valptr(xpath_resnode_t *resnode, val_value_t *valptr)

Get the next node val pointer from a result node struct.

Parameters:
  • resnode -- pointer of result node struct

  • valptr -- current value pointer to get next for

Returns:

the next val pointer or NULL if some error

Note that XPath result nodes may reference a value node header, so it is safest to iterate through the value instances in the resnode. If there is only one entry this code will still work:

 xpath_resnode_t *resnode = xpath_get_first_resnode(result);
 for (; resnode; resnode = xpath_get_next_resnode(resnode)) {

     val_value_t *valnode = xpath_first_resnode_valptr(resnode);
     for (; valnode; valnode = xpath_next_resnode_valptr(resnode, valnode)) {

         // do something with valnode.... e.g dump the value if debug2
         if (LOGDEBUG2) {
             log_debug2("\nGot resnode value:\n");
             val_dump_value(valnode, 0, DBG2);
         }
    }
}

Determining if a Value Node is Set

A val_value_t structure is usually allocated with val_new_value() and then initialized with val_init_from_template().

In order to determine if the value has been set by a client, and not set by default or simply initialized, use the API function 'val_is_value_set'.

boolean val_is_value_set(val_value_t *val)

Check if a value has been set by a client It has to be initialized and not set by default to return true.

Parameters:

val -- value to check

Returns:

true if value has been set

false if has not been set, or set to default, or the code cannot tell if the value is more than initialized

Determining if the SIL Callback is the Deepest for the Edit

A datastore edit includes data for a subtree of configuration data. It is possible that multiple SIL callbacks will be invoked for the edit operation.

The function 'agt_val_silcall_is_deepest' from file agt/agt_val_silcall.h can be used to determine if the current callback is the deepest SIL invocation for the edit.

boolean agt_val_silcall_is_deepest(rpc_msg_t *msg)

Check if the current nested SIL callback is at the deepest level for the current edit.

Example

In the example above, the code is generated as EDIT2 code so there are no callbacks for leafs

This function would return TRUE for list 'tlist3' and FALSE for all ancestor node callbacks (tnest, tlist, list2)

FALSE if the current SIL callback is not at the deepest level for the edit that contains the silcall record. There is at least one nested_silcall involved in the current edit that is at a child level compared to the current node being called

FALSE if there is no current silcall set

Parameters:

msg -- RPC message to check

Returns:

TRUE if the current SIL callback is at the deepest level for the edit that contains the silcall record