YANG Push

The YANG Push feature allows data to be sent to a client automatically, using NETCONF notifications.

A separate mechanism for managing dynamic subscriptions is used by YANG Push, which is intended to replace the existing <create-subscription> RPC operation defined in RFC 5277. This new set of RPC operations (defined in RFC 8639) is used for current event stream subscriptions. It is also used for new datastore subscriptions (defined in RFC 8641).

The YANG Push support will be provided in multiple phases:

Dynamic subscriptions over NETCONF in XML
1. Periodic subscriptions
2. Conventional datastore On-Change subscriptions
3. Simulated On-Change subscriptions for the Operational Datastore
4. On-Change subscriptions for the Operational Datastore from subsystem events
Dynamic subscriptions over RESTCONF in XML or JSON
Configured subscriptions for Binary Push support
1. gRPC for native protobuf files
2. gNMI ONCHANGE subscriptions
3. CBOR+SID encoding over UDP or TCP

RFCs and Features Supported

There are multiple ways to do the same thing in YANG Push, so not all of it is implemented at once.

The “dynamic subscriptions” functionality is implemented in phase 1.
- phase 1.4 is not available in this release
The “configured subscriptions” functionality is not needed or relevant for subscriptions that use NETCONF sessions. It is intended for a binary push protocol that is expected to be standardized in the near future. This is planned for phase 3.
RFC 8639: Subscription to YANG Notifications
- YANG Module: ietf-subscribed-notifications@2019-09-09.yang
- Supported Features
  - encode-xml
  - replay
  - subtree
  - xpath
- Unsupported Features
  - configured
  - dscp
  - encode-json
  - interface-designation
  - qos
  - supports-vrf
RFC 8640: Dynamic Subscription to YANG Events and Datastores over NETCONF
RFC 8641: Subscription to YANG Notifications for Datastore Updates
- YANG Module: ietf-yang-push@2019-09-09.yang
- Supported Features
  - on-change
- Unsupported Objects
  - /subscriptions data node is not available for NMDA access to <operational> values of dynamic subscriptions
  - anchor-time parameter is not implemented yet

Configuring YANG Push

There is no CLI parameter to turn YANG Push on or off. Instead, then “bundle” parameter is used.

A bundle named “yang-push” is used.
“yang-push” is a reserved library name in the server.
The SIL library (libyang-push.so) contains the RPC operation callbacks for YANG Push. The “agt” directory contains the server implementation code.
The server does not need to support or enable NMDA to use YANG Push. If the --with-nmda parameter is set to ‘false’ then the <operational> datastore will consist of config=false data nodes and their ancestor lists, list keys, and containers.
The source code must be built using WITH_YANG_PUSH=1 or EVERYTHING=1 make flag in order for the YANG Push code to be included in the target binaries.

netconfd-pro {
  bundle yang-push
}

YANG Push CLI Parameters

Parameter	Default	Description
--bundle=yang-push	not loaded	The bundle parameter is used to enable or disable the entire YANG Push feature
--push-max-operational	4	Specifies the maximum number of on-change push subscriptions that can be in use at once for the <operational> datastore
--push-max-periodic	16	Specifies the maximum number of periodic push subscriptions that can be in use at once
--push-min-dampening	100	Specifies the minimum value for the 'dampening-period' parameter that will be accepted for an on-change push subscription. (centiseconds)
--push-min-period	100	Specifies the minimum value for the 'period' parameter that will be accepted for a periodic push subscription. (centiseconds)
--push-simop-enabled	true	Specifies if the simulated on-change push subscriptions should be enabled for the <operational> datastore.
--push-simop-patch-update	true	Specifies the notification message that should be used for a simulated on-change push subscription.
--push-simop-period	500	Specifies the value for the 'period' parameter that will be used for simulated operational on-change push subscription.

Configuring Named Filters

The /filters container and /filters/stream-filter list are defined in RFC 8639.

+--rw filters
|  +--rw stream-filter* [name]
|  |  +--rw name                           string
|  |  +--rw (filter-spec)?
|  |     +--:(stream-subtree-filter)
|  |     |  +--rw stream-subtree-filter?   <anydata> {subtree}?
|  |     +--:(stream-xpath-filter)
|  |        +--rw stream-xpath-filter?     yang:xpath1.0 {xpath}?
|  +--rw yp:selection-filter* [filter-id]
|     +--rw yp:filter-id                         string
|     +--rw (yp:filter-spec)?
|        +--:(yp:datastore-subtree-filter)
|        |  +--rw yp:datastore-subtree-filter?   <anydata> {sn:subtree}?
|        +--:(yp:datastore-xpath-filter)
|           +--rw yp:datastore-xpath-filter?     yang:xpath1.0 {sn:xpath}?

Named filters are fully supported, using the /filters subtree:

An event stream subscription can use the “stream-filter” list
A datastore subscriptions can use the selection-filter list.
Even though the contents of each list are similar, the content of the filters is not.
A “stream-filter” is a notification filter, and the content is exactly like the <filter> element used in the “create-subscription” operation.
A “selection-filter” is a retrieval filter, and the content is exactly like the <filter> element used in the “get” operation.

Filter Configuration Restrictions

A filter can be created at any time.
A filter can be modified at any time. If any subscriptions are using the filter then they will updated to use the new filter as soon as possible. This is usually as soon as any current message on that session is completed.
- If the new filter is not acceptable for a subscription using the filter then that subscription will be terminated. Use caution when modifying a filter that is in use.
A filter can only be deleted if no subscription is using it.
An empty filter is allowed, which means only a key leaf was provided and not a filter leaf. This allows placeholders to be used. It is not recommended to use this feature. An empty filter is the same as no filter, and if that is not allowed for a subscription, then it will be rejected (or deleted if already active).
Sending anydata for a subtree filter, or an xpath1.0 leaf for an XPath filter can be complex or even unsupported in the client. XPath filters will be accepted without prefixed identifiers. If prefixes are used, then they must be properly mapped to a module namespace URI using an “xmlns” attribute. If multiple nodes match the same local name then prefixes are required to pick only one of them.

Filter Examples

A subtree filter for an event stream that selects the “netconf-session-end” event:

<filters xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
  <stream-filter>
    <name>filter1</name>
    <stream-subtree-filter>
      <netconf-session-end xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-notifications" />
    </stream-subtree-filter>
  </stream-filter>
</filters>

An XPath filter for an event stream that selects the “netconf-session-start” event:

<filters xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
  <stream-filter>
    <name>filter2</name>
    <stream-xpath-filter xmlns:n="urn:ietf:params:xml:ns:yang:ietf-netconf-notifications"
       >/n:netconf-session-start</stream-xpath-filter>
  </stream-filter>
</filters>

A subtree filter for a datastore that selects the /nacm/groups subtree.

<filters xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
  <selection-filter xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
    <filter-id>filter2</filter-id>
    <datastore-subtree-filter>
      <nacm xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-acm">
        <groups />
      </nacm>
    </datastore-subtree-filter>
  </selection-filter>
</filters>

An XPath filter for a datastore that selects the /nacm/groups subtree:

<filters xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
  <selection-filter xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
    <filter-id>filter3</filter-id>
    <datastore-xpath-filter
      xmlns:n="urn:ietf:params:xml:ns:yang:ietf-netconf-acm"
      >/n:nacm/n:groups</datastore-xpath-filter>
  </selection-filter>
</filters>

YANG Push RPC Operations

The following RPC Operations are supported

establish-subscription
modify-subscription
delete-subscription
kill-subscription
resync-subscription

YANG Push Notification Events

The following Notifications are supported:

push-update
push-change-update
replay-completed
subscription-modified
subscription-resumed
subscription-suspended
subscription-terminated

Periodic Subscriptions

A periodic push subscription request that is granted will cause the specified data to be pushed at a specified interval.

The data is sent whether it changed or not. Every interval, a push-update notification with the specified data will be sent to the subscription.

The --push-max-periodic CLI parameter controls the number of these push subscriptions that can be enabled at once. This is the server total, so one subscription could use up all the subscriptions. The server does not prevent a session from establishing multiple subscriptions.

Use the <establish-subscription> operation to start a periodic subscription. The “datastore”, and “period” parameters must be provided.

There are no filter restrictions for a periodic subscription. Any filter that is allowed for a <get> operation may be used for a periodic subscription. If a named filter is used then it can be modified while a periodic subscription is active. The subscription output will be updated as soon as possible after a filter reference in use is modified.

The datastore parameter can be set to “candidate”, “operational”, or “running”. It must be set to “operational” to monitor config=false data nodes.

Example: Retrieve the /interfaces-state/interface entry for interface “lo” every 30 seconds.

The client might send this request:

<?xml version="1.0" encoding="UTF-8"?>
<rpc message-id="2" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <establish-subscription xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
    <datastore xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push"
      xmlns:ds="urn:ietf:params:xml:ns:yang:ietf-datastores">ds:operational</datastore>
    <datastore-subtree-filter xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
      <interfaces-state xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
        <interface>
          <name>lo</name>
        </interface>
      </interfaces-state>
    </datastore-subtree-filter>
    <periodic xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
      <period>3000</period>
    </periodic>
  </establish-subscription>
</rpc>

The server sends the subscription-id in the reply if the subscription was accepted.

This is important since multiple subscriptions per session are allowed. The <push-update> and <push-change-update> notifications each contain an <id> leaf that matches the value returned in the <rpc-reply> for the <establish-subscription> request.

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply message-id="2" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <id xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">1</id>
</rpc-reply>

The server will send an initial “push-update” notification right away, and then periodically every 30 seconds

<?xml version="1.0" encoding="UTF-8"?>
<notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
  <eventTime>2020-10-11T19:02:59Z</eventTime>
  <push-update xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
    <id>1</id>
    <datastore-contents>
      <interfaces-state xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
        <interface>
          <name>lo</name>
          <type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type"
            >ianaift:softwareLoopback</type>
          <admin-status>up</admin-status>
          <oper-status>up</oper-status>
          <if-index>1</if-index>
          <phys-address>00:00:00:00:00:00</phys-address>
          <speed>0</speed>
          <statistics>
            <in-octets>163797125</in-octets>
            <in-unicast-pkts>866265</in-unicast-pkts>
            <in-multicast-pkts>0</in-multicast-pkts>
            <in-discards>0</in-discards>
            <in-errors>0</in-errors>
            <out-octets>163797125</out-octets>
            <out-unicast-pkts>866265</out-unicast-pkts>
            <out-discards>0</out-discards>
            <out-errors>0</out-errors>
          </statistics>
        </interface>
      </interfaces-state>
    </datastore-contents>
  </push-update>
</notification>

Conventional On-Change Subscriptions

A conventional on-change subscription request that is granted will cause the specified data to be pushed when the data has changed. The “candidate” and “running” datastores can be monitored.

There is no internal overhead such as polling to support this subscription type. The server will check these subscriptions when a successful commit to the target datastore has been completed. Error operations are not reported to the subscription.

If the sync-on-start option is selected then a <push-update> will be sent when the subscription starts. After that, only <push-change-update> events will usually be sent. The resync-subscription operation will cause a new <push-update> notification to be generated as soon as possible.

The “on-change” presence container must be provided, and a “dampening-period” parameter must be provided. This must be greater or equal to the --push-min-dampening CLI parameter value.

The filter parameter is restricted. Two modes of operation are provided:

object monitoring mode: The filter must represent one configuration data node. All instances of that one object will be monitored.
datastore replication mode: No filter at all is used

Example: Object Monitoring Mode

Retrieve the /nacm/groups/group entry from <running> if any group is edited

Assume the client has created a named filter called “nacm-group”

<config>
  <filters xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
    <selection-filter xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
      <filter-id>nacm-group</filter-id>
      <datastore-xpath-filter
        xmlns:n="urn:ietf:params:xml:ns:yang:ietf-netconf-acm"
        >/n:nacm/n:groups/n:group</datastore-xpath-filter>
    </selection-filter>
  </filters>
</config>

The client might send this request using a “selection-filter-ref” instead of an inline filter:

The dampening period is set to 10 seconds in this example.

<rpc message-id="5" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <establish-subscription xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
    <datastore xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push"
      xmlns:ds="urn:ietf:params:xml:ns:yang:ietf-datastores">ds:running</datastore>
    <selection-filter-ref
      xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">nacm-group</selection-filter-ref>
    <on-change xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
      <dampening-period>1000</dampening-period>
      <excluded-change>move</excluded-change>
      <sync-on-start>true</sync-on-start>
    </on-change>
  </establish-subscription>
</rpc>

Assume the server sent an <id> response with a value of “1” like the example above.

If the “sync-on-start” parameter is set to “true” (the default) then the server will send a <push-update> notification as soon as the subscription starts:

<notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
  <eventTime>2020-10-11T19:39:21Z</eventTime>
  <push-update xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
    <id>1</id>
    <datastore-contents>
      <nacm xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-acm">
        <groups>
          <group>
            <name>test1</name>
          </group>
        </groups>
      </nacm>
    </datastore-contents>
  </push-update>
</notifications>

The server will send a <push-change-update> notification when the targeted entry is changed

<?xml version="1.0" encoding="UTF-8"?>
<notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
  <eventTime>2020-10-11T19:42:01Z</eventTime>
  <push-change-update xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
    <id>1</id>
    <datastore-changes>
      <yang-patch xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
        <patch-id>0</patch-id>
        <edit>
          <edit-id>E1</edit-id>
          <operation>create</operation>
          <target>/ietf-netconf-acm:nacm/groups/group=monitor</target>
          <value>
            <group xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-acm">
              <name>monitor</name>
              <user-name>admin1</user-name>
              <user-name>admin2</user-name>
            </group>
          </value>
        </edit>
      </yang-patch>
    </datastore-changes>
  </push-change-update>
</notification>

Example: Datastore Replication Mode

Retrieve the patch for <running> if any config is edited

The dampening period is set to 10 seconds in this example.

The client might send this request without any filter parameter:

<?xml version="1.0" encoding="UTF-8"?>
<rpc message-id="2" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <establish-subscription xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
    <datastore xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push"
           xmlns:ds="urn:ietf:params:xml:ns:yang:ietf-datastores">ds:running</datastore>
     <on-change xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
       <dampening-period>1000</dampening-period>
     </on-change>
  </establish-subscription>
</rpc>

The server sends an <rpc-reply> with a new subscription ID, since another is in progress:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply message-id="2" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <id xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">2</id>
</rpc-reply>

The server will send the initial <push-update> notification right away

<?xml version="1.0" encoding="UTF-8"?>
<notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
  <eventTime>2020-10-11T19:50:29Z</eventTime>
  <push-update xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
    <id>2</id>
    <datastore-contents>
      <filters xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
        <selection-filter xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
          <filter-id>nacm-group</filter-id>
          <datastore-xpath-filter xmlns:n="urn:ietf:params:xml:ns:yang:ietf-netconf-acm"
            >/n:nacm/n:groups/n:group</datastore-xpath-filter>
        </selection-filter>
      </filters>
      <nacm xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-acm">
        <groups>
          <group>
            <name>test1</name>
          </group>
          <group>
            <name>monitor</name>
            <user-name>admin1</user-name>
            <user-name>admin2</user-name>
          </group>
        </groups>
      </nacm>
    </datastore-contents>
  </push-update>
</notification>

If the <running> datastore changes at all the server will send a <push-change-update> notification:

In this example, the /nacm/rule-list entry is edited:

<?xml version="1.0" encoding="UTF-8"?>
<notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
  <eventTime>2020-10-11T20:03:37Z</eventTime>
  <push-change-update xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
    <id>2</id>
    <datastore-changes>
      <yang-patch xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
        <patch-id>1</patch-id>
        <edit>
          <edit-id>E1</edit-id>
          <operation>create</operation>
          <target>/ietf-netconf-acm:nacm/rule-list=allow-nacm-read</target>
          <value>
            <rule-list xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-acm">
              <name>allow-nacm-read</name>
              <group>monitor</group>
              <rule>
                <name>r1</name>
                <module-name>ietf-netconf-acm</module-name>
                <access-operations>read</access-operations>
                <action>permit</action>
                <comment>allow monitor group to read nacm</comment>
              </rule>
            </rule-list>
          </value>
        </edit>
      </yang-patch>
    </datastore-changes>
  </push-change-update>
</notification>

Simulated Operational On-Change Subscriptions

A simulated operational on-change subscription request that is granted will cause the specified data to be pushed when the data has changed. The “operational” datastore can be monitored. These subscriptions are only allowed if the --push-simop-enabled parameter is set to ‘true’.

There is an internal polling and processing overhead required to support this subscription type. The server will check these subscriptions when the --push-simop-period interval expires. If the “dampening-period” parameter is provided then the maximum of these two parameters is used for the internal polling interval, when dampening is required.

If the sync-on-start option is selected then a <push-update> will be sent when the subscription starts. After that, only <push-change-update> Event notifications will usually be sent. The <resync-subscription> operation will cause a new <push-update> Event notification to be generated as soon as possible.

The “on-change” presence container must be provided, and a “dampening-period” parameter must be provided. This must be greater or equal to the --push-min-dampening CLI parameter value.

There are no filter restrictions for a simulated operational subscription. Any filter that is allowed for a <get> operation may be used for a simulated operational subscription. If a named filter is used then it can be modified while a subscription is active. The subscription output will be updated as soon as possible after a filter reference in use is modified.

Example: Retrieve the /netconf-state/backup-files subtree if it changes

In this example, the <datastore-xpath-filter> does not use any prefixes. The server will accept this filter and find the correct data nodes even if the prefixes are missing.

In this example, the dampening period is 1 second.

<?xml version="1.0" encoding="UTF-8"?>
<rpc message-id="9" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <establish-subscription xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
    <datastore xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push"
      xmlns:ds="urn:ietf:params:xml:ns:yang:ietf-datastores">ds:operational</datastore>
    <datastore-xpath-filter xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push"
      >/netconf-state/backup-files</datastore-xpath-filter>
    <on-change xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
      <dampening-period>100</dampening-period>
    </on-change>
  </establish-subscription>
</rpc>

The server will send a <push-update> notification right away for this filter:

<notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
  <eventTime>2020-10-11T20:20:21Z</eventTime>
  <push-update xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
    <id>4</id>
    <datastore-contents>
      <netconf-state xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring">
        <backup-files xmlns="http://yumaworks.com/ns/yumaworks-system">
          <backup-file>
            <name>push1.xml</name>
            <backup-time>2020-08-29T20:10:23Z</backup-time>
          </backup-file>
          <backup-file>
            <name>push2.xml</name>
            <backup-time>2020-08-29T20:50:35Z</backup-time>
          </backup-file>
        </backup-files>
      </netconf-state>
    </datastore-contents>
  </push-update>
</notification>

If any backup file changes then the entire <backup-files> container is returned in the <push-change-update> notification:

Note that the operation will always be “replace” for a simulated operational on-change subscription.

<?xml version="1.0" encoding="UTF-8"?>
<notification xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
  <eventTime>2020-10-11T20:25:35Z</eventTime>
  <push-change-update xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
    <id>4</id>
    <datastore-changes>
      <yang-patch xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
        <patch-id>0</patch-id>
        <comment>simop</comment>
        <edit>
          <edit-id>E1</edit-id>
          <operation>replace</operation>
          <target>/</target>
          <value>
            <data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
              <netconf-state xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring">
                <backup-files xmlns="http://yumaworks.com/ns/yumaworks-system">
                  <backup-file>
                    <name>push1.xml</name>
                    <backup-time>2020-08-29T20:10:23Z</backup-time>
                  </backup-file>
                  <backup-file>
                    <name>push2.xml</name>
                    <backup-time>2020-08-29T20:50:35Z</backup-time>
                  </backup-file>
                  <backup-file>
                    <name>test-snap.xml</name>
                    <backup-time>2020-10-11T20:25:34Z</backup-time>
                  </backup-file>
                </backup-files>
              </netconf-state>
            </data>
          </value>
        </edit>
      </yang-patch>
    </datastore-changes>
  </push-change-update>
</notification>

Operational On-Change Subscriptions

An operational on-change subscription request that is granted will cause the specified data to be pushed when the subsystem responsible for pushing object updates sends a push-update to the main server. This mode is not yet implemented.