Binary Push
Note
This feature is only available in the following release trains:
22.10T
23.10T
The source code is present in the 23.10 release train but it is not enabled because the standards and tools are not finalized yet.
This feature allows all NETCONF notifications to be sent using binary encoding instead of XML. The <notification> message size can be drastically reduced.
Uses Standard CBOR Encoding
Encoding of Data Modeled with YANG in the Concise Binary Object Representation (CBOR) (RFC 9254)
Standard YANG Push
All supported YANG Push functionality can be used with binary encoding. Simply set the 'encoding' parameter to
encode-yang-cborin the <establish-subscription> operation.> establish-subscription on-change/dampening-period=500 \ datastore-xpath-filter="/interfaces-state/interface[name='eno1']" \ datastore=operational \ encoding=encode-yang-cbor
Extending NETCONF
If the client selects CBOR encoding then the notifications sent to the session will be encoded in CBOR instead of XML.
All other messages are encoded in XML. The client will examine the incoming messages to determine the message encoding.
This extension to the NETCONF standard is only used if the client opts-in to request binary encoding.
Preserves Application Layer
There are no special protocols or new programming requirements needed to use Binary Push. The internal code (such as Server Instrumentation Libraries) do not need any changes at all to support binary encoding. The protocol operations for notifications and YANG Push are unchanged.
Centralized Code, not Per Message
YANG CBOR does not need any special files or generated code to support specific YANG Push reports. A small amount (200 Kbytes) is used to support an arbitrary number of YANG modules. The encode and decode algorithms are centralized so no extra code is needed for each 'Push report'.
Faster Smarter Parser Code
CBOR encoding uses a deterministic message framing mechanism, so processing is fast and does not require much memory. There are no cases where strings have to be examined, looking for special character sequences that need to be converted for actual use. XML and JSON both require such processing.
Maximum Flexibility with Subtree and XPath FIltering
There are no 'canned' reports in YANG Push. Instead, standard NETCONF filters for Subscribed Notifications or YANG Push are used.
Zero Impact on Server Instrumentation Code
The SIL "GET2" callback does not require any changes to support CBOR encoding.
Better Message Fidelity than XML or JSON
The YANG CBOR encoding requires less bytes than XML or JSON but manages to provide better decoding information for certain corner-cases:
YANG union: CBOR tags help distinguish the actual member type used
YANG anydata: descendant nodes within an 'anydata' or 'anyxml' node can be easily identified by their SID value. XML and YANG encoding rely on hacks and conventions to treat an 'anydata' node as a 'root' (such as the <config> element). CBOR precisely identifies any data node in this case.
Binary Push Overview
CBOR (RFC 8949) is an efficient binary encoding standard used in constrained environments. It can be used with YANG Push to maximize the network efficiency.
YANG CBOR (RFC 9254) defines the specific encoding rules to encode and decode YANG data in CBOR. The main source of the efficient encoding is the way data node identifiers are encoded. Instead of using a hierarchy of strings matching the YANG node hierarchy, an integer is assigned to every possible data node.
YANG SID (draft) provides a file format and procedures for creating and maintaining YANG Schema Item Identifier (SID) files for each corresponding YANG module.
Note
The SID file standards and procedures are still in progress in the IETF. This mode of operation will be supported in the future when it is ready.
LSID is a YumaWorks adaptation of the SID file approach. It uses Localized Schema Item Identifiers instead of Global identifiers. In this mode the server assigns SID values to all the relevant schema nodes. There is no attempt to share the LSID assignments across servers. The client must retrieve the LSID mappings from the server using the <get-yang-sid-data> operation, (similar to the <get-schema> operation for YANG files).
There are some advantages to this approach, for use with YANG Push:
The number space for LSID is not hard-wired to 64 bits. A 16, 32, or 64 bit sized LSID can be used. Currently, only the 32-bit variant is supported.
The server can assign sequential SID values, without any wasted identifiers in the number space. This allows CBOR encoding to be more efficient.
The server can assign LSID values that are not supported by the SID standard, such as the <notification> protocol message nodes, which are not modelled in YANG.
The server can assign LSID values for schema mounted objects. This cannot be supported by the SID standard.
The 'lsid-size' typedef explains the different LSID usage modes. The server is built with the default mode 'lsid32'.
typedef lsid-size {
type enumeration {
enum lsid16 {
description
"The SID width is 16 bits.
The maximum SID value is 15 bits (32K values)";
}
enum lsid32 {
description
"The SID width is 32 bits.
The maximum SID value is 31 bits (2G values)";
}
enum lsid64 {
description
"The SID width is 64 bits.
The maximum SID value is 63 bits (9.2 x 10^^18 values)";
}
}
description
"If the yang-sid-mode is 'lsid' or 'lsid-file' then the server
implementation will pick a SID size at build-time.";
}
Using Binary Push
This feature must be enabled in the server so it can be used. It is not enabled by default at this time.
For source code users:
All programs must be built with the WITH_YANG_CBOR=1 and WITH_YANG_PUSH=1 flags
There are 2 new CLI parameters that must be set to enable Binary Push in the server:
--with-yang-cbor: set to 'true'
--yang-sid-mode: set to 'lsid'
bundle yang-push
with-yang-cbor true
yang-sid-mode lsid
When a yangcli-pro session is started, it will check if the yumaworks-yang-cbor.yang module is listed in the server capabilities or YANG library. If so, then yangcli-pro will attempt to invoke the <get-yang-sid-data> operation to read the SID mappings for the server.
Note
yangcli-pro does not read the /yang-sid-info data at this time.
It does not need to know the server SID mode because only 'lsid'
is currently supported.
To request CBOR encoding, simply set the 'encoding' parameter in the <establish-subscription> operation.
Example 'NETCONF' stream subscription:
In this simple example, the 'NETCONF' event stream subscription is requested with CBOR encoding. All notifications sent on this subscription will use CBOR encoding.
yangcli-pro command:
user@server1> establish-subscription stream=NETCONF encoding=encode-yang-cbor
NETCONF XML request message:
<?xml version="1.0" encoding="UTF-8"?>
<rpc message-id="4"
xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<establish-subscription xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
<stream>NETCONF</stream>
<encoding xmlns:ycbor="urn:yumaworks:params:xml:ns:yang:yumaworks-yang-cbor">ycbor:encode-yang-cbor</encoding>
</establish-subscription>
</rpc>
Example 'netconf-config-change' notification:
Incoming msg for session 1
***binary***
buff cnt: 1
cbor_parse_buffer: unspecified object
Binary buffer dump (130 of 130)
A1 D8 2F 19 06 4F BF 01 ../..O..
78 1B 32 30 32 32 2D 31 x.2022-1
30 2D 31 32 54 30 31 3A 0-12T01:
34 38 3A 31 34 2E 39 37 48:14.97
38 33 35 35 5A 39 03 31 8355Z9.1
BF 01 BF 05 64 61 6E 64 ....dand
79 06 04 07 69 31 32 37 y...i127
2E 30 2E 30 2E 31 FF 09 .0.0.1..
00 0A 9F BF 01 78 35 2F .....x5/
6E 61 63 6D 3A 6E 61 63 nacm:nac
6D 2F 6E 61 63 6D 3A 67 m/nacm:g
72 6F 75 70 73 2F 6E 61 roups/na
63 6D 3A 67 72 6F 75 70 cm:group
5B 6E 61 63 6D 3A 6E 61 [nacm:na
6D 65 3D 22 61 64 6D 69 me="admi
6E 32 22 5D 02 02 FF FF n2"]....
FF FF
The yangcli-pro log display for this notification:
notification {
eventTime 2022-10-12T01:48:14.978355Z
netconf-config-change {
changed-by {
username andy
session-id 4
source-host 127.0.0.1
}
datastore running
edit {
target /nacm:nacm/nacm:groups/nacm:group[nacm:name="admin2"]
operation create
}
}
}
Binary Push Example
In this example the ietf-interfaces.yang module is used in a YANG Push periodic subscription. One interface will be monitored and the data is sent every 6 seconds.
The --yang-sid-mode CLI parameter is set to 'lsid'. The 'debug4' log file shows the SID assignments for the server.
yang_lsid: Assign SID '632' to object 'interfaces-state'
yang_lsid: Assign SID '633' to object 'interface'
yang_lsid: Assign SID '634' to object 'name'
yang_lsid: Assign SID '635' to object 'type'
yang_lsid: Assign SID '636' to object 'admin-status'
yang_lsid: Assign SID '637' to object 'oper-status'
yang_lsid: Assign SID '638' to object 'last-change'
yang_lsid: Assign SID '639' to object 'if-index'
yang_lsid: Assign SID '640' to object 'phys-address'
yang_lsid: Assign SID '641' to object 'higher-layer-if'
yang_lsid: Assign SID '642' to object 'lower-layer-if'
yang_lsid: Assign SID '643' to object 'speed'
yang_lsid: Assign SID '644' to object 'statistics'
yang_lsid: Assign SID '645' to object 'discontinuity-time'
yang_lsid: Assign SID '646' to object 'in-octets'
yang_lsid: Assign SID '647' to object 'in-unicast-pkts'
yang_lsid: Assign SID '648' to object 'in-broadcast-pkts'
yang_lsid: Assign SID '649' to object 'in-multicast-pkts'
yang_lsid: Assign SID '650' to object 'in-discards'
yang_lsid: Assign SID '651' to object 'in-errors'
yang_lsid: Assign SID '652' to object 'in-unknown-protos'
yang_lsid: Assign SID '653' to object 'out-octets'
yang_lsid: Assign SID '654' to object 'out-unicast-pkts'
yang_lsid: Assign SID '655' to object 'out-broadcast-pkts'
yang_lsid: Assign SID '656' to object 'out-multicast-pkts'
yang_lsid: Assign SID '657' to object 'out-discards'
yang_lsid: Assign SID '658' to object 'out-errors'
The yangcli-pro program will automatically invoke the <get-yang-sid-data> operation and assign the SID values to the YANG objects for this session.
Example snippet from the XML response:
<sid-item>
<sid>632</sid>
<namespace>data</namespace>
<identifier>/ietf-interfaces:interfaces-state</identifier>
</sid-item>
<sid-item>
<sid>633</sid>
<namespace>data</namespace>
<identifier>/ietf-interfaces:interfaces-state/interface</identifier>
</sid-item>
<sid-item>
<sid>634</sid>
<namespace>data</namespace>
<identifier>/ietf-interfaces:interfaces-state/interface/name</identifier>
</sid-item>
<sid-item>
<sid>635</sid>
<namespace>data</namespace>
<identifier>/ietf-interfaces:interfaces-state/interface/type</identifier>
</sid-item>
<sid-item>
<sid>636</sid>
<namespace>data</namespace>
<identifier>/ietf-interfaces:interfaces-state/interface/admin-status</identifier>
</sid-item>
<sid-item>
<sid>637</sid>
<namespace>data</namespace>
<identifier>/ietf-interfaces:interfaces-state/interface/oper-status</identifier>
</sid-item>
<sid-item>
<sid>638</sid>
<namespace>data</namespace>
<identifier>/ietf-interfaces:interfaces-state/interface/last-change</identifier>
</sid-item>
The yangcli-pro 'establish-subscription' command may look as follows:
establish-subscription periodic/period=600 \
datastore-xpath-filter="/interfaces-state/interface[name='eno1']" \
datastore=operational \
encoding=encode-yang-cbor
The XML request message received by the server for this command:
<?xml version="1.0" encoding="UTF-8"?>
<rpc message-id="4"
xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<establish-subscription xmlns="urn:ietf:params:xml:ns:yang:ietf-subscribed-notifications">
<encoding xmlns:ycbor="urn:yumaworks:params:xml:ns:yang:yumaworks-yang-cbor">ycbor:encode-yang-cbor</encoding>
<datastore xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push" xmlns:ds="urn:ietf:params:xml:ns:yang:ietf-datastores">ds:operational</datastore>
<datastore-xpath-filter xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">/interfaces-state/interface[name='eno1']</datastore-xpath-filter>
<periodic xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push">
<period>600</period>
</periodic>
</establish-subscription>
</rpc>
The server will send binary encoded buffers for this subscription. The following example 'debug4' buffer shows a complete push update notification for the subscription:
Incoming msg for session 3
***binary***
buff cnt: 1
cbor_parse_buffer: unspecified object
Binary buffer dump (146 of 146)
A1 19 06 4F BF 01 78 1B ...O..x.
32 30 32 32 2D 31 30 2D 2022-10-
31 32 54 30 31 3A 32 38 12T01:28
3A 31 36 2E 33 35 33 33 :16.3533
38 34 5A 39 01 21 BF 01 84Z9.!..
01 02 BF 39 02 B6 BF 01 ...9....
9F BF 01 64 65 6E 6F 31 ...deno1
02 19 01 0B 03 01 04 01 ........
06 02 07 71 30 34 3A 39 ...q04:9
32 3A 32 36 3A 62 66 3A 2:26:bf:
63 30 3A 62 64 0A 1A 3B c0:bd..;
9A CA 00 0B BF 02 1B 00 ........
00 00 2E 04 8C 10 B9 03 ........
1A 09 7F 4A 6F 05 1A 00 ...Jo...
2E 95 2C 06 01 07 00 09 ..,.....
1B 00 00 00 4B 61 7B 39 ....Ka{9
C4 0A 1A 0F 02 DE F2 0D ........
00 0E 00 FF FF FF FF FF ........
FF FF
The following notification is displayed after it is decoded into a val_value_t tree (internal reprepsentaion).
Got 1 data nodes returned from CBOR parse
Parsed CBOR message on session 3 OK
ncEvent:notification {
eventTime 2022-10-12T01:28:16.353384Z
yp:push-update {
id 1
datastore-contents {
if:interfaces-state {
interface eno1 {
name eno1
type ianaift:ethernetCsmacd
admin-status up
oper-status up
if-index 2
phys-address 04:92:26:bf:c0:bd
speed 1000000000
statistics {
in-octets 185
in-unicast-pkts 159337071
in-multicast-pkts 3052844
in-discards 1
in-errors 0
out-octets 196
out-unicast-pkts 251846386
out-discards 0
out-errors 0
}
}
}
}
}
}