YANG Automation Introduction
YANG Automation is an industry term to describe a Network Management methodology used in modern computer networking devices. It is also called "model-based" network management. YANG (RFC 7950) is the data modeling language that is used.
Maybe you have heard of YANG data modeling. Maybe you are already using it.
What is YANG?
What is YANG Automation?
What actual problems are getting solved with YANG Automation?
Basic Terminology
A "network management interface" can be described as all the protocol procedures and operations used to interchange data between components in a distributed networking system.
A "data model" is the entire conceptual management interface.
A "data model module" or "module" is one part of the data model. Modules can import definitions from other modules and extend definitions in other modules.
A 'YANG Protocol" is a networking protocol that uses a YANG data model. There are 2 standard protocols that are in use at this time that use YANG:
What is YANG?
YANG is a data modeling language, like Java is a programming language.
YANG is source code for describing Application Programming Interfaces (APIs)
The APIs are based on a standard network management architecture
The “programs” are called YANG modules.
They express conceptual models, not instructions for a computer to execute.
The YANG modules are the templates that tell the automation software what to do.
Many different tools can use the YANG source code, just like a programming language
All tools work together automatically, using the YANG modules to drive program behavior
YANG is not a protocol. Several network management protocols use YANG.
YANG data does not have a fixed encoding format.
Several popular encodings are supported, including Extensible Markup Language (XML) and JavaScript Object Notation (JSON).
YANG is not Information Modeling like Unified Markup Language (UML)
Data modeling is more constrained to a specific use case (e.g., network management)
Why Does Automation Matter?
Maybe it is not obvious why procedures handled automatically by software are better than having a programmer write custom software to achieve the same procedures.
Save programming effort and computer resources by reusing the procedure instead of many copies
Improve consistency because all uses of the procedure will be the same
Distributed systems are very prone to errors related to the data exchange between components.
Formal models describing the data exchanges between components help solve this problem
The entire tool chain, including documentation, can be achieved without duplication of effort
The YANG data model is source code written once, but it is reused in many applications
Sounds like SNMP MIB Modules
The Simple Network Management Protocol (SNMP) was the first generation of network management data modeling. It uses a data modelling language called the Structure of Management Information, Version 2 (SMIv2).
YANG represents the second generation of network management. It turns out that the SNMP data modelling language was not that simple after all, and not that easy for people or tools to use. SNMP is still used for monitoring. Protocols using YANG provide complete management interfaces.
Networking equipment is very complex and the need for formal models describing the data was recognized many decades ago, which led to the SNMP standards in the 1980s. But SMIv2 data is not structured data, and the APIs supported by SNMP are not extensible either. The Extensible Markup Language (XML) has both, and it was the primary factor causing the shift away from SNMP.
Programmability in now King
There was a time when the Command Line Interface (CLI) was King. Configuring a router meant a human typing CLI commands over local or remote console terminal.
Configuring lots of routers meant a human in front of a computer with an open terminal to each router command line. Monitoring lots of routers means some SNMP polling but mostly a human typing “show” commands at the CLI. This completely non-standard, ad-hoc approach is inefficient, unreliable, and slow.
The new network management architecture is based on programmability. Complete management of a networking device is now required. This requires a standard data modeling language that can describe every operation that can be done on any device, even proprietary operations. YANG supports these requirements and more. It started out as just a replacement for SMIv2 data modeling, but it has also allowed extremely detailed refinements (called YANG extensions) which are used to drive YANG automation tools.
Network Management Tasks
What does an operator or application program with YANG?
Remote Procedure Call (RPC) Operations: Provides access to all the Command Line Interface (CLI) operations used to control a networking device. Tasks such as backup, restore, software update are supported with RPC operations.
Configuration: Information for the device to setup services, IP addresses, users and passwords, etc. is managed using a “datastore architecture”. Routers have thousands of configurable settings. A modular, robust, standard description of this configuration data is critical to avoid programming mistakes, network down time, security breaches, etc. The configuration datastore contains all the settings for the server.
Monitoring: Information about the health and activity of a networking device is called operational data. A new generation of YANG data models have replaced SNMP data models for monitoring. YANG data is hierarchical and much more efficient to retrieve than SNMP data.
Notifications: Important information about the health and activity of a networking device is called notification data. It is delivered right away to the client instead of waiting for a monitoring request to check the health of the device.
YANG Automation Provides Programmer and Operator Guardrails
One important use for YANG automation is the support for mechanisms to prevent programmers and operators from making mistakes. By using a data model abstraction with the appropriate metadata, the operations on the data can be protected.
For example, YANG has metadata built-in to identify passwords and other sensitive data. The management system can automatically treat this data with extra special care to prevent disclosure and misuse.
YANG also has built-in access control mechanisms to prevent operations from being used without special permission. Operations such as 'reboot' are often root-protected on an operating system, and YANG provides the same sort of protection.
Differences Between Model-Based and Non-Model-Based Network Management
To understand YANG Automation, it is helpful to understand the "old way" of network management, without any data models.
Without YANG Modeling:
Human Readable documentation is used be each developer to manually design and implement the "networking protocol"
Capabilities are specific to the protocol and version and are not extensible
Protocol Operations are specific to the protocol and version and are not extensible
Adding any new feature or change to any existing feature is considered a "protocol change" which requires a lot of time to design, implement, and deploy.
Ad-hoc design with no structured reuse or sharing of protocol components.
Monolithic design that is not modular. Instead the feature set is determined by the protocol version.
With YANG Modeling:
Human and Machine Readable documentation used to create an abstraction of the entire network management interface that can be implemented automatically or manually (usually some combination of both).
Standard structured modeling language allows all kinds of tools to automate tasks related to network management.
Capabilities are defined in the protocols and the data models.
Protocol-specific features are still defined in the protocol document
Application-related features are defined in the YANG data model
Protocol Operations are defined in the data model and are not hard-wired to any specific protocol.
Multiple northbound protocols can support the same management feature using the same YANG data model
YANG Actions allow protocol operations that are specific to data, in addition to the general protocol operations.
Adding new features is must faster than before:
YANG is modular so there are many existing modeling components that can be easily reused.
No protocol changes are required at all. All YANG protocols are designed to work with an arbitrary set of YANG modules. This allows networking device vendors to easily change the YANG module set used by each device.
Many management tasks such as configuration and monitoring can be fully or partially automated:
No developer effort required
Centralized implementation improves software consistency and memory efficiency
Structured Server and Application SDKs allow module-specific instrumentation to be easily added to the management system.
YANG Automation Uses
The YANG language includes many ways to specify conditions for database validity, which traditionally are only documented in description statements and external developer documentation.
Even with all the machine-readable properties defined in YANG (see the YANG-based Automation section), there are many standard and non-standard extensions to YANG, to increase the ability to automate tool behavior related to YANG data models.
YANG supports external statements, which allows anybody to add new statements to the standard syntax, in a way that allows all tools to either use the extension or skip over the extension. A YANG 1.1 compliant tool MAY ignore any external statement, but it MUST be able to at least parse it and ignore it.
Extension statements are used two ways in YANG:
Metadata associated with other YANG statements
Abstract data structure definitions
YANG Extensions as Metadata
The main mechanism available in YANG for tool automation is the extension statement, when used to define metadata that is associated with the parent statement. This is usually a data definition statement, but it could be any YANG statement.
Servers and clients and other tools can code to the metadata instead of hard-wired to specific YANG identifiers. All objects using the same metadata get the same treatment.
Define the YANG extension (or use an existing one)
extension my-extension { description " - Define the purpose of the extension - Define an argument if needed - Define which parent statements are allowed - Provide an example of the statement usage "; }
Use the extension as an external statement in other YANG statements
container data1 { ex:my-extension; leaf leaf1 { type string; } // ... }
Parse and save the extension statements as metadata in the server
Use the metadata to control various server and protocol behavior
YANG Extensions as Abstract Data
YANG is very useful for defining data structures that can be used within client and server implementations. They are also used to define file and artifact representations of YANG data.
Unfortunately, plain YANG data definition statements (e.g., container, list) cannot be used because these statements define YANG and NMDA datastore contents. Only data structures for datastore contents can use these statements as top-level statements in YANG.
Note
Abstract data structure names cannot conflict with any other structure or plain YANG data definition statement.
Use the sx:structure extension as an external top level statement in a YANG module:
sx:structure mydata1 { leaf leaf1 { type string; } }
The server will automatically process the 'structure' extension and save a top-level object named 'mydata1'
Use the object access APIs to retrieve the obj_template_t for the data structure so it can be used within the program (e.g., parse input or generate output).