Multi-Instance Mode
Note
This feature is only available in the 23.10T release train.
The system must be properly configured so each server instance has a unique ID and IP endpoints.
The netconfd-pro server can be deployed so that multiple instances of the server can be run on the same host. This is called Multi-Instance Mode. There are some advantages to this deployment mode:
Allows deployment of micro-services as independent server applications.
Allows independent security and access control configuration
Allows complete isolation of YANG data and internal libraries
There are some usage restrictions and configuration requirements in this mode:
Each server instance is independent of all other server instances
Each server instance maintains its own data
All subsystem instances must belong to a specific server instance
The normal operating mode is called Single-Instance Mode.
The host running the server cannot mix server instances using different modes.
How It Works
Each server instance must be configured with the --server-root CLI parameter, and assigned a unique name for the server instance. This is a static assignment. The same ID must be used across reboots to maintain server instance data.
Different IP endpoints are used for each server instance.
A Server Root Map File is used to configure indirect sockets (e.g. SSH or HTTP/S) to access each server instance
Various CLI parameters are used to configure direct sockets (e.g., sil-sa-app) to access each server instance
Each server instance should have its own configuration;
The default config file is
/etc/yumapro/netconfd-pro.conf
The --config CLI parameter should be used instead of the default to provide a different configuration to each server instance.
Server Instance Configuration
Each server instance is completely independent of all other instances. There are many configuration requirements to consider, and some existing CLI parameters affect how Multi-Instance Mode behaves on the system.
The entire server system must be properly configured if multiple server instances are used. E.g. each subsystem that connects to the server must be configured with the proper server root ID.
Each server instance must be carefully configured to listen on different IP addresses and/or TCP ports for client sessions
For NETCONF over SSH, the OpenSSH server must be used, and properly configured to listen on multiple addresses and/or ports. The server root map file must be present.
For NETCONF over TLS, the server listens directly so the --netconf-tls-address and/or --netconf-tls-port parameters must be set to unique values in each server instance.
For RESTCONF: the WEB server (Apache2 or NGINX) must be used and properly configured to listen on multiple addresses and/or ports. The server root map file must be present.
For subsystems like sil-sa-app or db-api-app, the --server-root parameter must be used to select the server instance that the subsystem will join
For yp-shell, the OpenSSH server must be used, and properly configured to listen on multiple addresses and/or ports. The server root map file must be present.
Configurable Components
There are many system components that could be shared, or be independent, depending on the configuration.
Dynamic Libraries
YP System Library (libyp-system.so)
SIL and SIL-SA libraries
Some shared library locations can be configured using the --runpath CLI parameter.
The default location for these libraries is the
/usr/lib/yumapro
directory.
YANG Modules
The YANG modules used by the server can be shared if desired.
The --modpath CLI parameter can be used to select non-default module search path locations.
The default location for YANG modules is the
/usr/share/yumapro/modules
directory.
Independent Components
The various components can be shared or configured to be independent of each other:
command line parameters are always independent
netconfd-pro.conf
sm-config.json
YANG modules
server data files
startup-cfg.xml
factory-startup-cfg.xml
backups
sockets
server log
audit log
Server Root Map File
The netconf-subsystem-pro program will now look for a server root map file when connecting a NETCONF over SSH, RESTCONF, or other local connection to the netconfd-pro server.
This file is checked first so make sure it is only found if it is in use. The server will not operate correctly if this file is found in Single-Instance Mode.
The netconf-subsystem-pro program will first check an environment variable for the complete filespec. If this variable is not present, then the default server map file will be checked
SERVER_ROOT_MAPFILE
environment variableContains a full filespec for the location of the server root map file
This variable is in the environment of sshd child process.
Use the 'SetEnv' directive in the 'sshd_config' file to set this environment variable.
SetEnv "SERVER_ROOT_MAPFILE=/opt/yumapro/conf/server-map.txt"
The 'netconf-subsystem-pro' program will check its environment for this variable.
/etc/yumapro/server-root-map.txt
The 'etc/yumapro' directory is checked for a file named 'server-root-map.txt'
There must be exactly one Server Root Map file used on each host
Each server instance must be handled by the mapping file or no sessions to that server will be possible
Example sshd_config File
The following example shows:
ports configured for this system
server root map environment variable set
Port 22
Port 830
Port 9000
Port 9001
Port 9002
SetEnv "SERVER_ROOT_MAPFILE=/etc/yumapro/my-server-root.txt"
Example Server Map File
The following example shows:
3 server instances named 'sr1', 'sr2', and 'sr3'.
Server 'sr1' is handling all the default ports
TCP port mapping is used, so all IP address fields are shown as the wildcard character.
NETCONF over SSH port mappings for ports 830, 9000, 9001, and 9002 are shown
RESTCONF port mapping for port 80 is shown
yp-shell mapping for port 22 is shown
# server root map
# IP PORT SRID
* 9000 sr1
* 9001 sr2
* 9003 sr3
* 830 sr1
* 80 sr1
* 22 sr1
Server Root Map Syntax
The server root map file uses UTF-8 text encoding
A newline 'n' is expected at the end of each line
A line beginning with the '#' character is considered to be a comment and is skipped
A non-comment line contains one server root mapping
Each mapping contains 3 fields:
Fields 1 and 2 are followed by one space ' ' character.
Field 3 is followed by one newline '\n' character if there are more lines.
Field 1: IP
A single asterisk indicates a wildcard and will match all IP addressed
An unquoted string will be checked for a match against the server IP address used by the client connection. An exact match is required.
Field 2: PORT
A single asterisk indicates a wildcard and will match all TCP port numbers
An unquoted string will be checked for a match against the server TCP port used by the client connection. An exact match is required.
Field 3: Server Root ID (SRID)
An unquoted string is expected. This is the server root ID string that will be used when selecting a local socket for the connection.
This string must conform to the 'NcxName' data type.
The length must be 1 to 64 characters.
This string will be used in directory and file names to avoid naming collisions when multiple server instances are running at once
Socket Search Order
Search With a Server Root ID
If a server root ID is found for the server 'IP:PORT' used in the session
Look for TCP port mapping file (e.g. SRID = sr1)
/tmp/netconfd-pro-subsys-info.sr1.txt
If found then use this info for TCP connect
Look for the AFLOCAL + SRID socket
/tmp/ncxserver.sr1.sock
If found then use this socket for AFLOCAL connect
Look for the FHS AFLOCAL + SRID socket
/var/run/netconfd-pro/ncxserver.sr1.sock
If found then use this socket for AFLOCAL connect
If not found then connection fails
Normal Search
If a server root ID is not found then the netconf-subsystem-pro program will assume Single-Instance Mode is being used, and the normal search for the socket is done:
Look for TCP port mapping file
/tmp/netconfd-pro-subsys-info.txt
If found then use this info for TCP connect
Look for the AFLOCAL socket
/tmp/ncxserver.sock
If found then use this socket for AFLOCAL connect
Look for the FHS AFLOCAL socket
/var/run/netconfd-pro/ncxserver.sock
If found then use this socket for AFLOCAL connect
If not found then connection fails
Server File Locations
The Server Root ID (SRID) is used to create unique names in two ways:
SRID Used In a Directory Path
The server data files are kept in a directory using the value provided in the --server-root CLI parameter.
Pattern:
$HOME/.yumapro/<SRID>/backups
Example: (SRID == 'sr1')
$HOME/.yumapro/sr1/backups
SRID Used In a File Name
In some cases, the SRID is added to a filename to prevent naming collisions with other server instances.
Pattern:
/tmp/ncxserver.<SRID>.sock
Example: (SRID == 'sr1')
/tmp/ncxserver.sr1.sock
CLI Parameters Affecting File Locations
The --server-root and --fileloc-fhs CLI parameters affect the directory locations and file names used by the server.
There are four different modes created from these parameters:
Server File Types
There are several file types listed in the tables below:
server log: Main server log file
Override default with the --log CLI parameter
audit log: Server event audit log file
Override default with the --audit-log CLI parameter
Disable in FHS mode with the --no-audit-log CLI parameter
startup file: Server startup datastore contents
Override default with the --startup CLI parameter
Disable with the --no-startup CLI parameter
Default location depends on --fileloc-fhs setting
Search path depends on the --datapath setting
factory startup: Server startup datastore contents if a factory restart is done
Override default with the --startup-factory-file CLI parameter
Default location depends on --fileloc-fhs setting
Search path depends on the --datapath setting
TXID file: Server edit transaction ID file
Not configurable
Location depends on --fileloc-fhs setting
backups: Server backups directory to store files created with the <backup> operation
Not configurable
Location depends on --fileloc-fhs setting
PID file: Text file containing the process ID of the server instance
Not configurable
Location depends on --fileloc-fhs setting
AF socket: Socket file used for AFLOCAL connections
Not configurable
Location depends on --fileloc-fhs setting
Normal Mode
This is the default mode:
--fileloc-fhs='false'
--server-root not used
File Type |
Default Value |
---|---|
server log |
STDOUT |
audit log |
STDOUT |
startup file |
$HOME/.yumapro/startup-cfg.xml |
factory startup |
$HOME/.yumapro/factory-startup-cfg.xml |
TXID file |
$HOME/.yumapro/startup-cfg-txid.txt |
backups |
$HOME/.yumapro/backups/ |
PID file |
$HOME/.yumapro/netconfd-pro.pid |
AF socket |
/tmp/ncxserver.sock |
FHS Mode
This is a common usage mode to run the server as 'root':
--fileloc-fhs='true'
--server-root not used
File Type |
Default Value |
---|---|
server log |
/var/log/netconfd-pro/server.log |
audit log |
/var/log/netconfd-pro/audit.log |
startup file |
/var/lib/netconfd-pro/startup-cfg.xml |
factory startup |
/var/lib/netconfd-pro/factory-startup-cfg.xml |
TXID file |
/var/lib/netconfd-pro/startup-cfg-txid.txt |
backups |
/var/lib/netconfd-pro/backups/ |
PID file |
/var/run/netconfd-pro/netconfd-pro.pid |
AF socket |
/var/run/netconfd-pro/ncxserver.sock |
SRID Mode
This is the normal way to run in Multi-Instance Mode:
--fileloc-fhs='false'
--server-root is used (e.g., 'sr1')
File Type |
Default Value |
---|---|
server log |
STDOUT |
audit log |
STDOUT |
startup file |
$HOME/.yumapro/sr1/startup-cfg.xml |
factory startup |
$HOME/.yumapro/sr1/factory-startup-cfg.xml |
TXID file |
$HOME/.yumapro/sr1/startup-cfg-txid.txt |
backups |
$HOME/.yumapro/sr1/backups/ |
PID file |
$HOME/.yumapro/netconfd-pro.sr1.pid |
AF socket |
/tmp/ncxserver.sr1.sock |
FHS SRID Mode
This is the way to run in Multi-Instance Mode as 'root':
--fileloc-fhs='true'
--server-root is used (e.g., 'sr1')
File Type |
Default Value |
---|---|
server log |
/var/log/netconfd-pro/server.sr1.log |
audit log |
/var/log/netconfd-pro/audit.sr1.log |
startup file |
/var/lib/netconfd-pro/sr1/startup-cfg.xml |
factory startup |
/var/lib/netconfd-pro/sr1/factory-startup-cfg.xml |
TXID file |
/var/lib/netconfd-pro/sr1/startup-cfg-txid.txt |
backups |
/var/lib/netconfd-pro/sr1/backups/ |
PID file |
/var/run/netconfd-pro/netconfd-pro.sr1.pid |
AF socket |
/var/run/netconfd-pro/ncxserver.sr1.sock |