Multi-Instance Mode

Note

This feature is only available in the 23.10T release train.
The system must be properly configured so each server instance has a unique ID and IP endpoints.

The netconfd-pro server can be deployed so that multiple instances of the server can be run on the same host. This is called Multi-Instance Mode. There are some advantages to this deployment mode:

Allows deployment of micro-services as independent server applications.
Allows independent security and access control configuration
Allows complete isolation of YANG data and internal libraries

There are some usage restrictions and configuration requirements in this mode:

Each server instance is independent of all other server instances
Each server instance maintains its own data
All subsystem instances must belong to a specific server instance
The normal operating mode is called Single-Instance Mode.
The host running the server cannot mix server instances using different modes.

How It Works

Each server instance must be configured with the --server-root CLI parameter, and assigned a unique name for the server instance. This is a static assignment. The same ID must be used across reboots to maintain server instance data.

Different IP endpoints are used for each server instance.

A Server Root Map File is used to configure indirect sockets (e.g. SSH or HTTP/S) to access each server instance
Various CLI parameters are used to configure direct sockets (e.g., sil-sa-app) to access each server instance

Each server instance should have its own configuration;

The default config file is /etc/yumapro/netconfd-pro.conf
The --config CLI parameter should be used instead of the default to provide a different configuration to each server instance.

Server Instance Configuration

Each server instance is completely independent of all other instances. There are many configuration requirements to consider, and some existing CLI parameters affect how Multi-Instance Mode behaves on the system.

The entire server system must be properly configured if multiple server instances are used. E.g. each subsystem that connects to the server must be configured with the proper server root ID.
Each server instance must be carefully configured to listen on different IP addresses and/or TCP ports for client sessions
- For NETCONF over SSH, the OpenSSH server must be used, and properly configured to listen on multiple addresses and/or ports. The server root map file must be present.
- For NETCONF over TLS, the server listens directly so the --netconf-tls-address and/or --netconf-tls-port parameters must be set to unique values in each server instance.
- For RESTCONF: the WEB server (Apache2 or NGINX) must be used and properly configured to listen on multiple addresses and/or ports. The server root map file must be present.
- For subsystems like sil-sa-app or db-api-app, the --server-root parameter must be used to select the server instance that the subsystem will join
- For yp-shell, the OpenSSH server must be used, and properly configured to listen on multiple addresses and/or ports. The server root map file must be present.

Shared Components

There are some components in the system that do not change and a single instance of of the component is still used:

NETCONF over SSH: OpenSSH Server
yp-shell over SSH: OpenSSH Server
RESTCONF over HTTP/S: Apache2 or NGINX server

These servers must be properly configured to listen on multiple IP addresses or multiple TCP ports.

The Server Root Map File must be configured and contain the mappings based on the IP addresses and/or TCP ports configured in the SSH or HTTP/S server.

Example: sshd_config to listen on ports 9000 - 9003, and the default port of 830

Port 22
Port 830
Port 9000
Port 9001
Port 9002

Configurable Components

There are many system components that could be shared, or be independent, depending on the configuration.

Dynamic Libraries
- YP System Library (libyp-system.so)
- SIL and SIL-SA libraries
- Some shared library locations can be configured using the --runpath CLI parameter.
- The default location for these libraries is the /usr/lib/yumapro directory.
YANG Modules
- The YANG modules used by the server can be shared if desired.
- The --modpath CLI parameter can be used to select non-default module search path locations.
- The default location for YANG modules is the /usr/share/yumapro/modules directory.

Independent Components

The various components can be shared or configured to be independent of each other:
- command line parameters are always independent
- netconfd-pro.conf
- sm-config.json
- YANG modules
- server data files
  - startup-cfg.xml
  - factory-startup-cfg.xml
  - backups
- sockets
- server log
- audit log

Server Root Map File

The netconf-subsystem-pro program will now look for a server root map file when connecting a NETCONF over SSH, RESTCONF, or other local connection to the netconfd-pro server.

This file is checked first so make sure it is only found if it is in use. The server will not operate correctly if this file is found in Single-Instance Mode.

The netconf-subsystem-pro program will first check an environment variable for the complete filespec. If this variable is not present, then the default server map file will be checked

SERVER_ROOT_MAPFILE environment variable
- Contains a full filespec for the location of the server root map file
- This variable is in the environment of sshd child process.
- Use the 'SetEnv' directive in the 'sshd_config' file to set this environment variable.
```
SetEnv "SERVER_ROOT_MAPFILE=/opt/yumapro/conf/server-map.txt"
```
- The 'netconf-subsystem-pro' program will check its environment for this variable.
/etc/yumapro/server-root-map.txt
- The 'etc/yumapro' directory is checked for a file named 'server-root-map.txt'
There must be exactly one Server Root Map file used on each host
Each server instance must be handled by the mapping file or no sessions to that server will be possible

Example sshd_config File

The following example shows:

ports configured for this system
server root map environment variable set

Port 22
Port 830
Port 9000
Port 9001
Port 9002
SetEnv "SERVER_ROOT_MAPFILE=/etc/yumapro/my-server-root.txt"

Example Server Map File

The following example shows:

3 server instances named 'sr1', 'sr2', and 'sr3'.
Server 'sr1' is handling all the default ports
TCP port mapping is used, so all IP address fields are shown as the wildcard character.
NETCONF over SSH port mappings for ports 830, 9000, 9001, and 9002 are shown
RESTCONF port mapping for port 80 is shown
yp-shell mapping for port 22 is shown

# server root map
# IP PORT SRID
* 9000 sr1
* 9001 sr2
* 9003 sr3
* 830 sr1
* 80 sr1
* 22 sr1

Server Root Map Syntax

The server root map file uses UTF-8 text encoding
A newline 'n' is expected at the end of each line
A line beginning with the '#' character is considered to be a comment and is skipped
A non-comment line contains one server root mapping
Each mapping contains 3 fields:
- Fields 1 and 2 are followed by one space ' ' character.
- Field 3 is followed by one newline '\n' character if there are more lines.
- Field 1: IP
  - A single asterisk indicates a wildcard and will match all IP addressed
  - An unquoted string will be checked for a match against the server IP address used by the client connection. An exact match is required.
- Field 2: PORT
  - A single asterisk indicates a wildcard and will match all TCP port numbers
  - An unquoted string will be checked for a match against the server TCP port used by the client connection. An exact match is required.
- Field 3: Server Root ID (SRID)
  - An unquoted string is expected. This is the server root ID string that will be used when selecting a local socket for the connection.
  - This string must conform to the 'NcxName' data type.
  - The length must be 1 to 64 characters.
  - This string will be used in directory and file names to avoid naming collisions when multiple server instances are running at once

Socket Search Order

Search With a Server Root ID

If a server root ID is found for the server 'IP:PORT' used in the session

Look for TCP port mapping file (e.g. SRID = sr1)
- /tmp/netconfd-pro-subsys-info.sr1.txt
  
  If found then use this info for TCP connect
Look for the AFLOCAL + SRID socket
- /tmp/ncxserver.sr1.sock
  
  If found then use this socket for AFLOCAL connect
Look for the FHS AFLOCAL + SRID socket
- /var/run/netconfd-pro/ncxserver.sr1.sock
  
  If found then use this socket for AFLOCAL connect
If not found then connection fails

Normal Search

If a server root ID is not found then the netconf-subsystem-pro program will assume Single-Instance Mode is being used, and the normal search for the socket is done:

Look for TCP port mapping file
- /tmp/netconfd-pro-subsys-info.txt
  
  If found then use this info for TCP connect
Look for the AFLOCAL socket
- /tmp/ncxserver.sock
  
  If found then use this socket for AFLOCAL connect
Look for the FHS AFLOCAL socket
- /var/run/netconfd-pro/ncxserver.sock
  
  If found then use this socket for AFLOCAL connect
If not found then connection fails

Server File Locations

The Server Root ID (SRID) is used to create unique names in two ways:

SRID Used In a Directory Path

The server data files are kept in a directory using the value provided in the --server-root CLI parameter.

Pattern:

$HOME/.yumapro/<SRID>/backups

Example: (SRID == 'sr1')

$HOME/.yumapro/sr1/backups

SRID Used In a File Name

In some cases, the SRID is added to a filename to prevent naming collisions with other server instances.

Pattern:

/tmp/ncxserver.<SRID>.sock

Example: (SRID == 'sr1')

/tmp/ncxserver.sr1.sock

CLI Parameters Affecting File Locations

The --server-root and --fileloc-fhs CLI parameters affect the directory locations and file names used by the server.

There are four different modes created from these parameters:

Normal Mode
FHS Mode
SRID Mode
FHS SRID Mode

Server File Types

There are several file types listed in the tables below:

server log: Main server log file
- Override default with the --log CLI parameter
audit log: Server event audit log file
- Override default with the --audit-log CLI parameter
- Disable in FHS mode with the --no-audit-log CLI parameter
startup file: Server startup datastore contents
- Override default with the --startup CLI parameter
- Disable with the --no-startup CLI parameter
- Default location depends on --fileloc-fhs setting
- Search path depends on the --datapath setting
factory startup: Server startup datastore contents if a factory restart is done
- Override default with the --startup-factory-file CLI parameter
- Default location depends on --fileloc-fhs setting
- Search path depends on the --datapath setting
TXID file: Server edit transaction ID file
- Not configurable
- Location depends on --fileloc-fhs setting
backups: Server backups directory to store files created with the <backup> operation
- Not configurable
- Location depends on --fileloc-fhs setting
PID file: Text file containing the process ID of the server instance
- Not configurable
- Location depends on --fileloc-fhs setting
AF socket: Socket file used for AFLOCAL connections
- Not configurable
- Location depends on --fileloc-fhs setting

Normal Mode

This is the default mode:

--fileloc-fhs='false'
--server-root not used

File Type	Default Value
server log	STDOUT
audit log	STDOUT
startup file	$HOME/.yumapro/startup-cfg.xml
factory startup	$HOME/.yumapro/factory-startup-cfg.xml
TXID file	$HOME/.yumapro/startup-cfg-txid.txt
backups	$HOME/.yumapro/backups/
PID file	$HOME/.yumapro/netconfd-pro.pid
AF socket	/tmp/ncxserver.sock

FHS Mode

This is a common usage mode to run the server as 'root':

--fileloc-fhs='true'
--server-root not used

File Type	Default Value
server log	/var/log/netconfd-pro/server.log
audit log	/var/log/netconfd-pro/audit.log
startup file	/var/lib/netconfd-pro/startup-cfg.xml
factory startup	/var/lib/netconfd-pro/factory-startup-cfg.xml
TXID file	/var/lib/netconfd-pro/startup-cfg-txid.txt
backups	/var/lib/netconfd-pro/backups/
PID file	/var/run/netconfd-pro/netconfd-pro.pid
AF socket	/var/run/netconfd-pro/ncxserver.sock

SRID Mode

This is the normal way to run in Multi-Instance Mode:

--fileloc-fhs='false'
--server-root is used (e.g., 'sr1')

File Type	Default Value
server log	STDOUT
audit log	STDOUT
startup file	$HOME/.yumapro/sr1/startup-cfg.xml
factory startup	$HOME/.yumapro/sr1/factory-startup-cfg.xml
TXID file	$HOME/.yumapro/sr1/startup-cfg-txid.txt
backups	$HOME/.yumapro/sr1/backups/
PID file	$HOME/.yumapro/netconfd-pro.sr1.pid
AF socket	/tmp/ncxserver.sr1.sock

FHS SRID Mode

This is the way to run in Multi-Instance Mode as 'root':

--fileloc-fhs='true'
--server-root is used (e.g., 'sr1')

File Type	Default Value
server log	/var/log/netconfd-pro/server.sr1.log
audit log	/var/log/netconfd-pro/audit.sr1.log
startup file	/var/lib/netconfd-pro/sr1/startup-cfg.xml
factory startup	/var/lib/netconfd-pro/sr1/factory-startup-cfg.xml
TXID file	/var/lib/netconfd-pro/sr1/startup-cfg-txid.txt
backups	/var/lib/netconfd-pro/sr1/backups/
PID file	/var/run/netconfd-pro/netconfd-pro.sr1.pid
AF socket	/var/run/netconfd-pro/ncxserver.sr1.sock