../_images/logo.png

YumaPro gRPC User Guide

This document describes the gRPC integration within the netconfd-pro server and ypgrpc-go-app application.

../_images/Server_Architecture_gRPC_02-01-2022.png

gRPC Introduction

The gRPC protocol uses protobuf for data transfer between the client and server.

The gRPC server and netconfd-pro server integration is deployed with help of ypgrpc-go-app application that transfers information between integrated gRPC server and gRPC clients and netconfd-pro server. This is an application that provides faster and easier platform to implement gRPC Services. It is similar to db-api-app where users create instrumentation for their Services and RPCs.

Note

This feature is only available in the 22.10 (and later) release trains.
The yumapro-gnmi package contains both ypgnmi-app and ypgrpc-go-app applications.

gRPC Features

The main YumaPro gRPC functionality and integration features include:

  • Works with any developer-provided .proto files

  • The platform is the ypgrpc-go-app application, which is written in Golang

  • The client gRPC code can be maintained in any language and can be used by any other tool to send a gRPC request for the data, it can be auto generated code, or some GUI tools, or CLI tools, etc

  • Client sends gRPC requests directly to the ypgrpc-go-app

  • Subsystem reports when gRPC stream starts and ends to the netconfd-pro server for monitoring information

  • Stub code is generated using the 'protoc' tool

  • Stub code is integrated into ypgrpc-go-app similar to db-api-app

    • All possible gRPC examples provided for faster and easier deployment, including:

      • Empty request and Empty response RPC

      • gRPC that represent single request and response

      • gRPC that represent single request and a streaming response

      • gRPC that represent a sequence of requests and a single responses

      • gRPC that represent a sequence of requests and responses with multiple different scenarios

The netconfd-pro server is a controller that provides more functionality to the gRPC client – server communication. The netconfd-pro server has the following interaction with gRPC server:

  • ypgrpc-go-app application registers its capabilities and all the information about gRPC Services

    • List of available Services

    • List of available RPCs

    • List of open streams and when they were started

    • Counters to keep track of open and closed streams

    • List of supported .proto files

    • Name, address and the port number of the gRPC server and when it was started

  • <grpc-shutdown> RPC operation to shutdown the gRPC server

ypgrpc-go-app Program

The ypgrpc-go-app application is YControl subsystem (similar to db-api-app) that communicates with the netconfd-pro server and also it is a gRPC server that communicates with RPC clients. The main role of the ypgrpc-go-app application is to host gRPC server and provide common place to implement and instrument gRPC services and also provide monitoring and control using netconfd-pro server.

ypgrpc-go-app Features

The ypgrpc-go-app application provides following features:

  • Common place to implement and instrument .proto Services and RPCs

  • Single gRPC Server to handle

  • Subsystem reports to the netconfd-pro server with available capabilities

  • Subsystem reports when subscriptions start and end to the netconfd-pro server for monitoring information

  • Remote monitoring and control of gRPC server using netconfd-pro server

  • Possibility to remotely shutdown the gRPC server using netconfd-pro server

ypgrpc-go-app Processing

../_images/ypgrpc_arch_diagram.png

The above diagram illustrates deployment of the gRPC server, all its Services and messages handling and how the netconfd-pro server is integrated into this deployment.

The ypgrpc-go-app application is written in GO language and talks to the netconfd-pro server via socket and acts as a YControl subsystem (similar to db-api-app).

gRPC clients can be written in any languages that are supported for gRPC clients. The client part is out of the scope of this document and the current gRPC protocol integration does not include client part. The clients communicate to the gRPC server with help of the ypgrpc-go-app application and send gRPC request to the application.

The core of the ypgrpc-go-app is the gRPC server and integrated stub code from auto generated files from .proto files:

  • Handle integrated stub code callbacks. Callbacks that are integrated from stub code that were generated from .proto files using protoc tool

  • Register gRPC server for the protobuf message handling and gRPC Services callback invocation

  • Run main Serve loop that handles all the client/server communication

The processing between gRCP client to the netconfd-pro server can be split into following components:

  • gRPC clients to the ypgrpc-go-app processing: includes message parsing, gRPC Services callback invocation

  • ypgrpc-go-app application to netconfd-pro processing: includes YControl messages exchange and stream information exchange when a new stream opens or closes

  • netconfd-pro internal processing: includes subsystem registration, subsystem messages handling and parsing, gRPC monitoring information handling (gRPC server and streams status)

The ypgrpc-go-app implements multiple goroutines to manage the replies and the clients. All of this managers are goroutines. They run in parallel and asynchronously. The following goroutines are implemented in the ypgrpc-go-app:

  • Reply Manager goroutine: This manager is responsible for any already parsed messages from the netconfd-pro server or gRPC client, it stores any not processed messages that are ready to be processed

  • Message Manager goroutine: This manager is responsible for storing any ready to be processed messages that are going to the netconfd-pro server and that are coming back from the server

Startup Procedure

The ypgrpc-go-app application has the following startup steps for the gRPC server component:

  • Initialize all the prerequisites and parse all the CLI parameters

  • Open TCP socket to listen for clients requests

  • Serve any incoming gRPC messages from gRPC clients and send open-stream-event or close-stream-event to the netconfd-pro server if needed with help of all the goroutine managers.

The ypgrpc-go-app acts as a YControl subsystem (similar to db-api-app), however, it does not terminate after one edit or get request. Instead it continuously listens to the netconfd-pro server and keeps the AF_LOCAL or TCP socket open to continue communication whenever it's needed.

The communication is terminated only if the ypgrpc-go-app application is terminated, the netconfd-pro server terminates, or the netconfd-pro sends the request to terminate the ypgrpc-go-app application. All the message definitions described in the yumaworks-yp-grpc.yang YANG module.

The ypgrpc-go-app application has the following startup steps to initialize connection with the netconfd-pro server:

  • Initialize all the prerequisites and parse all the CLI parameters

  • Based on the --proto CLI parameter load all the .proto files and create capability structure for provided .proto files

  • Open socket and send <ncx-connect> request to the server with

    • transport = netconf-aflocal

    • protocol = yp-grpc

  • Register yp-grpc service

    • Send <register-request> to the server

    • Register ypgrpc-go-app subsystem and initialize all corresponding code in the netconfd-pro server to be ready to handle ypgrpc-go-app application requests

  • Send 'capability-ad-event' message to the netconfd-pro server to advertise all available Services, Methods and Streams

  • Keep listening socket until terminated

ypgrpc-go-app Configuration Parameter List

The following configuration parameters are used by ypgrpc-go-app. Refer to the CLI Reference for more details.

ypgrpc-go-app CLI Parameters

Parameter

Description

--ca

Specifies the gRPC server CA certificate file

--cert

Specifies the gRPC server certificate file

--fileloc-fhs

Specifies whether the ypgrpc-go-app should use File system Hierarchy Standard (FHS) directory locations to create, store and use data and files

--insecure

Directs to skip TLS validation

--key

Specifies the gRPC server private key file

--log

Specifies the log file for the ypgrpc-go-app application

--log-console

Directs that log output will be be sent to STDOUT, after being sent to the log file and/or local syslog daemon

--log-level

Controls the verbosity level of messages printed to the log file or STDOUT, if no log file is specified

--port

Specifies the port value to use for gRPC server connections

--proto

Specifies the .proto file for the ypgrpc-go-app application to use

--protopath

Specifies the file search path for .proto files

--server-address

Specifies the netconfd-pro server IP address

--subsys-id

Specifies the subsystem identifier (gRPC Server ID) to use when registering with the netconfd-pro server

ypgrpc-go-app Source Files

The following table lists the files that are included within the netconf/src/ypgrpc/src/ypgrpc directory.

Directory

Description

cli

Handle the CLI parameters for ypgrpc-go-app application

credentials

Package credentials loads certificates and validates user credentials.

examples

Stub code example for .proto files (helloworld and example .protos)

log

Handle the Logging for ypgrpc-go-app application

message_handler

Auto-generated gostruct representation of the yumaworks-yp-grpc.yang file. Used for message handling

netconfd_connect

Handler for the netconfd-pro connection with ypgrpc-go-app

proto

.proto files handling, parsing, search and storing

utils

Generic utility functions

ycontrol

Utilities to handle the netconfd-pro YControl messages and connections

The 'ypgrpc-go-app.go' file that can be found in the netconf/src/ypgrpc/src/ypgrpc directory is a source code file that contains a main function that provides gRPC server functionality, connectivity to the netconfd-pro server and stub code gRPC Services callback handling.

ypgrpc-go-app Installation

The following sections describe the steps to install and test ypgrpc-go-app application.

ypgrpc-go-app Prerequisites

  1. Install the Go programming language.

    Version 'go1.19' or higher is required. To verify the installation and to verify the version of the installed GO run the following:

    mydir> go version
    
    go version go1.19 linux/amd64
    
  2. Install the Protocol Buffer Compiler

    To verify the installation and to verify the version of the installed compiler run the following:

    mydir> protoc --version # Ensure compiler version is 3+
    
  3. Install the Go plugins for the protocol compiler:

    Install the protocol compiler plugins for Go using the following commands:

    $ go install google.golang.org/protobuf/cmd/[email protected]
    
    $ go install google.golang.org/grpc/cmd/[email protected]
    

Update your PATH so that the 'protoc' compiler can find the plugins:

$ export PATH="$PATH:$(go env GOPATH)/bin"

The gRPC client applications are out of the scope of this document and the current gRPC protocol integration does not include client applications. Refer to the gRPC Tools page for more details.

To send the request to the gRPC server the following tool can be used as an example:

gRPC client cli Tool - generic gRPC command line client.

In order to install this tool download the binary and install it to /usr/local directory:

> sudo curl -L https://github.com/vadimi/grpc-client-cli/releases/download/v1.10.0/grpc-client-cli_darwin_x86_64.tar.gz \
   | sudo tar -C /usr/local/bin -xz

> GO111MODULE=on go get -u  github.com/vadimi/grpc-client-cli/cmd/grpc-client-cli@latest

To verify the installation and to verify the version of the installed client tool run the following:

> grpc-client-cli --version

grpc-client-cli version 1.10.0

You may need to update your $GOPATH/bin in order to run grpc-client-cli from you current directory, or run it as follows:

> $HOME/go/bin/grpc-client-cli --version

grpc-client-cli version 1.10.0

yp-grpc Binary Package Installation

If you do not have source code and need to install the YumaPro with a binary package then the application will be installed in the default /usr/bin location. If you'd like to use a different directory move the binary to your desired location.

The YumaPro gRPC functionality is installed with yumapro-gnmi binary package. Refer to the yumapro-gnmi Binary Package Installation section for details on installing this package.

Binary Package ypgrpc-go-app Code Installation

The source code for ypgrpc-go-app application will be installed in /usr/share/yumapro/src/ypgrpc/src/ypgrpc directory for further modifications. However, there is no need to modify or run the application from that location if you want to test the application with out any modifications. The application has two .proto files implementations build in to illustrate its functionality. The .proto files are location at:

  • /usr/share/yumapro/src/ypgrpc/src/ypgrpc/examples/example/example.proto

  • /usr/share/yumapro/src/ypgrpc/src/ypgrpc/examples/helloworld/helloworld.proto

There are other auto-generated files such as 'pb.go' and '_grpc.pb.go'

In order to use the application with modifications and new .proto files implementations the ypgrpc-go-app application can be modified and rebuild from /usr/share/yumapro/src/ypgrpc/src/ypgrpc directory or copied to the default $GOPATH location for modifications, for example:

mydir> mkdir -p $HOME/go/src
mydir> cp -r /usr/share/yumapro/src/ypgrpc/src/ypgrpc $HOME/go/src/ypgrpc

After that the ypgrpc-go-app application can be modified, updated and run from $HOME/go/src/ypgrpc directory.

  • Compile and execute the ypgrpc-go-app code:

    $HOME/go/src/ypgrpc> go mod tidy
    $HOME/go/src/ypgrpc> go run ypgrpc-go-app.go --log-level=debug --fileloc-fhs \
       --insecure --proto=helloworld --protopath=$HOME/protos
    

Setting up the gRPC Tools

Refer to the yumapro-gnmi Binary Package Installation section for additional details on setting up the Go tools needed for development.

The following steps are required:

  • create the go workspace directory

  • setup $GOBIN and $GOPATH variables

  • install all dependencies

  • Create the workspace directory, $HOME/go

  • Set the $GOPATH environment variable

mydir> mkdir -p ~/go

The $GOPATH can be any directory on your system. $HOME/go is the default $GOPATH on Unix-like systems since Go 1.8. Note that $GOPATH must not be the same path as your Go installation.

Edit ~/.bash_profile (or ~/.bashrc if that is present instead) to add the following lines:

export GOPATH=$HOME/go
export GOBIN=$GOPATH/bin

Save and exit your editor. Then, source this file

mydir> source ~/.bash_profile

Using the 'go get' to install the following. Note that in this case $GOBIN and $GOPATH should be already setup:

Install

If you have installed the YumaPro from source code then you need to build and install using WITH_GRPC=1 and WITH_YCONTROL=1 build variables. Build the netconfd-pro server with gRPC support:

make WITH_YCONTROL=1 WITH_GRPC=1
sudo make WITH_YCONTROL=1 WITH_GRPC=1 install

Refer to the Setting up a Custom GO Workspace to use a custom workspace.

Source Code ypgrpc-go-app Installation

The source code for ypgrpc-go-app application will be in /netconf/src/ypgrpc directory. However, there is no need to modify or run the application from that location if you want to test the application with out any modifications. The application has two .proto files. Tools setup is described earlier, in the Binary Package ypgrpc-go-app Code Installation section.

Generate the client and server certificates if gRPC client uses TLS validation. Refer to Generate the CA Certificates for more details.

Running ypgrpc-go-app

Run the server with --with-grpc=true CLI parameter as follows:

mydir> sudo netconfd-pro --log-level=debug4 --with-grpc=true -- fileloc-fhs=true

Start the ypgrpc-go-app application. Note that you have to provide the certificates to start the application:

mydir> ypgrpc-go-app --cert=~/certs/server.crt --ca=~/certs/ca.crt \
   --key=~/certs/server.key --fileloc-fhs --protopath=$HOME/protos \
   --proto=helloworld --proto=example

It can also be run in “insecure” mode for test or verification:

mydir> ypgrpc-go-app --log-level=debug --fileloc-fhs --insecure \
  --protopath=$HOME/protos --proto=helloworld --proto=example

After this step the gRPC server starts to listen for any gRPC client requests and will handle all the request for the provided example.proto and helloworld.proto .proto files and will advertise gRPC capabilities and example.proto, helloworld.proto Services to the netconfd-pro server.

Closing ypgrpc-go-app

The ypgrpc-go-app can be shut down by typing Ctrl-C in the window that started the application.

If the netconfd-pro server is not running when ypgrpc-go-app is started the application will terminate with an error message stating that the netconfd-pro server is not running.

If the netconfd-pro server is shut down then ypgrpc-go-app will also shutdown.

The netconfd-pro server has <grpc-shutdown> NETCONF operation that can be triggered to shut down the ypgrpc-go-app application.

Proto Search Path

The ypgrpc-go-app uses configurable search paths to find .proto files that are needed during operation.

  • If the --protopath parameter is specified with a path, that search path is tried, relative to the current working directory. If it is not found then the search terminates in failure. Sub-directories will be searched.

    --protopath=../../protos
    
  • If the --proto parameter is specified without the --protopath path, then the $HOME/protos directory is checked by default. Sub-directories will be searched.