YumaPro gRPC User Guide
This document describes the gRPC integration within the netconfd-pro server and ypgrpc-go-app application.
gRPC Introduction
The gRPC protocol uses protobuf for data transfer between the client and server.
The gRPC server and netconfd-pro server integration is deployed with help of ypgrpc-go-app application that transfers information between integrated gRPC server and gRPC clients and netconfd-pro server. This is an application that provides faster and easier platform to implement gRPC Services. It is similar to db-api-app where users create instrumentation for their Services and RPCs.
Note
gRPC Features
The main YumaPro gRPC functionality and integration features include:
Works with any developer-provided .proto files
The platform is the ypgrpc-go-app application, which is written in Golang
The client gRPC code can be maintained in any language and can be used by any other tool to send a gRPC request for the data, it can be auto generated code, or some GUI tools, or CLI tools, etc
Client sends gRPC requests directly to the ypgrpc-go-app
Subsystem reports when gRPC stream starts and ends to the netconfd-pro server for monitoring information
Stub code is generated using the 'protoc' tool
Stub code is integrated into ypgrpc-go-app similar to db-api-app
All possible gRPC examples provided for faster and easier deployment, including:
Empty request and Empty response RPC
gRPC that represent single request and response
gRPC that represent single request and a streaming response
gRPC that represent a sequence of requests and a single responses
gRPC that represent a sequence of requests and responses with multiple different scenarios
The netconfd-pro server is a controller that provides more functionality to the gRPC client – server communication. The netconfd-pro server has the following interaction with gRPC server:
ypgrpc-go-app application registers its capabilities and all the information about gRPC Services
List of available Services
List of available RPCs
List of open streams and when they were started
Counters to keep track of open and closed streams
List of supported .proto files
Name, address and the port number of the gRPC server and when it was started
<grpc-shutdown> RPC operation to shutdown the gRPC server
ypgrpc-go-app Program
The ypgrpc-go-app application is YControl subsystem (similar to db-api-app) that communicates with the netconfd-pro server and also it is a gRPC server that communicates with RPC clients. The main role of the ypgrpc-go-app application is to host gRPC server and provide common place to implement and instrument gRPC services and also provide monitoring and control using netconfd-pro server.
ypgrpc-go-app Features
The ypgrpc-go-app application provides following features:
Common place to implement and instrument .proto Services and RPCs
Single gRPC Server to handle
Subsystem reports to the netconfd-pro server with available capabilities
Subsystem reports when subscriptions start and end to the netconfd-pro server for monitoring information
Remote monitoring and control of gRPC server using netconfd-pro server
Possibility to remotely shutdown the gRPC server using netconfd-pro server
ypgrpc-go-app Processing
The above diagram illustrates deployment of the gRPC server, all its Services and messages handling and how the netconfd-pro server is integrated into this deployment.
The ypgrpc-go-app application is written in GO language and talks to the netconfd-pro server via socket and acts as a YControl subsystem (similar to db-api-app).
gRPC clients can be written in any languages that are supported for gRPC clients. The client part is out of the scope of this document and the current gRPC protocol integration does not include client part. The clients communicate to the gRPC server with help of the ypgrpc-go-app application and send gRPC request to the application.
The core of the ypgrpc-go-app is the gRPC server and integrated stub code from auto generated files from .proto files:
Handle integrated stub code callbacks. Callbacks that are integrated from stub code that were generated from .proto files using protoc tool
Register gRPC server for the protobuf message handling and gRPC Services callback invocation
Run main Serve loop that handles all the client/server communication
The processing between gRCP client to the netconfd-pro server can be split into following components:
gRPC clients to the ypgrpc-go-app processing: includes message parsing, gRPC Services callback invocation
ypgrpc-go-app application to netconfd-pro processing: includes YControl messages exchange and stream information exchange when a new stream opens or closes
netconfd-pro internal processing: includes subsystem registration, subsystem messages handling and parsing, gRPC monitoring information handling (gRPC server and streams status)
The ypgrpc-go-app implements multiple goroutines to manage the replies and the clients. All of this managers are goroutines. They run in parallel and asynchronously. The following goroutines are implemented in the ypgrpc-go-app:
Reply Manager goroutine: This manager is responsible for any already parsed messages from the netconfd-pro server or gRPC client, it stores any not processed messages that are ready to be processed
Message Manager goroutine: This manager is responsible for storing any ready to be processed messages that are going to the netconfd-pro server and that are coming back from the server
Startup Procedure
The ypgrpc-go-app application has the following startup steps for the gRPC server component:
Initialize all the prerequisites and parse all the CLI parameters
Open TCP socket to listen for clients requests
Serve any incoming gRPC messages from gRPC clients and send open-stream-event or close-stream-event to the netconfd-pro server if needed with help of all the goroutine managers.
The ypgrpc-go-app acts as a YControl subsystem (similar to db-api-app), however, it does not terminate after one edit or get request. Instead it continuously listens to the netconfd-pro server and keeps the AF_LOCAL or TCP socket open to continue communication whenever it's needed.
The communication is terminated only if the ypgrpc-go-app application is terminated, the netconfd-pro server terminates, or the netconfd-pro sends the request to terminate the ypgrpc-go-app application. All the message definitions described in the yumaworks-yp-grpc.yang YANG module.
The ypgrpc-go-app application has the following startup steps to initialize connection with the netconfd-pro server:
Initialize all the prerequisites and parse all the CLI parameters
Based on the --proto CLI parameter load all the .proto files and create capability structure for provided .proto files
Open socket and send <ncx-connect> request to the server with
transport = netconf-aflocal
protocol = yp-grpc
Register yp-grpc service
Send <register-request> to the server
Register ypgrpc-go-app subsystem and initialize all corresponding code in the netconfd-pro server to be ready to handle ypgrpc-go-app application requests
Send 'capability-ad-event' message to the netconfd-pro server to advertise all available Services, Methods and Streams
Keep listening socket until terminated
ypgrpc-go-app Configuration Parameter List
The following configuration parameters are used by ypgrpc-go-app. Refer to the CLI Reference for more details.
ypgrpc-go-app CLI Parameters
Parameter |
Description |
|---|---|
Specifies the gRPC server CA certificate file |
|
Specifies the gRPC server certificate file |
|
Specifies whether the ypgrpc-go-app should use File system Hierarchy Standard (FHS) directory locations to create, store and use data and files |
|
Directs to skip TLS validation |
|
Specifies the gRPC server private key file |
|
Specifies the log file for the ypgrpc-go-app application |
|
Directs that log output will be be sent to STDOUT, after being sent to the log file and/or local syslog daemon |
|
Controls the verbosity level of messages printed to the log file or STDOUT, if no log file is specified |
|
Specifies the port value to use for gRPC server connections |
|
Specifies the .proto file for the ypgrpc-go-app application to use |
|
Specifies the file search path for .proto files |
|
Specifies the netconfd-pro server IP address |
|
Specifies the subsystem identifier (gRPC Server ID) to use when registering with the netconfd-pro server |
ypgrpc-go-app Source Files
The following table lists the files that are included within
the netconf/src/ypgrpc/src/ypgrpc directory.
Directory |
Description |
cli |
Handle the CLI parameters for ypgrpc-go-app application |
credentials |
Package credentials loads certificates and validates user credentials. |
examples |
Stub code example for .proto files (helloworld and example .protos) |
log |
Handle the Logging for ypgrpc-go-app application |
message_handler |
Auto-generated gostruct representation of the yumaworks-yp-grpc.yang file. Used for message handling |
netconfd_connect |
Handler for the netconfd-pro connection with ypgrpc-go-app |
proto |
.proto files handling, parsing, search and storing |
utils |
Generic utility functions |
ycontrol |
Utilities to handle the netconfd-pro YControl messages and connections |
The 'ypgrpc-go-app.go' file that can be found in the
netconf/src/ypgrpc/src/ypgrpc directory is a source code file that
contains a main function that provides gRPC server functionality,
connectivity to the netconfd-pro server and stub code gRPC Services
callback handling.
ypgrpc-go-app Installation
The following sections describe the steps to install and test ypgrpc-go-app application.
ypgrpc-go-app Prerequisites
Install the Go programming language.
Version 'go1.15' or higher is required. To verify the installation and to verify the version of the installed GO run the following:
mydir> go version go version go1.10 linux/amd64
Install the Protocol Buffer Compiler
To verify the installation and to verify the version of the installed compiler run the following:
mydir> protoc --version # Ensure compiler version is 3+
Install the Go plugins for the protocol compiler:
Install the protocol compiler plugins for Go using the following commands:
$ go install google.golang.org/protobuf/cmd/[email protected] $ go install google.golang.org/grpc/cmd/[email protected]
Update your PATH so that the 'protoc' compiler can find the plugins:
$ export PATH="$PATH:$(go env GOPATH)/bin"
The gRPC client applications are out of the scope of this document and the current gRPC protocol integration does not include client applications. Refer to the gRPC Tools page for more details.
To send the request to the gRPC server the following tool can be used as an example:
gRPC client cli Tool - generic gRPC command line client.
In order to install this tool download the binary and install it to
/usr/local directory:
> sudo curl -L https://github.com/vadimi/grpc-client-cli/releases/download/v1.10.0/grpc-client-cli_darwin_x86_64.tar.gz \
| sudo tar -C /usr/local/bin -xz
> GO111MODULE=on go get -u github.com/vadimi/grpc-client-cli/cmd/grpc-client-cli@latest
To verify the installation and to verify the version of the installed client tool run the following:
> grpc-client-cli --version
grpc-client-cli version 1.10.0
You may need to update your $GOPATH/bin in order to run grpc-client-cli from you current directory, or run it as follows:
> $HOME/go/bin/grpc-client-cli --version
grpc-client-cli version 1.10.0
yp-grpc Binary Package Installation
If you do not have source code and need to install the YumaPro with a
binary package then the application will be installed in the default
/usr/bin location. If you'd like to use a different directory move
the binary to your desired location.
The YumaPro gRPC functionality is installed with yumapro-gnmi binary package. Refer to the yumapro-gnmi Binary Package Installation section for details on installing this package.
Binary Package ypgrpc-go-app Code Installation
The source code for ypgrpc-go-app application will be installed in
/usr/share/yumapro/src/ypgrpc/src/ypgrpc directory for further modifications.
However, there is no need to modify or run the application from that
location if you want to test the application with out any modifications.
The application has two .proto files implementations build in to
illustrate its functionality. The .proto files are location at:
/usr/share/yumapro/src/ypgrpc/src/ypgrpc/examples/example/example.proto/usr/share/yumapro/src/ypgrpc/src/ypgrpc/examples/helloworld/helloworld.proto
There are other auto-generated files such as 'pb.go' and '_grpc.pb.go'
In order to use the application with modifications and new .proto files
implementations the ypgrpc-go-app application can be modified and
rebuild from /usr/share/yumapro/src/ypgrpc/src/ypgrpc directory or
copied to the default $GOPATH location for modifications, for example:
mydir> cp -r /usr/share/yumapro/src/ypgrpc/src/ypgrpc $HOME/go/src/ypgrpc
After that the ypgrpc-go-app application can be modified, updated
and run from $HOME/go/src/ypgrpc directory.
Compile and execute the ypgrpc-go-app code:
$HOME/go/src/ypgrpc> go run ypgrpc-go-app.go --log-level=debug --fileloc-fhs \ --insecure --proto=helloworld --protopath=$HOME/protos
Setting up the gRPC Tools
Refer to the yumapro-gnmi Binary Package Installation section for additional details on setting up the Go tools needed for development.
The following steps are required:
create the go workspace directory
setup $GOBIN and $GOPATH variables
install all dependencies
Create the workspace directory,
$HOME/goSet the $GOPATH environment variable
mydir> mkdir -p ~/go
The $GOPATH can be any directory on your system. $HOME/go is
the default $GOPATH on Unix-like systems since Go 1.8. Note that
$GOPATH must not be the same path as your Go installation.
Edit ~/.bash_profile (or ~/.bashrc if that is present instead)
to add the following lines:
export GOPATH=$HOME/go
export GOBIN=$GOPATH/bin
Save and exit your editor. Then, source this file
mydir> source ~/.bash_profile
Using the 'go get' to install the following. Note that in this case $GOBIN and $GOPATH should be already setup:
Install Using make
This example assumes the ypwork source tree is in the home directory.
> ~/ypwork/netconf/src/ypgrpc$ make goget
Install using goget
The packages can be installed directly using these commands:
mydir> GO111MODULE=off go get github.com/aws/aws-sdk-go/aws
mydir> GO111MODULE=off go get google.golang.org/grpc
mydir> GO111MODULE=off go get google.golang.org/grpc/codes
mydir> GO111MODULE=off go get google.golang.org/grpc/status
mydir> GO111MODULE=off go get google.golang.org/protobuf/types/known/emptypb
mydir> GO111MODULE=off go get golang.org/x/text/encoding/unicode
mydir> GO111MODULE=off go get golang.org/x/text/transform
mydir> GO111MODULE=off go get github.com/jessevdk/go-flags
mydir> GO111MODULE=off go get github.com/openconfig/goyang/pkg/yang
mydir> GO111MODULE=off go get github.com/openconfig/ygot/ygot
mydir> GO111MODULE=off go get github.com/openconfig/ygot/ytypes
mydir> GO111MODULE=off go get github.com/jhump/protoreflect/desc
mydir> GO111MODULE=off go get github.com/jhump/protoreflect/desc/protoparse
mydir> GO111MODULE=off go get github.com/clbanning/mxj
mydir> GO111MODULE=off go get github.com/golang/protobuf/proto
mydir> GO111MODULE=off go get github.com/davecgh/go-spew/spew
mydir> GO111MODULE=off go get github.com/golang/protobuf/protoc-gen-go/descriptor
If you have installed the YumaPro from source code then you need to build and install using WITH_GRPC=1 and WITH_YCONTROL=1 build variables. Build the netconfd-pro server with gRPC support:
make WITH_YCONTROL=1 WITH_GRPC=1
sudo make WITH_YCONTROL=1 WITH_GRPC=1 install
Refer to the Setting up a Custom GO Workspace to use a custom workspace.
Source Code ypgrpc-go-app Installation
The source code for ypgrpc-go-app application will be in
/netconf/src/ypgrpc directory. However, there is no need to modify
or run the application from that location if you want to test the
application with out any modifications. The application has two .proto
files. Tools setup is described earlier,
in the Binary Package ypgrpc-go-app Code Installation section.
Generate the client and server certificates if gRPC client uses TLS validation. Refer to Generate the CA Certificates for more details.
Running ypgrpc-go-app
Run the server with --with-grpc=true CLI parameter as follows:
mydir> sudo netconfd-pro --log-level=debug4 --with-grpc=true -- fileloc-fhs=true
Start the ypgrpc-go-app application. Note that you have to provide the certificates to start the application:
mydir> ypgrpc-go-app --cert=~/certs/server.crt --ca=~/certs/ca.crt \
--key=~/certs/server.key --fileloc-fhs *--protopath=$HOME/protos \
--proto=helloworld --proto=example
It can also be run in “insecure” mode for test or verification:
mydir> ypgrpc-go-app --log-level=debug --fileloc-fhs --insecure \
--protopath=$HOME/protos --proto=helloworld --proto=example
After this step the gRPC server starts to listen for any gRPC client requests and will handle all the request for the provided example.proto and helloworld.proto .proto files and will advertise gRPC capabilities and example.proto, helloworld.proto Services to the netconfd-pro server.
Closing ypgrpc-go-app
The ypgrpc-go-app can be shut down by typing Ctrl-C in the window that started the application.
If the netconfd-pro server is not running when ypgrpc-go-app is started the application will terminate with an error message stating that the netconfd-pro server is not running.
If the netconfd-pro server is shut down then ypgrpc-go-app will also shutdown.
The netconfd-pro server has <grpc-shutdown> NETCONF operation that can be triggered to shut down the ypgrpc-go-app application.
Proto Search Path
The ypgrpc-go-app uses configurable search paths to find .proto files that are needed during operation.
If the --protopath parameter is specified with a path, that search path is tried, relative to the current working directory. If it is not found then the search terminates in failure. Sub-directories will be searched.
--protopath=../../protos
If the --proto parameter is specified without the --protopath path, then the
$HOME/protosdirectory is checked by default. Sub-directories will be searched.