AppNet: Expressive, Easy-to-build, and High-performance Application Networks
What is AppNet?
AppNet is a framework designed for constructing high-performance application networks for microservices. At its core, AppNet offers a high-level abstraction that facilitates the creation of expressive and performant application networks. Users can define rich, possibly stateful, layer-7 (RPC) processing through match-action rules. AppNet's compiler compiles these specifications and generates high-performance code by optimizing where and how to execute different RPC processing functions.
Details are available in our arxiv paper and talk. (To be added)
Architecture Overview
AppNet consists of three main components - the AppNet program, the control plane and the data plane. TThe AppNet program orchestrates the network functionality among microservices through a sequence of elements, detailing each element with match-action rules that govern RPC content and element state.
The control plane includes a compiler that processes RPC definitions and chain specifications to produce code modules and a controller has global knowledge (acquired via cluster managers such as Kubernetes) of the network topology, service locations, and available AppNet data plane processors. It provisions network processing on available processors.
The data plane is composed of processors—such as sidecars, middleboxes, and RPC libraries—that execute the elemental operations at a low level. Each processor retrieves the compiled RPC processing logic from the control plane and periodically transmits logs, traces, and runtime statistical data back to the controller.
Supported Data Plane Processors
Getting help
Please get in touch with Xiangfeng Zhu (xfzhu@cs.washington.edu).
Installation Guide
Welcome to the AppNet installation guide. This document provides step-by-step instructions on how to set up AppNet and its dependencies.
Requirements:
First, clone the AppNet repo:
git clone git@github.com:appnet-org/appnet.git --recursive
cd appnet
conda create -y -n appnet python=3.10
conda activate appnet
Install the CLI
appnetctl is a command line program to manage the AppNet control plane.
To install the CLI, run
. ./install.sh
Once installed, verify the CLI and other tools is running correctly with:
appnetctl version
appnetctl verify
Lastly, install the CRDs into the cluster:
make install
Requirements
Kubernetes
To install a Kubernetes cluster, we recommend using kubeadm. Follow the steps below:
- Install the Control Plane:
. ./utils/k8s_setup.sh
- (Optional) Set Up Worker Nodes:
- First, prepare the worker nodes:
. ./utils/k8s_setup_worker.sh
- Then, join the cluster using kubeadm join. Run the following command on the control plane node to get the join command:
kubeadm token create --print-join-command
Note: if you plan to use a multi-node cluster, make sure you can ssh other nodes from the control plane node.
- Verify Installation:
kubectl version
For additional installation methods (e.g., KIND, Minikube), visit this page
(Optional) We highly recommend installing k9s for visualizing your clutser.
Istio
Istio can be installed in either sidecar mode or ambient mode. Choose the one that best fits your requirements:
- Sidecar Mode
. ./utils/istio_setup_sidecar.sh
- Ambient Mode
. ./utils/istio_setup_ambient.sh
Go
See this page for installation instructions.
Rust
Install Rust by running the following command:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Protoc
Install the Protocol Buffers Compiler and the necessary Go plugins with these commands:
sudo apt -y install protobuf-compiler
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
Conda
See this page for installation instructions.
For ubuntu users:
wget https://repo.anaconda.com/miniconda/Miniconda3-py310_23.3.1-0-Linux-x86_64.sh -O Miniconda.sh
bash Miniconda.sh
QuickStart
This guide will walk you through:
- Deploying a simple echo application.
- Running a simple element chain on the frontend to server communication edge.
Echo Application
The Echo application is a simple application developed using Go and gRPC. The client sends messages to the frontend, which then relays the messages to the Echo server. Finally, the server echoes the request back to the frontend. The architecture is as follows:
Deploy
Run the following command to deploy the echo application.
kubectl apply -f config/samples/echo/echo.yaml
Then, verify the deployment:
user@h1:~/appnet$ kubectl get pods
NAME READY STATUS RESTARTS AGE
echo-frontend-6f9cf6db74-tjvfc 2/2 Running 0 14m
echo-server-594b4797d-9t6gn 2/2 Running 0 14m
user@h1:~/appnet$ curl http://10.96.88.88?key=hello
You've hit server-986b8c7c6-725kd
Example element chain
We will deploy the following chain to the frontend to server edge.
Run the AppNet controller
First, you need to run the AppNet controller
make run
For this element chain the AppNet configurations is as follows:
apiVersion: api.core.appnet.io/v1
kind: AppNetConfig
metadata:
name: sample-echo # Name of the AppNetConfig
spec:
backend: sidecar # Name of the backend (sidecar/ambient/grpc)
appName: echo # Name of the application
clientService: frontend # Name of the client service (must be a valid service in the same namespace as the AppNetConfig)
serverService: server # Name of the server service (must be a valid service in the same namespace as the AppNetConfig)
method: echo # Name of the RPC method (defined in the proto file)
appManifestFile: <APPNET_DIR_PATH>/config/samples/echo/echo.yaml # Path to the application manifest file
clientChain:
- name: fault # Name of the first element in the client chain
file: <APPNET_DIR_PATH>/config/samples/echo/fault.appnet # Path to the fault injection element file
- name: logging # Name of the second element in the client chain
file: <APPNET_DIR_PATH>/config/samples/echo/logging.appnet # Path to the logging element file
serverChain:
- name: firwall # Name of the first element in the server chain
file: <APPNET_DIR_PATH>/config/samples/echo/firewall.appnet # Path to the firewall element file
anyChain:
- name: metrics # Name of the first element in the any(unconstraint) chain
file: <APPNET_DIR_PATH>/config/samples/echo/metrics.appnet # Path to the metrics element file
proto: <APPNET_DIR_PATH>/config/samples/echo/echo.proto # Path to the protobuf definition of client service to server service communication
Next, in a seperate terminal, replace <APPNET_DIR_PATH> with your AppNet directory path and apply this yaml file:
sed -i 's|<APPNET_DIR_PATH>|'"$(pwd)"'|g' config/samples/echo/sample_echo_sidecar.yaml
kubectl apply -f config/samples/echo/sample_echo_sidecar.yaml
You should some logs in the controller indicating it is reconciling, which should finish in a few minutes.
Finally, test the installation by running:
user@h1:~/appnet$ curl http://10.96.88.88?key=hello
You've hit server-6646d696cb-mx95h
user@h1:~/appnet$ curl http://10.96.88.88?key=test
Echo server returns an error.
The test request will be blocked by the firewall element.
Clean Up
When you're finish experimenting with the echo application, uninstall and clean it up using the following command:
kubectl delete all,pvc,pv,envoyfilters,appnetconfigs --all
istioctl experimental waypoint delete --all
Next Steps
AppNet Tutorials
Overview
AppNet provides high-level abstractions for developers to realize application networks. The abstractions enable easy specification of desired network functionality and enable optimizations that help realize the functionality with low overhead.
In AppNet, network functionality between a pair of microservices is specified as a chain of elements on the RPC request and response paths. Element processing is specified as match-action rules that operate over RPC fields. RPC fields (both metadata and payload) are represented as key-value pairs, which the elements can then easily read/write without worrying about (de)serialization. The state is also represented as key-value pairs.
AppNet’s compiler takes RPC definitions and chain specification to generate code modules. A module may combine the processing of multiple elements (to reduce invocation overhead) and may also re-order element processing based on inter-element dependencies (e.g., RPC fields read/written). The compiler also determines how to partition the state and if parts of shared state can be local (to reduce synchronization overhead). Finally, to further reduce the overhead, the compiler selectively bypasses the sidecar proxy processing.
Next Steps
Developer's Guide
Controller
Modifications to AdnconfigSpec
Changes can be made in this file.
make generate
Run controller locally
make run
Compiler
Chain Compiler
sed -i 's|<COMPILER_DIR>|'"$(pwd)"'|g' examples/chain/echo.yaml
python compiler/main.py --spec examples/chain/echo.yaml --backend envoy -v --opt_level no
usage: main.py [-h] -s SPEC_PATH [-v] [--pseudo_property] [--pseudo_impl] -b {mrpc,envoy}
[--mrpc_dir MRPC_DIR] [--dry_run] [--opt_level {no,ignore,weak,strong}]
[--no_optimize] [--replica REPLICA] [--opt_algorithm OPT_ALGORITHM] [--debug]
options:
-h, --help show this help message and exit
-s SPEC_PATH, --spec_path SPEC_PATH
Path to user specification file
-v, --verbose If added, request graphs (i.e., element chains) on each edge will be
printed on the terminal
--pseudo_property If added, use hand-coded properties instead of auto-generated ones
--pseudo_impl If added, use hand-coded impl instead of auto-generated ones
-b {mrpc,envoy}, --backend {mrpc,envoy}
Backend name
--mrpc_dir MRPC_DIR Path to mrpc repo
--dry_run If added, the compilation terminates after optimization (i.e., no
backend scriptgen)
--opt_level {no,ignore,weak,strong}
optimization level
--no_optimize If added, no optimization will be applied to GraphIR
--replica REPLICA #replica for each service
--opt_algorithm OPT_ALGORITHM
--debug Print debug info
The compiler will automatically install elements on all the nodes and
- Generate attach_all.sh and detach_all.sh in graph/gen if the backend is mRPC.
- Generate manifest files if the backend is Envoy. Use kubectl apply -f
to run the application
Element Compiler
Follow these steps if you want to interact with the element compiler directly.
The element compiler convert AppNet program to an IR. From IR, we can infer the element property (used by graph compiler). The element compiler also generates backend code for each element.
python compiler/element_compiler_test.py --element examples/elements/echo_elements/fault.appnet --backend envoy --placement client --proto ping.proto --method_name PingEcho
usage: element_compiler_test.py [-h] -e ELEMENT_PATH [-v] -p PLACEMENT -r PROTO -m METHOD_NAME
-b BACKEND
options:
-h, --help show this help message and exit
-e ELEMENT_PATH, --element_path ELEMENT_PATH
(Element_path',') *
-v, --verbose Print Debug info
-p PLACEMENT, --placement PLACEMENT
Placement of the generated code
-r PROTO, --proto PROTO
Filename of the Protobuf definition (e.g., hello.proto)
-m METHOD_NAME, --method_name METHOD_NAME
Method Name (must be defined in proto)
-b BACKEND, --backend BACKEND
Backend Code Target