Configuration of a development environment with standalone processes v2 - grupogita/ONOSP4-tutorial GitHub Wiki
As an alternative to the approach based on docker containers, and aiming at providing an environment where the developer has a more complete control of the elements forming the infrastructure (i.e. ONOS Controller, Mininet and stratum_bmv2 switches), we present the following procedure to configure a development environment with the components installed within the base Operating System.
As first step, you must install a base Operating System. Although any linux distribution should work, for this tutorial we indicate Ubuntu Server 22.04.3, where we have tested all the commands and procedures indicated. This Operating System installation, either physical or virtual machine, must have at least 30GB of free hard disk, 4GB of RAM memory and 2 CPU cores.
Note 0: For convenience, you might want to download this Virtual Machine Image which has all the components preinstalled. It is based on Ubuntu Server 22.04. Username is p4 and password is tutorial. The virtual machine is equipped with two network interfaces. One is configured as NAT interface in order to provide Internet Connectivity. The second one is connected to the internal network of the virtualization software (In the case of VirtualBox, this is a host-only adapter), and it is configured to receive an IP address via DHCP. This interface is used for the management of the virtual machine. Please follow the particular procedures of your virtualization software for the deployment of the virtual machine on your particular environment.
Note 1: If installing a virtual machine, it is recommended to create two network interfaces: A NAT interface for external communication and a host-only interface to access the instance for management and interaction.
Note 2: If installing Ubuntu Server, keep in mind that it uses by default logical volumes for disk layout. The root filesystem is created in a logical volume called ubuntu-lv which, regardless the disk capacity, is configured with a size of 14GB. In order to avoid issues associated to disk space exhaustion when installing and configuring components, it is recommended to configure at installation time this volume to use the maximum disk capacity. In case you forget this step, you might need to update the root volume. This step is out of the scope of this tutorial. Anyway, be careful, since this step might destroy your virtual machine.
Proceed with the usual installation procedure. You might consider to install OpenSSH server at installation time. No other packages are required at installation time.
Supporting packets
In order to compile stratum_bmv2 and ONOS, some Operating Systems packets are required. Using the corresponding software installation procedure of your particular distribution, make sure that the following packets are installed:
docker.io docker-compose build-essential cmake flex bison autoconf automake libtool libgc-dev libfl-dev libgmp-dev libboost-dev libboost-iostreams-dev pkg-config mininet python2.7 zip
Although this development environment is not intended to be based on docker images, there are two particular cases which at the time of writing this tutorial still rely on this approach. See below the further explanation.
Note: The prompt indicates what privileges you require to execute the command. $ means that the command may be executed as regular user. # means that the commands needs to be executed with root privileges.
Installing p4c
We rely on the p4c available at repositories. Follow the following steps to install the p4c compiler
# source /etc/lsb-release
# echo "deb http://download.opensuse.org/repositories/home:/p4lang/xUbuntu_${DISTRIB_RELEASE}/ /" | sudo tee
# /etc/apt/sources.list.d/home:p4lang.list
# curl -fsSL https://download.opensuse.org/repositories/home:p4lang/xUbuntu_${DISTRIB_RELEASE}/Release.key | gpg --dearmor | sudo
# tee /etc/apt/trusted.gpg.d/home_p4lang.gpg > /dev/null
# apt-get update
# apt install p4lang-p4c
Installing stratum bmv2
After installing p4c, then the stratum bmv2 switch can be installed. The installation procedure is based on the instructions contained at the Stratum repository. This procedure depends on a docker image.
- Enroll your operating system user in the docker group:
$ sudo -E usermod -aG docker $USER
Note: $USER is an environment variable which contains the username of your current user. After performing this step, you need to log out and log in again. Keep in mind that if you execute the command directly as super user (e.g. you got the super user privileges by using su(1M)), you must replace $USER with the login of the non privileged user.
- Clone the git repository
$ git clone https://github.com/stratum/stratum
- Update the code:
$ git submodule update --init --recursive
- Init and execute the development environment
$ cd stratum
$ ./setup_dev_env.sh
- (Hack) Update references to package versions to guarantee compatibility with Ubuntu 22.04
The default source code creates dependencies with some libraries which are not available on Ubuntu 22.04. In order to be able to generate the stratum_bmv2 package so that it can be installed in this Ubuntu release, some dependencies should be updated so that they point to the versions actually available. Edit the following files within the stratum_bmv2 environment and modify the version which is currently 1.67.0 to 1.74.0 for the corresponding libraries. We indicate the line numbers to make the change, in the versions available as of September 20th 2023
stratum/hal/bin/bcm/standalone/BUILD (Lines 198 and 224)
stratum/hal/bin/bmv2/BUILD (Lines 94, 95 and 96)
This procedure inits a docker image where the compilation of stratum bmv2 is performed. Just wait for the completion of this step. When completing, you will be within the environment of the docker image where the compilation will be performed.
- Build the stratum bmv2 component
Stratum bmv2 uses the bazel framework.
$ bazel build //stratum/hal/bin/bmv2:stratum_bmv2_deb
After the building process finishes, a deb package is generated. Hence, you can either run the stratum_bmv2 switch within the docker image, or you can install it in the base Operating System. In this tutorial we follow the same approach.
When finished the installation, the .deb package should be copied onto a directory shared with the base Operating System
$ cp -f /stratum/bazel-bin/stratum/hal/bin/bmv2/stratum_bmv2_deb.deb /stratum/
The stratum directory is a directory of the base Operating System which is mounted within the docker image. Thus, it allows moving files between the docker image and the base Operating System.
After performing the copy, exit from the docker image and return to the base operating system. Then, proceed to install the deb package which was previously built.
# apt-get update
# apt-get install -y --reinstall ./stratum_bmv2_deb.deb
- (Hack) Create symbolic links to satisfy remaining dependencies
It is possible that the installation fails due to an issue related to some libraries. Proceed with the following hack:
# apt install libatomic1 libboost-system1.74.0 libboost-filesystem1.74.0 libboost-program-options1.74.0 libboost-thread1.74.0 libboost-thread1.74.0 libjudydebian1 libpcap-dev
# cd /usr/lib/x86_64-linux-gnu
# ln -s libboost_thread.so.1.74.0 libboost_thread.so.1.67.0
# ln -s libboost_filesystem.so.1.74.0 libboost_filesystem.so.1.67.0
# ln -s libboost_program_options.so.1.74.0 libboost_program_options.so.1.67.0
# ln -s libboost_system.so.1.74.0 libboost_system.so.1.67.0
# ldconfig -v
If required, reinstall the stratum_bmv2 packet with apt --reinstall
After installing the deb package, you can try to run stratum_bmv2 switch to check whether it was successfully installed. Try the following command:
$ stratum_bmv2 \
-chassis_config_file=/etc/stratum/chassis_config.pb.txt \
-bmv2_log_level=debug
Some errors might appear related to the add of interfaces. However, if you see a line with the content: "Stratum external facing services are listening to 0.0.0.0:9339 ... " then it means that stratum_bmv2 switch was successfully installed
ONOS
For the compilation of stratum_bmv2 and ONOS, you must install bazel. To simplify the process, use bazelisk, at least 1.18.0.
# wget -t0 https://github.com/bazelbuild/bazelisk/releases/download/v1.18.0/bazelisk-linux-amd64
# cp bazelisk-linux-amd64 /usr/local/bin
# mv /usr/local/bin/bazelisk-linux-amd64 /usr/local/bin/bazel
# chmod +x /usr/local/bin/bazel
- (Hack) Python interpreter
# ln -s /usr/bin/python2.7 /usr/bin/python
# wget -O onos.tar.gz -t0 https://gerrit.onosproject.org/plugins/gitiles/onos/+archive/a821487ebbac55f2d7703e9dc36839ae60800288.tar.gz
# mkdir onos
# mv onos.tar.gz onos
# tar xvfz onos.tar.gz
# bazel build onos
- Create an Operating System user to configure the execution of ONOS with an unprivileged user:
# adduser sdn --system --group
After completing the compilation, there should be a file named onos.tar.gz in the bazel-bin directory. This file should be moved and unpacked at the /opt directory
# cp bazel-bin/onos.tar.gz /opt
# cd /opt
# tar xvfz onos.tar.gz
# mv onos-2.7.0 onos
# chown -R sdn:sdn onos
- (Hack) Reuse the JDK used for the compilation of ONOS
The compilation process of ONOS uses a remote JDK. We recommed to use that JDK instead of installing one of the Open JDK packages available in the Operating System repositories. This JDK will be installed within the bazel cache. Hence, you need to find the exact location to copy the folder containing the JDK. Use the find command:
# find / -name javac
Then, copy the folder containing the JDK. Below, you can find one example:
# find / -name javac
/var/lib/docker/overlay2/c9de0262f5e32bab1f6010bf48eab86a02953fa6f5f7fe0247636d79a71c2acc/diff/usr/local/lib/jvm/bin/javac
/home/p4/.cache/bazel/_bazel_p4/8754fd6c342dce2ed7bdb9a209206ad0/external/remotejdk11_linux/bin/javac
(output continues)
In our case, we copied the directory reported in the second line of the ouput
# cp -ir /home/p4/.cache/bazel/_bazel_p4/8754fd6c342dce2ed7bdb9a209206ad0/external/remotejdk11_linux /opt/java
- Define system wide environmental variables associated to Java (JDK_HOME and JRE_HOME)
# cat >> /etc/environment <<EOL
JAVA_HOME=/opt/java
JRE_HOME=/opt/ava
EOL
- Copy the init script of onos to the system location
# cp /opt/onos/init/onos.initd /etc/init.d/onos
- Create an options file for ONOS
# cat >> /opt/onos/options <<EOL
ONOS_USER=sdn
ONOS_APPS=
EOL
- (Hack) Small fix in the init script of ONOS
Add the following command before the definition of the start() method in the /etc/init.d/onos file:
cd $ONOS_HOME
After these steps, you can start/stop your ONOS installation by invoking /etc/init.d/onos { start | stop }
You can verify the onos log with the following command until you see a line containing the string "Started HttpServiceContext".
$ tail -f /opt/onos/log/karaf.log
When ONOS starts, you can access its graphical user interface at the URL http://A.B.C.D:8181/onos/ui where A.B.C.D is the IP address of the machine (either physical or virtual) where you installed ONOS.
Installing Maven
Apache Maven is a tool which allows the centralized management of Software Projects. The compilation and building of ONOS applications, which are written in Java, is done by using this tool. At the time of writing this tutorial, the latest version available is 3.8.6. Hence, this is the one that we will install. Nevertheless, look for the latest version, download it and install it.
- Download Maven and install at a convenient location
# cd /opt
# wget -t0 https://archive.apache.org/dist/maven/maven-3/3.8.6/binaries/apache-maven-3.8.6-bin.tar.gz
# tar xvfz apache-maven-3.8.6-bin.tar.gz
- Rename the installation directory to a more convenient name
# cd /opt
# mv apache-maven-3.8.6 maven
- Update the PATH environmental variable, either globally or for the current user
# echo export "PATH=$PATH:/opt/maven/bin" >> /etc/environment"
If you completed successfully these points, then your development environment where you will be able to integrate stratum_bmv2 p4-programmable switch with ONOS.
Then, you can go to the first exercise to get familiar with the development of P4 applications integrated with ONOS.