FeG Deployment - caprivm/virtualization GitHub Wiki

caprivm ([email protected])

Description

This page explains all the steps required to deploy FeG in a Ubuntu VM with Docker. Please consider all the steps performed as installation. All steps that expose this section has been tested in a Machine using the following OS preparation requeriments.

Feature Value
OS Used Ubuntu 18.04 LTS
vCPU 4
RAM (GB) 8
Disk (GB) 50
Home user ubuntu
Magma Tag v1.6

The contents of the page are:

Prerequisites

Before starting this guide, you should have installed the following tools. You can check the adjacent links if you haven't already:

Environment Variables

In this case, consider the next environment variables before continue the procedure:

export MAGMA_ROOT=~/magma_v1.6
export CUSTOM_MAGMA_TAG=v1.6
export CUSTOM_REGISTRY=docker.io/caprivm

Changes in Source Code

Any changes to the source code must be made before compiling the images to create the containers in the registry. This means that changes should be made at this point, and do not continue until all changes are made.

Build the FeG Containers

Based on the Federation Gateway page, the first step is building the FeG. Please, download the repo and checkout the tag or branch that you want. In this case, we checkout the v1.6 tag.

sudo apt update && sudo apt upgrade -y
cd && git clone https://github.com/magma/magma.git $MAGMA_ROOT
cd $MAGMA_ROOT
git checkout tags/$CUSTOM_MAGMA_TAG -b $CUSTOM_MAGMA_TAG

Start up your Docker daemon, cd to where you've cloned, checkout Magma and edit the .env file for considering your custom configuration.

cd ~/$MAGMA_ROOT/feg/gateway/docker
sudo vi .env

.env (example) 
# Copyright 2020 The Magma Authors.

# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.

# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

COMPOSE_PROJECT_NAME=feg
DOCKER_REGISTRY=feg_
DOCKER_USERNAME=
DOCKER_PASSWORD=
IMAGE_VERSION=latest
GIT_HASH=master
BUILD_CONTEXT=../../../

ROOTCA_PATH=../../../.cache/test_certs/rootCA.pem
CONTROL_PROXY_PATH=../configs/control_proxy.yml
SNOWFLAKE_PATH=../../../.cache/feg/snowflake
CONFIGS_DEFAULT_VOLUME=../configs
CONFIGS_TEMPLATES_PATH=../../../orc8r/gateway/configs/templates

CERTS_VOLUME=gwcerts
CONFIGS_VOLUME=gwconfigs

LOG_DRIVER=journald
docker-compose build --parallel

Registry the FeG Containers

Registry your Docker containers. In your ~/.bashrc file, put the next variables:

export PUBLISH=$MAGMA_ROOT/orc8r/tools/docker/publish.sh
export REGISTRY=$CUSTOM_REGISTRY
export MAGMA_TAG=$CUSTOM_MAGMA_TAG

Next, perform source ~/.bashrc. Now, start up the Docker daemon to log in to the Docker Hub account. In password put your Docker Hub access token.

docker login registry.gitlab.com
# Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
# Username: caprivm
# Password:
# Login Succeeded

Make the next changes in the publish.sh file sudo vi $PUBLISH:

@@ -56,7 +56,7 @@ PROJECT=${COMPOSE_PROJECT_NAME}

 # Find the image ID for the latest build
 DESIRED_IMAGE="${PROJECT}_${IMAGE}"
-IMAGE_ID=$(docker images "${DESIRED_IMAGE}:latest" --format "{{.ID}}")
+IMAGE_ID=$(docker images "${DESIRED_IMAGE}:${VERSION}" --format "{{.ID}}")
 if [[ -z "${IMAGE_ID}" ]]; then
   exitmsg "[Error] project ${PROJECT} missing image ${DESIRED_IMAGE}: please build the image"
 fi

Execute the next command to registry the FeG containers images.

NOTE: This step is necessary only if the images have not been published before.

cd $MAGMA_ROOT/feg/gateway/docker
source .env
COMPOSE_PROJECT_NAME=feg ${PUBLISH} -r ${REGISTRY} -i gateway_python -v ${IMAGE_VERSION}
COMPOSE_PROJECT_NAME=feg ${PUBLISH} -r ${REGISTRY} -i gateway_go -v ${IMAGE_VERSION}
COMPOSE_PROJECT_NAME=feg ${PUBLISH} -r ${REGISTRY} -i gateway_go_base -v ${IMAGE_VERSION}

Installing Federation Gateway

Based on the official documentation, there are three required files that are deployment-specific. These are described below:

  • rootCA.pem: This file should match the rootCA.pem of the Orchestrator that the Federation Gateway will connect to.
  • control_proxy.yml: This file is used to configure the magmad and control_proxy services to point toward the appropriate Orchestrator.
  • .env: This file provides any deployment specific environment variables used in the docker-compose.yml of the Federation Gateway.

control_proxy.yml (example) 
# Copyright 2020 The Magma Authors.

# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.

# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# nghttpx config will be generated here and used
nghttpx_config_location: /var/tmp/nghttpx.conf

# Location for certs
rootca_cert: /var/opt/magma/certs/rootCA.pem
gateway_cert: /var/opt/magma/certs/gateway.crt
gateway_key: /var/opt/magma/certs/gateway.key

# Listening port of the proxy for local services. The port would be closed
# for the rest of the world.
local_port: 8443

# Cloud address for reaching out to the cloud.
cloud_address: controller.magma.svc.cluster.local
cloud_port: 8443

bootstrap_address: bootstrapper-controller.magma.svc.cluster.local
bootstrap_port: 8444

fluentd_address: fluentd.magma.svc.cluster.local
fluentd_port: 24224

# Option to use nghttpx for proxying. If disabled, the individual
# services would establish the TLS connections themselves.
proxy_cloud_connections: True

# Allows http_proxy usage if the environment variable is present
allow_http_proxy: True

.env (example) 
# Copyright 2020 The Magma Authors.

# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.

# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

COMPOSE_PROJECT_NAME=feg
DOCKER_REGISTRY=registry.gitlab.com/everis_factory/fbc/magma/
DOCKER_USERNAME=caprivm
DOCKER_PASSWORD=
IMAGE_VERSION=latest
GIT_HASH=master
BUILD_CONTEXT=../../../

ROOTCA_PATH=/var/opt/magma/certs/rootCA.pem
CONTROL_PROXY_PATH=/etc/magma/control_proxy.yml
SNOWFLAKE_PATH=/etc/snowflake
CONFIGS_DEFAULT_VOLUME=/etc/magma
CONFIGS_TEMPLATES_PATH=/etc/magma/templates

CERTS_VOLUME=/var/opt/magma/certs
CONFIGS_VOLUME=/var/opt/magma/configs

LOG_DRIVER=journald

# This section is unnecessary if using host networking
# S6A_LOCAL_PORT=3868
# S6A_HOST_PORT=3868
# S6A_NETWORK=sctp
# SWX_LOCAL_PORT=3869
# SWX_HOST_PORT=3869
# SWX_NETWORK=sctp
# GX_LOCAL_PORT=3870
# GX_HOST_PORT=3870
# GX_NETWORK=tcp
# GY_LOCAL_PORT=3871
# GY_HOST_PORT=3871
# GY_NETWORK=tcp

[email protected]
OCS_SERVICE_IDENTIFIER_OVERWRITE=200:200
PCRF_91_COMPLIANT=1

rootCA.pem (example) 
-----BEGIN CERTIFICATE-----
MIIDTTCCAjWgAwIBAgIUF/aIL9e3VgoUTtyleH2/AeS6YpAwDQYJKoZIhvcNAQEL
BQAwNjELMAkGA1UEBhMCVVMxJzAlBgNVBAMMHnJvb3RjYS5tYWdtYS5zdmMuY2x1
c3Rlci5sb2NhbDAeFw0yMTA4MTcyMTIyNTZaFw0zMTA4MTUyMTIyNTZaMDYxCzAJ
BgNVBAYTAlVTMScwJQYDVQQDDB5yb290Y2EubWFnbWEuc3ZjLmNsdXN0ZXIubG9j
YWwwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDPDlEquQfdrDI1IBxp
jyHFLvyxJdzu1cUFGFZXo3XZbHWOUG68fHis+YPBIQUGlrqAKT48SM3xwYK2+lzp
soka7SKlNkJbZ9ngiBFS/md8VWupmBnkSjMG6SjF54kNFIauNBJdhbcqcCMXbI0Q
rqFOVDbgIV3J2+YghmC/DaDJf4m+hRR5RpO9yqFVjb8469d8i7vhRksR49lw0iVb
sUIPGieg+6+PJyf9+n7ZKJKTttYU3V/lrm3hLD7pYgoSyIO7AduuW8HEtSXJfIYQ
qxQ6Bhjez77+Lar61oqXaM9uI4LcQ5r5fuUVMX0ZGPZpSI/sqBjPQXEs9WqOPikk
chOlAgMBAAGjUzBRMB0GA1UdDgQWBBTU4qYYNP7NmdEJqy2psJHabKsMfzAfBgNV
HSMEGDAWgBTU4qYYNP7NmdEJqy2psJHabKsMfzAPBgNVHRMBAf8EBTADAQH/MA0G
CSqGSIb3DQEBCwUAA4IBAQAdA2W1+kYR/NxF7QHnVy9aIDocTspswHw5pP8wAPsw
Ahs/Mue3vo2uW2nFmD4FdAmvU8ktPzuCTMvk5mLG4TOk1tuGmvhdloQEmBQrC7RP
jOuK7aLxj8xfeB7aCDWLik1O3Z0e5s09hv7kHUi2Km19lU9Lw68BbSuj+Sae8bB6
1wgkWQ/dw+ttS8SDNtHJ7y1d6sdbQpR33KW1pRoKjLNqpd8pa6qwT44SKO9Z0Fti
H83Snc1zrxb+hnt3g3Zegy32wmRtNNtglSfrgP6xUp7V7eI4IQ69If40kFpN2Fep
uO1Khv54UwVgdzHpdwJ8OIaXg2bCM1vu5CQnCQLKw/Xe
-----END CERTIFICATE-----

Installation

The installation is done using the install_gateway.sh script located at $MAGMA_ROOT/orc8r/tools/docker. To install, copy that file and the three files described above into a home directory on the install host, for instance feg_install. For this, execute:

mkdir -p ~/feg_install && cd ~/feg_install
sudo cp $MAGMA_ROOT/orc8r/tools/docker/install_gateway.sh .
# Create or copy the above control_proxy.yml, .env and rootCA.pem files

NOTE: Create or copy the above control_proxy.yml, .env and rootCA.pem files. Before install, edit the next lines in install_gateway.sh file:

# Ensure this script hasn't changed
-if ! cmp "$INSTALL_DIR"/magma/orc8r/tools/docker/install_gateway.sh install_gateway.sh; then
-   echo "This 'install_gateway.sh' script has changed..."
-   echo "Please copy this file from $INSTALL_DIR/magma/orc8r/tools/docker/install_gateway.sh and re-run"
-   exit
-fi
# Ensure this script hasn't changed
+if ! cmp "$INSTALL_DIR"/magma/orc8r/tools/docker/install_gateway.sh install_gateway.sh; then
+   echo "This 'install_gateway.sh' script has changed..."
+   echo "Please copy this file from $INSTALL_DIR/magma/orc8r/tools/docker/install_gateway.sh and re-run"
+   exit
+fi

Execute the installation script:

cd ~/feg_install
sudo ./install_gateway.sh feg

Show the Containers Status

After installation, be sure that your containers are running.

docker ps -a
# CONTAINER ID   IMAGE                                                               COMMAND                  CREATED      STATUS                PORTS     NAMES
# 8e84e9a0321b   registry.gitlab.com/everis_factory/fbc/magma/gateway_go:latest       "envdir /var/opt/mag…"   6 days ago   Up 9 hours                      csfb
# a0bf5e364297   registry.gitlab.com/everis_factory/fbc/magma/gateway_go:latest       "envdir /var/opt/mag…"   6 days ago   Up 6 days                       eap_aka
# c8cf2d72e231   registry.gitlab.com/everis_factory/fbc/magma/gateway_go:latest       "envdir /var/opt/mag…"   6 days ago   Up 6 days                       health
# 5675afcfe3b7   registry.gitlab.com/everis_factory/fbc/magma/gateway_python:latest   "python3.8 -m magma.…"   6 days ago   Up 6 days                       magmad
# baded46fcc8a   registry.gitlab.com/everis_factory/fbc/magma/gateway_go:latest       "envdir /var/opt/mag…"   6 days ago   Up 6 days                       swx_proxy
# a7faf0fdf768   registry.gitlab.com/everis_factory/fbc/magma/gateway_go:latest       "envdir /var/opt/mag…"   6 days ago   Up 6 days                       s6a_proxy
# 702d85f4b20b   registry.gitlab.com/everis_factory/fbc/magma/gateway_python:latest   "python3.8 -m magma.…"   6 days ago   Up 6 days                       eventd
# 7a3ac03a7115   registry.gitlab.com/everis_factory/fbc/magma/gateway_go:latest       "envdir /var/opt/mag…"   6 days ago   Up 6 days                       feg_hello
# 73bdd4003e04   registry.gitlab.com/everis_factory/fbc/magma/gateway_go:latest       "envdir /var/opt/mag…"   6 days ago   Up 6 days                       s8_proxy
# 5a02a0cecbc0   registry.gitlab.com/everis_factory/fbc/magma/gateway_go:latest       "envdir /var/opt/mag…"   6 days ago   Up 6 days                       aaa_server
# 24796306ec31   registry.gitlab.com/everis_factory/fbc/magma/gateway_go:latest       "envdir /var/opt/mag…"   6 days ago   Up 6 days                       eap_sim
# 7673fa10d57e   registry.gitlab.com/everis_factory/fbc/magma/gateway_python:latest   "/bin/bash -c '/usr/…"   6 days ago   Up 2 days                       control_proxy
# 7bf546f037c1   registry.gitlab.com/everis_factory/fbc/magma/gateway_go:latest       "envdir /var/opt/mag…"   6 days ago   Up 6 days                       session_proxy
# 2d952888cfcf   registry.gitlab.com/everis_factory/fbc/magma/gateway_go:latest       "envdir /var/opt/mag…"   6 days ago   Up 6 days                       radiusd
# dd6e24d3c357   registry.gitlab.com/everis_factory/fbc/magma/gateway_python:latest   "/bin/bash -c '/usr/…"   6 days ago   Up 6 days                       redis
# bbbaa8c7d04a   registry.gitlab.com/everis_factory/fbc/magma/gateway_python:latest   "/bin/bash -c '/usr/…"   6 days ago   Up 6 days (healthy)             td-agent-bit

Consider that you use hostnames, add the next lines to /etc/hosts.

NOTE: Consider changing the IP 127.0.0.1 to the IP of each service (controller, bootstrapper and fluentd) based on the configuration of your environment.

127.0.0.1     controller.magma.svc.cluster.local
127.0.0.1     bootstrapper-controller.magma.svc.cluster.local
127.0.0.1     fluentd.magma.svc.cluster.local

Add the session_proxy environment variables

For the session_proxy to take the necessary changes for the FeG deployment, modify the following lines in the service that uploads docker-compose.

  session_proxy:
    <<: *goservice
    environment:
      USE_GY_FOR_AUTH_ONLY: ${USE_GY_FOR_AUTH_ONLY}
      GY_SUPPORTED_VENDOR_IDS: ${GY_SUPPORTED_VENDOR_IDS}
      GY_SERVICE_CONTEXT_ID: ${GY_SERVICE_CONTEXT_ID}
      DISABLE_REQUESTED_SERVICE_UNIT_AVP: ${DISABLE_REQUESTED_SERVICE_UNIT_AVP}
+     OCS_SERVICE_IDENTIFIER_OVERWRITE: ${OCS_SERVICE_IDENTIFIER_OVERWRITE}
+     PCRF_91_COMPLIANT: ${PCRF_91_COMPLIANT}
+     MAGMA_PRINT_GRPC_PAYLOAD: 1
    container_name: session_proxy
    command: envdir /var/opt/magma/envdir /var/opt/magma/bin/session_proxy -logtostderr=true -v=0

After this change, restart the containers using:

cd /var/opt/magma/docker
docker-compose up -d

Verify again the status using docker ps -a.

Registration

Once the FeG has been installed, it must be registered in the orchestrator. For this, consider first taking the necessary information for registration. For this, run the following commands on the machine where the FeG was installed:

cd /var/opt/magma/docker
docker-compose exec magmad /usr/local/bin/show_gateway_info.py
# Hardware ID:
# ------------
# a4235f23-90e1-4b56-b0d9-990e4ef9dd90
# 
# Challenge Key:
# -----------
# MHYwEAYHKoZIzj0CAQYFK4EEACIDYgAESkk59Hqmh0gvyAq3sPRW8qqjDx3i88241syEF4dQESrD7VdWkOzi/OF8JBPkm7VlXoDRHI2I8z4io0T46gwWrbtIUwBqr4ICalSMJ1SKpHjCmZv3yyqqbLlc8s7RjMvp

Go to the Orchestrator API. Based on the FeG documentation, registration from the NMS GUI is in progress. Now, once you enter the API, you should see the following graphical interface on the screen:

FeG API Interface

Go to the Federation Networks part and create a new Network using a POST method, as shown in the following figure. Make sure the ID you are using for the new network is unique. In this case, a FeG Network with ID network_2 is created:

Federation Network Creation

You can validate the creation of the FeG Network by using the GET method as shown in the following figure:

Validate FeG Network Creation

Next to this, create a new Tier using the ID of the network you just created. Go to the Upgrades part and run a POST like the one seen in the following figure. Make sure to use the same ID and, for this case, remove the gw1 gateway that comes by default in the body.

Tier Creation

Then create the FeG using a POST in the Federation Gateways section. Make sure to use the same Network ID, Hardware ID, Challenge Key, and Tier from the previous steps. The following image shows this POST done from the Orchestrator UI API.

Federation Gateway Creation

Validate the Registration Process

You can validate the creation of the FeG by means of a GET or by executing the following command from the machine where you created the FeG.

cd /var/opt/magma/docker
docker-compose exec magmad /usr/local/bin/checkin_cli.py
# 1. -- Testing TCP connection to controller.magma.svc.cluster.local:8443 --
# 2. -- Testing Certificate --
# 3. -- Testing SSL --
# 4. -- Creating direct cloud checkin --
# 5. -- Creating proxy cloud checkin --
# Success!

If an error occurs in the execution of the above command, make sure you have in your /etc/hosts file the IPs of the services controller.magma.svc.cluster.local and bootstrapper-controller.magma.svc.cluster.local. Also, ensure that the control_proxy.yml information in the /var/opt/magma/configs/ and /etc/magma/ folders, point to the ports that the controller exposes for the aforementioned services. In this case, the are ports 8443 and 8444 respectively.

Then, so that the FeG can be seen in the organization's interface in the NMS, it is necessary to enable it from the interface of the master NMS. For this, go to https://master.nms.magma.svc.cluster.local, search for the organization magma-test and in its options, enable the view of the newly created FeG as shown in the following figure:

Enable FeG in NMS

Save your changes and now go to your organization's NMS, in this case https://magma-test.nms.magma.svc.cluster.local/. Login to the interface (default credentials [email protected]/password1234) and select the network you just created, as shown in the following figure:

NMS FeG choosing

When selected, you should see the FeG ID plus its Hardware ID on the screen, as shown in the following figure:

FeG NMS View

Remember that if you have multiple FeG registered, the Orchestrator will select one of these as the primary. In case it is the primary, a star will appear on the screen next to its ID.

Upgrade the FeG deployment

To upgrade the FeG using new containers, you must download the already deployed containers and create a new installation. Follow the steps below for that:

sudo apt update
cd /var/opt/magma/docker
docker-compose down
docker ps -a    # <-- Validate that all docker containers are down
cd ~/feg_install
sudo vi .env    # <-- Modify the version for the new container
# [...]
# IMAGE_VERSION=latest
# [...]
sudo ./install_gateway.sh feg   # <-- Install the FeG
cd /var/opt/magma/docker/

The last step is to add the session_proxy environment variables and male a docker-compose up -d.

⚠️ **GitHub.com Fallback** ⚠️