The virtual router provides to open source solutions for load balancing TCP services:

1. Keepalived + LVS (IPVS).
2. HAProxy.

In this guide we will assume the following configuration:

        public network
      ┌───────────────   Users
      │
   ┌──┴─┐  10.0.1.1
┌──┤eth0├────────┐
│  └────┘        │   LB service (http:80)
│ Virtual Router │
│                │   10.0.1.1:80 ┌────► 172.20.0.104:8080
│ Load Balancer  │               │
│  ┌────┐        │               └────► 172.20.0.105:8080
└──┤eth1├────────┘
   └─┬──┘
     │   private network
     └────┬─────────────────┬───
     172.20.0.104      172.20.0.105
       ┌──┴──┐           ┌──┴──┐
       │ LB1 │           │ LB2 │
       └─────┘           └─────┘

Additionally we will consider two LB deployment modes:

• Static, when the number LB servers are fixed and known
• Dynamic, when LB servers can be added/removed from the backend

To configure the LB services you need to define the public (user-facing) IP. This IP is only known once the Virtual Router is deployed and the IP assignment is resolved. Usually, you can refer to the public facing IP of the Virtual router as ETH0_EP0

For advanced scenarios, OpenNebula provides you with some convenient placeholders that are resolved automatically at runtime:

• <ETHx_IPy> means "interpolate (in-place) the y-th IP of the x-th NIC"
• <ETHx_VIPy> means "interpolate (in-place) the y-th Virtual IP of the x-th NIC"
• <ETHx_EPy> means "interpolate (in-place) the y-th EndPoint of the x-th NIC"

Values of ETHx_IPy and ETHx_VIPy are always merged together to produce ETHx_EPy. When ETHx_VIPy is undefined, then ETHx_EPy is always set to ETHx_IPy as a fallback, that way you can always have some valid "endpoints" even in non-HA scenarios (no VIPs defined).

To configure keepalived LVS you need to define:

• The front-end IP and port to expose the service ONEAPP_VNF_LBx_IP, ONEAPP_VNF_LBx_PORT
• The IP and port of the backend severs ONEAPP_VNF_LBx_SERVERy_HOST, ONEAPP_VNF_LBx_SERVERy_PORT set at deployment time
• The protocol and mode where keepalive will use to load balance the services (e.g. ONEAPP_VNF_LBx_PROTOCOL)

For example, to create a static LVS-based TCP LB, for the topology above, you can use:

CONTEXT = [
  ...
  ONEAPP_VNF_LB_ENABLED = "YES",

  ONEAPP_VNF_LB0_IP        = "<ETH0_EP0>", # Interpolate the first "endpoint".
  ONEAPP_VNF_LB0_PORT      = "80",
  ONEAPP_VNF_LB0_PROTOCOL  = "TCP",
  ONEAPP_VNF_LB0_METHOD    = "NAT",
  ONEAPP_VNF_LB0_SCHEDULER = "rr", # "Round-robin".

  # Static backends:

  ONEAPP_VNF_LB0_SERVER0_HOST = "172.20.0.104",
  ONEAPP_VNF_LB0_SERVER0_PORT = "8080",
  ONEAPP_VNF_LB0_SERVER1_HOST = "172.20.0.105",
  ONEAPP_VNF_LB0_SERVER1_PORT = "8080",
  ...
]

You can see in the example above, that we defined the LB0 by providing the IP / PORT pair with some extra settings. Then two (static) backends are defined for the LB0.

Similarly for HAProxy you need to define:

• The front-end IP and port to expose the service ONEAPP_VNF_HAPROXY_LBx_IP, ONEAPP_VNF_HAPROXY_LBx_PORT
• The IP and port of the backend severs ONEAPP_VNF_HAPROXY_LBx_SERVERy_HOST, ONEAPP_VNF_HAPROXY_LBx_SERVERy_PORT

For example, to create a HAProxy-based TCP LB for the topology above, you can use:

CONTEXT = [
  ...
  ONEAPP_VNF_HAPROXY_ENABLED = "YES",

  ONEAPP_VNF_HAPROXY_LB0_IP   = "<ETH0_EP0>", # Interpolate the first "endpoint".
  ONEAPP_VNF_HAPROXY_LB0_PORT = "80",

  # Static backends:

  ONEAPP_VNF_HAPROXY_LB0_SERVER0_HOST = "172.20.0.104",
  ONEAPP_VNF_HAPROXY_LB0_SERVER0_PORT = "8080",
  ONEAPP_VNF_HAPROXY_LB0_SERVER1_HOST = "172.20.0.105",
  ONEAPP_VNF_HAPROXY_LB0_SERVER1_PORT = "8080",
  ...
]

The procedure is as follows:

1. Add ONEAPP_VNF_LB_ONEGATE_ENABLED="YES" or ONEAPP_VNF_HAPROXY_ONEGATE_ENABLED="YES" to the Virtual Router context, to set one or both LBs in the dynamic mode.
2. Create a new Virtual Machine that includes BACKEND = YES in its CONTEXT attribute (not required in OneFlow).
3. Update the VM to include the IP/HOST, and PORT pairs with the target IP

Let's assume you already have deployed a Virtual Router in static mode with backends 172.20.0.104 and 172.20.0.105, as described above. To add a new backend. Create a new Virtual Machine, for example you could use the following template:

NAME = MyBackend
...
NIC = [ NETWORK = "private" ]

# The ONEGATE_* values below are created inside the "User Template" of the backend VM.

ONEGATE_LB0_IP = "<ETH0_EP0>"
ONEGATE_LB0_PORT = 80

CONTEXT = [
  ...
  NETWORK = YES,
  SSH_PUBLIC_KEY = "$USER[SSH_KEY]",
  BACKEND = YES, # NOTE: This isn't required when running inside OneFlow.
  ...
]

Once the VM is created look for the assigned IP in the private network (for example let's assume it is 172.20.0.133) and update the user template contents with the IP/PORT pairs (e.g. onevm update):

ONEGATE_LB0_SERVER_HOST = "172.20.0.133"
ONEGATE_LB0_SERVER_PORT = 8080

Equivalently you'll need to create a VM that includes the following attributes in its template:

NAME = MyBackend
...
NIC = [ NETWORK = "private" ]

# The ONEGATE_* values below are created inside the "User Template" of the backend VM.

ONEGATE_HAPROXY_LB0_IP = "<ETH0_EP0>"
ONEGATE_HAPROXY_LB0_PORT = 80

CONTEXT = [
  ...
  NETWORK = YES,
  SSH_PUBLIC_KEY = "$USER[SSH_KEY]",
  BACKEND = YES, # NOTE: This isn't required when running inside OneFlow.
  TOKEN = YES
  ...
]

And then update it with the target IP/PORT pairs (e.g. onevm update):

ONEGATE_HAPROXY_LB0_SERVER_HOST = "172.20.0.133"
ONEGATE_HAPROXY_LB0_SERVER_PORT = 8080

This can be added as a start script on the VM template, so the VM is configured on instantiation:

onegate vm update --data "ONEGATE_HAPROXY_LB0_SERVER_HOST=\"$ETH0_IP\""
onegate vm update --data "ONEGATE_HAPROXY_LB0_SERVER_PORT=\"8080\""

The Virtual Router appliance can deployed in three different ways:

1. As a proper OpenNebula Virtual Router instance.
2. As a standalone VM, but inside an OneFlow service instance.
3. As a standalone VM. (not supported)

When running both types of LBs (static and dynamic) in modes 1. and 2. the context and OneGate interface is exactly the same (with the important caveat, that the mode 1. requires backends to have the CONTEXT = [..., BACKEND = "YES", ...] flag defined).

Under the hood the source of updates is different, it both cases OneGate is used, but in the mode 1. all VNETs attached to the VR are scanned (recursively), in the mode 2. OneFlow provides API responses.

In that sense mode 1. is superior to mode 2., because a "proper" VR can reverse-proxy to any VMs in its attached VNETs (also to VMs running in OneFlow instances), at the same time mode 2. allows for backends from within its OneFlow instance only.