Virtual Network Topology - maggiemsft/SONiC GitHub Wiki

As we mentioned earlier, we are using high performance servers to create virtual network routers and virtual Linux machines, then use virtual vSwitch to establish a virtual network through tap interfaces, then use fanout switches layer2 VLAN to connect the virtual network to SONiC DUT.

Before we start to go deep in virtual topology, here are some terminologies we are going to use when we talk about how to set up virtual topology:

  • PTF: PTF is the abbreviation of open source project “Packet Test Framework”, PTF is a Python based dataplane test framework. It is based on unittest, which is included in the standard Python distribution. PTF docker is a docker built for running ptf which is focus on dataplane testing. SONiC testbed virtual network is using ptf as a packer generator and capture tool in test cases. Detailed please see PTF

  • VM: Here VM in sonic testbed is refer to virtual machines in testing servers running vEOS operating system from Arista Networks. It’s a virtual EOS switch/router that is part of the virtual test network.

Test Server

Test server is the place where virtual network topology created and connected to SONiC DUT. The testbed server has 2 network ports:

  • Trunk port (Mellanox MT27700 Family CX4) to root fanout switch
  • Server management port connecting to management network to manage the server, VMs and PTF containers on the server

Following figure is what's inside a test server that we use to setup virtual network and virtual topology.

  • We use Arista vEOS VMs as virtual routers for now and later will migrate to container based cEOS. They are used to setup protocol test such as BGP, LACP, LLDP. Each EOS VM uses about 2G of RAM and has 6 network interfaces.

    • 4 front panel ports. These ports are connected to openvswitch bridges, which are connected to vlan interfaces. The vlan interfaces are connected to the fanout switch (through physical port).
    • 1 back panel port. All VMs in one topology have their 5th port connected to this backplane network.
    • 1 management port. This port is used to connect to the VMs for management

  • PTF container is used to send and receive data packets to validate the DUT data plane. For any testing topology, PTF container interface number is exactly the same as SONiC DUT front panel ports, so each DUT front panel port connects to one of the PTF interface in port index order from both side.

Topologies

There are 2 type of virtual network connections when we build virtual testbed topology:

  • PTF with direct port for PTF topology

ptf-32 or ptf-64 is this type of topology.

For this type of topology, DUT front panel port is directly connected to one of PTF container ports. Usually eth0 port of PTF container connects Ethernet0(index0) port of DUT, eth1 port of PTF container connects Ethernet4(index1) port of DUT and so on. This is usually used in PTF topologies to connect DUT ports to PTF container ports.

  • PTF with injected port for VM+PTF topology

T0 or T1, T1-lag topology are this type of topology.

For this type of topology, DUT front panel port is directly connected to one of VMs interfaces. But also we have a tap into this connection. Packets coming from the physical vlan interface are sent to both the VMs and the PTF docker. Packets from the VM and PTF docker are sent to the vlan interface. It allows us to inject packets from the PTF host to DUT and maintain a BGP session between VM and DUT at the same time.

Virtual topology types

All Virtual topology definition files are YAML format description file under Virtual Topology. You may go through these definition file to understand how each topology is defined.

PTF

ptf-32 and ptf-64 are PTF type topologies. There is no VMs in ptf topology.

PTF Testbed

T0

T0 is one type of injected connection VM+PTF topology. It's simulate a Top of Rack switch(T0) in a datacenter.

The concept of T0 topology is:

SONiC DUT has a few uplink ports configured and connected to EOS-VMs and ptf interface, form LACP port channels and BGP neighbors with EOS-VMs. Rest of ports that are not connecting to VMs are put into one large Vlan interface and connect to PTF interfaces only.

Following figure is the first 32 ports t0 topology.

Depends on DUT actual port numbers and how uplink and downlink are connected, there are a few different T0 topologies being derivative from this topology but keep the same concept, such as t0-116; t0-64; t0-52 etc.

How VMs connects to DUT

T0 topology

T0 Testbed

T1/T1-lag

T1 topology is another type of injected connection VM+PTF topology. It's simulate a switch in a datacenter T1 layer to connect ToR switch(T0) and to T2 fabric.

The concept of T1 topology is:

SONiC DUT has all ports configured and connected to EOS-VMs and ptf interface, form BGP neighbors with EOS-VMs. Some of connections are configured with LACP portchannels and some are not.

There are 2 type of T1 topologies:

  • T1: No LACP portchannel interfaces configured in DUT
  • T1-lag: There are LACP lag interfaces configured in DUT. Following figures are first 32 ports t１ and t1-lag topology.

Depends on DUT actual port numbers and how uplink and downlink are connected, there are a few different T1 topology being derivative from original T1 and T1-lag topology but keep the same concept, such as t1-64-lag; t1-64

T1

How VMs connect to DUT

T1 Testbed

T1 topology

T1 Testbed

**T1-lag **

How VMs connect to DUT

T1-lag Testbed

T1-lag topology

T1-lag Testbed