vRouter in Gateway Mode - ganeshahv/Contrail_SRE GitHub Wiki
This will describe the procedure to configure vRouter in GW mode and few tests around it.
All credits to Yuvaraja Mariappan([email protected]) for the procedure and explanation.
1]. Configure the following in the /etc/contrail/common_vrouter.env on the compute host:
VROUTER_AGENT__DEFAULT__gateway_mode=server
2]. Recreate the agent docker and see that the change is reflected in the agent.conf file:
cd /etc/contrail/vrouter/
docker stop vrouter_vrouter-agent_1
docker rm vrouter_vrouter-agent_1
docker-compose up -d
docker exec -it vrouter_vrouter-agent_1 bash
(vrouter-agent)[root@r2ru1 /]$ cat /etc/contrail/contrail-vrouter-agent.conf | grep gateway
gateway=100.100.100.100
gateway_mode = server
(vrouter-agent)[root@r2ru1 /]$
3]. Inside the host, create a bridge
brctl addbr iotbridge
ip link set dev iotbridge up
4]. Get the instances running inside the host:
[root@r2ru1 ~]# virsh list
Id Name State
----------------------------------------------------
1 instance-00000039 running
2 instance-0000003c running
[root@r2ru1 ~]#
5]. Attach a new interface to the running instance and add it to the iotbridge:
[root@r2ru1 ~]# virsh attach-interface --domain 2 bridge iotbridge
6]. Observe that inside the VM, a new interface is created:
root@blue-vm:/home/ubuntu# ifconfig
eth0 Link encap:Ethernet HWaddr 02:e3:1c:b9:98:9b
inet addr:192.168.20.3 Bcast:192.168.20.255 Mask:255.255.255.0
inet6 addr: fe80::e3:1cff:feb9:989b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:641 errors:0 dropped:0 overruns:0 frame:0
TX packets:909 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:79111 (79.1 KB) TX bytes:84041 (84.0 KB)
eth1 Link encap:Ethernet HWaddr 52:54:00:91:30:3c
inet6 addr: fe80::5054:ff:fe91:303c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:72 (72.0 B) TX bytes:2358 (2.3 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
root@blue-vm:/home/ubuntu#
7]. Login to the webUI.
- Create a physical router with the exact name of the compute which is in Gateway mode.
- Provide only the VTEP and set the
Associated Virtual Routersto typeEmbedded. - Create a physical interface with the name
iotbridge - Create a logical interface with the parent physical interface as
iotbridge. - The
Logical Interface Typeis aServer. - Provide a VLAN-ID and select the correct Virtual-Network.
- Add the MAC address of the eth1 interface under the
Server MAC. - Observe a new VMI is created with IP from the VN provided for the logical interface.
8]. Observe a new interface, whose parent is iotbridge getting created in the vRouter
[root@r2ru1 ~]# vif --get 8
Vrouter Interface Table
Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, L=MAC Learning Enabled
Proxy=MAC Requests Proxied Always, Er=Etree Root, Mn=Mirror without Vlan Tag, HbsL=HBS Left Intf
HbsR=HBS Right Intf, Ig=Igmp Trap Enabled
vif0/8 93258913-a87f-460e-b955-b67a47b2e840 Vlan(o/i)(,S): 5/5, 52:54:00:91:30:3c Bridge Index: 476 Parent:vif0/3 NH: 25
Type:Virtual(Vlan) HWaddr:fe:54:00:91:30:3c IPaddr:192.168.10.7
Vrf:2 Mcast Vrf:2 Flags:PL3L2DErMn QOS:-1 Ref:6
RX packets:0 bytes:0 errors:0
TX packets:47 bytes:2162 errors:0
ISID: 0 Bmac: 52:54:00:91:30:3c
Drops:0
[root@r2ru1 ~]# vif --get 3
Vrouter Interface Table
Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, L=MAC Learning Enabled
Proxy=MAC Requests Proxied Always, Er=Etree Root, Mn=Mirror without Vlan Tag, HbsL=HBS Left Intf
HbsR=HBS Right Intf, Ig=Igmp Trap Enabled
vif0/3 OS: iotbridge NH: 20
Type:Physical HWaddr:fe:54:00:91:30:3c IPaddr:0.0.0.0
Vrf:0 Mcast Vrf:65535 Flags:TcL3L2PrVntEr QOS:-1 Ref:3
RX packets:19 bytes:6050 errors:19
TX packets:50 bytes:2300 errors:0
Drops:19
[root@r2ru1 ~]#
9]. Get inside the VM
- Create a sub-intf with the same vLAN tag as that of the logical-interface.
root@blue-vm:/home/ubuntu# modprobe 8021q
root@blue-vm:/home/ubuntu# vconfig add eth1 5
Added VLAN with VID == 5 to IF -:eth1:-
root@blue-vm:/home/ubuntu# ifconfig eth1.5
eth1.5 Link encap:Ethernet HWaddr 52:54:00:91:30:3c
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:168 (168.0 B) TX bytes:0 (0.0 B)
- Request for DHCP on that sub-interface. The same IP assigned for the logical-interface should be seen on the sub-intf
root@blue-vm:/home/ubuntu# dhclient eth1.5
root@blue-vm:/home/ubuntu# ifconfig eth1.5
eth1.5 Link encap:Ethernet HWaddr 52:54:00:91:30:3c
inet addr:192.168.10.7 Bcast:192.168.10.255 Mask:255.255.255.0
inet6 addr: fe80::5054:ff:fe91:303c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:115 errors:0 dropped:0 overruns:0 frame:0
TX packets:103 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:5631 (5.6 KB) TX bytes:5538 (5.5 KB)
root@blue-vm:/home/ubuntu#
10]. The vRouter has the following info:
- For the BMS(connected behind the QFX)
(vrouter-agent)[root@r2ru1 /]$ rt --dump 2 --family bridge | grep 0:25:90:c6:da:7b
108412 0:25:90:c6:da:7b LDf 10 35 9048
(vrouter-agent)[root@r2ru1 /]$ nh --get 35
Id:35 Type:Tunnel Fmly: AF_INET Rid:0 Ref_cnt:7 Vrf:0
Flags:Valid, Vxlan, Etree Root,
Oif:0 Len:14 Data:88 e6 4b 61 fb 63 90 e2 ba db 2a 48 08 00
Sip:100.100.100.101 Dip:1.1.1.7
- For the iotbridge/logical interface
(vrouter-agent)[root@r2ru1 /]$ rt --dump 2 --family bridge | grep fe:54:0:91:30:3c
74904 fe:54:0:91:30:3c Df - 3 1091
(vrouter-agent)[root@r2ru1 /]$ nh --get 3
Id:3 Type:L2 Receive Fmly: AF_INET Rid:0 Ref_cnt:15 Vrf:0
Flags:Valid, Etree Root,
- For the local VM
(vrouter-agent)[root@r2ru1 /]$ rt --dump 2 --family bridge | grep 13
243596 2:71:ad:98:e7:13 - 55 8238
(vrouter-agent)[root@r2ru1 /]$ nh --get 55
Id:55 Type:Encap Fmly:AF_BRIDGE Rid:0 Ref_cnt:3 Vrf:2
Flags:Valid, Policy, Etree Root,
EncapFmly:0806 Oif:6 Len:14
Encap Data: 02 71 ad 98 e7 13 00 00 5e 00 01 00 08 00
(vrouter-agent)[root@r2ru1 /]$ vif --get 6
Vrouter Interface Table
Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, L=MAC Learning Enabled
Proxy=MAC Requests Proxied Always, Er=Etree Root, Mn=Mirror without Vlan Tag, HbsL=HBS Left Intf
HbsR=HBS Right Intf, Ig=Igmp Trap Enabled
vif0/6 OS: tap71ad98e7-13 NH: 47
Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:192.168.10.3
Vrf:2 Mcast Vrf:2 Flags:PL3L2DEr QOS:-1 Ref:6
RX packets:9949 bytes:888769 errors:0
TX packets:10385 bytes:914102 errors:0
Drops:8
(vrouter-agent)[root@r2ru1 /]$