vrouter agent not running modprobe: ERROR: could not insert 'vrouter': Cannot allocate memory - lujiang2016/contrail GitHub Wiki

vrouter-agent is not loading, vrouter is showing not present, agent process is showing failed (exited too quickly)

all1r42c007:~$ sudo contrail-status -d
vRouter is NOT PRESENT

== Contrail vRouter ==
supervisor-vrouter:           active
contrail-vrouter-agent        failed              Exited too quickly (process log may have details)
contrail-vrouter-nodemgr      active              pid 168310, uptime 0:02:02


all1r42c007:~$ sudo contrail-version
Package                                Version Build-ID | Repo | Package Name
-------------------------------------- ------------------------------ ----------------------------------
contrail-lib                           3.2.8.0-66 66
contrail-nodemgr                       3.2.8.0-66 66
contrail-nova-vif                      3.2.8.0-66 66
contrail-openstack-vrouter             3.2.8.0-66 66
contrail-setup                         3.2.8.0-66 66
contrail-utils                         3.2.8.0-66 66
contrail-vrouter-agent                 3.2.8.0-66 66
contrail-vrouter-common                3.2.8.0-66 66
contrail-vrouter-dkms                  3.2.8.0-66 66
contrail-vrouter-init                  3.2.8.0-66 66
contrail-vrouter-utils                 3.2.8.0-66 66
nova-common 1:2015.1.0-1~u14.04.1-main34471+r3.0.3.4566
nova-compute 1:2015.1.0-1~u14.04.1-main34471+r3.0.3.4566
nova-compute-kvm 1:2015.1.0-1~u14.04.1-main34471+r3.0.3.4566
python-contrail                        3.2.8.0-66.1 66
python-contrail-vrouter-api            3.2.8.0-66 66
python-neutronclient 1:2.4.0-1~u14.04.1-main769+r3.0.3.6  66
python-nova 1:2015.1.0-1~u14.04.1-main34471+r3.0.3.4566
python-opencontrail-vrouter-netns      3.2.8.0-66 66

This issue is caused by system failure while inserting vrouter module in kernel, we can tell from the following log snippet.

Tue Jan 29 22:11:36 UTC 2019 : Removing old vrouter module
+ rmmod vrouter
rmmod: ERROR: Module vrouter is not currently loaded
+ modprobe vrouter
modprobe: ERROR: could not insert 'vrouter': Cannot allocate memory
+ '[' 1 '!=' 0 ']'
++ date
+ echo 'Tue Jan 29 22:11:37 UTC 2019 : Error inserting vrouter module'
Tue Jan 29 22:11:37 UTC 2019 : Error inserting vrouter module
+ return 1
Tue Jan 29 22:11:37 UTC 2019: Value __VGW_SUBNET_IP__

If there is no one recompiled the vrouter module inproperly, this should be same issue as below, which can be fixed by applying command “free && sync && echo 3 > /proc/sys/vm/drop_caches && free” then restart vrouter, or you can reboot the whole compute node to fix it. Please let me know if it works, thanks a lot.

VRouter memory requirements VRouter will allocate few blocks of memory during its initialisation. The memory is normally available during bootup of compute node.

If the vrouter module is reloaded on a running compute node, there are chances that vrouter may not get all required memory blocks and insertion of vrouter module can fail.

Running command below will free-up os-cache and lets vrouter allocate memory blocks it needs.

root@nodel2:~$ free && sync && echo 3 > /proc/sys/vm/drop_caches && free
 total used free shared buffers cached
Mem: 263785456 16921100 246864356 1312 73132 139468
-/+ buffers/cache: 16708500 247076956
Swap: 268324860 0 268324860

 total used free shared buffers cached
Mem: 263785456 16749356 247036100 1312 1144 39532
-/+ buffers/cache: 16708680 247076776
Swap: 268324860 0 268324860

https://github.com/Juniper/contrail-controller/wiki/Vrouter-Module-Parameters