DevStack 2 interfaces - hpaluch/hpaluch.github.io GitHub Wiki

DevStack with 2 interfaces

Both OpenStack and DevStack expects at least 2 interfaces (and networks) to offer VMs reachable from remote hosts:

Management interface (IP defined as HOST_IP in DevStack)
Public interface (defined as PUBLIC_INTERFACE)

If you use only 1 interface, DevStack will just allocate internal bridge br-ex with NAT, so VMs will have access to Internet but you will be not able to reach VM from remote machines unless you will manually setup DNAT rules...

My scenario

I now use nested LibVirt under openSUSE 15.6 Host where Ubuntu VM running DevStack is installed. VM parameters:

OS Ubuntu 24.04 LTS
4 CPUs, 10 GB RAM, 64 GB disk, 60 GB root filesystem, 4 GB swap
2 networks and interfaces:
- eth0 NAT network called default, net: 192.168.122.0/24 - will be used for DevStack management
- eth1 NAT network called default2, net: 192.168.123.0/24 - cloned from default, will be used as "public" network

Network configuration inside Ubuntu VM:

netplan /etc/netplan/50-cloud-init.yaml

network:
    ethernets:
        eth0:
            dhcp4: true
        eth1:
            link-local: [ ]
    version: 2

2nd NIC (eth1) must be just Enabled Up, but without (!) IP address as described on https://docs.openstack.org/devstack/latest/guides/neutron.html
NOTE: using just eth1: {} will not work properly, because networkd will wait for IPv6 link-local address that will be never established that will cause systemd-networkd-wait-online.service fail and thus even Nova service fail - because systemd will wait until all interfaces have assigned IP address (which will never happen).
to generate above netplan configuration run netplan --debug generate and reboot system.

here is relevant output from networkctl:

# networkctl
IDX LINK       TYPE     OPERATIONAL SETUP
  1 lo         loopback carrier     unmanaged
  2 eth0       ether    routable    configured
  3 eth1       ether    carrier     configured

Here are details about devstack version I tested:

cd ~/projects/devstack
git remote -v

  origin	https://github.com/openstack/devstack.git (fetch)
  origin	https://github.com/openstack/devstack.git (push)

git branch -v

    master        97ea68ec Fix the db user for mariadb in ubuntu 24.04
  * stable/2024.2 ae4e1d62 Cap stable/2024.2 network, swift, volume api_extensions for tempest

And my ~/projects/devstack/local.conf:

[local](/hpaluch/hpaluch.github.io/wiki/localrc)
ADMIN_PASSWORD=Secret123
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD

Q_USE_SECGROUP=True
PUBLIC_INTERFACE=eth1
FLOATING_RANGE=192.168.123.0/24
PUBLIC_NETWORK_GATEWAY=192.168.123.1
Q_FLOATING_ALLOCATION_POOL=start=192.168.123.20,end=192.168.123.30

Q_USE_PROVIDERNET_FOR_PUBLIC=True

WARNING! After installation verify that there is NOT running default network, it is NOT used by DevStack and it may clash with it. Using:

virsh net-list
virsh net-destroy default # destroy = stop
virsh net-autostart --disable default

After setup you need to:

enable incoming traffic in default security group - from https://docs.openstack.org/devstack/latest/networking.html

openstack security group rule create --proto icmp --dst-port 0 default
openstack security group rule create --proto tcp --dst-port 22 default

associate public (also known as floating) IP address to VM
you can find complete example on https://docs.openstack.org/devstack/latest/networking.html

below is example of such running VM:

cd ~/projects/devstack
source openrc admin
openstack server list

+-----------------------------+-------+--------+-----------------------------+--------------------------+---------+
| ID                          | Name  | Status | Networks                    | Image                    | Flavor  |
+-----------------------------+-------+--------+-----------------------------+--------------------------+---------+
| 952b50f7-8bd1-45e6-98dd-    | test1 | ACTIVE | private=10.0.0.56,          | N/A (booted from volume) | m1.nano |
| 786d88e026b0                |       |        | 192.168.123.22, fd45:80bd:5 |                          |         |
|                             |       |        | e16:0:f816:3eff:fe6f:fab4   |                          |         |
+-----------------------------+-------+--------+-----------------------------+--------------------------+---------+

you have to use target 192.168.123.22 from remote machine - in my case open SUSE 15.6 host (because this site is also NATed).
for default cirros openstack VM using [email protected], password gocubsgo
WARNING! This access works from Host only but not inside Ubuntu (so you may NOT test SSH inside you Ubuntu with DevStack installed - only from external host). I guess that it is because some required forward rule

Outdated DevStack docs

As reported on https://bugs.launchpad.net/devstack/+bug/2091266 official DevStack docs still use /etc/network/interfaces for Ubuntu setup which is wrong, because now only Netplan is available.

Issues

DNS in Nested Cirros VM is wrong:

cat /etc/resolv.conf

nameserver 127.0.0.53

There is note in official DevStack docs:

As workaround I did:

# using DNS from *public* subnet
openstack subnet set --dns-nameserver 192.168.123.1 private-subnet

After Stop and Start of instance we can test inside cirros VM:

nslookup -type=any www.google.com

NOTE! When running nslookup with default parameters (without -type=any) it prints correct answer but later reports timeout - not sure why - tcpdump did no reveal anything incorrect.

Resources

https://docs.openstack.org/devstack/latest/networking.html