Parent CloudStack Environment - shapeblue/Trillian GitHub Wiki
Introduction
The Trillian framework needs to know about the environment which it is deploying into. The pre-reqs and configuration to allow this to happen will be listed and explained here.
Throughout the documentation we will refer to PARENT and NESTED CloudStack environments. Confusion can occur when the two are mixed up, especially when it comes to networking. So - to state the obvious, the PARENT CloudStack is the environment which looks after the lifecycle of the VMs created. The NESTED environment is the one created by Trillian which runs the internal CloudStack infrastructure VMs and hypervisors.
Management, public and guest networks for nested environments
https://raw.githubusercontent.com/wiki/shapeblue/Trillian/TrillianNetworking.png
The nested hypervisors must be attached to networks supplied by the parent CloudStack instance. For ease of use, the management, guest and public networks are expected to be created on a shared network in the parent CloudStack instance.
The nested guest and public networks may be on a number of different VLANs, therefore the shared network created in the parent CloudStack instance utilizes VMware ESXi's ability to configure a vSwitch as VLAN 4095. This is then interpreted by the ESXi hosts as 'trunk all VLANs from the guests', and this traffic is propagated across all ESXi hosts in the cluster.
Isolated networks for the nested management planes, while great for isolation, pose an number of issues with regards to communicating with hosts which are NAT'd. To get around this we have also used a shared networks for nested cloud management and storage IP traffic.
IP ranges and VLANs
A number of nested environments must co-exist in the same IP space for public and management networks, and the same VLAN space in the case of guest networks. To facilitate this the available L2 and L3 networking space is carved into smaller ranges which are allocated to nested environments on demand. On nested environment tear down these are returned back to the to the free pool of networking resources.
For instance, a management range of say 192.168.1.0/24 might be split into smaller chunks of 12 IP addresses each with the shared gateway of (say) 192.168.1.254
The first environment to be created would get 192.168.1.1 to 192.168.1.12 to use for its 'reserved system range' the next environment would get 192.168.1.13 to 192.168.1.24 and so on.
This is also done for public IP addresses and guest VLAN ranges for the nested environments. These ranges are stored in a MySQL database in the parent CloudStack environment.
Projects
Each nested CloudStack instance created by Trillian is hosted in it's own parent CloudStack project. This allows for easier management of each environment as well as allowing each environment to be shared with selected parent CloudStack users.
Templates
The building of VMs naturally requires the creation of templates. The template utilised for any given VM in the nested environments (management server, hypervisor host, etc.) can be set as a default in the Trillian plays, alternatively they can be overridden using Ansible commandline arguments.