setup manila development environment in ubuntu vm - csabahenk/manila GitHub Wiki

Hi,

GlusterFS-Manila team wanted to find a quick, easy way to launch and use a reliable Manila setup for development purposes. We've been perenially plagued by our inability to quickly setup a new development environment, and reuse the setup. This has had a direct adverse effect of prolonging our development work cycle.

The Manila setup we refer to is an all-in-one setup in a VM. We chose this setup for it's simplicity, where all the OpenStack services and their dependencies are running in a single node. And the setup - installation, configuration, and initiation - of various services is done by DevStack's shell scripts.

Reusing even this seemingly simple setup has been difficult due to the following issues:

• setup not coming back to the original state after restarting the OpenStack services.

• insufficient memory/RAM.

• shortage of disk space.

Besides these issues there remains an unsubstantiated but nevertheless present
concern that Fedora may not be the best base operating system to run OpenStack services. Ubuntu is more widely used by the Manila developers (also other OpenStack developers?) and testers. So it's easy to believe that the Manila services run with lesser glitches in a Ubuntu machine than in a Fedora machine especially when using the latest source code.

The following sections try to address these problems.

GlusterFS-Manila team used to work with Fedora VMs hosted by Rackspace(paid service) and RHS Bangalore(free internal service). Initially, we weren't familiar with the minimum requirements to run an all-in-one setup. And we repeatedly ran out of RAM and disk space (Note: RabbitMQ, AMQP silently errors out when disk space drops below 1G.). So we decided to use VMs with RAM >= 4G and disk space >=20G. But we had issues with the VM host services. The RHS service wasn't very stable and Rackspace's paid service wasn't inexpensive. Luckily, the recently launched RedHat's OS1 internal cloud seemed to solve our problems - a free internal service with better resources than RHS service. With OS1, one can launch VMs with sufficient RAM and disk space, upload images, take snapshots of running instances. We've been using OS1 services for a few months now. While it has solved our problems with mininimum VM requirements, it's had its own sporadic issues - networking issues, VMs not booting, etc. But the OS1 cloud team has so far been quick to resolve these issues.

For reasons mentioned before, it made sense for us to have a Ubuntu based setup handy. This was done as documented below. In OS1 cloud, a working Ubuntu setup was snapshotted to produce an image. This image can now be used to launch VMs when needed, and by following a few steps the VM can be made an all-in-one OpenStack host.

The steps to easily re-use a setup were also figured out after several trials. These have been recorded in the section, "Reuse setup after host reboot".

A VM was launched using the image, name: ubuntu-12.04_jeos image, id: 6c4fb0d2-bf32-4b68-8357-5daf2a05198c, available publicly in the OS1 cloud, with the flavor 'm1.large'(8G RAM, 4VCPUs, 20G ephemeral disk). Various Open Stack services were installed and configured using DevStack to create a Manila generic driver setup. The setup was tested. Then the OpenStack services were killed (by running unstack.sh script of DevStack) to allow easier and quicker snapshotting. Snapshot was taken by clicking on the 'Create Snapshot'button in the 'Actions' field of the instance on the OS1 dashboard.

Launch a VM from the snapshot, name: Ubuntu-12_04-DevStack@BASE, id: 982e6ef3-7f6c-472e-af22-d18574a02171, with the m1.large flavor.

• Login to the VM as user 'stack' whose password is 'redhat'

  $ ssh [email protected]
  

• Once inside the VM you will find that the repos and config files to run OpenStack are in the '/opt/stack' and '/etc/' folder respectively. But running the 'rejoin-stack.sh' script in the 'devstack' folder will not restart the services as you might expect. The ip address used to identify end-points of the various services is that of host VM that was snapshotted. This needs to be changed to the ip of the new host of the various OpenStack service, i.e. the ip of your VM's eth0. You can do this by running 'stack.sh' script. But before doing that checkout the localrc file that controls the run of the stack.sh script.

$ cd devstack $ cat localrc | grep OFFLINE OFFLINE=True $ cat localrc | grep RECLONE RECLONE=no

The OFFLINE and RECLONE flags have been set to 'True' and 'no' respectively
to quicken the setup process by not accessing the internet and making use of
the existing repos. But to download and use the latest sources set the
OFFLINE and RECLONE flags to 'False' and 'yes' respectively. Now begin the
setup.

$ ./stack.sh

Once the script completes check whether all the OpenStack services are
running by rejoining and browsing the various service screens.

$ ./rejoin-stack.sh

- Login as OpenStack user 'demo' within the tenant 'demo' (The tenant and
the user were created by 'stack.sh' script).

$ source openrc demo demo

- Choose a tenant subnet that will have exclusive L2 network access to a share.
You can choose the existing subnet of the 'demo' tenant's 'private' network.
The subnet's information will be stored as a part of a share-network object.

$ manila share-network-create --neutron-net-id=<'private' network id> --neutron-subnet-id=

- Create a 1G NFS share associated with the share-network created above.

$ manila create --name 'myshare00' --share-network NFS 1

'myshare00' is created and an isolated L2 network access with the tenant's subnet is
established.

- Spawn a nova instance within the subnet of the 'private' network with ssh and
ping access to the instance.

$ nova keypair-add --pub_key ~/.ssh/id_rsa.pub mykey $ nova secgroup-add-rule default tcp 22 22 0.0.0.0/0 $ nova boot --flavor m1.micro --image ubuntu_1204_nfs_cifs --key-name mykey --security-groups default myvm00

- Allow the guest instance to access the share, and test mounting the share 
within the instance. The username and password of the guest is 'ubuntu'.

$ manila access-allow ip $ ssh [email protected] $ mount.nfs4

### Reuse setup after host reboot

The OpenStack services are not persistent after reboot. And they need to be
restarted. If active Manila shares and access rules were present prior to the
reboot it might get tricky to restore the state of the host machine.

- A loopback device serves a physical volume in the volume group 'stack-volumes'
used by Cinder services. This device does not persist after reboot. So
recreate it.

$ sudo losetup -f /opt/stack/data/stack-volumes-backing-file

- Restart the services

$ cd devstack $ ./rejoin-stack.sh

- Browse through the various screens to check whether the OpenStack services
are running. Manila's generic driver uses Nova, Neutron, and Cinder services
to facilitate network based multi-tenancy. So make sure that compute,
networking, and block storage services are running before expecting Manila
services to work. After debugging issues (if any) in the services Manila
depends on, restart the Manila services.

- As mentioned before, bringing back the shares (created by the generic driver)
that were in active state prior reboot is not straight forward. Manila tries
to bring back up these shares listed as active in its database. But
service instances that serve the shares might be down after the reboot. These
instances need to be active again before restarting the Manila services. This
can be done by logging into the tenant 'service' to which the instances
belong and start them up. The service instances can be identified by the
network in which they reside, 'manila_service_network'.

$ source openrc nova service $ nova list $ nova start <service_instance_vm>

Now restart the Manila services. The shares with access rules (specified in
the database) should be created.


For a more detailed write up on setting up and trouble shooting the generic
driver setup please refer the following link,
https://wiki.openstack.org/wiki/Manila/docs/Setting_up_DevStack_with_Manila_on_Fedora_20#Enable_DevStack_to_use_Manila