FAQ D - ibmcb/cbtool GitHub Wiki

Development

DQ1: I want to add a new Cloud Adapter for CBTOOL, and I verified that this new Cloud Model is fully supported by LibCloud (https://libcloud.apache.org/). How should I proceed?

DA1: CBTOOL’s layered architecture was intended to allow the framework to be incrementally expanded in a non-intrusive (i.e., minimal to no changes to the existing “core” code) and incremental manner. As of now (early 2017), it is highly recommended that any new Cloud Adapter developed for CBTOOL should follow the guidelines laid out here.

While multiple Cloud Adapters are already available, new adapters are constantly added. In order to facilitate this work, a new generic "LibCloud" meta-adapter (~/cbtool/lib/clouds/libcloud_common.py) was made available, which allows a new Cloud-specific adapter to be developed by writing as little as 50 lines of code. This new meta-adapter using the libcloud-based method includes support for the following CBTOOL operations/features:

a. create/destroy instances

b. image capture (from a running instance) and deletion.

c. multi-tenancy (using multiple accounts at the same time)

d. create (and attach)/destroy block storage volumes

e. multi-region support

f. cloud-init support for both SSH-keys and OpenVPN.

The things that our simplified libcloud-support does NOT include:

I. floating IP addresses

II. block storage snapshot management

III. other esoteric functions provided by individual libcloud-based cloud providers.

For example, if you have some “special” feature supported by your cloud which is not standard, such as, docker-specific commands, or auto-configuration of a secret feature that only your cloud knows about, then the simplified libcloud-approach is not for you. You will want to go with the native approach. Assuming that an adapter for a “New Awesome Cloud” (nac) will be written, here we have the required steps summarized:

Using ~/cbtool/configs/templates/_digitalocean.txt as an example, create ~/cbtool/configs/templates/_nac.txt: A simple cp ~/cbtool/configs/templates/_digitalocean.txt ~/cbtool/configs/templates/_nac.txt should be enough.
Using ~/cbtool/lib/clouds/do_cloud_ops.py as an example, create ~/cbtool/lib/clouds/nac_cloud_ops.py. Again, a simple cp ~/cbtool/lib/clouds/do_cloud_ops.py ~/osgcloud/cbtool/lib/clouds/nac_cloud_ops.py should be sufficient.
Read the documentation in the __init__ function at the beginning of the file ~/cbtool/lib/clouds/libcloud_common.py. Edit your own file ~/cbtool/lib/clouds/nac_cloud_ops.py and adapt the options to the __init__ function to match the features supported by your libcloud-based cloud. For example, if you don’t need SSH keys via cloud-init, or you don’t support cloud-init at all, then set those respective options to False. Most options are optional. While the DigitalOcean adapter makes more complete use of all the libcloud features, your cloud may not need them all.
In the pre_create_vm function, this is where you would add any additional overrides to the function which actually launches VMs to your cloud. Many clouds provide special python keyword arguments to the libcloud create_node() function which are special to your individual cloud. If this is true, then add the respective keyword arguments to the pre_create_vm function. See the example used in the DigitalOcean adapter for such an override: ~/cbtool/lib/clouds/do_cloud_ops.py.
Once you are done, you should have a very short cloud adapter. The recommended way to test the new adapter is to start with a simple “cldattach npc TESTNPCCLOUD”, followed by “vmcattach all”, directly on CBTOOL’s CLI.

This operation will ensure that vmccleanup and vmcregister methods are properly implemented.
At this point, select an image that contains a "vanilla" Ubuntu/Centos/RHEL/Fedora image. You can test the instance creation with vmattach check:<SELECTED IMAGEID>, and instance destruction with vmdetach youngest.

If you are running into trouble adding the new adapter, we will be more than happy to help you on the mailing lists. Join the lists and let us know how it goes.

Back

DQ2: I want to add a new Cloud Adapter for CBTOOL, and I cannot use LibCloud (https://libcloud.apache.org/). How should I proceed?]

DA2: CBTOOL’s layered architecture was intended to allow the framework to be incrementally expanded in a non-intrusive (i.e., minimal to no changes to the existing “core” code) and incremental manner. First of all, make sure that you cloud is indeed not covered on the extensive list provided by LibCloud.

While multiple Cloud Adapters are already available, new adapters are constantly added. These adapters can be divided in two broad classes, following the Cloud’s classification, white-box and black-box (i.e., public). It is recommended that, for the addition of a new cloud adapter, one uses either OpenStack (in case of white box) or EC2 (in case of black-box) as examples.

Assuming that an adapter for a “New Public Cloud” (npc) will be written, here is the summarization of the required steps.

Using ~/cbtool/configs/templates/_ec2.txt as an example, create ~/cbtool/configs/templates/_npc.txt:
A simple cp ~/cbtool/configs/templates/_ec2.txt ~/cbtool/configs/templates/_npc.txt should be enough Using ~/cbtool/lib/clouds/ec2_cloud_ops.py as an example, create ~/cbtool/lib/clouds/npc_cloud_ops.py. Again, a simple cp ~/cbtool/lib/clouds/ec2_cloud_ops.py ~/cbtool/lib/clouds/npc_cloud_ops.py should be enough. Open the file ~/cbtool/lib/clouds/npc_cloud_ops.py and start by changing lines 37-38 (import New Public Cloud’s native python client) and line 40 (rename Ec2Cmds into NpcCmds)
CBTOOL’s abstract operations are mapped to five mandatory methods in the (newly created by the implementer) class NpcCmds:
- vmccleanup(obj_attr_list):
  - Lists all instances (and if supported, volumes) named by CloudBench and removes these
  - Instances (and Volumes) created by CB are always prefixed by cb-<USERNAME>-<CLOUD NAME>-
  - This method is invoked when --hard_reset or --soft_reset is used to start the tool
- vmcregister(obj_attr_list):
  - Creates a record, inside CloudBench's ObjectStore, for each of the specified VMC (i.e., Virtual Machine Container, one of the four Concrete Objects).
  - If supported by the Cloud's API (e.g., OpenStack), individual host discovery is performed within this method.
  - May invoke the vmccleanup method if instructed to do so (i.e., CB was invoked with --hard_reset or --soft_reset
- vmcunregister(obj_attr_list):
  - Removes the record of a given VMC Concrete Object from CB's ObjectStore
  - Usually, invokes the vmccleanup method (this behavior is controlled by the cleanup_on_detach VMC attribute
- vmcreate(obj_attr_list):
  - Creates a single instance on a given Cloud Region/Zone/DataCenter (i.e., VMC)
  - Instances are always named with the aforementioned cb-<USERNAME>-<CLOUD NAME>- prefix
  - If the Cloud's API supports so (list of capabilities), one additional data volume per VM (controlled by the VM attribute cloud_vv=<SIZE IN GB> is also created.
  - During the multiple steps of Virtual Machine/Application Instance creation, the following methods will be invoked: connect, is_vm_running, is_vm_ready, get_ip_address, get_images, get_networks
- vmdestroy(obj_attr_list):
  - Deletes a single instance on a given Cloud Region/Zone/DataCenter (i.e., VMC)
  - If the instance has a data volume associated to it, it is also deleted
  - During the deletion, the following methods will be invoked: connect, is_vm_running
In addition to the mandatory mapping methods, the following methods are also part of each Cloud adapter:
- get_description()
  - Returns a string specifying the Cloud's "long name"
- connect(access, credentials, vmc_name, ...)
  - Establishes a connection to the Cloud's API Endpoint
  - Typically one connection per VMC
  - Some native python clients are not thread-safe, and might required the storing of individual, per-instance connection in a python dictionary (look for osk_cloud_ops.py for an example)
  - Method invoked by many other methods
- check_networks(vmc_name, vm_defaults)
  - During the initial Cloud attachment operation (at CB's startup), check if the networks specified in the cloud configuration file are present.
  - Used by the method test_vmc_connection
- check_images(vmc_name, vm_defaults)
  - During the initial Cloud attachment operation (at CB's startup), check how many of the images specified in the cloud-specific template (e.g., ~/cbtool/configs/templates/_npc.txt) are present on the cloud.
  - If none is found, CB will drop into WALKTHROUGH MODE
  - Used by the method test_vmc_connection
- check_ssh_key(vmc_name, vm_defaults)
  - During the initial Cloud attachment operation (at CB's startup), check if the SSH key specified in the cloud configuration file is present.
  - If the key is not present, CBTOOL will automatically create a the ssh key on the cloud (by invoking the method create_ssh_key)
  - Used by the method test_vmc_connection
- discover_hosts
  - If the Cloud API's supports it, the method vmcregister will invoke this method to perform host discovery
- test_vmc_connection(cloud_name, vmc_name, access, credentials, key_name, security_group_name, vm_templates, vm_defaults, vmc_defaults)
  - During the initial Cloud attachment operation (at CB's startup) perform all the sanity checks (networks, SSH keys, images) needed to make sure that this cloud is useable with the credentials supplied by the Experimenter.
  - During the testing, the following methods will be invoked: connect, check_networks, check_ssh_key, check_images
- is_vm_running(obj_attr_list)
  - This method, invoked in a polling manner multiple methods, ensure that a given instance exists and started its boot process (e.g., it is state, as reported from the cloud) is RUNNING or ACTIVE)
  - Used by both vmcreate and vmdestroy methods.
- is_vm_ready(obj_attr_list)
  - This method, invoked in a polling manner multiple methods, ensure that a given instance exists, is running and has an IP address (assigned by the Cloud) associated with it.
  - Used by the method vmcreate
- get_ip_address(obj_attr_list)
  - Get an instance's IP address
  - Used by the method is_vm_ready
- get_instances(obj_attr_list)
  - Returns a python object that represents a created instance.
  - Used by the method is_vm_ready
- get_images(obj_attr_list)
  - Before creating the instance, queries the cloud and obtains information about the image which is about to be used
  - In order to put more stress on the Cloud's API, this is method is invoked, during vmcreate for each individual VM
- get_networks(obj_attr_list)
  - Before creating the instance, queries the cloud and obtains information about the network which is the VM about to be attached to
  - In order to put more stress on the Cloud's API, this is method is invoked, during vmcreate for each individual VM
In addition to the “mandatory” methods one might opt for (as shown in the aforementioned table of already existing Cloud Adapters) to implement “optional” operations, such as “vmcapture” and “vmrunstate” (both additional methods in the same class).
- IMPORTANT: In case your Cloud API does not support this operation, you can simply return 0, "NOT SUPPORTED"
- vm_placement(obj_attr_list)
  - If the Cloud API supports the identification and addressing of individual hosts, this method will be invoked during vmcreate to place specific instances into specific hosts.
- vmcapture(obj_attr_list):
  - Takes a running instance, stops it, creates a new bootable image and then deletes it.
- vmrunstate(obj_attr_list)
  - Translates the CBTOOL API call/CLI command vmrunstate into start/stop/save/resume/suspend operations on the cloud.
- vmmigrate(obj_attr_list)
  - Allows instances to be removed from one host to another.
- vmresize(obj_attr_list)
  - Allows instances to have its number of virtual CPUs or Memory size dynamically altered
- imgdelete(obj_attr_list)
  - Allows images (created by a Cloud user, for instance by issuing the CBTOOL API call/CLI command vmcapture) to be deleted
It is also possible to add the ability for persistent storage attachment and detachment (i.e., “virtual volumes”) through the methods “vvcreate” and “vvdestroy”.
- vvcreate
  - Create an additional data volume to be attached to a just created instance
  - Used by the method vmcreate
- vvdestroy
  - Delete the additional data volume attached to an already created instance
  - Used by the method vmdestroy
Finally, it is important to remember that the parameters in _npc.txt will have to be changed taking into account specific features on this cloud.
The recommended way to test the new adapter is to start with a simple “cldattach npc TESTNPCCLOUD”, followed by “vmcattach all”, directly on CBTOOL’s CLI.

This operation will ensure that vmccleanup and vmcregister methods are properly implemented.

At this point, select an image that contains a "vanilla" Ubuntu/Centos/RHEL/Fedora image. You can test the instance creation with vmattach check:<SELECTED IMAGEID>, and instance destruction with vmdetach youngest.

Back

DQ3: I want to add a new Workload for CBTOOL. How should I proceed?

DA3: The inclusion of new Workloads - i.e., new Application Instance (AI) types - is a six-step process, whose complexity is entirely dependent on the complexity of the desired application. For reference, adding a new AI type that executes a "synthetic" benchmark (e.g., netperf, bonnie) can normally be done with simple configuration and execution scripts. A more complex AI type, such as hadoop of hpcc might require providing initial automation to fully form a (logical) cluster (of instances), as required by the application.

Create a new subdirectory in https://github.com/ibmcb/cbtool/blob/master/scripts/, naming it after the desired AI type (in this example "New Application Type for CBTOOL" (natc)
Create a file called dependencies.txt in cbtool/scripts/, listing all required software dependencies, its order and method of installation, which kinds of tests are required to check the installation, and which versions should be used. Examples of this file can be found in https://github.com/ibmcb/cbtool/blob/master/scripts/parboil/dependencies.txt (one of the simplest) and https://github.com/ibmcb/cbtool/blob/master/scripts/open_daytrader/dependencies.txt (one of the most complex)
Write a Dockerfile to automate the installation of all required software packages in the image. This file will be used by the CBTOOL's automated installer to automatically install all dependencies in any of the supported clouds (and not only docker). For instance, the automated installer was used to configure an Instance with the requirements for the nullworkload AI type (~/cbtool/install -r workload --wks nullworkload) while in the initial configuration step Preparing a VM to be used by with CBTOOL on a real Cloud.
Write a new "Workload Definition File"
Write configuration scripts
Write execution scripts

DQ4: I am creating a new Virtual Application (VApp) type for CBTOOL. How can I debug my application-specific "setup" configuration scripts?

DA4: Debug instructions

DQ5: I have implemented a new core feature, Cloud Adapter or Virtual Application (VApp) for CBTOOL. How can I run a regression test?

DA5: Regression test instructions

Back

FAQ D - ibmcb/cbtool GitHub Wiki

Development

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️