FAQ S - ibmcb/cbtool GitHub Wiki
SQ1: What do I need in order to start using CBTOOL?
SA1: Three things.
a) Access to an IaaS (public or private) cloud (see the list of clouds supported here.
b) One or more images that are prepared to be used with CBTOOL (see how to do it here).
c) A VM or HOST (with network access to the cloud), were CBTOOL can be installed (see how to do it here).
SQ2: What is my "private" cloud configuration file? Where is it located?
SA2: Your "private" configuration file contains configuration changes made on CBTOOL when running on your environment, as explained here. By the default, CBTOOL looks for a file called (your username)_cloud_definitions.txt in ~/cbtool/configs, but you can indicate any other file by just running the tool's CLI with the --config option (e.g. ~/cbtool/cb --config <another cloud configuration file>.txt).
SQ3: I have an external Redis/MongoDB node (or set of nodes) that I would like to use as Object Store/Metric Store for CBTOOL. Which configurations do I need to change to "point" CBTOOL to these?
SA3: UPDATE: The "shared" attribute is now the default for Object and Metric Stores.
Assuming that the Object and Metric Stores are reachable through IP address "ABC", add the following line to your private configuration file:
...
[OBJECTSTORE]
HOST = ABC
USAGE = shared
...
[METRICSTORE]
HOST = ABC
USAGE = shared
...
[LOGSTORE]
HOST = ABC
USAGE = shared
....
Please note the "shared" attribute means that CBTOOL will stop controlling the Stores' state (i.e., it will stop trying to automatically start it every time the CLI is run). Also keep in mind that it is important for the network latency between the CBTOOL orchestrator node, this external node that houses the Object Store/Metric Store and the instantiated VMs might play an important role, specially when performance data is being collected at a high frequency.
SQ4: Do you have any sizing guidelines for the CBTOOL orchestrator node?
SA4: The CBTOOL code is being constantly optimized, so expect these guidelines to change frequently. The sizing of the orchestrator node is closely related to the size of the experiment (in numbers of simultaneously active VMs) to be executed. In CBTOOL the primary source of resources consumption is the Object Store and Metric Store. The resource amount consumed by this source is closely related to the number of running VMs, since each VM keeps updating/accessing both stores continuously. A second source of resource consumption are the Virtual Application Submitters. Since each VAppS is managed by a pair of python daemons running on the orchestrator node, the resource amount is directly related to the number of simultaneously running VAppS on a given experiment.
A good rule of thumb for processing power sizing is two cores for all CBTOOL daemons (list of daemons here), plus 1 core for every 2000 VMs (again, simultaneously active VMs), plus 1 core for every 200 VAppS. For memory, the rule of thumb is 2 GB of RAM for all daemons, plus 5 MB for every VM, plus 100 MB for every VAppS.
An alternative deployment architecture, where an external Redis server is used as the Object Store and an external MongoDB is used as the Metric Store, is recommended in order to make CBTOOL much more scalable (both Redis and MongoDB can be scaled-out). Please check the details on how to configure CBTOOL to use external servers. Evidently, in this particular case, the CBTOOL orchestration node sizing requirements can be greatly reduced.
SQ5: Can I run multiple instances of CBTOOL on the same orchestrator node?
SA5: Yes, as long as each instance is ran under a different user (this is extremely important). For each user, change the following parameters on your private configuration file, picking different values for each user:
[OBJECTSTORE]
DBID = 10
[LOGSTORE]
PORT = 5137
[FILESTORE]
PORT = 10037
[VPN]
SERVER_PORT = 11037
[API_DEFAULTS]
PORT = 7037
[GUI_DEFAULTS]
PORT = 8037
SQ6: I want to run the CBTOOL Orchestrator Node (ON) outside the cloud/I want to run a single large CBTOOL ON that benchmarks multiple clouds. How should I configure CBTOOL?
SA6: First of all, please keep in mind that for Public clouds (e.g., Amazon EC2, Digital Ocean), it is possible to instruct CBTOOL to configure the instances through the public network, while letting the actual load generating processes on the Virtual Application (VApp) to be executed over the private network. For instance, adding the following parameters to your private configuration file will allow you to run the ON locally
[VM_DEFAULTS : EC2_CLOUDCONFIG]
RUN_NETNAME = private
PROV_NETNAME = public
A more detailed explanation of these two parameters can be found here.
Please note that this configuration assumes that actual instances that form a VApp can send data back to the Orchestrator Node outside of the cloud. For private clouds, such as OpenStack and CloudStack, there is a whole gamut of options described on this how to will show you how to do this.
IMPORTANT : keep in mind that this parameter can be changed in any of the 3 ways discussed here and here (i.e., by changing the private configuration file, by issuing the CLI/API cldalter vm_defaults <parameter> <value> or by dynamically overriding the parameters during an vmattach or aiattach CLI/API call).
SQ7: Are there any tools available to help with the building of images containing all the requirements for different Virtual Application types?
SA7: Yes, CB has tooling which allows the creation of Docker and KVM/QEMU (QCOW2) images for all supported workloads plus the Orchestrator Node. Full details are discussed here
SQ8: Is it possible to programmatically/automatically create a private cloud configuration file?
SA8: Yes, it is. Basically, you can define environment variables (all starting with CB_) and then invoke the script ~/cbtool/configs/build_cloud_config_file_from_env.sh. This script will use the "vanilla" cloud configuration file and will create a new file ~/cbtool/configs/<USERNAME>_cloud_definitions.txt.
The cloud model (i.e., which Cloud Adapter Identifier is specified by the variable CB_STARTUP_CLOUD. Cloud-agnostic parameters are expressed in the form CB_<SECTION>_NAME (e.g., CB_METRICSTORE_DATABASE=metrics). Cloud-specific parameters are expressed in the form CB_<OBJECT NAME>_<CLOUD MODEL>_<ATTRIBUTE> (e.g., CB_VMC_DEFAULTS_PLM_DISCOVER_HOSTS=True)
Here follows an illustrative example: export CB_STARTUP_CLOUD=MYPDM; export CB_CLOUD_RENAME=TESTDOCKERCLOUD; export CB_PDM_ACCESS=172.17.0.1; ~/cbtool/configs/build_cloud_config_file_from_env.sh
SQ9: Can I execute the CBTOOL Orchestrator as a Docker container?
SA9: Yes, you can. First of all, the configuration of the private configuration file can be done via environment variables, exactly as explained in the previous question. Once all environment variables are properly set, you can do cd cbtool; ./cbdocker --help. Some relevant command line options:
-
cbdocker: attach a cloud and leaves the container running in background -
cbdocker -d: attach a cloud leaves it a CB's prompt (interactive) -
cbdocker -k: kill currently running docker -
cbdocker -ebash : start the container interactively and drop to a bash prompt.
IMPORTANT: This is a self-contained "fat" docker, running copies of all CBTOOL stores processes (e.g., Redis and MongoDB). While this can be used for small-scale experiments (few hundreds of VMs), larger ones (e.g., tens of thousands of VMs) might require dedicated nodes just for the stores, given the resource requirements. You might be willing to consider at least a dedicated node for all CBTOOL stores, as explained here