Quick Start Guide - cloud-barista/cb-spider GitHub Wiki

CB-Spider Quick Start Guide

CB-Spider β€” One unified API to control multiple clouds. This guide shows you how to start the server and control VMs on AWS and GCP with identical API calls.

Prerequisites

This guide is based on a Linux environment (Ubuntu recommended).

  • Cloud Credentials: AWS Access Key + GCP Service Account Key (see How to get CSP Credentials)
  • curl / jq: for API verification (sudo apt install -y curl jq)
  • Docker: Installed and running (curl -fsSL https://get.docker.com | sh)

Don't have both CSPs? You can follow along with just AWS or just GCP β€” skip the other provider's steps.

Part 1: Start CB-Spider Server

Step 1: Start Server

Open Terminal 1 and run:

sudo docker run --rm -p 1024:1024 \
  -v ${HOME}/cb-spider/meta_db:/root/go/src/github.com/cloud-barista/cb-spider/meta_db \
  -e SERVER_ADDRESS="your-server-ip:1024" \
  -e SPIDER_USERNAME="admin" -e SPIDER_PASSWORD="your-secure-password" \
  -e MC_INSIGHT_API_TOKEN="c72df1af9b1f0c29" \
  --name cb-spider \
  cloudbaristaorg/cb-spider

⚠️ Set your-server-ip to your host machine’s IP address(e.g., 1.2.3.4:1024).
⚠️ Change the default password before deploying to production!


Wait for output:

<CB-Spider> Multi-Cloud Unified Interface Framework >> One-Code, Multi-Cloud
   - AdminWeb: http://your-server-ip:1024/spider/adminweb
   - Swagger UI: http://your-server-ip:1024/spider/api

Step 2: Verify Server

Open Terminal 2 and verify:

curl -X GET http://localhost:1024/spider/readyz
# List all supported cloud providers
curl -sX GET -u admin:your-secure-password \
  http://localhost:1024/spider/cloudos | jq '.cloudos'

Output includes: "AWS", "GCP", "AZURE", and more.

Part 2: Multi-Cloud Control β€” AWS & GCP with One API

This section demonstrates CB-Spider's core value: the same API controls different clouds. The only thing that changes between AWS and GCP is the ConnectionName.

Step 3: Set Your Credentials

In Terminal 2, set variables for your credentials. This is the only manual input required.

# ── CB-Spider auth ────────────────────────────────────────
export SPIDER_AUTH="admin:your-secure-password"
export URL="http://localhost:1024/spider"

# ── AWS Credentials ───────────────────────────────────────
# From: AWS Console > IAM > Users > Security credentials
export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

# ── GCP Credentials ───────────────────────────────────────
# From: GCP Console > IAM > Service Accounts > Keys (JSON key file)
export GCP_PROJECT_ID="my-gcp-project-id"
export GCP_CLIENT_EMAIL="[email protected]"
# Paste private key (replace literal newlines with \n)
export GCP_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\nMIIE...\n-----END RSA PRIVATE KEY-----\n"

πŸ’‘ GCP Private Key tip: Open your JSON key file, copy the private_key value (the \n-escaped string) and paste it directly.

Step 4: Register Cloud Connections

Register drivers, credentials, regions, and connection configs for both clouds.

4-1. Register AWS Connection

# (1) Register driver
curl -sX POST -u $SPIDER_AUTH $URL/driver \
  -H 'Content-Type: application/json' \
  -d '{"DriverName":"aws-driver","ProviderName":"AWS","DriverLibFileName":"aws-driver-v1.0.so"}' | jq .DriverName

# (2) Register credential
curl -sX POST -u $SPIDER_AUTH $URL/credential \
  -H 'Content-Type: application/json' \
  -d "{
    \"CredentialName\": \"aws-cred\",
    \"ProviderName\": \"AWS\",
    \"KeyValueInfoList\": [
      {\"Key\": \"ClientId\",     \"Value\": \"$AWS_ACCESS_KEY_ID\"},
      {\"Key\": \"ClientSecret\", \"Value\": \"$AWS_SECRET_ACCESS_KEY\"}
    ]
  }" | jq .CredentialName

# (3) Register region
curl -sX POST -u $SPIDER_AUTH $URL/region \
  -H 'Content-Type: application/json' \
  -d '{"RegionName":"aws-us-east-1","ProviderName":"AWS","KeyValueInfoList":[{"Key":"Region","Value":"us-east-1"},{"Key":"Zone","Value":"us-east-1a"}]}' | jq .RegionName

# (4) Register connection config
curl -sX POST -u $SPIDER_AUTH $URL/connectionconfig \
  -H 'Content-Type: application/json' \
  -d '{"ConfigName":"aws-us-east-1","ProviderName":"AWS","DriverName":"aws-driver","CredentialName":"aws-cred","RegionName":"aws-us-east-1"}' | jq .ConfigName

4-2. Register GCP Connection

# (1) Register driver
curl -sX POST -u $SPIDER_AUTH $URL/driver \
  -H 'Content-Type: application/json' \
  -d '{"DriverName":"gcp-driver","ProviderName":"GCP","DriverLibFileName":"gcp-driver-v1.0.so"}' | jq .DriverName

# (2) Register credential
curl -sX POST -u $SPIDER_AUTH $URL/credential \
  -H 'Content-Type: application/json' \
  -d "{
    \"CredentialName\": \"gcp-cred\",
    \"ProviderName\": \"GCP\",
    \"KeyValueInfoList\": [
      {\"Key\": \"PrivateKey\",   \"Value\": \"$GCP_PRIVATE_KEY\"},
      {\"Key\": \"ClientEmail\",  \"Value\": \"$GCP_CLIENT_EMAIL\"},
      {\"Key\": \"ProjectID\",    \"Value\": \"$GCP_PROJECT_ID\"}
    ]
  }" | jq .CredentialName

# (3) Register region
curl -sX POST -u $SPIDER_AUTH $URL/region \
  -H 'Content-Type: application/json' \
  -d '{"RegionName":"gcp-us-central1","ProviderName":"GCP","KeyValueInfoList":[{"Key":"Region","Value":"us-central1"},{"Key":"Zone","Value":"us-central1-a"}]}' | jq .RegionName

# (4) Register connection config
curl -sX POST -u $SPIDER_AUTH $URL/connectionconfig \
  -H 'Content-Type: application/json' \
  -d '{"ConfigName":"gcp-us-central1","ProviderName":"GCP","DriverName":"gcp-driver","CredentialName":"gcp-cred","RegionName":"gcp-us-central1"}' | jq .ConfigName

Step 5: Create VM on AWS

Set connection and create resources in order:

export CONN_CONFIG="aws-us-east-1"

# (1) Create VPC
curl -sX POST -u $SPIDER_AUTH "$URL/vpc" \
  -H 'Content-Type: application/json' \
  -d "{\"ConnectionName\":\"$CONN_CONFIG\",\"ReqInfo\":{\"Name\":\"vpc-01\",\"IPv4_CIDR\":\"192.168.0.0/16\",\"SubnetInfoList\":[{\"Name\":\"subnet-01\",\"IPv4_CIDR\":\"192.168.1.0/24\"}]}}" \
  | jq '{name: .IId.NameId, cidr: .IPv4_CIDR}'

# (2) Create Security Group
curl -sX POST -u $SPIDER_AUTH "$URL/securitygroup" \
  -H 'Content-Type: application/json' \
  -d "{\"ConnectionName\":\"$CONN_CONFIG\",\"ReqInfo\":{\"Name\":\"sg-01\",\"VPCName\":\"vpc-01\",\"SecurityRules\":[{\"FromPort\":\"22\",\"ToPort\":\"22\",\"IPProtocol\":\"tcp\",\"Direction\":\"inbound\"}]}}" \
  | jq '{name: .IId.NameId}'

# (3) Create KeyPair
curl -sX POST -u $SPIDER_AUTH "$URL/keypair" \
  -H 'Content-Type: application/json' \
  -d "{\"ConnectionName\":\"$CONN_CONFIG\",\"ReqInfo\":{\"Name\":\"keypair-01\"}}" \
  | jq '{name: .IId.NameId}'

# (4) Create VM  (Ubuntu 24.04 LTS, t3.micro)
curl -sX POST -u $SPIDER_AUTH "$URL/vm" \
  -H 'Content-Type: application/json' \
  -d "{\"ConnectionName\":\"$CONN_CONFIG\",\"ReqInfo\":{\"Name\":\"vm-01\",\"ImageName\":\"ami-0e86e20dae9224db8\",\"VMSpecName\":\"t3.micro\",\"VPCName\":\"vpc-01\",\"SubnetName\":\"subnet-01\",\"SecurityGroupNames\":[\"sg-01\"],\"KeyPairName\":\"keypair-01\"}}" \
  | jq '{name: .IId.NameId, ip: .PublicIP}'

Step 6: Create VM on GCP β€” Identical API Calls!

Only CONN_CONFIG changes. Everything else is the same API structure.

export CONN_CONFIG="gcp-us-central1"

# (1) Create VPC  ← same command, different CONN_CONFIG
curl -sX POST -u $SPIDER_AUTH "$URL/vpc" \
  -H 'Content-Type: application/json' \
  -d "{\"ConnectionName\":\"$CONN_CONFIG\",\"ReqInfo\":{\"Name\":\"vpc-01\",\"IPv4_CIDR\":\"192.168.0.0/16\",\"SubnetInfoList\":[{\"Name\":\"subnet-01\",\"IPv4_CIDR\":\"192.168.1.0/24\"}]}}" \
  | jq '{name: .IId.NameId, cidr: .IPv4_CIDR}'

# (2) Create Security Group  ← same
curl -sX POST -u $SPIDER_AUTH "$URL/securitygroup" \
  -H 'Content-Type: application/json' \
  -d "{\"ConnectionName\":\"$CONN_CONFIG\",\"ReqInfo\":{\"Name\":\"sg-01\",\"VPCName\":\"vpc-01\",\"SecurityRules\":[{\"FromPort\":\"22\",\"ToPort\":\"22\",\"IPProtocol\":\"tcp\",\"Direction\":\"inbound\"}]}}" \
  | jq '{name: .IId.NameId}'

# (3) Create KeyPair  ← same
curl -sX POST -u $SPIDER_AUTH "$URL/keypair" \
  -H 'Content-Type: application/json' \
  -d "{\"ConnectionName\":\"$CONN_CONFIG\",\"ReqInfo\":{\"Name\":\"keypair-01\"}}" \
  | jq '{name: .IId.NameId}'

# (4) Create VM  (Ubuntu 24.04 LTS, e2-micro)
curl -sX POST -u $SPIDER_AUTH "$URL/vm" \
  -H 'Content-Type: application/json' \
  -d "{\"ConnectionName\":\"$CONN_CONFIG\",\"ReqInfo\":{\"Name\":\"vm-01\",\"ImageName\":\"https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/ubuntu-2404-noble-amd64-v20240423\",\"VMSpecName\":\"e2-micro\",\"VPCName\":\"vpc-01\",\"SubnetName\":\"subnet-01\",\"SecurityGroupNames\":[\"sg-01\"],\"KeyPairName\":\"keypair-01\"}}" \
  | jq '{name: .IId.NameId, ip: .PublicIP}'

Step 7: Unified Multi-Cloud View

Now query both clouds with the same API:

# List VMs on AWS
curl -sX GET -u $SPIDER_AUTH "$URL/vm?ConnectionName=aws-us-east-1" \
  | jq '[.vm[] | {name: .IId.NameId, ip: .PublicIP}]'

# List VMs on GCP  ← identical call, different connection
curl -sX GET -u $SPIDER_AUTH "$URL/vm?ConnectionName=gcp-us-central1" \
  | jq '[.vm[] | {name: .IId.NameId, ip: .PublicIP}]'

# Check status of all registered connections
curl -sX GET -u $SPIDER_AUTH "$URL/connectionconfig" \
  | jq '[.connectionconfig[] | {name: .ConfigName, provider: .ProviderName}]'

This is CB-Spider's power: one codebase, all clouds.

Step 8: Cleanup

# Terminate VMs
for CONN in "aws-us-east-1" "gcp-us-central1"; do
  curl -sX DELETE -u $SPIDER_AUTH "$URL/vm/vm-01" \
    -H 'Content-Type: application/json' \
    -d "{\"ConnectionName\":\"$CONN\"}" \
    | jq '{status: .Status}'
done

# Delete KeyPairs
for CONN in "aws-us-east-1" "gcp-us-central1"; do
  curl -sX DELETE -u $SPIDER_AUTH "$URL/keypair/keypair-01" \
    -H 'Content-Type: application/json' \
    -d "{\"ConnectionName\":\"$CONN\"}" | jq .
done

# Delete Security Groups
for CONN in "aws-us-east-1" "gcp-us-central1"; do
  curl -sX DELETE -u $SPIDER_AUTH "$URL/securitygroup/sg-01" \
    -H 'Content-Type: application/json' \
    -d "{\"ConnectionName\":\"$CONN\"}" | jq .
done

# Delete VPCs
for CONN in "aws-us-east-1" "gcp-us-central1"; do
  curl -sX DELETE -u $SPIDER_AUTH "$URL/vpc/vpc-01" \
    -H 'Content-Type: application/json' \
    -d "{\"ConnectionName\":\"$CONN\"}" | jq .
done

Part 3: Managing the Server

Stop Server

# Ctrl+C in Terminal 1, or:
sudo docker stop cb-spider

Run in Background

sudo docker run -d --rm -p 1024:1024 \
  -v ${HOME}/cb-spider/meta_db:/root/go/src/github.com/cloud-barista/cb-spider/meta_db \
  -e SERVER_ADDRESS="your-server-ip:1024" \
  -e SPIDER_USERNAME="admin" -e SPIDER_PASSWORD="your-secure-password" \
  -e MC_INSIGHT_API_TOKEN="c72df1af9b1f0c29" \
  --name cb-spider \
  cloudbaristaorg/cb-spider

View Logs

sudo docker logs -f cb-spider

Reset Metadata

# Reset all metadata (removes all registered clouds/resources)
sudo docker stop cb-spider && rm -rf ${HOME}/cb-spider/meta_db/*

Troubleshooting

Problem Cause Solution
Server won't start Missing credentials Set both SPIDER_USERNAME and SPIDER_PASSWORD
Port conflict Port 1024 in use sudo lsof -i :1024 then stop the conflicting process
Docker permission denied User not in docker group sudo usermod -aG docker $USER && newgrp docker
VM creation fails Wrong image/spec name Check available images: GET /spider/vmimage?ConnectionName=...
GCP credential error Malformed private key Ensure \n line endings in the key value (not literal newlines)
Add a custom footer
⚠️ **GitHub.com Fallback** ⚠️