Using Ansible with Astra - datastaxdevs/awesome-astra GitHub Wiki

🏠 Back to home | Written by Madhavan Sridharan, Last Update 2/24/2022

📘 This content has been retrieved from Ansible and Astra DB APIs — A Match Made in Heaven DataStax Medium Blog

📋 On this page

A - Overview

📘 What is Ansible?

Ansible is a predictive, simple, and reliable tool that has an agentless architecture that delivers simple IT automation that ends (once for all!) repetitive tasks and frees up DevOps teams for more strategic work. It dramatically reduces the burden in managing the lifecycle of your infrastructure by automating the repetitive work which is the drudgery of IT admins in their day-to-day tasks and boosts productivity. As Ansible documentation tells us, “Complexity kills productivity!”.

📘 Why do we need a tool like Ansible?

Every business is a digital business today. Technology is your innovation engine, and delivering your applications faster helps you win. Historically, that required a lot of manual effort and complicated coordination. But today, there is Ansible, the simple, yet powerful IT automation engine that thousands of companies are using to drive complexity out of their environments and accelerate DevOps initiatives.

📘 What to do with DataStax AstraDB and Ansible?

DataStax Astra DB offers a rich set of APIs. The DevOps API makes it easy to script the creation and management of an Astra DB database, roles, security tokens, access lists, and private links for your database. And, with Stargate, Astra users have easy access to built-in APIs (REST, gRPC, GraphQL, and Document) to work with the data.

You may have queried APIs with a Postman, a web browser, or cURL, but one of the overlooked capabilities of Ansible is how well it can leverage APIs as part of the playbook. This is extremely useful because the number of APIs being built and offered both by Astra and across the global internet-scale companies is exponentially increasing day by day. There’s even a public-apis GitHub repo listing hundreds of free APIs over a dozen categories.

In this post, we are going to walk you through how to use Ansible to leverage Astra DB’s DevOps APIs and Stargate Data APIs. In this repo, we give you the Ansible playbook to interact with Astra APIs for getting the database up and running with the required table(s) for your application(s) to begin kicking the tires.

B - Configuration

With just one single playbook, we will be demonstrating the operations needed to begin leveraging your database. These steps will help you get your Astra DB up and running and the tables ready to go for your applications:

  1. Create a token to use the Astra DevOps API endpoints (this is only supported for classic tier Astra DBs).
  2. Return supported regions and availability for a given user and org combination.
  3. Create a new Astra database & create a keyspace on the created DB.
  4. Find the created Astra database using its ID.
  5. Obtain the secure connect bundle for the created Astra database using its ID.
  6. Create a token to use the Data API for the Astra database using its ID.
  7. Return all keyspaces in the Astra database using its ID.
  8. Add a table in the created Astra database.
  9. Return all tables in the Astra database using its ID.
  10. Resize the already created Astra database (this is only supported for non-free tier classic tier Astra DBs).
  11. Park the already created Astra database (this is only supported for classic tier Astra DBs).
  12. Unpark the already parked Astra database (this is only supported for classic tier Astra DBs).

📘 Getting set up with the necessary prerequisites

  1. Download and set Ansible up on the control machine from where you’ll be executing the Ansible playbook. In this example, we used version 2.9.13.
  2. Sign into DataStax Astra DB portal and create an Application Token. We are going to use the API Admin Svc Account role here to generate the token. Save the generated token details by downloading the CSV. Tip: This step can also be created using the DevOps API, but we wanted to show you the Astra DB portal mechanism as well here!
  3. Download the astra_api.yml file to your Ansible control machine. If you’re curious, you could dive into this playbook to understand the operations. All that’s needed is Ansible’s core uri module to achieve what we want.

C - Executing the Ansible playbook

Run the astra_api.yml Ansible playbook using the below command on the Ansible control machine. You’ll experience all the actions/tasks being executed sequentially as defined in the playbook.

ansible-playbook astra_api.yml

You’ll be asked to provide answers/values for the questions one after the other which are required to execute the playbook. Just hit enter for the optional ones. See the below table for additional details.

Alternatively, one could pass in the answers/values via command-line arguments by running the following:

ansible-playbook astra_api.yml --extra-vars
'astra_devops_api_auth_token=<your_iam_token_from_prerequisite_step2> 
astra_db_name=<name_for_your_db> astra_db_keyspace=<name_for_your_keyspace> 
astra_db_cloudProvider=<public_cloud_provider_name_AZURE_GCP_AWS> 
astra_db_tier=<astra_db_tier> astra_db_capacityUnits=<capacity_unit_in_numeric> 
astra_db_region=<your_astra_db_region> astra_db_username=<your_astra_username> 
astra_db_tableName=<name_for_the_table>'

Let’s see what’s all these Ansible playbook input variables mean to us:

Property Name Required? Notes & References
astra_devops_api_auth_token Yes An application token to connect to the DB
astra_db_name Optional Default: astra
astra_db_keyspace Optional A container to hold to your tables within the DB. Default: astra
astra_db_cloudProvider Optional Default: GCP
astra_db_tier Optional Default: serverless
astra_db_region Optional Default: us-east1 (based on GCP as the cloud)
astra_db_tableName Yes Your AstraDB’s table name
astra_db_username Optional Only applicable for clusters created prior to the introduction of IAM capabilities. Default: astra
astra_db_password Optional Only applicable for clusters created prior to the introduction of IAM capabilities. Use this along with astra_db_username property
astra_db_capacityUnits Optional A Group of replicas. N/A for Serverless tier DB. Default: 1

And, voila!! ✨ We’ve successfully performed a series of operations leveraging Astra DevOps API and the built-in Stargate APIs to spin up a free-tier database. Below, you’ll see an example of what the last line of the output will look like once your playbook has been completed successfully. There’s also a longer example of the output at the end of this post, which we’ve shortened for the sake of brevity. Note that the playbook would have skipped agenda items 1, 10, 11, and 12 here during this run, as this operation is not required with serverless tier Astra DBs, but we wanted to showcase what’s possible for classic tier Astra DBs.

This is the last line of the playbook’s run output:

PLAY RECAP   ************************************************************************************************************************
localhost                  : ok=19   changed=0    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0

Upon the run completion, you’ll see the following in the Astra DB portal:

And, when you click on the “CQL Console” menu/section to explore the created data model schema by describing it, you’ll notice output similar to below:

And, here’s a simplified example of the run output:

$ ansible-playbook astra_api.yml - extra-vars 
'astra_devops_api_auth_token=AstraCS:CHANGE:ME astra_db_name=astra 
astra_db_keyspace=astra astra_db_cloudProvider=GCP 
astra_db_tier=serverless astra_db_capacityUnits=1 astra_db_region=us-east1 
astra_db_username=astra astra_db_tableName=astra'

PLAY [localhost] 
*****************************************************************************************************************************************************

TASK [Returns supported regions and availability for a given user and org combination] 
****************************************************************************************************************************
ok: [localhost] => {…},
 "json": [{"cloudProvider": "GCP","cost": {…}, …,"description": "Free tier: Try Astra with no obligation. Get 5 GB of storage, free forever.",
 "region": "us-east1", "regionContinent": "North America","regionDisplay": "Moncks Corner, South Carolina", "tier": "serverless"},{…}],
 "msg": "OK (unknown bytes)", "status": 200,"url": "https://api.astra.datastax.com/v2/availableRegions"}

TASK [Create a new Astra database]    *****************************************************************************************************************************************************
ok: [localhost] => {…,"location": "https://api.astra.datastax.com/v2/84be8c7b-5241-4eb1-99d0-2414d7b5908f","msg": "OK (0 bytes)"}

TASK [Find the created Astra database using its ID]    ***************************************************************************************************************************************************** 
ok: [localhost] => {…},
 "json": { "availableActions": […],"cost": { …},…,
 "info": {"capacityUnits": 1,"cloudProvider": "GCP", "datacenters": [{…}], "keyspace": "astra", "keyspaces": ["astra"], "name": "astra", "region":    "us-east-1", "tier": "serverless"},"metrics": {…},"orgId": "219acc40-f1aa-4d66–81ec-1ce64e1020e1", "ownerId": "llafqviyGJuaxMHUNmCnwjtY","status": "ACTIVE",    "storage": {…},},"msg": "OK (3613 bytes)","url": "https://api.astra.datastax.com/v2/databases/84be8c7b-5241-4eb1-99d0-2414d7b5908f"}

TASK [Obtain the secure connect bundle for the created Astra database using its ID]    *******************************************************************************************************************************
ok: [localhost] => { …,"status_code": ["200"],"url": "https://api.astra.datastax.com/v2/databases/84be8c7b-5241-4eb1-99d0-2414d7b5908f/secureBundleURL",...
 }},
 "json": {
 "downloadURL": "https://datastax-cluster-config-prod.s3.us-east-2.amazonaws.com/84be8c7b-5241-4eb1-99d0-2414d7b5908f/secure-connect-astra. zip?   X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA2AIQRQ76TUCOHUQ4%2F20210316%2Fus-east-2%2Fs3%2Faws4_request& X-Amz-Date=20210316T024510Z&   X-Amz-Expires=300&X-Amz-SignedHeaders=host& X-Amz-Signature=3575a25b299c083aa29e003824f93c13f2989bbd2bcf74f0ad56266055cd65de",…
 },"msg": "OK (1811 bytes)","url": "https://api.astra.datastax.com/v2/databases/84be8c7b-5241-4eb1-99d0-2414d7b5908f/secureBundleURL"}

TASK [Return all keyspaces in the Astra database using its ID]    ****************************************************************************************************************************************************
ok: [localhost] => {…"url": "https://84be8c7b-5241-4eb1-99d0-2414d7b5908f-us-east-1.apps.astra.datastax.com/api/rest/v1/keyspaces",...}},
 "json": […,"astra",…],"msg": "OK (365 bytes)","status": 200,"url": "https://84be8c7b-5241-4eb1-99d0-2414d7b5908f-us-east-1.apps.astra.datastax.com/api/rest/   v1/keyspaces"}

TASK [Add a table in the created Astra database]    ***************************************************************************************************************************************************** 
ok: [localhost] => {
 "invocation": {"module_args": {"body": {"columnDefinitions": [
 {"name": "pk1","static": false,"typeDefinition": "int"},
 {"name": "pk2","static": false,"typeDefinition": "text"},
 {"name": "ck1","static": false,"typeDefinition": "double"},
 { "name": "ck2", "static": false, "typeDefinition": "date"},
 {"name": "c1", "static": false, "typeDefinition": "timeuuid" }],
 "ifNotExists": true,"name": "astra","primaryKey": {"clusteringKey": ["ck1","ck2"],"partitionKey": ["pk1","pk2"]},
 "tableOptions": {
 "clusteringExpression": [{"column": "ck1","order": "ASC"},{"column": "ck2","order": "DESC"}],"defaultTimeToLive": 0}},
 "body_format": "json",…,"status_code": ["201"],"url": "https://84be8c7b-5241–4eb1–99d0–2414d7b5908f-us-east-1.apps.astra.datastax.com/api/rest/v2/schemas/   keyspaces/astra/tables"}},
 "json": {"name": "astra"},"msg": "OK (17 bytes)","url": "https://84be8c7b-5241-4eb1-99d0-2414d7b5908f-us-east-1.apps.astra.datastax.com/api/rest/v2/schemas/   keyspaces/astra/tables"}

TASK [Return all tables in the Astra database using its ID]    *******************************************************************************************************************************************************
ok: [localhost] => {
 "invocation": {"module_args": {"url": "https://84be8c7b-5241-4eb1-99d0-2414d7b5908f-us-east-1.apps.astra.datastax.com/api/rest/v2/schemas/keyspaces/astra/   tables",}},
 "json": {"data": [
 {"columnDefinitions": [
 {"name": "pk1","static": false,"typeDefinition": "int"},
 {"name": "pk2","static": false,"typeDefinition": "text"},
 {"name": "ck1","static": false,"typeDefinition": "double"},
 { "name": "ck2", "static": false, "typeDefinition": "date"},
 {"name": "c1", "static": false, "typeDefinition": "timeuuid"}],
 "keyspace": "astra","name": "astra","primaryKey": {
 "clusteringKey": ["ck1","ck2"],"partitionKey": ["pk1","pk2"]},
 "tableOptions": {"clusteringExpression": [{"column": "ck1","order": "ASC"},{"column": "ck2","order": "DESC"}],"defaultTimeToLive": 0}}]},
 "msg": "OK (548 bytes)","status": 200,
 "url": "https://84be8c7b-5241–4eb1–99d0–2414d7b5908f-us-east-1.apps.astra.datastax.com/api/rest/v2/schemas/keyspaces/astra/tables"
}

TASK [Resize the already created Astra database]    ***************************************************************************************************************************************************** 
skipping: [localhost] => {"changed": false,"skip_reason": "Conditional result was False"}

TASK [Park the already created Astra database]    *****************************************************************************************************************************************************
skipping: [localhost] => {"changed": false,"skip_reason": "Conditional result was False"}

TASK [Find the parked Astra database using its ID]    ****************************************************************************************************************************************************
skipping: [localhost] => {"changed": false,"skip_reason": "Conditional result was False"}

TASK [Unpark the already created Astra database]    ***************************************************************************************************************************************************** 
skipping: [localhost] => {"changed": false,"skip_reason": "Conditional result was False"}

TASK [Print park the already created Astra database response for debugging]    ***************************************************************************************************************************************
ok: [localhost] => {"unpark_astra_db": {"changed": false,"skip_reason": "Conditional result was False","skipped": true}}

PLAY RECAP    *****************************************************************************************************************************************************
localhost : ok=19 changed=0 unreachable=0 failed=0 skipped=4 rescued=0 ignored=0 
$

We’ve only scratched the surface in this post in terms of what’s possible. Now that you’ve seen the power of leveraging Ansible with Astra APIs, if you’re a Terraform enthusiast, we’ve got you covered there, too.

With the ever-increasing capabilities of the Astra DevOps API and Stargate data APIs built into Astra, there’s a lot more that you can do with it, including creating the users, their tokens, access list, private link, and more! If you’re not already using Astra DB, maybe it’s time to take it for a spin. You can sign up for free here!

D - Other References