bash, curl, and jq - AquaticInformatics/examples GitHub Wiki
The JSON-based REST APIs of AQUARIUS Time-Series can also be easily exercised using standard *NIX command line tools.
This section shows some examples of simple bash command lines, using curl to issue REST API requests to your AQTS app server and jq to manipulate the JSON responses.
All three tools are available on Linux, mac OS, and Windows systems.
Using curl to make API requests
Here's a minimum request to fetch all the parameters in a system. This sends a large-ish 30KB JSON response to the console.
$ curl -f -u admin:admin "http://myappserver/AQUARIUS/Publish/v2/GetParameterList"
{"Parameters":[{"Identifier":"Stage","UnitGroupIdentifier":"Length","UnitIdentifier":"m","DisplayName":"Height of Gauge (River Stage)","InterpolationType":"1","RoundingSpec":"DEC(3)"},{"Identifier":"Ice Thickness","UnitGroupIdentifier":"Length","UnitIdentifier":"cm","DisplayName":"Ice thickness","InterpolationType":"7","RoundingSpec":"SIG(3)"},{"Identifier":"Snow Depth","UnitGroupIdentifier":"Length","UnitIdentifier"
Commonly used curl options for REST API requests
You'll need to set these options for every curl request.
| Option | Description |
|---|---|
-f |
Fail if the response is not an HTTP 2xx status code. |
-u username:password |
Set your AQUARIUS credentials using HTTP Basic Authentication. |
See curl --help for the (huge!) list of supported options.
Making POST, PUT, or DELETE requests
curl makes GET requests by default. To use a different HTTP verb, you'll need to set a few more command line options.
| Option | Description |
|---|---|
-X POST |
Set the HTTP request verb to something other than GET. Useful for POST, PUT, and DELETE requests. |
-d '{json data}' |
Sets the request payload with a JSON document. |
-H "Content-Type: application/json" |
Sets the content-type header. Used in conjuction with the -d option to tell the server that the request payload is JSON data. |
Here's a PUT request which changes the name of a location tag:
$ curl -f -s -u admin:admin -H 'Content-Type: application/json' -X PUT -d '{"Name":"WebPortal.Engineering"}' "http://doug-vm2012r2/AQUARIUS/Provisioning/v1/tags/location/811181fe3a93418c983e2a3215ff747d"
Useful curl options for debugging your requests
| Option | Description |
|---|---|
-v |
Enables verbose output from curl, showing headers and cookie information. |
-x127.0.0.1:8888 |
Routes the request through the given HTTP proxy, like the awesome Fiddler web proxy. Use this when -v is not enough. |
-s |
Silent mode, which disables the textual response progress bar on the terminal. |
Using jq to process JSON responses
The jq command is a powerful tool for manipulating and transforming JSON response documents.
You typically pipe the output of your curl request into jq for processing.
jq with no other arguments just pretty-prints the received JSON
Here's the first request example, piped into jq. That single-line 30 KB JSON document is now spread out over about 1100 lines.
$ curl -f -u admin:admin "http://myappserver/AQUARIUS/Publish/v2/GetParameterList" | jq
{
"Parameters": [
{
"Identifier": "Stage",
"UnitGroupIdentifier": "Length",
"UnitIdentifier": "m",
"DisplayName": "Height of Gauge (River Stage)",
"InterpolationType": "1",
"RoundingSpec": "DEC(3)"
},
...
Use jq to pull out one item from an array
jq command lines are a set of filters in a single-quoted string, with pipes separating each filter.
We'll find the Discharge parameter by using two filters:
.Parameters[]to extract the Parameters array from the responseselect(.Identifier = "Discharge")to return the one parameter named "Discharge"
$ curl -f -u admin:admin -s "http://doug-vm2012r2/AQUARIUS/Publish/v2/GetParameterList" | jq '.Parameters[] | select(.Identifier == "Discharge")'
{
"Identifier": "Discharge",
"UnitGroupIdentifier": "Volumetric Flow",
"UnitIdentifier": "m^3/s",
"DisplayName": "Discharge",
"InterpolationType": "1",
"RoundingSpec": "SIG(3)"
}
Extract all the locations in a system
This command line fetches all the locations in a system, pulls out their identifiers, one per line.
$ curl -f -u admin:admin -s "http://doug-vm2012r2/AQUARIUS/Publish/v2/GetLocationDescriptionList" | jq '.LocationDescriptions[] | .Identifier'
"01FB002"
"01AH005"
"03OA003"
...
Each location identifier is still surrounded by quotes, so use the -r option to get raw output instead.
$ curl -f -u admin:admin -s "http://doug-vm2012r2/AQUARIUS/Publish/v2/GetLocationDescriptionList" | jq -r '.LocationDescriptions[] | .Identifier'
01FB002
01AH005
03OA003
...
Transform a JSON array into a CSV stream
jq supports a @csv filter to transforms an array into a CSV line.
jq -r '.someArray[] | [.field1, .field2, .field3] | @csv'
That cryptic filter sequence will:
- select the someArray JSON array from the response
- compose a new array using the 3 fields
- convert it to a CSV
- And spit it out in raw mode
Let's apply that to the list of locations to get a CSV of location names and identifiers.
$ curl -f -u admin:admin -s "http://doug-vm2012r2/AQUARIUS/Publish/v2/GetLocationDescriptionList" | jq -r '.LocationDescriptions[] | [.Identifier, .Name] | @csv'
"01FB002","SOUTHWEST MARGAREE RIVER AT MARGAREE FORKS"
"01AH005","MAMOZEKEL RIVER NEAR CAMPBELL RIVER"
"03OA003","MCPHADYEN RIVER NEAR THE MOUTH"
"01CB004","WILMOT RIVER NEAR WILMOT VALLEY"
"01AO006","PORTOBELLO STREAM DOWNSTREAM OF BURNS BRIDGE"
"01EE007","WHITEBURN BROOK BELOW POLLOCK LAKE"
...