bash jq - ghdrako/doc_snipets GitHub Wiki

To test queries live use https://jqplay.org/

Output Formatting - Prettify JSON

If you do only care about output formatting (pretty print) run

$ echo '{"fruit":{"name":"apple","color":"green","price":1.20}}' | jq '.'
{
  "fruit": {
    "name": "apple",
    "color": "green",
    "price": 1.2
  }
}

jq . my.json

Note: for redirection you need to pass a filter too to avoid a syntax error:

jq . my.json > output.json

Being able to prettify JSON is particularly useful when we want to retrieve data from an API and see the response in a clear, readable format.

curl http://api.open-notify.org/iss-now.json | jq '.'

Select Elements - Accessing Properties

Getting the first element

output=$(curl ... 2>/dev/null)
output=$(echo $output | jq .[0]

Getting a value only for specific key

question=$(echo $output | jq .question)

$ curl https://api.github.com/repos/stedolan/jq

{
  "id": 5101141,
  "node_id": "MDEwOlJlcG9zaXRvcnk1MTAxMTQx",
  "name": "jq",
  "full_name": "stedolan/jq",
  "private": false,
  "owner": {
    "login": "stedolan",
    "id": 79765
  },
  "html_url": "https://github.com/stedolan/jq",
  "description": "Command-line JSON processor",
  "stargazers_count": 19967,
  "watchers_count": 19967,
  "language": "C",
  "license": {
    "key": "other",
    "name": "Other",
    "spdx_id": "NOASSERTION",
    "url": null,
    "node_id": "MDc6TGljZW5zZTA="
  }
}

$ curl https://api.github.com/repos/stedolan/jq | jq ' .name' 
"jq"

$ curl https://api.github.com/repos/stedolan/jq | jq ' .owner' 
{
    "login": "stedolan",
    "id": 79765
}

$ https://api.github.com/repos/stedolan/jq | jq ' .owner.login' 
"stedolan"

Select FROM JSON Arrays

curl https://api.github.com/repos/stedolan/jq/issues?per_page=5 | jq '.[4]'

$ echo "[1,2,3,4,5]" | jq '.[2:4]'
[3,4]

$ echo "[1,2,3,4,5]" | jq '.[2:]'
[3,4,5]

$ echo "[1,2,3,4,5]" | jq '.[-2:]'
[4,5]

$ curl https://api.github.com/repos/stedolan/jq/issues?per_page=5 | jq '.[4].title'

All elements

curl https://api.github.com/repos/stedolan/jq/issues?per_page=5 | jq '.[].title'

Array-Index - Accessing by Index

jq lets you select the whole array [], a specific element [3], or ranges [2:5] and combine these with the object index if needed.

It ends up looking something like this:

jq '.key[].subkey[2]

Slicing

$ echo '[1,2,3,4,5,6,7,8,9,10]' | jq '.[6:9]'
[
  7,
  8,
  9
]

$ echo '[1,2,3,4,5,6,7,8,9,10]' | jq '.[:6]' | jq '.[-2:]'
[
  5,
  6
]

Getting Keys

$ jq '.fruit | length' fruit.json

Returning the Length

$ jq '.fruit.name | length' fruit.json
5  #   fruit name property has five characters: “apple”.

Mapping Values

applying the has function to each item in the array and looking to see if there is a name property

jq 'map(has("name"))' fruits.json

use the map function to apply operations to the elements in an array

jq 'map(.price+2)' fruits.json

Min and Max

minimum or maximum element of an input array, we can utilize the min and max functions

jq '[.[].price] | min' fruits.json

Selecting Values

Selects all the fruit with a price greater than 0.5.

$ jq '.[] | select(.price>0.5)' fruits.json

$ jq '.[] | select(.color=="yellow")' fruits.json
$ jq '.[] | select(.color=="yellow" and .price>=0.5)' fruits.json
{
  "name": "banana",
  "color": "yellow",
  "price": 0.5
}

 jq --arg branch $branch '.[] | select(.name|match($branch)) | .name '

Support for Regular Expressions

price of all the fruit whose name starts with the letter “a”.

$ jq '.[] | select(.name|test("^a.")) | .price' fruits.json

Finding Unique Values

Use the map function to create a new array containing only colors. Then we pass each color in the new array to the unique function using a pipe |.

$ jq 'map(.color) | unique' fruits.json
[
  "green",
  "yellow"
]

Deleting Keys From JSON

to remove a key and corresponding value from JSON objects

$ jq 'del(.fruit.name)' fruit.json
# outputs the fruit object without the deleted key:

{
  "fruit": {
    "color": "green",
    "price": 1.2
  }
}

Simple Extraction

Object Identifier-Index jq '.key.subkey.subsubkey'

Consider this example document

{
    "timestamp": 1234567890,
    "report": "Age Report",
    "results": [
        { "name": "John", "age": 43, "city": "TownA" },
        { "name": "Joe",  "age": 10, "city": "TownB" }
    ]
}

jq '.[] | {timestamp,report}' # To extract top level attributes “timestamp” and “report”
jq '.results[] | {name, age}'  # To extract name and age of each “results” item

# Filter this by attribute
jq '.results[] | select(.name == "John") | {age}'      # Get age for 'John'
jq '.results[] | select(.name | contains("Jo"))'       # Get complete records for all names with 'Jo'
jq '.results[] | select(.name | test("Joe\s+Smith"))'  # Get complete records for all names matching PCRE regex 'Joe\+Smith'

Changing values

Merging/overwriting keys

echo '{ "a": 1, "b": 2 }' |\
jq '. |= . + {
  "c": 3
}'

Adding elements to lists

echo '{ "names": ["Marie", "Sophie"] }' |\
jq '.names |= .+ [
   "Natalie"
]'

Handle Empty Arrays

When you want to iterate and an array you access is empty you get something like

jq: error (at :3): Cannot iterate over null (null)

To workaround the optional array protect the access with

select(.my_array | length > 0)

Using jq in Shell Scripts

From https://www.terraform.io/docs/providers/external/data_source.html

Parsing JSON into env vars

To fill environment variables from JSON object keys (e.g. $FOO from jq query “.foo”)

export $(jq -r '@sh "FOO=\(.foo) BAZ=\(.baz)"')

To make a bash array

read -a bash_array < <(jq -r .|arrays|select(.!=null)|@tsv)

JSON template using env vars

To create proper JSON from a shell script and properly escape variables:

jq -n --arg foobaz "$FOOBAZ" '{"foobaz":$foobaz}'
  • --arg name value:

This option passes a value to the jq program as a predefined variable. If you run jq with --arg foo bar, then $foo is available in the program and has the value "bar". Note that value will be treated as a string, so --arg foo 123 will bind $foo to "123".

URL Encode

Quick easy way to url encode something

date | jq -sRr @uri
⚠️ **GitHub.com Fallback** ⚠️