How to parse a json file from Linux command line using jq - JohnHau/mis GitHub Wiki

The JSON (JavaScript Object Notation) format is widely used to represent data structures, and is frequently used to exchange data between different layers of an application, or by the use of API calls. We probably know how to interact with json-formatted data with the most used programming languages such as parsing JSON with python, but what if we need to interact with it from the command line, or in a bash script? In this article we will see how we can accomplish such a task by using the jq utility and we will learn its basic usage.

In this tutorial you will learn:

How to install jq in the most used Linux distributions or compile it from source How to use jq to parse json-formatted data How to combine filters using “,” and “|” How to use the length, keys, has and map functions

If we have a preference for the Red Hat family of distributions, such as Fedora, CentOS or RHEL, we can install jq via the dnf package manager (in recent versions of those distributions it superseded yum). To install the package we would run:

$ sudo dnf install jq Installing jq on Archlinux is just as easy. The distribution package manager is pacman, and the package is available in the community repository. We can perform the installation with the following command:

$ sudo pacman -S install jq If we can’t, or for some reason we don’t want to use a pre-built binary package, we can compile jq from source. In the following lines we describe the needed steps.

Building and installing from source To build and install jq from source, the first thing we must do is to download a release tarball. At the moment of writing, the latest available release is 1.6. To download the tarball without leaving the terminal, we can use wget:

$ wget https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz Once the download is complete, we must decompress and extract the tarball:

$ tar -xzf jq-1.6.tar.gz The next step is to enter the jq-1.6 directory, created as a result of the last command:

$ cd jq-1.6

Usage Once we have jq installed, we can use it to parse json files from the command line. For the sake of this tutorial we will work with a simple data structure which contains some details about three characters from Lord Of The Rings book. The data is saved in to the characters.json file.

The jq utility works by applying filters on a stream of json data. As a first thing, we will use the most simple filter, ., which returns the input data unchanged but pretty printed. For this characteristic, it can be used to format data in a more readable way:

$ jq . characters.json The command above produces the following output:

We obtain a slice (the first two letters) of the “Aragorn” string: "Ar".

Access array elements separately In the examples above we printed the content of the “characters” array, which consist of three objects that describe fantasy characters. What if we want to iterate over said array? We must make so that elements contained in it are returned separately, so we must use [] without providing any index:

$ jq .characters[] characters.json The output of the command is:

In this example we have two filters. At the left of the operator we have the .characters[] filter, which as we previously saw, let us obtain the elements of the “characters” array as separate results. In our case, each result is an object with the "name" and "race" properties. The .name filter at the right of the | operator is applied to each one of the objects, so we obtain the following result:

"Aragorn" "Gimli" "Legolas" Functions The jq utility includes some very useful functions we can apply to the json – formatted data. We will now see some of them: length, keys, has and map.

The length function The first one we will talk about is length, which, as the name suggests, let us retrieve the length of objects, arrays and strings. The length of objects is the number of their key-value pairs; the length of arrays is represented by the number of elements they contain; the length of a string is the number of characters it is composed of. Let’s see how to use the function. Suppose we want to know the length of the “characters” array, we run:

$ jq '.characters | length' characters.json As expected, we obtain 3 as result, since it is the number of elements in the array. In the same way, to obtain the length of the first object in the array we could run:

$ jq '.characters[0] | length' characters.json This time we obtain 2 as result, since it is the number of value pairs contained in the object. As we already said, the same function applied to a string, returns the number of characters contained in it, so, for example, running:

$ jq '.characters[0].name | length' characters.json We receive 7 as result, which is the length of the “Aragorn” string.