Docs and tests - troyp/jq GitHub Wiki

In an inspired and original move, jq's docs are to be found in the docs folder.

Generating the docs requires a working Ruby setup, read on for how to get one. If you just want to document a feature that you've added and you don't want to figure out Ruby, then you can skip this section, and just edit the YAML file. If you make a pull request with something that at least smells like valid YAML, we'll figure it out (or, failing that, annoy you into installing Ruby and fixing it).

Installing Ruby and dependencies

The docs folder is a website built with Bonsai. To work on it, you'll need a working Ruby install (get one here). I'm using version 1.9.3 (rvm install 1.9.3). In theory it shouldn't matter, but Ruby is notorious for breaking backward compatibility.

To install the dependencies for the website, run bundle install from the docs folder.

make jq.1 makes the roff manual. nroff -man jq.1 | $PAGER displays it as man jq would.

rake build from the docs folder generates the website as normally seen in the (jq home)[https://stedolan.github.io/jq] in the output folder, and rake serve will start a mini webserver displaying the website.

Editing the manual

When you add a new builtin or other feature, it should get an entry in the manual. The manual is found at docs/content/3.manual/manual.yml, and it's a big YAML file. The format is fairly straightforward, just find an entry documenting something vaguely similar to your new feature and use that as a template. Note that YAML is sensitive to indentation and closing of quotes and so on -- if rake build works then you almost certainly got it right.

The manpage jq.1 is generated from the same source as the HTML manual (use make jq.1 from the project root, you'll need a working Ruby as above), so you should stick to basic Markdown formatting in the manual. In particular, don't use raw HTML tags in manual.yml; they won't come out right in the manpage.

You'll notice that almost all manual entries have an examples section which look something like this:

        examples:
          - program: '.foo, .bar'
            input: '{"foo": 42, "bar": "something else", "baz": true}'
            output: ['42', '"something else"']

These are important. As well as docs, they're tests. In particular, be aware that program is a YAML string, input is a YAML string, and output is a YAML list of YAML strings.

Tests

Examples from the manual are extracted and tested when you run make check. There are other tests as well outside the manual.

The testsuite runs four kinds of checks:

• examples taken from the manual
• tests/jq.test (jq-coded tests)
• tests/onig.test (jq-coded tests of regular expression functionality)
• tests/shtest (shell-coded tests that cannot be purely jq-coded, such as I/O, the module system, and fuzzing)

There's no need to duplicate tests between the manual and tests/jq.test. The latter is for testing tricky edge-cases which are too fiddly to be useful examples in the manual, but need to be checked for correctness.

The .test file format is particularly simple. Each testcase has the program on one line, the input on the next, and the output(s) on the following line(s). There may be multiple lines of output if the program produces multiple results. Testcases are terminated by a blank line, and lines starting with # are ignored (i.e., comments). The set of tests is futher terminated by an empty line, so the file should end in two (or more) empty lines; EOF also terminates the last test and the set of tests, but the preferred form is to end on two empty lines.

To run the tests, do make check from the project root. The tests are run under Valgrind (it's optional, but you should install it in order to develop any patches for jq), and make check will fail if any of the tests don't give the expected output, or if Valgrind reports that they leak memory or use it invalidly. The GCC and LLVM address sanitizer is also supported in master after the jq-1.5 release.

Besides the jq-coded tests in the manual, tests/jq.test, and tests/onig.test, and the shell-coded tests in tests/shtest, there are also some C-coded tests in src/jq_test.c that test the (jv API)[./C-API:-jv).