Command Line Tool Usage Guide - GateNLP/cloud-client GitHub Wiki

We provide a command-line tool for easy access to the GATE Cloud APIs for one-off uses or to use in scripts. The tool is open-source and written in Java, and is made available as a single executable JAR file:

• gate-cloud-cli-1.2.jar (3.8MB)

The tool requires a Java 8 or later JRE to run.

To run the command-line tool, simply use java -jar:

java -jar gate-cloud-cli.jar <command> <arguments>

The first argument after the JAR file name must be the command you want to run -- if run without arguments the list of available commands is displayed. The first time you use the tool you must start with the configure command in order to provide your API key, which will be stored in a configuration file.

$ java -jar gate-cloud-cli.jar configure
Configuration
-------------

This client requires an API Key to authenticate to the GATE Cloud
APIs.  You can generate one from your account settings page on
https://cloud.gate.ac.uk - ensure you enable the appropriate
permissions for the API actions you intend to call.

API key id: gcnxxxxxxxxxx
API key password: 
Writing configuration to /...../client.conf
Configuration saved successfully.

The password is not echoed to the screen.

Starting with version 1.2, the tool has the option to provide its output in JSON for easier scripting and integration with other tools. To enable this simply provide --json as the first option, after the JAR file name but before the command.

This section is a summary of the most commonly used commands available in the tool. The help command will display a general help message, and help {command} provides detailed help for each command including details of any parameters required. For the full list of all available commands, simply run gate-cloud-cli.jar with no arguments.

List all the items (i.e. pipelines) available in the shop, optionally filtered by tag

$ java -jar gate-cloud-cli.jar list-items "English"
    ID  Name                              Price
----------------------------------------------------------
     2  ANNIE Named Entity Recognizer     £0.80 per hour
...

Reserve an annotation job based on a particular pipeline. The pipeline can be specified by ID (as returned by list-jobs). A name for the new job may optionally be given as a second parameter.

$ java -jar gate-cloud-cli.jar reserve-job 2 "First ANNIE pipeline test"
Successfully reserved job.
  ID: 15
Name: First ANNIE pipeline test

The same command can reserve items that refer to cloud machines rather than pipelines (the alias reserve-machine is provided for convenience). Reserving machines is an asynchronous process, so the command cannot immediately return you the machine identifier - the new machine should appear in your list-machines output and your GATE Cloud dashboard shortly.

List all your persistent data bundles.

$ java -jar gate-cloud-cli.jar list-bundles
    ID  Name                                      Notes
----------------------------------------------------------
    27  Uploaded news articles
    35  Results of J-000012 (French web pages ...

Upload files to create a new persistent data bundle that can subsequently be processed by annotation jobs. You can upload several files to a single bundle, but they must all be of the same type (ZIP, TAR, ARC, etc.) and use the same configuration parameters (see the main API documentation).

$ java -jar gate-cloud-cli.jar upload-bundle "Uploaded news articles" \
>    -type ZIP -mimeTypeOverride text/html -encoding ISO-8859-1 \
>    /path/to/news/*.zip
Created bundle 27
Uploading /path/to/news/part01.zip
Uploading /path/to/news/part02.zip
Closing bundle...
Bundle 27 uploaded

Configure an annotation job to take input from a data bundle (either one you uploaded yourself or one you created from one of the available data sources on https://cloud.gate.ac.uk).

$ java -jar gate-cloud-cli.jar input-from-bundle 15 27
Configuring job 15 to take input from bundle 27
Created https://cloud.gate.ac.uk/api/job/15/input/206

Define an output specification for a job to write its output to files in a particular format.

$ java -jar gate-cloud-cli.jar add-file-output 15 -type GATE_XML \
>    -extension .gate.xml
Created output https://cloud.gate.ac.uk/api/job/15/output/12

Start a fully-configured job running.

$ java -jar gate-cloud-cli.jar start-job 15

Read the log messages generated by a job as it runs. The -watch option polls the server continuously to display new messages as they appear.

$ java -jar gate-cloud-cli.jar execution-log 15 -watch

Note: in --json mode, the -watch option causes the tool to print a stream of JSON log messages, one per line, rather than a single well-formed JSON array.

Information about a specific job. If the job has finished running this will include the URL to the job's output data bundle.

$ java -jar gate-cloud-cli.jar job-details 15
             ID: 15
           Name: First ANNIE pipeline test
          State: COMPLETED
          Price: £0.80 per hour
   Date created: 03 July 2014 12:37:04 BST
 Date completed: 06 July 2014 13:20:26 BST
    Expiry date: 13 July 2014 13:20:26 BST
  Result bundle: https://cloud.gate.ac.uk/api/data/bundle/35
Processing time: 01:53:10.482 (charged 02:00:00.000 so far) 
 Data processed: 5MiB (charged 0B so far)
   Job progress: 100%

Download all the files in a data bundle to your local machine.

$ java -jar gate-cloud-cli.jar download-bundle \
>    https://cloud.gate.ac.uk/api/data/bundle/35
part01.zip-out.zip (file 1 of 2)
part02.zip-out.zip (file 2 of 2)

Use -d to specify an output location other than the current directory - see help download-bundle for full details.

Download supplementary files from a completed job into local files.

$ java -jar gate-cloud-cli.jar download-job-files 15

List all your cloud machines, optionally filtered by "state".

$ java -jar gate-cloud-cli.jar list-machines

Start a stopped machine or stop a running one.

$ java -jar gate-cloud-cli.jar start-machine 282