Rosetta Batch Processing API - Unidata/rosetta GitHub Wiki

Currently, batch processing data conversion is done via web API. The entry point to the batch processing service is

https://<server>/rosetta/batchProcess

To perform batch processing, one must do an HTTP Post of a zip to the batchProcess endpoint. The response of the server will be a zip file containing the converted netCDF files along with a log file of what took place during the conversion for each data file in the original zip file.

The zip file used in the HTTP POST must contain the following:

  • One or more data files to convert
  • A Rosetta Template file

The Rosetta template file is a JSON file and, at a minimum, must include the following:

{
    "format": <FORMAT>
}

<FORMAT> must be one of the following:

  • "tuff"
  • "esc"
  • "custom"

The Rosetta template may contain other properties, but format is absolutely required. It is highly recommended to use the template produced by the Rosetta wizard when dealing with a "custom" conversion.

Optional

For each data file contained within the zip file, an optional file with the same name (but with extension metadata) may be included. The .metadata file serves two purposes:

  1. add global metadata specific to an individual file
  2. overwrite global metadata defined in the template, but with values appropriate for the individual file

The .metadata file is a simple text file with key:value pairs on each line. The key is the global attribute name, and the value is the value of that attribute name.

For example, let's say we have a datafile called mydata.txt, and the contents of mydata.metadata looks like:

title: My Dataset
author: John Doe

In this scenario, the netCDF produced by the conversion will have both title and author added as global attributes. If there is a Rosetta template included in the zip file, and both title and author are defined within the template, the values for title and author from the .metadata file will be used instead of the values contained within the template.

Note: Currently, the Spring controller for doing batch processing isn't included in the re-write. The most recent version can be found here

⚠️ **GitHub.com Fallback** ⚠️