Transforms - full360/sneaql GitHub Wiki

Transforms and Steps

SneaQL procedures are structured in the following hierarchy:

• Transform - Collection of step files (SQL) and configuration files
  • Step - A single file containing many SneaQL commands and SQL statements
    • Command - A single SneaQL command tag, which may or may not precede a SQL statement

Transform

A transform is typically created as a directory structure containing the following:

• sneaql.json file at the root of the directory structure.
• One or more step files. These files can be nested in a directory structure.

A typical transform directory structure might look like this:

transform-dir/
  sneaql.env
  sneaql.json
  lib/
    init.sql
    validate.sql

NOTE the sneaql.env file is not mandatory, but is helpful when you are developing your transform locally. You should NEVER check the sneaql.env file into your source code repository.

The sneaql.json file tells SneaQL the step files in your transform, as well as the order in which to execute them. Below is an example that would apply to our theoretical transform:

[
  {"step_number": 1, "step_file": "init.sql"},
  {"step_number": 2, "step_file": "validate.sql"}
]

Note that the order of the steps is determined by the step_number, not the ordering of the steps in the file.