Create a Dataset - resource-watch/ocean-watch-data GitHub Wiki

Uploading datasets for Ocean Watch

For the data work flow to upload a published Resource Watch dataset, refer to the Resource Watch Data Workflow Manual. Published datasets must meet all of Resource Watch's data, content, and styling standards. The Resource Watch Data Team should provide any Ocean Watch Data Team members with extensive training.

It is important to note that unpublished datasets do not need to meet all of Resource Watch's data, content, and styling standards. Read more about the exceptions below. Each section below corresponds to a step in the Resource Watch Data Workflow Manual.

Document Data

As stated above, key metadata information for unpublished datasets is documented in this repository, not on Resource Watch. This is due to the lack of an "explore" page for the dataset. To document a new dataset, create a new folder named unique id_title within the datasets folder. Copy the template, change the name to README.MD and fill in the key information. This repository is configured to run scripts on a server. If a dataset needs to update on the server, include the same components inside the folder as those inside the Resource Watch near-real-time dataset folders (see resource-watch/nrt-scripts).

Prepare Data

All data workflows should be well documented and reproducible to enable regular updates. Notebooks containing workflows and python scripts are stored in the the dataset folder and described in the folder's README.md.

Visualize Data

After the dataset has been connected to Resource Watch, the dataset will be visualized as a layer, a widget, or both. This depends on how the dataset fits into the Ocean Watch data narrative. If the dataset will be visualized in a map, create layers according to the Resource Watch Data Workflow Manual. Since many of the unpublished datasets are tabular, they will not have layers. The visualizing step of the RW data workflow can be skipped.

Analyze Data

If the dataset will be analyzed in a widget, follow the information in the widgets wiki