Indian Eco-Acoustics Network (IEN) working group 2025, sent out a bio acoustics data inventory survey to rough size the amount of data present with individuals and institutions (data owners) and gauge their interest in contributing to crowdsourced data for India's first bio acoustics database. 50+ data owners are interested in sharing bio-acoustics data for Indian taxa for the centralized database.

The working group proposes an IEN annotation standard described in sections below: a combination of folder/file structure and labeling schema that the contributors can follow.This wiki details out the workflow for data upload of media with varying degrees of labels. In following sections annotation templates that make it easy for data owners to both upload and annotate their data at scale are shared. Multiple options are provided to data owners to help them balance the trade-off between upload effort and compliance to annotation best practices. As a result the onus of adherence to a strict data-model of labels is offloaded to data engineering exercise.

The workflow will be first tested with a smaller sample of data owners to collect feedback and iterate. We will use Git wiki questions and zoom sessions to take questions and feedback.

Data upload workflows

Data owners are allowed to choose from the following options while uploading their data:

• Type A [Preferred] : Uploaded data will adhere to IEN annotation standard. This may require them to re-structure data labels to some extent
• Type B: Uploaded data will be annotated but NOT adhere to the IEN annotation standard. Missing columns and custom file formats are accepted.
• Type C: Uploaded data lacks meta-data or acoustic annotations or both. While not immediately useful, such data is a prime candidate for crowd-sourcing annotations and can be used for unsupervised learning and creating augmented training data-sets.

An owner can choose any combination of the above options to categorize their media during the upload process. Data engineering pipeline integrating LLM apis will be used to restructure Type B and Type C annotations to IEN's data model. For the uploaded data to be useful Type A uploads are preferred, so the group will create training docs and videos to educate the contributors.

IEN Annotation Standard

Folder Structure

Primary and metadata information of the media is suggested to be split into two different kind of files, i.e. metadata and acoustic labels. Audio, metadata and annotation files should follow the following file structure. An example of the project structure is example_project[https://github.com/chimarkhi/bioac_annotations/tree/main/Examples/projects/example_project]

root_folder
├── project_1
│   ├── media
|   │   ├── audio_file_1.wav
|   │   ├── audio_file_2.wav
|   │   ├── audio_file_3.flac
|   │   └── audio_file_4.mp3
│   ├── meta
|   │   ├── meta_0012.txt
|   │   └── meta_0014.csv
│   ├── annotation
|   │   ├── annotation_1.csv
|   │   └── annotation_0235.csv
├── project_2
│   ├── media
|   │   ├── audio_file_1.wav
|   │   ├── audio_file_2.wav
|   │   ├── audio_file_15.wav
|   │   └── audio_file_4.mp3
│   ├── meta
|   │   ├── meta_1242.txt
|   │   └── meta_0254.csv
│   ├── annotation
|   │   ├── annotation_4.csv
|   │   └── annotation_as13.csv

While sharing the data, it is recommended that each cloud upload be limited to less than 10Mb size and should contain all meta/annotation/media files being referenced within. This can be done by structuring data into separate projects with all three type of file self contained within it and making sure that the zipped size of a project is not greater that 10Mb.

Media Metadata:

A metadata file name must start with meta (eg. meta_20jan25.csv) and contain information about the recording event(s). csv, tsv, xls, xlsx formats are accepted. There can be one or more metadata files describing any number of media files.

A meta data file should have the following keys/columns:

Column Name	Internal Name	Mandatory	Description
Media File Name/Media URL	media_file_url	Yes	unique name of the media file or downloadable url, if the media is uploaded on an open access platform
Location	location	Yes	recording location geo-coords (latitude, longitude) eg. (40.7812, 71.2552)
Recording Datetime	recording_datetime	Yes	datetime of recording in ISO_8601 format, eg. '2024-11-20T10:23:00+5:30`
Authors	authors	Yes	name of the contributor/data-owner
Data License	license	Yes	One of the data licensing options described here
Location Name	location_name	No	free text location name
Recording Type	recording_type	No	PAM/focal recording
Channels	channels	No	# Channels in acoustic file
Is Annotated?	is_annotated	Yes	Yes/No based on whether the file has accompanying acoustic annotations
Sampling Interval	sampling_interval	No	free text description of sampling rate (eg. 10mins every hour)
Recording Event Description	event_description	No	description of the recording event
Notes	notes	No	location/event/climate information in free text or json

Acoustic Annotations:

An annotation file name must start with annotation (eg. annotation_rec123_20jan25_morning.csv) and contains acoustic labels identifying specific calls. csv, tsv, xls, xlsx formats are accepted.

A single file can contain information about multiple media files and should have the following keys/columns:

Column Name	Mandatory	Description
media_file	Yes	unique name of the media file
Begin Time (s)	Yes	Start time of selection in seconds
End Time (s)	Yes	End time of selection in seconds
LowFreq (kHz)	No	Lower frequency bound of selection
HighFreq (kHz)	No	Upper frequency bound
Common Name	Yes	List of identified species common names separated with |
Species ID	No	List of identified species scientific names separated with |
Notes	Yes	Elaborate free text notes on species, behavior, call type, etc.

Data upload web-page (TBU)

Below is a tentative wireframe of the data-upload page, containing the above described 3 upload options. On hitting the submit button, the page will analyse the attached annotation and meta files to flag any inconsistencies with the prescribed data model if Type A upload option is chosen by the contributor.