Tracking Data Preparation - npolar/api.npolar.no GitHub Wiki
== JSON === Argos ARGOS data has been parsed and converted to JSON on linux using the Ruby programming language (version 1.9.3) and the argos-ruby command line tool
== Publishing Tracking data has been published to https://api.npolar.no/tracking using the npolar-api command line tool, also written in Ruby.
Source metadata of each individual Argos ARGOS publishing example $ argos-ruby --action=source seed/tracking/ivory_gull.ds | npolar-api --level=info http://api:9393/source -d@- $ argos-ruby seed/tracking/ivory_gull.ds | npolar-api --level=info http://api:9393/tracking -d@- curl http://localhost:9393/tracking/_bundle | npolar-api --level=debug http://localhost:8983/solr/tracking/update -d@-
== Storage The data is stored in a CouchDB database, mirrored in Solr search engine. There might be slight differences.
Traceability
The tracking dataset consists of documents extracted from files archived over a considerable time period, back to where floppy disks where the most used data carrier. The ARGOS portion of the data is created from over 700 files, sometimes with different variants.
Source-tracing Every tracking document can be traced to its source.
Each document contains a source identifier, the SHA-1 hash of the source's content, irrespective of filename, OS, and filesystem. (See http://en.wikipedia.org/wiki/SHA-1#Data_integrity for a little elaboration).
The SHA-1 hash and other source metadata is kept in a separate API, browsable at http://api.npolar.no/source/?q=
if you search for all Argos data from January 2007 you will see 3 sources http://api.npolar.no/tracking/?filter-technology=argos&q=&filter-measured=2007-01
The third source only contain two measurements http://api.npolar.no/tracking/?filter-measured=2007-01&filter-source=2a0ffa7e2b0b9bbf49bd2d0f7d92a23df250015e&filter-technology=argos&q=
Why? The Source API reveals that these two documents are stray cats in he February CD http://api.npolar.no/source/2a0ffa7e2b0b9bbf49bd2d0f7d92a23df250015e
The source above was created using the following command: argos-ruby --action=source /mnt/datasets/Tracking/ARGOS/archive/2007/2007-02/CD-ROM_2362_1_01_1_DB_DIAG.txt
On linux you can obtain the source identifier using the sha1sum command $ sha1sum /mnt/datasets/Tracking/ARGOS/archive/2007/2007-02/CD-ROM_2362_1_01_1_DB_DIAG.txt 2a0ffa7e2b0b9bbf49bd2d0f7d92a23df250015e
Edit-tracing All changes are traced and available in the following API browser: http://api.npolar.no/editlog/?q=&filter-endpoint=/tracking Revisions Each revision is stored forever who changed what when, but also the complete document
Parser-tracing parser-version
Consistency
Tracing consistenncy Storage bundle http://localhost:5984/tracking/_design/source/_view/bundle?key=%22f63f859ecb1340e84d71cf5522faf25453d64334%22&include_docs=true
Search... http://localhost:9393/tracking/?q=&filter-bundle=f63f859ecb1340e84d71cf5522faf25453d64334
Platform identification
The identity of all platforms are not known, ie. the information about what is being tracked (species, individual, object) is missing for a large portion of the data.
All known tracking platform metadata is available at the following service: See http://api.npolar.no/tracking/deployment/?q=
Argos-position inaccuracy
Positional data quality varies with technology, in particual Argos position are known to have a high degree of variability in accuracy. Argos positions vary greatly in positional accuracy, as inicated in the data by location class (lc).
It's possible to filter for location class = {1, 2, 3 } using a range query http://api.npolar.no/tracking/?filter-lc=1..3&fq=lc:G&filter-technology=argos&q=
For more information about location classes and Argos accuracy, see