FcigDM_ANWC - AtlasOfLivingAustralia/ala-datamob GitHub Wiki

Australian National Wildlife Collection: primary collection management system darwincore export

Introduction

This is an implementation of a darwincore export, for one of the FCIG-OZCAM participants.

Artefacts and synopsis

Item Short URL Details (or long URL)
This wiki page http://goo.gl/kbSG4 FcigDM_ANWC
Source data system
Collection management software bespoke SQL Server data store with MS-Access user interface
Exporter's execution environment windows command line and sql-server powershell
Adhoc query t-sql
Bulk export method powershell's 'Export-CSV' instruction
Schema reporting SQL Server Management Studio's 'Database tasks>Generate scripts' function
DwC mapping t-sql query
Compression, transmission 7zip, Putty's psftp to upload.ala.org.au
Output data Darwincore csv (simple-dwc) format with non-standard FCIG extensions

institutionCode "ANWC"
dcterms:type "PhysicalObject"
basisOfRecord "PreservedSpecimen"

Data availability:

ANWC data before export
ANWC data at export
ANWC data after atlas (biocache) ingest http://goo.gl/bDRLn
Completeness model http://goo.gl/kIhG0 Google docs -> Data management -> CompletenessDwC -> anwc.dwccm.26
Source code http://goo.gl/00wSg https://github.com/AtlasOfLivingAustralia/ala-datamob/tree/master/biodomains/fcig-ozcam/anwc
Usage doco http://goo.gl/jXIYd https://github.com/AtlasOfLivingAustralia/ala-datamob/tree/master/biodomains/fcig-ozcam/anwc/anwc%20cms%20doco.20130103.pdf
Final status report http://goo.gl/kA6Dm Google docs ➢ Communications ➢ Data management ➢ Mobilisation reports ➢ finalreport.anwc.odt
http://goo.gl/iT25T finalreport.anwc.pdf (under the same directory)

Behavioural diagrams

From usage documentation https://github.com/AtlasOfLivingAustralia/ala-datamob/tree/master/biodomains/fcig-ozcam/anwc/anwc%20cms%20doco.20130103.pdf...

There are two parts to the exporter:

anwc_dwc.bat

The first export component is a bash shell script, anwc_dwc.bat, which is the entry point for running an export. This batch script encourages powershell.exe to connect to the sqlserver database and obtain the result of the query (see anwc_dwc.sql) into darwincore-csv format; it then compresses the data using 7zip.exe and uploads the file to the upload.ala.org.au sftp server using psftp.exe.


Activity diagram for https://github.com/AtlasOfLivingAustralia/ala-datamob/tree/master/biodomains/fcig-ozcam/anwc/anwc_dwc.bat

anwc_dwc.sql

The second export component is the sqlsvr query, anwc_dwc.sql, which is called by the batch file and powershell.exe. This must reside in the same directory as the batch file. It will extract data, joining source tables and introducing constants where necessary. Output is to anwc-dwcdata.csv in the same directory as the batch file.



Activity diagrams for https://github.com/AtlasOfLivingAustralia/ala-datamob/tree/master/biodomains/fcig-ozcam/anwc/anwc_dwc.sql
⚠️ **GitHub.com Fallback** ⚠️