Ecodata Overview - AtlasOfLivingAustralia/ecodata Wiki

Overview

See the diagram below for an overview of ecodata components and dependencies.

Ecodata overview

Ecodata components

The table below provides a brief description of the component services provided by ecodata.

Component / Functional area Description
CRUD Services Ecodata exposes a JSON API to clients to perform CRUD operations on the domain entities.
Bulk data services Ecodata supports the download of system information in zip files, Microsoft Excel spreadsheets and ESRI shapefiles
Spatial services Ecodata interfaces with the ALA spatial portal to add contextual information to site / geographic data, and unpack and convert shapefiles. It also supports downloading site information in shapefile format
Search services Data in the database is indexed using elasticsearch. Ecodata provides a full text search and faceting API for this index
Occurrence data services Projects in ecodata can opt-in to allow species occurrence data to be exported to the ALA. To support this ecodata interfaces with the ALA Collectory to define data sets and provides a routine to export occurrence data in Darwin Core format for import into the ALA BioCache
Administration services Ecodata exposes a web UI for administration functions. These include caching, indexing and editing metadata
Security services Ecodata secures access to the API via use of an API key and IP address whitelist. The Administration services are secured via CAS (see below) and the ALA_ADMIN role
Audit services Changes to any entities in ecodata will be detected and logged to the AuditMessage table. This audit log can be search by clients
ecodata data store ecodata persists all data in a mongodb database
attached documents Documents attached to projects, including logo's and photo's are stored on the file system. The ecodata data store references the file system path to allow the document to be retrieved
elasticsearch An embedded instance of elasticsearch is used to support searching and faceting of data stored in the ecodata data store.
elasticsearch index The search index is stored on the file system

Dependencies

Ecodata has dependencies on several ALA services. These are illustrated in the diagram below.

System dependency Description Ref
CAS CAS is the ALA authentication system. MERIT redirects the user's browser to CAS when it detects they need to login and will subsequently check user credentials (CAS service granting tickets) with CAS to ensure they are valid. CAS
CAS Tickets The CAS application uses a mongodb database to store ticket granting tickets and service tickets.
User data User information, including system roles are stored in the user database
User details MERIT permissions are stored in ecodata as IDs. The User details application is used to lookup names and email addresses for an ID when that is required. Access to the User details system is restricted by both IP address and API key userdetails
API Key service ecodata uses the ALA API Key service to validate that API keys presented by client applications are valid. Access to the API Key service is restricted by IP address and API key apikey
Spatial service The ALA spatial service is used by ecodata to turn shapefiles into geojson (and converting the to the WGS84 CRS). When a site geometry in MERIT is modified, it is also used to intersect the site geometry against many of the layers in the spatial database to determine the State, Local Government Area, Electorate and more the site resides in.
geoserver Some sites (known shapes) in MERIT are displayed as image tiles generated and served by the ALA geoserver installation
Collectory The ALA Collectory maintains metadata about the data sets that have been imported into the ALA main database (biocache). Ecodata maps Organisations to Collectory institutions, and Projects to data resources. This supports referencing the source of occurrence data that is exported out of ecodata into the ALA biocache
ALA BioCache Management The ALA BioCache management system queries ecodata for the existence of data sets that contain occurrence data suitable for export into the ALA. This is configured on a per project basis and is opt-in. Ecodata provides occurance data in the form a Darwin Core Archive
SMTP Server Ecodata will send management emails to administrator under some error conditions
BioCollect BioCollect is an ALA project which provides a web front end to ecodata
MERIT MERIT is an ALA / Dept. of the Environment and Energy project which provides a web front end to ecodata focusing on natural resource management projects