Ecodata Overview - AtlasOfLivingAustralia/ecodata GitHub Wiki
Overview
See the diagram below for an overview of ecodata components and dependencies.
Ecodata components
The table below provides a brief description of the component services provided by ecodata.
| Component / Functional area | Description |
|---|---|
| CRUD Services | Ecodata exposes a JSON API to clients to perform CRUD operations on the domain entities. |
| Bulk data services | Ecodata supports the download of system information in zip files, Microsoft Excel spreadsheets and ESRI shapefiles |
| Spatial services | Ecodata interfaces with the ALA spatial portal to add contextual information to site / geographic data, and unpack and convert shapefiles. It also supports downloading site information in shapefile format |
| Search services | Data in the database is indexed using elasticsearch. Ecodata provides a full text search and faceting API for this index |
| Occurrence data services | Projects in ecodata can opt-in to allow species occurrence data to be exported to the ALA. To support this ecodata interfaces with the ALA Collectory to define data sets and provides a routine to export occurrence data in Darwin Core format for import into the ALA BioCache |
| Administration services | Ecodata exposes a web UI for administration functions. These include caching, indexing and editing metadata |
| Security services | Ecodata secures access to the API via use of an API key and IP address whitelist. The Administration services are secured via CAS (see below) and the ALA_ADMIN role |
| Audit services | Changes to any entities in ecodata will be detected and logged to the AuditMessage table. This audit log can be search by clients |
| ecodata data store | ecodata persists all data in a mongodb database |
| attached documents | Documents attached to projects, including logo's and photo's are stored on the file system. The ecodata data store references the file system path to allow the document to be retrieved |
| elasticsearch | An embedded instance of elasticsearch is used to support searching and faceting of data stored in the ecodata data store. |
| elasticsearch index | The search index is stored on the file system |
Dependencies
Ecodata has dependencies on several ALA services. These are illustrated in the diagram below.
| System dependency | Description | Ref |
|---|---|---|
| CAS | CAS is the ALA authentication system. MERIT redirects the user's browser to CAS when it detects they need to login and will subsequently check user credentials (CAS service granting tickets) with CAS to ensure they are valid. | CAS |
| CAS Tickets | The CAS application uses a mongodb database to store ticket granting tickets and service tickets. | |
| User data | User information, including system roles are stored in the user database | |
| User details | MERIT permissions are stored in ecodata as IDs. The User details application is used to lookup names and email addresses for an ID when that is required. Access to the User details system is restricted by both IP address and API key | userdetails |
| API Key service | ecodata uses the ALA API Key service to validate that API keys presented by client applications are valid. Access to the API Key service is restricted by IP address and API key | apikey |
| Spatial service | The ALA spatial service is used by ecodata to turn shapefiles into geojson (and converting the to the WGS84 CRS). When a site geometry in MERIT is modified, it is also used to intersect the site geometry against many of the layers in the spatial database to determine the State, Local Government Area, Electorate and more the site resides in. | |
| geoserver | Some sites (known shapes) in MERIT are displayed as image tiles generated and served by the ALA geoserver installation | |
| Collectory | The ALA Collectory maintains metadata about the data sets that have been imported into the ALA main database (biocache). Ecodata maps Organisations to Collectory institutions, and Projects to data resources. This supports referencing the source of occurrence data that is exported out of ecodata into the ALA biocache | |
| ALA BioCache Management | The ALA BioCache management system queries ecodata for the existence of data sets that contain occurrence data suitable for export into the ALA. This is configured on a per project basis and is opt-in. Ecodata provides occurance data in the form a Darwin Core Archive | |
| SMTP Server | Ecodata will send management emails to administrator under some error conditions | |
| BioCollect | BioCollect is an ALA project which provides a web front end to ecodata | |
| MERIT | MERIT is an ALA / Dept. of the Environment and Energy project which provides a web front end to ecodata focusing on natural resource management projects |