Infrastructure Requirements - AtlasOfLivingAustralia/documentation Wiki

Infrastructure Requirements for running a Living Atlas

The infrastructure required to run a Living Atlases depends on following factors:

We recommend the use of cloud infrastructures for Living Atlas installations. This could be a commercial provider (e.g. Amazon EC2, Google Cloud Engine, Microsoft Azure), or a cloud infrastructure within your country operated by an institution (e.g. an OpenStack based installation).

Basic Installation

A basic installation of the core components with support for up to 20 million records could be a single Ubuntu 18.04 server with 4-8 CPU, 32GB RAM and SSD storage. Ideally though, it is recommended that Cassandra and SOLR are ran on separate virtual machines, as both of these components require a reasonable amount of resources. Running Cassandra and SOLR separately will allow you to run data maintenance task (loading, processing, indexing) without impacting the performance of your web portal tools.

Recommended Installation for larger installations

For installations requiring the indexing of large amounts of data (over 50 million records and/or indexing of large number of spatial layers), we would recommend a clustered installation. This clustered installation is in use by Australia (75 million records and 500+ spatial layers) and UK (219 million records and 50+ spatial layers).

Clustering affects the installation of Apache SOLR, Apache Cassandra and the biocache command-line tools.

See the Cassandra requirements and solr requirements.

Core components for a Living Atlas

The core set of components that an Living Atlas will require as a starting point are the following:

These components will give a Living Atlas installation the following capabilities:

These components are installed as part of the ALA demo installation scripts. This is the recommended starting point for projects in the initial phase of looking at the Living Atlases components for their project.

Additional components - advanced installations

In addition to the core components, the following components can be setup to enhance an installation further:

Additional requirements