Getting started as a developer and data engineer - govau/observatory-website GitHub Wiki

Github
Cloud.gov.au
AWS
Observatory Data

Github

All code is stored in either the govau or ausdto organisation. To request access to these organisations, contact [email protected] and send them your github username.

Cloud.gov.au

All of our web products are stored on cloud.gov.au, which is built upon cloud foundry.

To get access to cloud.gov.au send the following email to [email protected]

G`Day cloud.gov.au team

Could you please create a cloud.gov.au account for me?

email: <your email here>
Mobile number: <your mobile number here>

Could I please have access to the Analytics space.

Regards,

Jane Doe
<your team name here>

You can read more on cloud.gov.au on their website.

AWS

Email [email protected] for access to AWS, with the role of datagovau-support. This is used to do maintenance work on data.gov.au/data

Data responsibility at Observatory

• Google Analytics data for whole of Government
• Government organisations data published on data.gov.au
• Geo dataset published on nationalmap.gov.au
• Other varied data sources, data pipelines outputs in Google Cloud Storage

Data store at Observatory

• The Google Analytics data is ingested in cloud data warehouse hosted on Google Cloud Platform BigQuery service.
• Data on data.gov.au is hosted using CKAN
• Geo dataset on nationalmap.gov.au is hosted on MAGDA platform which is maintained by Geoscience Australia.

Github Repository for publishing data scripts

The data scripts developed in SQL and Python are available in following Github repository.  The scripts are used to build data pipelines. • Galileo repository • Observatory Service – the details of data scripts can be found in documentation guide.