S3 Cloud Storage - FLARE-forecast/flare-forecast.github.io GitHub Wiki
The following data is stored on S3 buckets:
- Input data from sensors
- Driver data from NOAA
- Intermediary data generated during FLARE forecasts
- Forecast outputs and graphs produced at the end of the forecast workflow
To access the S3 storage, we can use different tools such as Rclone or MinIO client.
Rclone is a command-line program used to manage files on cloud storage, including S3 object stores.
Follow the official Rclone installation guide.
On Ubuntu, the Rclone config file is typically located at:
~/.config/rclone/rclone.confThe content of the file should be similar to this:
[s3flare]
type = s3
provider = Minio
access_key_id = <your_id>
secret_access_key = <your_password>
endpoint = https://s3.flare-forecast.orgA crontab job on Jetstream2 VM ensures that S3 storage is mounted on reboot:
@reboot sleep 30 && /usr/bin/rclone mount --read-only s3flare: ~/s3flare-directory🚀 Read-only mode is strongly recommended to prevent unwanted changes to the S3 objects.