Access S3 buckets from Fedora SIlverblue - joefidler/joefidler.github.io GitHub Wiki

Use s3fs-fuse to access S3 buckets on Amazon AWS and mount them locally on a Fedora Silverblue computer. You will need an AWS account and of course an S3 bucket.

• Credentials: An AWS IAM managed userid dedicated to your computer is a good idea, just in case the machine is lost/stolen, etc. The AWS IAM userid will need full S3 access, but nothing else. Keep a copy of the Access Key ID and Secret Access Key.

• Install:

  rpm-ostree install s3fs-fuse

  systemctl reboot

• Create a file called .passwd-s3fs to store the S3 credentials in your home directory.

  echo ACCESS_KEY_ID:SECRET_ACCESS_KEY > ${HOME}/.passwd-s3fs

  it should look something like

  echo 13randomstuff45:dfsmorerandomestuff123456789012345 > ${HOME}/.passwd-s3fs

• set owner-only permissions for the file:

  chmod 600 ${HOME}/.passwd-s3fs

• To run the command the format is:

  s3fs bucket_name /path/to/mountpoint -o passwd_file=${HOME}/.passwd-s3fs

  for example

      s3fs bucket_name /var/home/your_user_id/bucket_name -o passwd_file=${HOME}/.passwd-s3fs
  

• Make things a bit faster:

There are some helpful suggestions for options on the IBM Cloud Storage S3 page page that can help speed things up.

        use_cache=~/.s3fs_cache
        max_background=1000	
        max_stat_cache_size=100000
	multipart_size=52
	parallel_count=30
	multireq_max=30`
	use_sse=SSE-S3
	endpoint=us-west-1           (optional - your buckets AWS location)
	url=https://s3.amazonaws.com         (optional - forces HTTPS)

• Final command(s) look something like

s3fs bucket_name /var/home/your_user_id/bucket_name -o passwd_file=${HOME}/.passwd-s3fs \
-o use_cache=~/.s3fs_cache -o max_background=1000 -o max_stat_cache_size=100000 \
-o multipart_size=52 -o parallel_count=30 -o multireq_max=30 -o use_sse=1 \
-o url=https://s3.amazonaws.com

If things work, your buckets should start showing up in file manager at the mount points. If the buckets are large the initial sync can take a little while, so it might be worth trying a small bucket first.

You can access multiple buckets at the same time by entering more than one of these commands. The command(s) can be scripted into a bash script, which can be added to a .desktop file to create a desktop application, or a startup application.

Another option, which I didn't use, is to create an entry in in /etc/fstab so the buckets will be included in your system startup:

<bucket name> <mount point> fuse.s3fs _netdev, 0 0

• If you have problems first try entering a simple version of the s3fs command manually to see what happens. S3FS-fuse runs as a daemon so when running shows up in the system-monitor process list.

There is also a debug option:

s3fs imaginative_S3_bucket_name /var/home/your_user_id/imaginative_S3_bucket_name -o passwd_file=${HOME}/.passwd-s3fs -o dbglevel=info -f -o curldbg

If you need more information : https://github.com/s3fs-fuse/s3fs-fuse