Magic Castle EESSI 2023 09 26 - EESSI/meetings GitHub Wiki

Magic Castle clusters for EESSI

progress on setting up MC clusters in AWS
- x86_64 (Thomas)
  - https://github.com/EESSI/mc_aws_rocky8_x86_64_202309_test
  - no progress yet
- aarch64 (Kenneth)
  - https://github.com/EESSI/mc_aws_rocky8_aarch64_202309_test
  - full breakdown of how cluster was built in https://github.com/EESSI/mc_aws_rocky8_aarch64_202309_test/issues/1
  - deployment of (aarch64) login node didn't work correctly yet
    - no /mnt/home or /mnt/scratch yet
    - no Slurm yet
    - needs https://github.com/cmd-ntrf/puppet-consul_template/commit/26842577ae8b1c27d031fc256c662754687fb70d applied manually on mgtm1 + restart puppet on login1
  - autoscaling not set up yet, see https://github.com/ComputeCanada/magic_castle/blob/main/docs/terraform_cloud.md#enable-magic-castle-autoscaling
    - requires that TFE API token is stored in Git repo
    - can be encrypted, see https://github.com/ComputeCanada/magic_castle/blob/main/docs/README.md#4131-encrypting-hieradata-secrets
    - this means that Git repo should always be private!
    - and that we rotate TFE API tokens regularly
Azure
- Terje created a dedicated account/service principal
- should we reuse that?
- https://learn.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals?tabs=browser
bot should clean up subdirecties in jobs/ when PR is closed/merged
next sync meeting: Wed 4 Oct 2023 at 12:00 CEST