At NCI we use two separate servers - 'accessdev', which hosts UI & job control and 'vayu', the supercomputer. All communication between the two happens via SSH, no other ports are open. Accessdev can only talk to the login node, compute nodes can send messages to accessdev and the login node can communicate with either.

Security Restrictions

As NCI is a shared system used by many researchers across a wide range of fields security of accounts is important. The following is NCI's policy for SSH keys-

• Unrestricted SSH keys must have a passphrase
• Passphraseless keys can be used, but have to be restricted to only allow specific programs to run which don't affect the filesystem

Restricting available commands is done using the 'authorized_keys' file on the remote machine, which forces commands to be run in a restricted shell, e.g. lines like:

command="remote-job-submission forced",no-port-forwarding,no-x11-forwarding,no-agent-forwarding ssh-rsa ABC123...XYZ== user@host

instead of the unrestricted access given by

ssh-rsa ABC123...XYZ== user@host

The remote-job-submission script does the following:

• Gets the basename of the command to run
• Sets $PATH to only look in /opt/remote-submission/rbin
• Disables shell builtins with Bash's 'enable -n' function
• Runs the function in a restricted shell with no profile or rc files using

exec /bin/bash --restricted --noprofile --norc -c "$command"

Currently the available remote commands are 'cylc started/succeeded/failed'.

It is much easier to secure communication from the supercomputer to the UI than the other way around, simply because the whole purpose of the UI is to run arbitrary code in the supercomputer queue. On the other hand, securing the supercomputer is much more a priority than securing the UI server.

We can assume that a run starts with someone logged in, so we can use a secured ssh agent to start up a run & copy scripts over. Unattended running is really the issue here, for instance if a script wants to resubmit itself. Perhaps we can allow qsub'ing an existing script as a remote command.

SSH Agents

SSH agents have been an inconvenience historically, mostly due to them not being properly cleaned up when someone logs off the system. SSH agent forwarding may provide a work-around for this, security implications need to be understood before moving forward. Another alternative is to set up agent handling in /etc/bashrc & similar, so that it is not left up to users.