Prod Deployment (Python Coderunner) - UQ-eLIPSE/shifoo-docs GitHub Wiki

Follow steps

ssh <uq_username>@mango.eait.uq.edu.au
triton profile set shifoox
triton instance create --name=shifoo-py312x-001 --network=zones webproject z1-standard
triton instance tag set shifoox-py312x-001 triton.cns.services=shifoox-py312x

ssh into the zone Generate keys

ssh-keygen # Type /root/.ssh/id_rsa_coderunner_python
cat ~/.ssh/id_rsa_coderunner_python.pub

Copy key and place in deploy keys for coderunner-python repository. Repeat for coderunner-server

eval "$(ssh-agent -s)" # Start ssh agent
ssh-add ~/.ssh/id_rsa_coderunner_python

git clone [email protected]:UQ-eLIPSE/coderunner-python.git

Install python and its dependencies

sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 1
sudo update-alternatives --config python3 # Choose python3.12
python3 --version # Should be python3.12
apt install build-essential tcl tk tcl-dev tk-dev xvfb x11-apps x11-utils
python3.12 -m pip install numpy scipy matplotlib ipython jupyter pandas sympy nose
# If the above didn't work try: pip install numpy scipy matplotlib ipython jupyter pandas sympy nose

cd coderunner-python
cp config.json.example config.json
cat config.json

Should show

{
    "API_KEY": "SECRET_KEY",
    "PORT": 5000,
    "MAX_PROC_TIMEOUT": 10000,
    "PYTHON_PATH": "/home/monkey/dependencies",
    "CODERUNNER_TEMP_DIR": "/home/monkey/tmp",
    "REMOVE_LOGGING": true
}

If it doesn't please paste the above to the config.json file. Install dependencies

ssh-add -D # Clear identities
ssh-add ~/.ssh/id_rsa_coderunner_server
yarn

Create Code Monkey Executor User

USER=monkey
adduser --disabled-password --gecos "" ${USER}
USER_HOME=$(getent passwd monkey | cut -d: -f6)
yarn build
cp --recursive --target-directory=${USER_HOME} node_modules dist dependencies
mkdir ${USER_HOME}/tmp
chown -R ${USER}:${USER} ${USER_HOME}

nginx config

1. vim /etc/nginx/frameworks-enabled/monkey.conf
2. Paste the following

location /ping { access_log off; return 200 'pong'; }
location /run { if ($request_method = 'POST') { proxy_pass http://localhost:5000; } }

3. vim /etc/nginx/conf.d/auth.conf
4. Comment line 19 # default "allow:user:*";
5. Uncomment line 20 (disables SSO): default "allow:*";
6. After :wq Run nginx -t. Make sure the syntax in the conf file is ok and the test is successful
7. Finally, systemctl restart nginx

Monkey Service

1. vim /etc/systemd/system/monkey.service
2. Paste the following:

[Unit]
Description=ShiFoo Python3.12 Code Executor.
[Service]
Type=simple
User=monkey
ExecStart=/usr/bin/node /home/monkey/dist/src/index.js
Restart=always
Restart=10
[Install]
WantedBy=multi-user.target

3. systemctl enable monkey
4. systemctl daemon-reload
5. systemctl restart monkey
6. systemctl status monkey

Adding rsyslog

syslog is the default syslogd on Debian systems and allows us to help debug any future issues

apt-get install rsyslog -y
systemctl enable rsyslog
systemctl start rsyslog

vi /etc/rsyslog.conf Find and uncomment the following lines to make your server listens to the udp and tcp ports in the MODULES section as shown below; Make sure to also add the $template line below input(type="imtcp" port="514")

...
#################
#### MODULES ####
#################
...

# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="514")

# provides TCP syslog reception
module(load="imtcp")
input(type="imtcp" port="514")
$template RemInputLogs, "/var/log/remotelogs/%FROMHOST-IP%/%PROGRAMNAME%.log"
*.* ?RemInputLogs        ### ADDME

systemctl restart rsyslog
systemctl status rsyslog # Should be active
tail -f /var/log/syslog # To view logs

Load balancing

Make sure you're in mango and not in the zone.

triton image create --description='ShiFoo(Python3.11) Code Monkey Execution' --wait --json shifoox-py312x-001 shifoox-py312x 1.0.0
triton instance create --name=shifoox-py312x-002 --tag='triton.cns.services=shifoox-py312x' --network=zones --affinity='instance!=shifoox-py312x-001' shifoox-py312x z1-standard
triton instance create --name=shifoox-py312x-003 --tag='triton.cns.services=shifoox-py312x' --network=zones --affinity='instance!=shifoox-py312x-001' --affinity='instance!=shifoox-py312x-002' shifoox-py312x z1-standard
triton instance create --name=shifoox-py312x-004 --tag='triton.cns.services=shifoox-py312x' --network=zones --affinity='instance!=shifoox-py312x-001' --affinity='instance!=shifoox-py312x-002' --affinity='instance!=shifoox-py312x-003' shifoox-py312x z1-standard
triton instance create --name=shifoox-py312x-005 --tag='triton.cns.services=shifoox-py312x' --network=zones --affinity='instance!=shifoox-py312x-001' --affinity='instance!=shifoox-py312x-002' --affinity='instance!=shifoox-py312x-003' --affinity='instance!=shifoox-py312x-004' shifoox-py312x z1-standard

To test each zone, refer to https://github.com/UQ-eLIPSE/shifoo-docs/wiki/Coderunners-zone-testing

More

If old coderunners are set and needs to be ignored, its tags can be set to something else other than shifoox-py312x

triton instance tag set shifoox-py312x-001 triton.cns.services=shifoo-py311x-old