These are some of the steps and commands that we use to operate our instance of edX here at Stanford.

Installation

End-To-End Test

cd playbooks/edx-west
ANSIBLE_EC2_INI=./ec2.ini ansible-playbook  -c ssh -vvv -i ./ec2.py edxapp_prod.yml --list-hosts

playbook: edxapp_prod.yml

Should this playbook run database migrations? (Type 'yes' to run, anything else to skip migrations):

Normal Update -- run all plays

cd playbook/edx-west
ANSIBLE_EC2_INI=./ec2.ini ansible-playbook  -c ssh -vvv -i ./ec2.py edxapp_prod.yml

Deploy Only -- run only the "deploy" play

doesn't change any of the *.json files, ie. the configuration steps, so only want to do this when we're sure we just want new software.

Specify the branch here, variable is lms_version: configuration-secure/ansible/vars/edxapp_prod_vars.yml

and then run this:

cd playbook/edx-west
ANSIBLE_EC2_INI=./ec2.ini ansible-playbook  -c ssh -vvv -i ./ec2.py edxapp_prod.yml

Useful Commands

Restarts

cd playbooks/edx-west
ANSIBLE_EC2_INI=./ec2.ini ansible '~tag_Name_app(4)_prod' -s -c ssh -i ./ec2.py -m shell -a "sudo shutdown -g0 -r -y"

Useful line to detect connections piling up:

cd playbooks/edx-west
ANSIBLE_EC2_INI=./ec2.ini ansible '~tag_Name_app(10|20|11|21)_prod' -s -c ssh -i ./ec2.py -m shell -a "lsof -i |grep TCP | wc -l"

Example to aggregate cms gunicorn logs from the last 2 days onto local /tmp:

cd playbooks/edx-west
ANSIBLE_EC2_INI=./ec2.ini ansible '~tag_Name_app(10|20|11|21)_prod' -s -c ssh -i ./ec2.py -m shell -a "echo ========`hostname`========; find /var/log/upstart -iname 'cms.log.*' -a -mtime +2 -a -mtime -5 -exec zcat {} \;" > /tmp/var_run_upstart_cms.log

Example to grep cms logs from August 1st onto local /tmp:

cd playbooks/edx-west
ANSIBLE_EC2_INI=ec2.ini ansible "~tag_Name_app(10|11|20|21)_prod" -c ssh -v -i ./ec2.py -s -m shell -a 'echo ========`hostname`========; cat /logs/cms/edx.log | grep "^Aug *1"' > /tmp/mnt_logs_cms_edx.log

Example Staging Variable

ANSIBLE_CONFIG=stage-ansible.cfg ANSIBLE_EC2_INI=ec2.ini ansible-playbook ...

Example remove user Bob from Staging

ANSIBLE_CONFIG=stage-ansible.cfg ANSIBLE_EC2_INI=ec2.ini ansible tag_environment_stage -c ssh -s -i ./ec2.py -m shell -a 'export NAME=Bob; if [ -d /home/$NAME ]; then userdel $NAME; tar vczf /home/$NAME.tgz /home/$NAME --remove-files; chmod 400 /home/$NAME.tgz; fi'

Example to invoke a python shell

sudo -u www-data SERVICE_VARIANT=lms /opt/edx/bin/django-admin.py shell --settings=lms.envs.aws --pythonpath=/opt/wwc/edx-platform