To import the data for a given month, start a screen session on largeweb2 as the hello user:

screen -S import_{year}_{month}

Then run:

cd /webapps/openprescribing/openprescribing/
. ../.venv/bin/activate
python manage.py fetch_and_import {year} {month}

Fetching BNF code data requires solving a captcha. When prompted:

• visit https://applications.nhsbsa.nhs.uk/infosystems/data/showDataSelector.do?reportId=126
• solve the captcha
• get the JSESSIONID cookie
• in a new session (not a screen session):
  • cd /webapps/openprescribing/openprescribing/
  • . ../.venv/bin/activate
  • ./manage.py fetch_bnf_codes [JSESSIONID]
• when this finishes, hit in the other session
• detach from the screen session, and wait for bennettbot to report in Slack

When the pipeline has finished (after about 24 hours) a message will be sent to Slack.

If the import fails, a message will be sent to Slack.

You should reconnect to the screen session:

screen -r import_{year}_{month}

There should be a traceback which will help you understand the problem. The traceback will also be recorded in the formatted_tb attribute of the corresponding TaskLog object. Once you've understood and fixed the problem, you should add a comment to https://github.com/ebmdatalab/openprescribing/issues/571.

You should then re-run the fetch_and_import command, which will re-run the failed task and then continue. Note that many tasks are not idempotent and so you may need to delete some data or fudge the command.