Some versions got skipped! What do I do?

If you end up in a situation where you’ve skipped a few versions (due to server outages, API updates, etc.) this puts your database in a bad place and you should fix it before proceeding. (Tagging isn’t disastrous, but you have to fix it before calculating intervals, since that will mean you have to delete residence, reinsert, and repeat_status for every document after the gap in data, which is doable but not straightforward.) You can manually set the date/version after initializing the scraper and before running download_release():

scraper = ElasticEngToolsDownloader(ip=os.getenv('FW_IP'),
   username=os.getenv('FW_USERNAME'),
   password=os.getenv('FW_PASSWORD'),
   download_dir=os.getenv('DOWNLOAD_DIR'))
scraper.latest_version = '3061-3571'
scraper.latest_date = parser.parse('2019/08/05 04:04:44 PDT')
scraper.download_release()

Modifying this code at the bottom of notes_parser.py should fix your issue. Remove it after you’re done. Note that you should probably run all of this from Jenkins, since the Jenkins user reads its home directory differently from the paloalto user and will therefore download documents and read them from a different directory.