The COVIDScenarioPipeline repo should be cloned inside of a COVID19_spatial repo. The two repositories should be treated independently, meaning that code and branches should be pushed and pulled to them independently. You will need to make sure that both are at the correct commit in order to run the model as you intend. The directions below should be executed from the terminal.

1. Create a spatial repo from the COVID19_Minimal template by navigating to COVID19_Minimal and clicking "Use this template". For this example, we'll do Hawaii. We will name it COVID19_Hawaii and create the repository as yourgithubuser

2. Clone the spatial repo you just created to the device where you intend to run the model.

   git clone https://github.com/yourgithubuser/COVID19_Hawaii.git

3. Checkout COVIDScenarioPipeline repo within the spatial repo.

   cd COVID19_Hawaii
   git clone https://github.com/HopkinsIDD/COVIDScenarioPipeline.git
   

While working with this pipeline, we recommend that you edit files from your local machine and run scripts from the provided Docker container. The advantage of this container is that it already has all the packages installed, which takes some time.

If you prefer to run the model without the use of Docker, you can see all of the requirements in the COVIDScenarioPipeline repo. See the R requirements in packages.R and local_install.R, OS requirements in Dockerfile, and Python requirements in requirements.txt.

For most users, we strongly recommend the use of the Docker container and the instructions for its installation are described below.

1. Go to the Docker Hub website (https://hub.docker.com) and create an account.

2. Start the Docker service on your computer. One way is to download, install, and run Docker Desktop. Google searches can help you with this, potentially more than our support team can, but feel free to ask questions.

3. Open a terminal. For the docker commands in this section, if you encounter permissions errors, you will need to put sudo in front.

4. Pull the docker image from hub.docker.com. You'll only have to do this the first time.

   docker pull hopkinsidd/covidscenariopipeline:latest

5. Run the docker container with your current directory mounted as /home/app/covidsp

   On Linux or Mac:

    `docker run -v ~/mysrcdir:/home/app/covidsp -it hopkinsidd/covidscenariopipeline:latest`
   

   Replace mysrcdir with the path where the code is mounted on your machine (e.g., ~/myuser/COVID19_Hawaii)

   On Windows:

    `docker run -it --rm -v %CD%:/home/app/covidsp hopkinsidd/covidscenariopipeline:latest`
   

   You may need to change "%CD%" to your explicit directory

6. You are now in the docker container in /home/app. The directory you ran step #3 from is mapped to /home/app/covidsp inside the container.

   cd covidsp

7. The Docker container needs some local R packages installed. Run this:

   Rscript COVIDScenarioPipeline/local_install.R

If there's a prompt Enter one or more numbers, or an empty line to skip updates:, just hit <Enter>.

8. The Docker container requires local Python packages to be installed. Run:

   python COVIDScenarioPipeline/setup.py install

The config file in your spatial repository config.yml controls all of the options currently available. (See this page for more details.) This file has a tabbed outline structure. We will refer to keys using their full position in the outline. For example, we denote:

spatial_setup:
  ...
  geodata: minimal

as spatial_setup::geodata having a value of minimal

Delete the line this_file_is_unedited, or set it's value to FALSE. This is just to make sure people edit the config.yml before trying to run a model.

There are several tags in the config like <Your State Postal Code>. Fill these tags in with state-specific information.

Setup files include 1) geodata.csv, 2) mobility.csv, and 3) seeding.csv. These files indicate respectively, 1) the unique geoids that will be modeled, 2) their accompanying mobility matrix, and 3) the epidemic seeding for the simulations. For the US location example, these data may be pulled from the US Census Bureau. The mobility matrix will be taken from publicly available county-level commuting data.

1. Pull large data in repo. This may involve installing the git LFS tool and adding your Census API key. You may acquire a Census API key here.

cd COVIDScenarioPipeline
git lfs install
git lfs pull
cd ..
export CENSUS_API_KEY=<copy your key here>

2. Be aware that the model build code will run a script R/scripts/build_US_setup.R and create a geodata and mobility file according to the specifications in your config file. You will not need to run this script separately.

3. Be aware that the model build code will run another script R/scripts/create_seeding.R and create an epidemic seeding file according to the specifications in your config file. You will not need to run this script separately.

This shapefile must have a GEOID column and be located in the data directory.

To retrieve shapefile data for US states from the US census, run these commands in R:

config <- covidcommon::load_config("config.yml")
tidycensus::census_api_key(key = config$importation$census_api_key)
covidImportation::get_county_pops(c('HI'), 'HI')

If you do not already have the covidImportation package installed, you can see directions for its installation here.

1. From the spatial scenario directory, create the Makefile using the R script.

   Rscript COVIDScenarioPipeline/R/scripts/make_makefile.R -c config.yml

2. There is a bug in the Makefile that causes the report generation done by the make command in step 5 to fail. To get around this, steps 2-4.

Make a report directory and sub-directory because that is the .Rmd expects to be two directories below right now.

mkdir notebooks
cd notebooks
mkdir HI_today
cd ../..

3. Make the initial R markdown (.Rmd) by running the following command.

   Rscript -e 'rmarkdown::draft("notebooks/HI_today/HI_report.Rmd",template="state_report",package="report.generation",edit=FALSE)'

4. Write the render line into compile_Rmd.R. (compile_Rmd.R is later used by fancy automation.)

   echo 'rmarkdown::render("notebooks/HI_today/HI_report.Rmd", params=list(state_usps="HI"))' >compile_Rmd.R

5. Build and run.

   make

   This command will generate the setup files, run the model simulations, and generate a dummy report, as specified in config.yml.

If you see the following error during the make command, this is a known bug in the report generation step, which Rscript compile_Rmd.R bypasses.

Execution halted
Makefile:11: recipe for target 'notebooks/blue_hawaii_20200520/blue_hawaii_20200520_report.html' failed
make: *** [notebooks/blue_hawaii_20200520/blue_hawaii_20200520_report.html] Error 1

Commands:

make clean # Removes all the generated files so they can be re-generated
make
Rscript compile_Rmd.R # Getting around report generation bug

Tada! The data that you are looking for is in csv's in the folders model_output and hospitalization.

The report you are looking for is in notebooks/HI_today/HI_report.html. You should modify its associated Rmd file in order to add more details to your report. The html output file may be viewed in a web browser.

To exit without stopping the container so you can attach later, type Ctrl-p; Ctrl-q.

Otherwise, exit and remove the container with Ctrl-c.