Testing Local MacOS Pipeline - Display-Lab/scaffold GitHub Wiki

First Time Local Testing Setup (MacOS)

Installing Python

First, we need to install python locally. Open up python V 3.10.12 and download the correct installer executable. Run it and finish the install.

If you have Homebrew, you can use that to download a verified install of python easily with the terminal:

brew install [email protected]

If you already have another version of python like 3.12, you can use pyenv to manage multiple versions of python. Otherwise, continue on.

Check pip installation

We now need to verify pip was installed with your python download, as it always should be when downloading from python.org. Use the where pip command to verify that Python can find pip in its program files. This should return a local filepath if successful.

This may require changing directory, and can also involve using alternate commands:

python3 -m where pip

Installing Poetry

Now, we need to install poetry. We can use pip to do this.

Use one of the following commands:

pip install poetry

If that fails, try this instead:

python3 -m pip install poetry

Downloading pipeline (local copy)

Now, we need to download the codebase from github and store it locally.

Method 1: Manual Download

  1. Go to the precision feedback pipeline repo, then hit the green code menu (looks like <> Code v) and download the whole repo as a zip file.
  2. Extract the zip file's contents in a place you can remember (or copy/paste somewhere). !! Know where you are dropping it, you will need that filepath going forward !!

Method 2: Git Clone

  1. Inside a terminal window, change directory to where you intend to download the local repo copy. This can be somewhere like /usr/local/.../Code/:
cd /path/to/working/directory/
  1. Use the command below to clone the repo, noting that whatever directory your terminal is in will be where it will be cloned:
git clone  https://github.com/Display-Lab/precision-feedback-pipeline/tree/main

Running a PFP instance locally

  1. Navigate to the PFP API directory in a terminal window:
cd /path/to/working/directory/.../precision-feedback-pipeline
  1. Open up a virtual environment, in this instance a poetry shell:
poetry shell
  1. Finally, run a local PFP API instance with uvicorn. Use the following command in your terminal:
uvicorn main:app --reload

If the startup is successful, you should see an output like this:

CRITICAL:root:---total esteemer run time according python script 5.698204040527344e-05 seconds ---
INFO:     Started server process [2005]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

This lets you know that your local server has successfully started up.

Note: using the argument --reload when starting up the local server may be useful when making changes to the PFP code. This will run the startup process with auto-redeploy on - any changes you make to the pipeline scripts will cause the server to restart with those changes implemented automatically.

Troubleshooting

If you get an error message from the startup script, it will start with Traceback (most recent call last): Scroll on down to the bottom, and see what the error is. Below we will walk through some of the usual error suspects and how to correct them.

  1. Dependency Errors

If the console reads something like ModuleNotFoundError: No module named 'jeff', it means we need to install the dependency jeff in a way that poetry can find it. We can do this from the poetry shell:

pip install jeff

After the module is installed, run the server startup command (uvicorn main:app) again and install any other dependencies until the server starts up properly.

Sending Post Requests

There are two easy ways to send POST requests to the PFP - with Postman or with Leakdown Tester. To use Postman, check out the Postman Workflow setup wiki here if this is your first time working with Postman. It will save you a lot of time and walk you through how to use the program. If using the Leakdown Tester, see the readme file to learn how to use that script (included in PFP).

Posting with Postman

  1. Send a POST request to the ip address 127.0.0.1:8000/createprecisionfeedback/ with details present in input_message.json using the postman app

  2. If you are running the pipeline locally and want to see intermediate files have the debug="yes" in the post body.

  3. If you want to set export variable for the causal pathways in the pipeline, you can do so by using the command:

poetry export PATHWAYS=https://github.com/Display-Lab/knowledge-base/tree/main/causal_pathways
poetry export measures=https://raw.githubusercontent.com/Display-Lab/knowledge-base/main/measures.json
poetry export templates=https://github.com/Display-Lab/knowledge-base/tree/main/message_templates

Or you can use:

export PATHWAYS=/Users/username/knowledge-base/causal_pathways
export templates=/Users/username/knowledge-base/templates
export measures='file:///Users/username/knowledge-base/measures.json'