Postman Testing Workflow - Display-Lab/scaffold GitHub Wiki

Setting up a testing workflow in Postman

This wiki will walk you through a solid workflow to use while testing the pipeline with Postman. We use Postman for single-case testing of all of our pipeline instances, so keeping things organized is a big deal - housekeeping is important!

Why use Postman this way? There are a couple huge benefits:

  • Separate collections organize your POST request by which platform they are sent to
    • This keeps you from documenting a bug on the wrong platform, and from changing the URLs by hand on a frequent basis
  • Setting up a template makes certain that you have Vignette-accurate data sets to send the PFP instance you are testing on startup - positive controls are important!
  • Making new post requests for each test case helps immensely with debugging:
    • Saving a POST request that causes a bug lets you document it, but also lets you have it saved to come back to and verify our fix works when the bug fix gets implemented
    • Setting up separate case requests lets you easily share your input conditions or the entire post request itself to the team for verifying and fixing the bug
    • Most importantly, this setup lets you easily duplicate your bugged Post request and test it against the other pipeline instances with just one URL change

Install Postman

To send post requests for the pipeline to process, we use a program called Postman. You can use either the Postman program found here, or create an account and use the browser version here.

Create an account or download the app - either one works.

A shared Display-Lab team workspace is in development - if you want to use the shared workspace and post requests, ask Mack to send you an invite, and go to the appropriate wiki for whatever you want to test! Otherwise, follow the steps below to make your own private collection in your personal workspace.

First-time Personal Setup Steps

Setting up a new Local Collection

Here we will setup a collection in your workspaces that you can use repeatedly for testing the pipeline.

  1. Open Postman
  2. Go to the workspaces tab in the top left of your screen, and click on My Workspace to enter your own personal workspace.
  3. Create a new collection by clicking the second button down on the left sidebar that looks like a folder.
  4. Name your collection something related to whatever testing you are doing - Heroku, Local, Google Cloud, or something else. This helps keep your requests organized by destination!

Creating a Template Post Request

We want to now set up a template post request to make copies of and modify when we test different things against whichever pipeline instance we want. Start with the checklist below:

  1. Click the plus button on the top of your screen next to the tab that shows Overview and whatever collection you are in (you just made one, it should be there!). You can also hit the New button in the top left. This should open up a new blank post request called Untitled Request.
  2. In the new request, you'll see a dropdown menu under the title, next to the request URL field. It will read GET in green. Click the dropdown, and select POST to set the request method to POST.
  3. In the request URL field, we need to add the IP address of where we want the post request to go. Paste the link for the pipeline API hosted where you are testing:

Local

127.0.0.1:8000/createprecisionfeedback/

Heroku

https://pfpapi.herokuapp.com/createprecisionfeedback/

Google Cloud

https://pfp.test.app.med.umich.edu/

  1. Underneath the request URL bar, switch from the Params tab to the Body tab by clicking it.
  2. A dropdown menu should appear that shows none with a carrot. Click the drowpdown and set it to raw.
  3. A Blue dropdown menu reading Text should appear. Change that to select JSON. This sets your request body data type to JSON.
  4. Now we need to enter the body data in the field - click on this link to take you to the input_message.json file.
  5. On the right side of the top of the code block interface in GitHub, next to Raw is a copy symbol. Click it to copy the full raw code to your clipboard.
  6. Go back to Postman, and paste the raw code into the body data interface. If everything is set up correctly, the JSON syntax should highlight in different colors. If not, check your data type (step 6).
  7. Save your post request by clicking the Save button with the floppy disk. This should open up a whole menu. If not, click the carrot next to the save button and select Save as. Name your template whatever you would like, but I recommend Starting it with Template.

Creating a new Test Case Request

  1. Open up your template post request
  2. Click the carrot next to the save button, and select Save As.
  3. Change the request name to something that describes the test case you want to run. For instance, if I was testing data from one persona, with only one measure, and 12 months of data that has no change in the performance over the full year, I would name it something like Flatline -1p, 1m, 1y. This is all up to your discretion.
  4. In the field below that says Save to, find the appropriate collection or folder to put your request in.
  5. Click the orange button at the bottom that says Save. We will now have a duplicate of the template post request that we can modify as much as we like!
  6. Make whatever modifications to the body of the post request that you want to test. Make sure to save your work often.

We are now ready to use Postman! Hit the send button to send your post request.

Debugging Value

There is an option in the current main.py that looks for a defined "debug" value in the postman request. As of 7/18/23, this means you need to go into the postman request and define a variable as debug, and set a value to "yes" or "no".