Running Experiments - bblockwood/lab GitHub Wiki

We sometimes use tools like Qualtrics and TurkPrime to run online experiments.

TurkPrime

TurkPrime is a wrapper around Amazon's Mechanical Turk platform. Log into it using the "Sign in with Amazon" button, using the credentials for Ben's dedicated Amazon MTurk account.

Protocol for getting reimbursed

The Wharton Behavioral Lab (WBL) provides additional funding support for running experiments, but requires a specific approval protocol. WBL's protocol involves two steps: (1) requesting pre-approval to run a study and (2) submitting a reimbursement request. More information about this process, including where to submit pre-approval and reimbursement requests, can be found here.

Requesting pre-approval

Before running a study, a pre-approval request must be sent to the WBL Senior Research Coordinator for online studies who will then approve the study.

  • Title of study
  • IRB#
  • Faculty contact
  • Panel (M Turk, Prolific, etc.)
  • Paid by (Whose credit card was used for payment)
  • Approximate Cost
  • Total # of participants
  • Payment per participant
  • Account to be Charged (DARTS, grant, research fund, etc.)

Requesting reimbursement

After running an approved study, a reimbursement request should be drafted for submission to the WBL Senior Research Coordinator for online studies. You should provide the following information and supporting documentation.

  • Title of study
  • IRB#
  • Faculty contact
  • Paid by
  • Final Cost (for approved HITs only)
  • Total # of approved participants
  • Payment per participant
  • Account to be Charged (DARTS, grant, research fund, etc.)
  • The 26-digit budget code of the account to be charged

In addition, the following documentation is required for a reimbursement:

  • Batch Details: If the reimbursement request is for MTurk/Turk Prime, you must send screenshots of batch details. For each survey or survey wave, you should create a separate screenshot of the batch details and save it in PNG format. See below for two examples, one from MTurk and one from Turk Prime.

Images/MTurk Batch Details.png

Images/Turk Prime Batch Details.png

  • Bonus Details: If the MTurk/Turk Prime requests include bonuses paid out to participants, you must attach the spreadsheet detailing the bonuses. On MTurk, under “My Account,” there is a “Transaction History” tab. You can generate the spreadsheet by typing in the date range.

  • Itemized receipt: If the reimbursement request is for Prolific, please include the itemized receipt that includes the payment for participation and the invoice marked as paid.

  • Credit Card Receipt: Required as proof that the requester is paying for their studies (since funds must be loaded onto the platform up-front, the receipt total may not match the requested reimbursement amount).

If you are unsure, more information is better than less.