nodeGame Mturk v7 - nodeGame/nodegame GitHub Wiki

• status: complete
• version: 7.x

Amazon Mechanical Turk (AMT) is a major online platform for recruiting experimental participants. nodeGame currently offers partial integration with AMT via the nodeGame-mturk package.

The experimenter needs a valid AMT requester account. The HIT creation can be carried out using the AMT interface.

1. Before creating a new project, create a new Qualification. You will use this to identify who participated in your experiment. Go to Manage/Qualification Types/Create New Qualification Type.

2. Create a new project. Go to Create/New Project and choose the template called Survey Link.

3. Choose Project Name (for own reference), Title (what will be displayed to participants), Description, and Keywords.

4. Add all other settings as required by your experimental design. Keep in mind that AMT guarantees that one worker can do at most one assignment, as long as you have only one HIT in your group (exactly what you want, and what you achieve following this tutorial).

5. You are likely going to run multiple sessions of the same experiment, therefore you should avoid workers that participated in previous sessions join future ones. To achieve this, in the Worker Requirements select the Qualification that you have created at step

6. and choose the has not been granted option. You have now to remember to grant this qualification to your participants after each experimental session is completed.

7. Proceed to the Design Layout. Modify the existing template by clicking on Source and replacing the entire text with the content of the file mturklinkpage.html, from the nodeGame-mturk package. The template already includes a consent form, but if you prefer you can add it as the initial stage of your experiment following these guidelines. Important: this file requires that your experiment runs under the https protocol. Read this wiki page for more info.

8. Make sure to adapt the content mturklinkpage.html for your purpose. In particular:

   • Replace all text enclosed in curly brackets {} with the appropriate content relevant for your experiment.
   • In the enclosed JavaScript script replace X and Y variables with the minimum resolution you would like participants to have.
   • In the enclosed JavaScript script replace the variable host with the url of your nodeGame server.

9. Click Finish.

10. It is recommended to also create another project called Trouble Ticket with 0 dollars reward for submitting a HIT. In this project, participant can report problems with your experimental HIT, and you will be able to assign them a bonus, if needed. In fact, there is no other way to pay them if, for example, they fail to submit their HIT in time for any reason.

Make sure that your nodeGame server is up and running and properly configured. That is, your auth.settings.js should have authorization enabled and contain the following settings:

// Remote clients will be able to claim an id via GET request
// from the task description page.
claimId: true,

// Validates incoming requests.
claimIdValidateRequest: function(query, headers) {
    if ('string' !== typeof query.a || query.a === '') {
        return 'missing or invalid AssignmentId';
    }
    if ('string' !== typeof query.h || query.h === '') {
        return 'missing or invalid HITId';
    }
    return true;
},

// Stores information about worker in the database.
claimIdPostProcess: function(code, query, headers) {
    code.WorkerId = query.id;
    code.AssignmentId = query.a;
    code.HITId = query.h;
}

Finally, follow this checklist to make sure you got all the details right.

Make sure you are saving a results file at the end of your experimental session. This results file must be saved manually in comma separated value format (".csv") and should contain the following header:

"access","exit","WorkerId","hid","AssignmentId","bonus","Approve","Reject"

Every line should contain the earned bonus, and an "x" either under "Approve" or under "Reject".

Follow the instructions in the README.md of the nodeGame-mturk package to:

• approve/reject each assignment,
• grant a bonus to each assignment,
• assign the qualification

Important! Make sure to assign the qualification also to dropouts.

• Other Readings on AMT

• nodeGame-mturk package

• Home