5. Facilitating Reproducibility - openstax/kinetic-learner-characteristics-lib GitHub Wiki

Through Kinetic, our goal is to further open-source, open-science, and reproducible research. To that end, we encourage use of the following tools.

Study Pre-Registration

We encourage you to pre-register your studies.

GitHub

Our internal researchers use GitHub for version control and to make our scripts publicly available.

R and RStudio

The R package you will use is called qualtRics. You can paste the following code snippets (written like this) into any R file to access Qualtrics data.

For beginners, we also recommend using the tidyverse package to make dataframe management code more intuitive and readable. However, installing tidyverse is optional if you would prefer to manage your variables and data frames using base R code.

Installing Packages in R

install.packages("qualtRics")
library(qualtRics)

Here is the qualtRics documentation.

Read in Your Qualtrics Credentials

Once you’ve logged into Qualtrics, click on the circle in the top right corner of the screen. This will take you to your account settings. Then click Qualtrics IDs from the horizontal menu at the top of the screen. This page will contain a box that says API. Inside that box is the Token you need. This token is unique to your account and should not be shared with anyone else.

Copy the token and paste it into a .txt file (e.g., a Notepad file). The file should contain only this token. Save the file in the same folder as the scripts you will be running. Then, from within GitHub, add that filename or filetype to your .gitignore file. This will protect your token and ensure that it is not accidentally exposed on GitHub.

Now you can access the file you just created. We will call it qualtrics_cred.
qualtrics_cred <- readLines("qualtrics_cred.txt", warn=FALSE)
qualtrics_api_credentials(api_key = qualtrics_cred, base_url = "yourorganizationid.yourdatacenterid.qualtrics.com", install = FALSE)

Pulling Data from Qualtrics

First, pull in all surveys connected to your account surveys <- all_surveys()

Now you can filter your surveys by the survey name that you need, which will give you the Qualtrics ID for that survey. We provide an example using the tidyverse below.

your_survey_id <- surveys %>%
filter(grepl("Your Survey Name", name)) %>%
select(id) %>%
unlist(use.names=FALSE)

Then you can run the fetch_survey() command to download all survey responses from Qualtrics.

your_survey <- fetch_survey(surveyID=your_survey_id, verbose=TRUE, label=FALSE, convert=FALSE)

Now you can begin analyzing the data in the your_survey data frame. If you need more help setting up R, or need assistance running analyses on your data, please contact us at [email protected].

Python

The Python package you will use is called QualtricsAPI. To install it you can use the following terminal pip command. Install the Package

pip install QualtricsAPI
To import QualtricsAPI to your python code you can paste the following code snippet.

Importing QualtricsAPI
from QualtricsAPI.Setup import Credentials
from QualtricsAPI.JSON import Parser
from QualtricsAPI.Survey import Responses
from QualtricsAPI.Exceptions import Qualtrics500Error, Qualtrics503Error, Qualtrics504Error, Qualtrics400Error, Qualtrics401Error, Qualtrics403Error

Read in Your Qualtrics Credentials

To read in your Qualtrics credentials it is very similar to how it is done in R. Please refer to that section for how to create qualtrics_cred.txt. To access Qualtrics by using the text file you just created, use the following code snippet:

file = open("qualtrics_cred.txt", "r")
token_survey = file.read()
token_survey = token_survey.split(",")
file.close()
Credentials().qualtrics_api_credentials(token=token_survey[0], data_center='yourorganizationid.yourdatacenterid.qualtrics.com')

Pulling Data from Qualtrics

The following class will help you pull in all surveys connected to your account: class Surveys(Credentials):
'''This is a child class to the credentials class that handles survey functionality for the authenticated user.'''
def __init__(self):
return
def list_user_surveys(self):
'''This method provides functionality to share a survey within a given brand/organization.
:return: a Pandas DataFrame with the user's available surveys.
'''
surveys = pd.DataFrame(columns=['id', 'name', 'ownerId', 'lastModified', 'creationDate', 'isActive', 'nextPage'])
headers, url = self.header_setup(content_type=False, xm=False, path='surveys')\

    `def extract_page(surveys=surveys, url=url):`\
        `''' This method is a nested method that extracts a single page of surveys. '''`\
        `try:`\
            `request = r.get(url, headers=headers)`\
            `response = request.json()`\
            `if response['meta']['httpStatus'] == '500 - Internal Server Error':`\
                `raise Qualtrics500Error('500 - Internal Server Error')`\
            `elif response['meta']['httpStatus'] == '503 - Temporary Internal Server Error':`\
                `raise Qualtrics503Error('503 - Temporary Internal Server Error')`\
            `elif response['meta']['httpStatus'] == '504 - Gateway Timeout':`\
                `raise Qualtrics504Error('504 - Gateway Timeout')`\
            `elif response['meta']['httpStatus'] == '400 - Bad Request':`\
                `raise Qualtrics400Error('Qualtrics Error\n(Http Error: 400 - Bad Request):`\  _There was something invalid about the request._
            `elif response['meta']['httpStatus'] == '401 - Unauthorized':`\
                `raise Qualtrics401Error('Qualtrics Error\n(Http Error: 401 - Unauthorized):\ The Qualtrics API user could not be authenticated or does not have authorization to access the requested resource.')`
            `elif response['meta']['httpStatus'] == '403 - Forbidden':`\
                `raise Qualtrics403Error('Qualtrics Error\n(Http Error: 403 - Forbidden):\ The Qualtrics API user was authenticated and made a valid request, but is not authorized to access this requested resource.')`
        `except (Qualtrics500Error, Qualtrics503Error):`\
            `t.sleep(0.25)`\
            `extract_page(surveys=surveys, url=url)`\
        `except Qualtrics504Error:`\
            `t.sleep(5)`\
            `extract_page(surveys=surveys, url=url)`\
        `except (Qualtrics400Error, Qualtrics401Error, Qualtrics403Error) as e:`\
            `print(e)`\
        `except:`\
            `t.sleep(10)`\
            `extract_page(surveys=surveys, url=url)`\
        `else:`\
            `keys = ['id', 'name', 'ownerId', 'lastModified', 'creationDate', 'isActive']`\
            `lists = Parser().json_parser(response=response, keys=keys, arr=False)`\
            `single_page = pd.DataFrame(lists).transpose()`\
            `single_page.columns = keys`\
            `surveys = pd.concat([surveys, single_page]).reset_index(drop=True)`\
            `next_page = response['result']['nextPage']`\
            `return surveys, next_page`\

    `surveys, next_page = extract_page(surveys=surveys, url=url)`\

    `if next_page is None:`\
          `return surveys`\
    `else:`
        `while next_page is not None:`\
          `surveys, next_page = extract_page(surveys=surveys, url=next_page)`\
        `return surveys`

Now you can filter your surveys by the survey name that you need, which will give you the Qualtrics ID for that survey.

name_of_survey = 'Your Survey Name'
survey_id = Surveys().list_user_surveys().loc[Surveys().list_user_surveys()['name'] == name_of_survey ]["id"].item()

Then you can run the get_survey_responses() command to download all survey responses from Qualtrics.

your_survey = Responses().get_survey_responses(survey_id)

Now you can begin analyzing the data in the your_survey data frame. If you need more help setting up Python, or need assistance running analyses on your data, please contact us at [email protected].