4. Integration with Qualtrics - openstax/kinetic-learner-characteristics-lib GitHub Wiki

Kinetic powerful integration with Qualtrics helps you make your survey-building experience as seamless as possible.

Not a problem! We’re here to help. The next section will give you all the information you need to get started as soon as possible.

If you have used Qualtrics before, please feel free to skip to the Advanced Qualtrics for Kinetic chapter. There you will find instructions on how to integrate Kinetic and Qualtrics.

To create a new survey project, click the “Create a new project” button at the bottom left corner of your screen. When creating a new project you are given a choice between creating a survey “From scratch” or instead exploring one of the pre-existing templates the platform provides you with.

We’ll walk through how to create a survey from scratch, but if there is a template that fits your project, feel free to use it! Doing so may save you a lot of time.

When creating a new project, you’ll see this prompt: Fill it out, making sure you use a descriptive name for your project so as to avoid confusion in the future. Once that’s done, you’ll find yourself in Qualtrics’ survey builder. We will now review the functionality of the 4 main components of Qualtrics available on the left-hand side of your dashboard, as follows:

Builder Survey Flow Look and Feel

Builder is your default view, as it is the place where you create your survey. Here you will be able to design and structure your survey concept, and ultimately bring it into reality.

First, basic survey functionalities can be found under the “Tools” dropdown menu positioned at the left-hand side of the top of your dashboard. Here you can manage different versions of your survey, add collaborators, import/export your survey, and find more granular options such as survey “Review” (spell check, search and replace, etc.).

While on the right-hand side of the top of your dashboard you can also “Preview” your survey - allowing you to see how it might look when it gets published - and “Publish” it - making it available to the public.

A note on user experience… The preview function is incredibly important, as it gives you access to both a desktop and a mobile view, allowing you to ensure that your survey is compatible with different devices in the way that it’s designed and formatted. Why is this important? To help you reach your audience more broadly, it’s important to consider how different segments of your audience may access and experience your survey, ensuring that their survey experience is seamless no matter the type of device they use.

Now let’s explore “Blocks”, one of the most important tools available in the “Builder” dashboard. Blocks are essentially a set of questions in your survey. In Qualtrics, questions are often separated into blocks to facilitate the display of an entire block of questions (rather than one by one). The image below illustrates what a basic question block looks like in the builder dashboard

Blocks help you display different sets of questions on different pages/screens. This allows you to display your survey in bite sizes, reducing cognitive load and helping your audience go through your survey without getting too overwhelmed. This is an incredible tool to help you carefully design and structure how you want to present your survey. And because they know we love structure, Qualtrics gives you access to additional editing options both on the block level and on the question level - to access them, you can simply hover your mouse over either one of your blocks or questions to see the ellipsis appear On the block level you can find basic block options such as moving, copying, deleting or previewing the block. Very similar options are available for questions, with an additional one that should be considered, the “Page Break”. Adding a page break essentially means that your questions under the same block will not be displayed simultaneously on the same page/screen, but rather will appear in phases (within the same page/screen) once your audience clicks on the arrow to move forward. You can add more blocks by clicking the “Add Block” prompt under an already existing block, and within each block you can click the “Add new question” button to add questions or the “Import from library” to add questions from your library. Qualtrics allows you to import questions from their library or create a question from a long list of options. There are many different options to explore, whether from the default qualtrics options or ones inside the library, so explore and see what fits your project the best!

More editing options become available on the left-hand side of your screen when clicking on the survey questions. Here, you can change the question type, formatting, requirements, and much more. More advanced customization options can be added using the question behavior logic or by embedding JavaScript code. Qualtrics is quite detailed and comprehensive so it is important to explore all the options. Although we won’t be going into much more detail on this topic here, know that If you ever feel lost, you can find a great deal of documentation online that could be of support to your work.

Additionally, to add pictures to your survey questions: Click on the “Click to write the question text”.

Select “Rich Content Editor…” and click on the circled icon

And “Upload a New Graphic”

Adding the Kinetic consent form to your Qualtrics survey is very simple. First, create a question block, like the one below.

Add the following text: “Please read the consent form below and answer the consent question. By selecting that you consent, you agree that the above information has been explained to you and all your current questions have been answered. You understand that you may ask questions about any aspect of this research study during the course of the study and in the future. By selecting that you consent on this form, you agree to participate in this research study.”

Select “Rich Content Editor” and click on the icon highlighted with a circle below.

At the bottom of the text, add the following code:

<iframe width="720" src="https://docs.google.com/document/d/13EcFwMDaNAJjS6E2KTJfu8STkoLvD8QX/edit;embedded=true" height="420"></iframe>

Now your survey question should look something like this:

Because Kinetic does not coordinate parental consent forms for minors, Kinetic researchers can only use data from non-minor respondents (i.e., 18 years of age or older). Be sure that all of your studies include a question asking learners to confirm their age. If they are minors, you should still allow them to continue with the study, but you must exclude their data from your analyses.

One useful feature of Qualtrics is the ability to capture metadata alongside your survey questions, recording participants’ parameters such as device type, operating system, browser type and version, etc. You can add metadata collection as your first survey question, which will be invisible to the end user.

You can capture the total amount of time a participant spent on a particular page by adding a Timing Question to each of your blocks. As with metadata, this question will be invisible to the end user.

This section is dedicated to providing advanced information on the Qualtrics integration with Kinetic. Implementing the below information is essential to ensure that Kinetic works as intended.

The Survey Flow enables you to customize the order in which the participants interact with the elements of the survey, as well as include fields necessary for backend integration with Kinetic. Authentication

• Set the very first block in your survey flow to “Branch on Successful Authentication.”

• Click on “Survey Flow.”

• Click on “Add a New Element Here” (highlighted in green).

• Click on the “Authenticator” option

• Add a new element within the authentication block

Make the following changes to your “Branch on Successful Authentication” block:

• Change your “Authentication Type” to “SSO”
• Unselect “Associate Respondent With Panel”
• Select “Capture respondent identifying info”
• Click “Add Embedded Data”
• Add: research_id = research_id
• Click
• Add: return_to_url = return_to_url
• Click
• Add: is_testing = is_testing
• No changes to “SSO Type” as “Token”
• Change “Encryption Method” to “AES128”
• No changes to Mac Method as “md5”
• No changes to “Leeway” as “300”
• In the “Key” field click “Generate Key”
• (This will be the key you will used when prompted for a “Secret Key” in the Kinetic platform)

Below the “Branch on Successful Authentication” select “Embedded Data” and make the following changes: Click “Add a New Field” and select “Create New Field or Choose From Dropdown” and type in each of the following fields (followed by “Add a New Field”): research_id (collects the visitor’s research ID) return_to_url: ${e://Field/return_to_url}?consent=${e://Field/consent} (sends the visitor back to Kinetic, along with information on whether they consented) is_testing (this indicates whether the visit is from the staging server) consent (whether the participant consented)

Below are the embedded fields that are user-designed fields required to connect Kinetic with Qualtrics. Please make sure you have all of these in your Qualtrics survey.

• research_id
• return_to_url
• consent
• teacher_name - point to the question where you ask for the teacher name
• is_testing -
• return_to_url:

We suggest adding the authentication block at the top of the survey flow (as it’s shown above), but the researcher can choose an alternate location. Consent Flow Below the consent question, click “Add Below” to add a new element Click “Branch” Click “Add a Condition” Select the appropriate consent question and determine the appropriate IF statement Below that branch, make sure to set “Embedded Data” to determine the relationship between the statement and the consent state Repeat the above steps in response to the binary nature of this statement

Finally, at the end of your survey, click on the “End of Survey” element. Make the following changes:

• Click “Customize”
• Select “Override Survey Options”
• Change from “Default end of survey message” to “Redirect to a URL”
• Insert the following URL: ${e://Field/return_to_url}?consent=${e://Field/consent}

Look and Feel gives you extensive options to change how your survey looks. In this section you’re given the tools to change details such as button design, logos, font, etc. Kinetic provides a dynamic base theme that fits most researchers’ needs. This is a minimalistic yet modern style that includes a Kinetic banner at the top of the page. However, feel free to change the theme according to your preferences.

The Survey Options section enables you to customize elements of your project such as language, title, description, how the survey deals with certain responses, security, scoring, etc.

Is your study ready to post to Kinetic? Checklist Add embedded fields Authentication block End of Survey link Add required questions Consent form Age check

Adding Your Study to Kinetic Now that you have a Qualtrics study ready to deploy to Kinetic, it’s time to go to Kinetic to upload your study. Creating a Kinetic Account First, you will need an OpenStax login (free and quick to sign up for). https://openstax.org/accounts/i/login/ Then, contact XXX to add your email address to our list of Kinetic researchers, so that you will have access to the researcher portal when you log in to Kinetic. Logging in to Kinetic Once you are logged in to your Kinetic account and have received researcher access, visit kinetic.openstax.org/studies. You will see a dashboard that looks something like this: This is the researchers’ view of Kinetic studies. These studies are listed by the internal title the researcher has chosen; the title that students will see is set within each study. Click on the “...” on the far right to see the details on any existing study. The below screenshot is from one of Kinetic’s individual difference measures: general science knowledge. This is one of many measures in the Kinetic “library” that you can use data from when you analyze the results from your own studies.

Click on the + icon on the right side of the screen.

This will open up a page requesting all the details about your study.

All fields need to be filled out.

• Title for participants: The title that learners will see on their Kinetic dashboard
• Title for researchers: Your own internal, or working, title for the study
• Duration minutes: How long you expect the study to take, in minutes
• Participation points: Based on the type of task
• Tags: Labels that will help learners sort through potential studies. Choose all that apply to your study.
• Opens at: The date that you want your study to go live
• Closes at: The date that your study will no longer be accessible to learners
• Short description: The study details that will be visible on the study card on the learners’ dashboards.
• Long description: The study details that will be visible when the learner clicks on the study card to learn more.

Once you save the study, you will enter the details on the stages. If you are administering a study only once, you will only have stage 0. If you have multiple stages, you can enter the details for each of them, along with how long of a delay (in days) you want before Kinetic learners can access the next stage of your study.

This is where you will enter information from Qualtrics, to connect your Qualtrics form to your Kinetic study.

• Title of the stage
• Description of the stage
• Survey ID: This is the end of your survey link
• Secret key: This is the key generated within your authentication block in Qualtrics Once you have entered all your fields, click Preview at the top right of the screen and ensure that your survey loads correctly.

It may be helpful to understand how Kinetic learners see your and other studies on their dashboards. For each study, learners see a “card” containing:

• The title you selected for participants
• The short description
• The points assigned for completion
• The expected length of time to complete the study
• The study tags

Kinetic aims to provide an excellent user experience for participants. You can help by focusing on how you can add value to Kinetic participants’ lives.

• Target your study descriptions and title to the interests of Kinetic participants. How will you grab their attention, and how will you make it clear why their participation is important? Questions as titles do particularly well.
• If your study design offers any feasible way of providing personalized feedback, provide it! If you cannot provide personalized feedback, conclude your study with a brief statement on what constructs you were exploring.