Raw data: https://docs.google.com/document/d/1hVC5qEUjFjoF1z6XcBO2tKfT9ZOR1PH0rnFqXaS_J_E/edit?usp=sharing (access will need to be granted)

Testing the Initial Teacher Web-page Prototype

(see wire frame wiki to see prototype screenshots)

User Testing Sheet for Marvel Prototype

Prototype Tasks

• Task 1: Approve Tiffany T.’s comment.
• Task 2: Reject approval of Bryan D.’s comment.
• Task 3: Star Tiffany T.’s comment.
• Task 4: Navigate to the Maths section of the application.
• Task 5: Reject approval of Bryan D.’s comment.
• Task 6: Star a comment in the Maths section.
• Task 7: Go to English and then create a new tab for the class.

Additional Questions

1. How did you find navigating through this technology? (on a scale of 1 = super easy to 5 = extremely hard)
2. Was the colour scheme relevant to the target demographic (children and teachers)?
3. What do you think “starring” a students post would do?
4. Do you understand the difference in sections (review or live)? What did you think they represented?
5. Do you think it is necessary for teachers to have to approve messages the students try to upload?
6. What do you think the stars next to the student’s name represent?
7. Do you have any suggestions or improvements for this prototype?

Results of User Testing

Observations

User 1-3 completed all tasks successfully (3/10/2018)

• User 3 had some confusion with Task 3 on whether to click the star next to the child's name or the star at the bottom of the post.

Answered Questions

1. How did you find navigating through this technology? (on a scale of 1 = super easy to 5 = extremely hard)

General Comments: All users found it super easy (1) on the scale of 1-5.

2. Was the colour scheme relevant to the target demographic (children and teachers)?

General Comments: All users agreed that the colour scheme was relevant to the target demographic.

3. What do you think “starring” a students post would do?

General Comments: All users somewhat to fully understood the usage of "starring" a child's post.

4. Do you understand the difference in sections (review or live)? What did you think they represented?

General Comments: All users understood the teacher would have some sort of influence over the children's posts before uploaded.

5. Do you think it is necessary for teachers to have to approve messages the students try to upload?

General Comments: One user agreed that the teacher should filter the children's comments. Whilst the other responses commented that it might be a lot of work for the teacher to keep up with approving the comments and it might be easier to have all comments live and the teacher can choose to go through the comments and delete any they think are inappropriate. Another user commented that the team might get more of a lesson to kids about social media if we allow unfiltered comments, because the teacher could intervene and start the conversation about how they utilized social media in a negative way.

6. What do you think the stars next to the student’s name represent?

General Comments: The majority of user's did understand what the stars besides the student's names were however did comment that they could be easily confused with the starring a student's comment. The rest of users could not figure out what the stars meant.

7. Do you have any suggestions or improvements for this prototype?

General Comments: The main suggestions for improving the prototype were including profiles for the students, comments on posts and more interaction with the teacher and student through Digilit. It was also suggested that the students get more freedom to post whatever they want and also incorporating more elements of "the public image".

Summary/Analysis

The main insight gained form this user testing session was to make the system more educational on how to use social media through incorporating more elements that emulate social media. To better represent the public nature of content posted on social media, a public screen was incorporated into the design that would present the top comments from the general forum to the entire school. All students from every grade could then star comments to show how many people a post can reach. Even if the star button was spammed by one user, this would still be a representation of the potential virality of content and the loss of control once it is submitted.

To make the system more educational and provide the students with feedback, a input field for comments and feedback was incorporated for when a teacher decides to delete a comment, sending a message to a student explaining why their comment was deleted. This would hopefully allow students to gain an understanding of what is appropriate to post. The teacher being able to delete comments was critiqued as this doesn't accurately represent social media. However, as this system is aimed at training children on how to appropriately use social media, we believe this is an important feature for students to understand what is appropriate. However, from this feedback, we also realised it's important for children to be able to learn by mistake and see the real life consequences for their actions if they were to post something inappropriate. This involved incorporating into the design a review toggle switch which allows teachers to let all comments that are submitted by the students to immediately go to the public display board. This means trust would have to be built between the teacher and the students before switching off reviewing, and this trust could be violated if students were to post inappropriate content and may experience the ramifications of this.

Another feature we decided to alter was the stars that children received besides their name, this seemed to be confusing to many users. The team decided to change the stars besides the child's name to trophies, which indicate when the child gets over a certain number of likes on their comments.

This feature indicates that interacting online does have consequences. Users also suggested to add the ability to add comments on posts, this could possibility be implicated in the future, however for this prototype the team established it was not necessary to convey our point. In future iterations, however, comments may be a good way to encourage engagement with DigiLit and spark healthy debate. The team will conduct a another user testing round on our improved prototype to get some more feedback before the final presentation.