User Testing - adaptive-learning/flocs GitHub Wiki

Closely watched users

Characteristic: over 70 years old man, nearly without programming experience (last 50 years, but he remembers punched cards), basic computer skill.

he didn't know what to do (what is the purpose of the toolbox, what he should do with the maze, ...)
he completely ignored the instructions
even the first task was probably too difficult for him (he needed about 4 attempts to solve it)
he ignored "difficulty rating" dialog text and always clicked "Easy" although I it wasn't easy for him (as a consequence, he got difficult tasks, even though he was still very confused and would benefit from easy tasks)
he repeatedly tried to put the "body" of the loop/condition below the block (instead of inside)
he repeatedly tried to put conditions inside the body and commands inside the head
for a long time he thought that he can solve a part of the problem (e.g. first half of the path) by some program, than delete it and solve the second part by another program (he was really confused by "returning back")
he often thought that the program should do something different than what the program does
at one point there was a bug and the program ignored his "Run" clicks (and after the reload, he was given different task)
he was not sure about the robot orientation and ask me several times which direction is meant by "Step forward"
he was not able to recognize the "treasure chest" icon in the "not at the field with [treasue chest]" block

Characteristic: 55 years old man, without any programming experience but technically skilled thinking.

he read the instructions, but the information about robot getting to the treasure box was pretty clear without the instruction
he understood principle of creating program but didn't know how to construct it (failed to understand drag and drop)
was clicking on everything
tried to move robot by clicking on the blocks
put blocks on each other, next to each other, but connected them only after advice
he thought the program triggers by connecting blocks instead of clicking on RUN
he put blocks for turning in the row without "move forward" and thought the robot will move that way
didn't understand that the "move forward" will move the robot in the direction of its head
absolutely cannot watch both maze and highlighting blocks at the same time when programme is triggered

Characteristic: 20 years old lady, without previous programming experience