6.1.2 Think Aloud Protocol - nus-mtp/sashimi-note GitHub Wiki

Think Aloud Protocol

Goal

The purpose of this study is to investigate the user's experience when creating a document with Sashimi Note.

Method

The study were conducted with 3 participants on 14 April 2017. The study were conducted on sashimi-note v0.1.4. Each study span around 25 to 50 minutes.

Before the study

The study is conducted by requesting the participant to duplicate several parts of a document into Sashimi Note. These documents will be reviewed by the experimenter before the study.

  1. The document is chosen so that the participants will have to carry out these actions:
    • create and rename a file
    • writing basic markdown syntax: such as headings, paragraph, list and text formatting
    • writing mathamatic notaion with either LaTeX or ASCII Math
    • writing codes in code block
  2. After the document creation part, participants were also asked to:
    • switch between different view mods such as Pages, Slides and HTML mode.
    • print/produce a PDF format of the document

Finding

Findings are shown chronologically:

A.1 Commonly faced problems

Participants:

  1. felt lost when they first opened/see the application.
  2. needed constant guidance with using the application during the early part of the study.
  3. got lost in the features list provided.
  4. struggled to track the content between the Viewer and Editor
  5. struggled to find out how to print the document.
  6. expected to be able to right click on the File Manager to rename, print or download a file.

B.1 Commonly welcomed features

Participants:

  1. quickly discover the double click to open file/folder mechanism in the File Manager.
  2. felt happy when they were able to create some mathematics notation by themselves using ASCII math.
  3. managed to locate and switch between different view mode (split, editor-only, viewer-only) quickly.
  4. managed to locate and swtch between different file format (Pages, Slides, HTML) quickly.

Discussion

A.2 Why the commonly faced problems?

  1. The reason they got lost could be could be due to the lack of welcome screen. This application is completely new to them, therefore, some kind of guidance or focus point at the beginning would be necessary.
  2. Although Sashimi Note supports the CommonMark syntax (the most commonly used markdown syntax), all participants do not use markdown frequently. Therefore, it took all participants some times to get used to learning and using the application. However, they reduce their reliance on the features list towards the end of the study.
  3. Althought the features list are divided into different section, it could be hard for the user to navigate around the document as the document is very long. A table of content that is always accessible to the user could helps them in the navigation.
  4. The scroll between the editor and viewer mode is not synchronised automatically. User need to manually trace the markdown code on the left to find out the corresponding element in the Viewer. This process is tedious and frustrating especially in a long document like the feature list.
  5. The File Manager behave and look very smilar to the native OS's file manager. However, most native file manager will show a context menu upon user's right click. However, the behaviour shown by Sashimi Note's file manager does not match with the OS's native file manager's behaviour.

B.2 Why the commonly welcomed features?

  1. Possible reason why user were able to quickly get used to the double click mechanism of the File Manager is due to its resemblance of all native OS's file manager.
  2. Traditionally users will rely on using Microsoft Word Equation tool or similar tools to find and insert different math symbols together. However, ASCII Math allow these to be done solely by typing out the math in plain text.
    • This however do have some learning curve for more complex mathematic notation. Most participants will either need to look up the feature list or search online for the correct syntax to use.
  3. The icon used to represent the the view mode are properly highlighted. The icon chosen also resemblance the concept of edit and view.
  4. The menu to select different file format is also very accessible as there are not many other buttons available for the user to click on the navigation bar.

Post study

The problems are taken into consideration so that the usability of Sashimi Note can be improved in the next development phase.

Results

⚠️ **GitHub.com Fallback** ⚠️