CTAT Basics - CMUCTAT/CTAT GitHub Wiki
CTAT Basics
Table of Contents
CTAT Interface Overview
The Cognitive Tutor Authoring Tools (CTAT) interface is composed of modular windows that are grouped and arranged to form views. A view (also called a "perspective") is a collection of windows arranged in a specific layout for a specific task. Depending on your task, you may want to see some windows but not others in your view.
By default, CTAT has grouped and arranged the windows that are likely to be useful for the given type of tutor you're working on. This means that when you select a tutor type, your view will automatically switch for you to display to the tool windows most commonly used for the selected type of tutor. To switch views, select a new tutor type from the Tutor Types drop-down box. Your view should update to reflect the change.
Figure: CTAT Example-tracing tutor view
To modify your current view, you can add or remove windows, resize the current windows, and/or position the windows differently in or out of the main CTAT window. (Graph windows cannot be moved out of the main CTAT window.) After customizing your view, the new view will be saved in your preferences file for that tutor type so that opening CTAT again shows the same customized view. You can customize the view for each of CTAT's tutor types.
Tutor Types
CTAT enables you to create two main types of tutors: Example-tracing Tutors, which can be created without programming, but require problem-specific authoring; and Cognitive Tutors, which require programming a cognitive model of student problem solving, but support tutoring across a range of similar problems.
Figure: CTAT Tutor Types
There are a few factors that may influence your decision regarding which tutor type to build:
- the complexity of the tutor problems (i.e., the problems that students solve as they work with the tutor);
- whether many problems of the same type need to be developed;
- how many alternative solutions paths exist for each problem;
- whether subtle ordering constraints exist on the steps within the tutor problems;
- whether later steps in a problem depend on earlier steps; and
- whether in-house expertise in AI programming is available to develop a cognitive model.
If you are new to the process of creating a tutor, we recommend that you start with an Example-tracing Tutor.
For more information on the different types of tutors, see the Tutor Types page on the CTAT web site.
Example-tracing Tutor
An Example-tracing Tutor is based on a single problem representation stored in a behavior graph (a BRD file, or behavior recorder data, file). You create a behavior graph by first defining the problem's initial state in the student interface and storing the initial state in the "start state" (that is, by creating the first node in the behavior graph. Then, using CTAT's built-in Behavior Recorder, you demonstrate problem-solving steps in the student interface, creating links and states that correspond to each step. (See Example-tracing Tutors for more on creating an Example-tracing Tutor.)
An Example-tracing Tutor works by comparing student actions to steps recorded in a graph. If the student action is represented in the graph as a correct action, the tutor marks the student action as correct; if the student action is represented in the graph as an incorrect action, or if the student action is not in the graph, the tutor marks the student action as incorrect. In addition to storing whether or not a student action is correct, the behavior graph stores a representation of step order, hint messages, error feedback messages, and other parameters. For more information on behavior graph (BRD) files, see CTAT Behavior Graphs.
By default, the following tools are shown when the Example-tracing Tutor type is selected:
- Graph
- Group Editor
A Graph window is the only tool window required to create an Example-tracing Tutor. In addition, you will need a student interface built in HTML (or Java or Flash) in which to demonstrate behavior. (The Group Editor allows you to specify whether steps can be completed in any order or sequential order.)
To start creating an Example-tracing tutor, see Example-tracing Tutors, or try the CTAT tutorial on creating an example-tracing tutor for the domain of fraction addition.
Cognitive Tutor (Jess)
A Cognitive Tutor is based on a rule-based cognitive model of student problem-solving knowledge (modeled in Jess). This cognitive model is commonly applicable to various problems in the same domain. You create a cognitive model by writing production rules that characterize the variety of strategies and misconceptions students may acquire. (Cognitive Tutors are sometimes are also called "Production Rule Tutors", a reference to the production rules you create.)
During problem solving, the Cognitive Tutor follows the student in a process called model tracing. In model tracing, the tutoring system maintains a model of problem solving that is 'traced' (compared) against the student's actions. Feedback during problem-solving is given based on current state of the model (also called working memory) and the rules that represent student cognition and action.
By default, the following tools are shown when the Cognitive Tutor (Jess) type is selected:
- Graph
- Conflict Tree
- WME Editor
- Jess Console
Figure: CTAT Cognitive Tutor (Jess) tutor view
These tools are explained in detail in the section Cognitive Tutor (Jess) Tools.
Cognitive Tutor (TDK)
Note
The Cognitive Tutor (TDK) tutor type is not supported in this version of CTAT.
A Cognitive Tutor (TDK) is a tutor based on the TDK (Tutor Development Kit) production rule language. This tutor type requires access to a LISP rule engine, which is not included with CTAT.
The TDK modeling language was used to develop the Algebra and Geometry Cognitive Tutors.
Simulated Student
Note
The Simulated Student tutor type is not available in this version of CTAT.
Simulated Student, or Sim St, is a CTAT module that learns cognitive skills from model solutions demonstrated by human problem-solvers.
Using Sim St, an author demonstrates solutions to problems in a domain using a student interface connected to CTAT. The Simulated Student observes those demonstrations and generates a set of Jess production rules that replicate the problem-solving steps demonstrated. These production rules are generalizations of the observed behavior. If these generalizations are correct—i.e., are correct implementations of the actual task that is being taught—then they can be added to the cognitive model as correct rules. If the generalizations are incorrect but "plausible" (in the sense that a student might adopt the incorrect conception), then they can be added to the cognitive model as buggy rules. Both correct and buggy rules would form the cognitive model of a Cognitive Tutor, allowing the tutor to perform its usual functions, namely, model tracing and scaffolding.
More information on Simulated Student can be found at the project home page, http://www.simstudent.org/.
Authoring Modes
CTAT has three authoring modes for creating tutors. Since tutor development is inherently an iterative process, you will probably switch modes often while creating a tutor.
Figure: CTAT Authoring Modes
Set Start State
In Set Start State mode, you define the initial state for the tutor problem by interacting with the student interface. Each problem has an initial start state, which is shown as the first state in the behavior graph. For example, the given fractions (1/4 and 1/6) are defined as the initial state in the sample Fraction Addition tutor. The Set Start State mode is the default mode for all new behavior graphs.
You must define a start state for your graph before you can demonstrate steps for the problem. With your student interface connected to an empty graph window, you enter or select values in the student interface, then create the "start state" node in the graph.
You can modify the initial state of the problem at any time during authoring. To enter Set Start State mode, select Set Start State from the drop-down box at the top of the CTAT window. The tutor state will switch to its initial state, allowing you to enter or modify values in the student interface that will be shown to the student.
Caution
Changing the start state for an existing graph does not ensure that new values you enter for the start state will be consistent with values throughout the rest of the graph. For example, if you change the given values in the fraction addition tutor, the other steps in the graph will still refer to the solution of the old problem (fraction addition with the old given values). If this is the case for your tutor, edit the rest of the tutor so that the values are consistent, or re-create the behavior graph (useful if it's a short problem with few steps).
Demonstrate
In Demonstrate mode, you demonstrate student problem-solving by interacting with the student interface. Interactions you perform in the student interface will be recorded in the behavior graph connected to the student interface. For each interaction, the Behavior Recorder creates a new link between the current node and new child node.
Any sequence of demonstrated actions will be recorded in the graph as a single branch. To create new branches (for example, from a point in the middle of an existing branch), click a state in the graph--the student interface will update to reflect that state--and demonstrate a new path.
Test Tutor
Test Tutor mode previews how your tutor will behave when interacting with students. In this mode, you can play a student attempting to solve a problem. You can also test the tutor, or make changes to the graph and/or production rules.
CTAT Preferences
CTAT preferences control the behavior of the Authoring Tools environment. Set or change your default "CTAT workspace" at the top of the preferences window; the other preferences are divided into functional categories:
You can change the default settings of these preferences to define how you want the tools to operate. Open the CTAT Preferences window by choosing Tools > CTAT Preferences from the menu. All preference settings are global and changes take effect immediately when you click the OK or Apply buttons.
Figure: CTAT Preferences window
CTAT Workspace
This setting defines the folder in which you store your tutors. Typically, you would set your workspace to be the CTAT folder of your synced folder for Google Drive or Dropbox.
- Select a workspace: The default top-level folder for files created while authoring. Relative paths are relative to the folder from which the current student interface was loaded. If the value is empty or no interface is loaded, the location is relative to the current working directory.
General
These options define the general behavior of the Authoring Tools.
- Show Tutor Type Save Dialog: If enabled, attempting to save a graph with a new Tutor Type will prompt the display of a confirmation dialog. Enabled by default.
- Log Replay Tutors Folder: The default top-level folder for tutor files (student interfaces and behavior graphs) stored locally to support the Replay from Log and Replay from DataShop Export features. If the value is a relative pathname, its relative to the current directory where the authoring tools (or Tutoring Service) was started. The value "." (dot, by itself) indicates this directory.
Graph Window
These options define the default settings for editing graphs in the Graph window.
- Use Group Dragging: If enabled, dragging one node in the graph will drag all child nodes; otherwise, only the one node will move. Enabled by default.
- Allow Changing Preferred Path: If enabled, links in the graph can be designated as preferred, customizing the preferred problem-solving path. Enabled by default.
- Enable Link Traversal Counts: If enabled, the Behavior Recorder will count link traversals in demonstrate mode, and display them in the graph. Disabled by default.
- Always Link Similar States: While demonstrating an alternate path, you might demonstrate a step that matches another step already recorded in the graph. By default, the Behavior Recorder will always ask whether to link to a similar existing state, or to create a new state. If this preference is enabled, the Behavior Recorder will never prompt you about whether or not to link states when it detects that the current state of the student interface matches an existing state in the behavior graph; instead, the Behavior Recorder will always link the similar states. Disabled by default.
New Graphs
These options define the default settings to be used for all new behavior graphs you create. For example, if you prefer that all new graphs allow the student to perform steps in any order by default, you would enable the Unordered Mode option.
Important
Individual graphs can have settings that override these preferences if desired. To change any of these settings for a single graph, see CTAT Behavior Graphs.
- Case Sensitive Exact Matching: If enabled, student input during tutoring must match the case of the input recorded in the graph for the input to be considered correct. This setting applies to Exact Matches only; other types of input matching (e.g., wildcard, regular expression) are not affected. Disabled by default.
- Unordered Mode: If enabled, the top-level group of the behavior graph is unordered. In a graph with no other ordering constraints (i.e., groups), this means that a student can perform the actions recorded in the graph in any order; otherwise the actions must be performed in the order shown in the graph, from start state to done state. See Ordering Constraints for more information. Disabled by default.
- Feedback Policy: Determines if and when tutor feedback (hints and correct/incorrect indications) is displayed to the student.
- Show All Feedback: the tutor displays feedback immediately as the student works through each step. Enabled by default.
- Delay Feedback: the tutor will display feedback only when the student presses the Done button, and then only if the problem is incomplete because some steps are incorrect or left undone. No hints will be displayed, but correct/incorrect indications will be shown for all steps done so far.
- Hide All Feedback: the tutor never displays feedback.
- Hide feedback but require all steps: no feedback will be displayed, but if the student has not completed all steps, pressing the Done button displays a message instructing the student to complete the steps. This can be useful, for instance, in quizzes or tests.
- Hide feedback but enforce constraints: no feedback will be displayed, but ordering and traversal minimums and maximums are enforced.
- Hint Policy: Determines how hints are chosen in example-tracing tutors.
- Always Follow Best Path: the tutor's choice for a hint will depend on the behavior graph and ignore any prior error or current selection. Enabled by default.
- Bias Hints by Current Selection Only: the tutor's first choice for a hint will depend only on the user interface component presently selected. If no component is selected, default to show the hint on the best next step in the behavior graph.
- Bias Hints by Prior Error Only: the tutor's first choice for a hint will depend only a student's immediately-preceding error. If no component is selected, default to show the hint on the best next step in the behavior graph.
- Use Both Kinds of Bias: the tutor will first try to choose a hint that addresses a student's immediately-preceding error, then try for a hint on the user interface component presently selected, and finally default to show the hint on the best next step in the behavior graph.
- Lock Widgets on Correct Action: If enabled, a student's correct input to a component will lock that component. Otherwise, the student is able to change a component's value after a correct answer. Enabled by default.
- Highlight Right Selection: If enabled, a message such as "Please work on the highlighted step" will be shown to the student when the student interacts with a widget not specified on the next correct action link in the graph and the widget that the student should be working on is highlighted. This message can be changed for an individual graph by selecting the Edit Out of Order Message option from the Graph menu.
Note: Disabling this feature may be useful in cases where highlighting the correct widget gives away the answer (e.g., when the correct widget is a button). For unordered steps, the next correct action link is the next link within the current unordered group that is both preferred and untraversed.
Java Tutors
These options define the default settings for Java student interfaces. These settings apply to all Java tutors run in the Authoring Tools environment.
- Use Login Window: If enabled, a CTAT log-in window will be presented to the student before they can use the tutor. The log-in window prompts for a username and password for the purposes of logging, though only username is required. Neither username or password is authenticated. Using the log-in window in combination with the Log to Disk preference ensures that the student-entered username will be used in the file name of the log file. Disabled by default.
- Confirm Student Quit Question: Defines the text displayed on a confirmation dialog when the student tries to quit a Java tutor. If empty, then no confirmation dialog is shown.
Logging
These options define how logging of student-tutor interactions should work. These settings apply to Java and Flash tutors run in the Authoring Tools environment, and Flash tutors run in the CTAT Tutoring Service environment. HTML tutors should not rely on these preferences for logging.
Note:
For Flash tutors, these options can be set/over-ridden by parameters passed to the tutor at run-time.
- Log to Disk: (Java tutors only) If enabled, log messages are generated by the Java server and recorded in disk files. The location for storing these files is determined by the Disk Logging Directory preference. If the Use Login Window preference is enabled, disk logs will be named using the student's username as the first part of the file name; otherwise, the system log-in name will be used in the file name. Disabled by default.
- Enable Logging Service: (Flash tutors only) If enabled, log messages received by the Tutoring Service from the tutor are saved to disk. The location for storing these files is determined by the Disk Logging Directory preference. Enabled by default.
- Disk Logging Directory: (Java and Flash tutors only)The path name for the directory in which disk logging files will be stored. This can be an absolute or relative path (relative to the current directory where the authoring tools or the Tutoring Service was started). Set to "." by default.
- Log to Remote Server: (Java tutors only) If enabled, log messages are generated by the Java server and sent to a logging server URL. Disabled by default.
- Forward Logging Service Data: (Flash tutors only) If enabled, log messages received by the Tutoring Service are also forwarded to a logging server URL. Enabled by default.
- Logging Server URL: URL of the logging server. For PSLC experiments, this is typically a LearnLab server.
- Enable Author Logging: (Java and Flash tutors only) If set, then author actions are logged when logging is turned on. Disabled by default.
Jess Tools
These options affect the Jess tools.
- Cognitive Model Folder: The default location for CTAT to check for cognitive model files. Unless explicitly set, this folder is set to the folder from which the current Java student interface was loaded, or the folder from which the last behavior graph was loaded.
- Chain: Turns on chaining in the Jess Console. Enabled by default.
- Firable: Turns on Firable options in the Jess Console. Enabled by default.
- Break on Exceptions: Break when an exception occurs in the Jess Console. Enabled by default.
- Maximum Depth of Tree: The maximum number of times a rule can be chained to another in the Conflict Tree. Default number is 5.
- Maximum Number of Bug Rules: The maximum number of bug rules displayed in the Conflict Tree. Default number is 4.
- Use Salience: If enabled, the production system will interpret salience defined in production rules. Enabled by default. See the Jess Wiki page on salience usage tips for more information.
External Tools
Default preferences for external systems.
- Use Dialog System: Enables communication with a natural language system. Disabled by default.
- Invoke Browser on External URL: Invokes the computer's default browser whenever the student traverses a link marked with a URL in the Attach Dialogue dialog. The URL is formed by combining the External URL for Link Traversal (defined below) and link-specific data. Disabled by default.
- External URL for Link Traversal: The base URL used for links that specify invoking a web browser. This URL is only valid if the Invoke Browser on External URL option is turned on. Disabled by default.