Existing Research - adaptive-learning/flocs GitHub Wiki

Elo systém a modelování času řešení (Libor Vaněk, DP na FI)
Application of time decay functions and Elo system in student modeling (Pelanek, 2014)
Modeling students' memory for application in adaptive educational systems (Pelanek, 2015)
http://www.fi.muni.cz/adaptivelearning/?a=publications

Radek Pelánek, 2016, pdf

description of Elo and its variants, e.g. multiple parameters, uncertainty, solving times
comparison to other skill/difficulty estimation methods

Elo has advantages over other models for skill estimation - it's simple, flexible, online and achieves reasonably good performance (x IRT assumes constant skill; BKT and PFA are difficult to use in the online setting)
there are several variants of ELO which work with solving time - basic idea: skill/difficulty parameters denote the expected time for the user to solve an average task (and the expected time the task will be solved by an average student), for details see references in the article
we use uncertainty function for difficulty updates, maybe we should also use (another) uncertainty function for skill update. For a new user the uncertainty is high, so update should be high - but unlike difficulties, skills are assumed to change forever, so the uncertainty should not converge to zero. It can also work with time from the last update, for inspiration see Klinkenberg, 2011.

Radek Pelánek, Petr Jarušek, full version, shorter version (Analysis of Simple Model to Problem Solving Times)

timing information as the only measure of student performance (as in Problem Solving Tutor)
model of problem solving times (assumes an exponential relationship between a problem solving skill and time to solve a problem)
including multidimensional skill and learning
how to estimate parameters of the model (e.g. by stochastic gradient descent, incl. pseudocode)
how to deal with cold start problem
evaluation of predictions
automatic detection of concepts

showing predicted problem solving time to user as a motivational element (suitable challenge -> supports a state of flow)
tip for gradual start: create estimates of problem solving times (manually or using some heuristic from the content of the task) and add several fake users to the system with these estimated solving times
another tip for start: fix some parameters at the beginning and start using them only after there is enough data (when the predictions of more complex model become better than the simple model with fixed parameters)

29 concepts, 43 prerequisites (-> DAG)
student model: Bayes network (each concept is known/unknown with some probability)
after each concept (= reading article about the concept), student can self-report that he understands, doesn't understand or don't know (in that case, a quiz is used to determine if he understands)
student can freely navigate between concepts, the system just shows soft recommendation for each concept: already known, ready to learn, not ready to learn (traffic light colors)
already known: p(known | evidence) > 0.7
ready to learn: p(known | evidence) < 0.7 and all prerequisites are "already known"
system also can generate minimum prerequisite knowledge (sequence of concepts) for any concept which is "not ready to learn" but a student wants to learn

possibility of using DAG to model concepts and their relationship
possibility of using Bayes network as a model of student (but the assumption of known|unknown discrete states is not met in our settings)
sometimes, students know what they want to learn; useful feature can be to generate a learning sequence towards a concept selected by a student (minimum prerequisite knowledge sequence)

Robotanik in Problem Solving Tutor
canonical solutions
analysis of difficulty (what makes some tasks more difficult, e.g. length of the solution, number of conditions, recursion depth, length of Robotanik's path, number of canonical solutions)

1:1 tutoring is ideal, but not economically feasible
solution: ITS, problem: usually also high cost (lack of shareable, reusable components)
functions of ITS (how to help a student): assist (help, give hints), show them analysis of their errors, give them suitable problems (outer loop)
propose generic architecture (contains e.g. visualization module, pedagogical module, ... - each module is described)

worlstate (output) vs. codestate (code snapshot)
how to generate hint in any state (worldstate/codestate) using data of students which have successfully solved the task
too many codestates (-> normalization and canonization; or use worldstates)

in future, we should have a hint button (and possibly, if the student is not able to solve the task for a long time, remind him of the hint button)
how can hints be presented in our system? (maybe: partial path to take, a block to use)