Data processing: find tests results for the tasks - JetBrains-Research/task-tracker-post-processing GitHub Wiki

Description

This module allows to run the tests to calculate a rating the each task. The rating of the fragment is the percent of the number of correct testing on a set of precalculated input-output pairs for the fragment. You can find the available tasks in the const file (the TASK Enum class). After this step, the data will be divided into folders according to the languages in which the tasks were solved.

Note: the available languages are:

• Java
• Python
• Kotlin
• C++

If the file has an unknown extension, the language will be defined as undefined language.

Note: If you would like to add a new task, you should add a new item to the TASK Enum class. You should also add a new folder to the resources of the project. Finally, you should add a new folder with the task name to the tasks_tests folder. The folder has to contain in and out files for the task tests. Each pair of in_{n} and out_{n} files must contain the input data and expected output data for the nth test for the task.

Usage

Use run_tests method from tasks_tests_handler.py.

Argument	Description
path	path of the directory with files

An example of the root input directory structure before usage:

-root
 --user_N1
 ---task1
 ----user_N1_files
 --user_N2
 ---task1
 ----user_N2_files

An example of the main folder structure after usage:

-root
 --user_N1
  ---python
   ----task1
    -----user_N1_files
 --user_N2
  ---cpp
   ----task1
    -----user_N2_files

After this step each file will contain a TESTS_RESULTS column with an array of float values by the rates of tasks - ith rate corresponds to the ith task from the TASK Enum class.