PC2V9 Run Execution - pc2ccs/pc2v9 GitHub Wiki

Several kinds of frames (for example, the Admin and the Judge view frames) contain a "Runs Pane" displaying submitted runs (also called "submissions"). When a RunsPane is first created, as part of its initialization it creates three subordinate frames: an EditRunFrame, a ViewJudgementsFrame, and a SelectJudgementFrame.

When the SelectJudgementFrame is first created, it is initialized with a new instance of a pane of type SelectJudgementPaneNew. This SelectJudgementPaneNew object is subsequently initialized via a call from the RunsPane to its setContestAndController() method.

When a submission (Run) is selected for execution (from a RunsPane) by double-clicking or pressing "Request Run", it causes the current SelectJudgementFrame to be populated with the selected run and then made visible.

The SelectJudgementFrame contains an "Execute" button; pressing this button initiates execution of the selected run -- meaning, the submission is compiled, and if compilation succeeds then it is executed once for each Test Case configured in the corresponding problem. Execution is initiated by calling method SelectJudgementPaneNew.executeRun().

Method SelectJudgementPaneNew.executeRun() performs the following steps:

• construct a new object of type Executable
• disable the buttons on the SelectJudgementPane
• invoke the execute() method in the Executable (see below), which compiles the submission and executes it against the Test Case data files, producing an object of type RunTestCaseResult for each data test case. This method returns an object of type IFileViewer which can be used to display output of the execution process.
• fetch the names of the output files (team output and validator output) produced by the Executable.execute() method
• send the fetched file names to an instance of class TestCaseResultsFrame. If no TestCaseResultsFrame yet exists then one is created. Constructing a TestCaseResultsFrame includes populating it with an instance of class TestCaseResultsPane, which when constructed is initialized with an empty 12x7 Results JTable.
• send the submission data (run, runFiles, problem, and problem data files) to the TestCaseResultsFrame by calling TestCaseResultsFrame.setData(). This in turn (indirectly) calls populateGUI() in the TestCaseResultsPane, which populates the pane with the submission data -- including fetching the current test case results (which were generated by Executable.execute()) and loading these results into the TestCaseResultsPane JTable (via a call to getResultsTable()).
• make the TestCaseResultsFrame visible
• fetch the validator status produced by the Executable.execute() method and update the SelectJudgementPaneNew "Validator Recommends" controls if appropriate
• fetch the ExecutionData object produced by the Executable.execute() method and check whether it indicates "compilation error"; if so, update the "Validator Recommends" controls
• update the button status on the SelectJudgementPane as appropriate.

Method Executable.execute() performs the following steps when executing on a Judge (the steps when executing a Test Run on a Team machine are similar although there are slight differences):

• construct a new ExecutionData object
• get the name of the "execution directory" and insure that directory exists (creates it if necessary)
• clears the "execution directory"
• copies all files associated with the submission into the execution directory
• calls method compileProgram(). This method attempts to compile the submission and stores into the ExecutionData object data about the compilation process (e.g. compilation time, compiler stdout and stderr output, as well as the resulting executable program filename and the compiler result code). This method returns a boolean "true" if the submission compiled successfully (or if the language is "interpreted" and hence doesn't need to be compiled), or "false" if compilation failed for some reason.
• If the compilation process failed due to some system error, the execute() method displays an appropriate dialog to the Judge
• if the compilation process failed due to syntax errors in the submission, the compiler stdout and stderr is copied to a pane in the returned IFileViewer
• if the compilation process succeeded, the execute() method does the following:
  • fetches the judge's data files, if any
  • fetches the "stopOnFirstFailure" flag which indicates whether or not all test cases should be executed regardless of whether one or more fail
  • if there is at most one judge's data test case:
    • calls method executeAndValidateDataSet(0) to execute the submission against test case zero. Method executeAndValidateDataSet() returns a boolean "true" if the submission was determined to be correct; false if not.
    • if the submission failed, fetches and saves the validation results from the ExecutionData object
  • if there are multiple judge's test data cases:
    • repeatedly calls method executeAndValidateDataSet(dataSetNumber) (see below) to execute the submission against each test case, until either all test cases have been executed or, if the "stopOnFirstFail" flag was set, until the first time executeAndValidateDataSet() returns "false".
    • for each test case, if executeAndValidateDataSet() returned "false" (i.e., the submission failed on the test case), fetches and saves the validation results from the ExecutionData object.
    • copies the first test data set and corresponding judge's answer file back into the execution directory (because they will have been potentially overwritten by the loop which executes each subsequent test data set).
  • If at least one test case failed, stores the saved validation results into the ExecutionData object
• Adds the Compiler, Execution, and Validator stdout and stderr outputs to panes in the IFileViewer object
• If the submission generated a runtime error, updates the IFileViewer object with the runtime error code
• Returns the IFileViewer object (which may or may not even be used by the caller)

Method executeAndValidateDataSet() performs the following steps:

• calls method executeProgram(dataSetNumber) to execute the executable program (i.e. the result of the compilation process) against the specified data test case. Method executeProgram() returns true if the system was able to execute the program (regardless of what actually happened during execution; for example, if the program generated a runtime error method executeProgram() will still return true); it returns false if an error occurred such that the system was unable to execute the program.
• if executeProgram() returned true then method executeAndValidateDataSet() checks to see whether the problem has a Validator associated with it.
  • If there is a Validator, executeAndValidateDataSet() calls method validateProgram(dataSetNumber), which runs the Validator to check the output of the program execution.
  • if validateProgram() returns true (indicating that the program correctly solved the problem for the specified data test case), method executeAndValidateDataSet() selects true for its return value; if validateProgram() returns false (indicating that the program did not correctly solve the problem), method executeAndValidateDataSet() selects false for its return value. In other words, for the situation where there IS a Validator, the value returned by the Validator (true if the program solved the problem, false if not) becomes the value returned by executeAndValidateDataSet().
  • If there is no Validator, executeAndValidateDataSet() returns false (meaning, "it couldn't determine whether the program solved the problem because there is no Validator available to make the determination").

In summary: executeAndValidateDataSet returns true if (1) the program executed successfully AND (2) there was a Validator for the problem AND (3) the Validator ran successfully AND (4) the Validator returned an indication that the program's output was correct; executeAndValidateDataSet() returns false under all other combinations of conditions.

Prior to returning true or false, method executeAndValidateDataSet() also always does the following:

• Constructs an object of type JudgementRecord and stores into it the data associated with the execution of the run for the specified data test case.
• Constructs an object of type RunTestCase (which really should be called RunTestCaseResult) and stores into it the above JudgementRecord, along with the data test case number and the boolean value about to be returned by executeAndValidateDataSet(), plus additional information about the run execution such as the time the test case was run, how long it took, and so forth.
• Adds the new RunTestCaseResult to the current Run (Submission).

Thus, every call to executeAndValidateDataSet() (one for each data test case) results in a new RunTestCaseResult being added to the Submission.