Coderunners zone testing - UQ-eLIPSE/shifoo-docs GitHub Wiki

This doc outlines steps to test the coderunners deployed as triton instances. If you want to test the questions locally, see scripts/runLocalTests/README.md. This might be helpful to test only the logic of the tests, and not if the coderunners are functional

In the shifoo-questions repository, head to scripts folder and run yarn or npm i There are generally two ways of testing the coderunners If you want to test an individual test case, i.e. python_gui3 or just an individual zone, testAnswers.js is recommended. If you want to test multiple zones at once, the testAnswersWithZones.js is recommended.

testAnswersjs

Setup

After the imports there should be two variables JAVA_CODERUNNER and PYTHON_CODERUNNER. Simply change this to the zone you want to test. i.e.

const JAVA_CODERUNNER = "https://shifoox-java22x-001.uqcloud.net/run";
const PYTHON_CODERUNNER = "https://shifoox-py312x-001.uqcloud.net/run";

You can also test load balancing with by not specifying number

const JAVA_CODERUNNER = "https://shifoox-java22x.uqcloud.net/run";
const PYTHON_CODERUNNER = "https://shifoox-py312x.uqcloud.net/run";

This would pick a coderunner at random to test

Running

node testAnswersWithZones.js will run all the test case in the repository. This is also dependent on which branch you're currently in. If you only want to test java OR python test cases, you need to comment out line 84 and change the includes parameter. i.e. If I want to only test java test cases:

const allQuestions =
  params.length > 2
    ? params.slice(2)
    : // : repo.filter((a) => a.includes("python") || a.includes("java"));
      repo.filter((a) => a.includes("java"));

If you want to test specific test cases, simply append the file name as a commandline argument. i.e. If I want to test python_gui3 test case

node testAnswers.js python_gui3
node testAnswers.js python_while_loop_menus_1 python_while_loop_menus_2 # Testing multiple files

If a test failed because of the zone and you want more information, add a console.log ~line 110, just above ❌ ${question} failed!. i.e.

    if (res.payload.failed > 0 || res.payload.exitcode === 1) {
      console.log(res.payload);
      console.info(`❌ ${question} failed!`);
    } else {
      console.info(`✅ ${question} passed!`);
    }

testAnswersWithZonesjs

Setup

Just after the imports, there are two arrays codeRunners and javaCodeRunners. Populate these arrays with the zones you want to test. i.e.

const codeRunners = [
  'https://shifoox-py312x-001.uqcloud.net/run',
  'https://shifoox-py312x-002.uqcloud.net/run',
  'https://shifoox-py312x-003.uqcloud.net/run',
  'https://shifoox-py312x-004.uqcloud.net/run',
  'https://shifoox-py312x-005.uqcloud.net/run',
];

const javaCodeRunners = [
  'https://shifoox-java22x-001.uqcloud.net/run',
  'https://shifoox-java22x-002.uqcloud.net/run',
  'https://shifoox-java22x-003.uqcloud.net/run',
  'https://shifoox-java22x-004.uqcloud.net/run',
  'https://shifoox-java22x-005.uqcloud.net/run',
];

Running

node testAnswersWithZones.js

This will test two test cases (python and java) in the script for all the zones.