Testing exercise answers - Real-Skill/support GitHub Wiki
Our environment
We provide you with complete runtime environment inside our JenkinsCI docker image. You can use any of the technologies listed below:
- MongoDB
- NPM & NodeJS
- Java Development Kit
- Webdriver
Real browser support
For Webdriver / Protractor usage you can also use Chrome or Firefox browser, to make the best use of the functions that they provide.
How it works
Here is an example of how we run tests on your exercise:
1. User create trial.
Trials can be created from autoinvite link for both private and public tests, while on the other hand, other user can invite you to take on his or even random public test.
2. User should be able to clone (using git version control system), implement and run tests locally by provided commands.
Every scaffolding for specific technology is using certain library to test exercise solutions. According to Realskill policy users should be able to test their code with a single command.
3. When user is done with his answer, he pushes specific branch with answer to our remote.
It is important to note that force pushes are not allowed.
4. When new branch is pushed our server check if this is first answer.
We only allow for one answer per branch, if that is true it is queued in our JenkinsCI - otherwise it is rejected.
5. The status of trial branch is updated to ANSWERED
6. When JenkinsCI is done with all requests that were issued before the user has pushed his answer, we run a build against this specific branch:
To provide your application with all necessary dependencies:
JavaScript:
npm install
If bower.json configuration is found:
bower install
To test the application (main npm task to run tests in your application):
npm test
Your npm test configuration can include any tests as long they doesn't require any external binaries(you can, however use binaries from npm repository).
We encourage you to use Grunt for running your tests, however you are free to do as you like.
Java:
To clean old builds and compile your code run:
mvn clean compile
To test your solutions run:
mvn test
7. Having the results from the tests we provide user with reports for unit and e2e tests as well as tests coverage and jshint report. Specific technologies may have additional reports provided, they are available to see on scaffolding for that technology.
Branch status is updated according to build status:
User pushed answer with all tests passed:
- the build is marked as Success
User pushed answer with any incorrect answer:
- the build is marked as Failed
User pushed answer with correct answer but incorrect jshint and exceeded the threshold:
- the build is marked as Unstable, but RealSkill show this value as Failed
Having all the reports parsed, our server provide you with specific data for the pushed branch, all of that is integrated into RealSkill trial details view.