vrx_2022 phase2_dress_rehearsal - osrf/vrx GitHub Wiki
For the 2022 Phase 2 - Dress Rehearsal, teams will submit their system so that it can be run against previously-unseen scenarios. Teams have already been exposed to all of the tasks that will be present in the Dress Rehearsal scenarios. A single system must be designed to solve any of the VRX 2022 tasks.
The VRX 2022 Task Descriptions and Technical Guide, available at the VRX 2022 Competition website, contain all the details about the tasks and about the environmental envelope expected in the unseen competition scenarios.
To ensure that your system can adapt to previously-unseen scenarios, teams should test their system at least against all released sample tasks.
To help you prepare, we have provided the Phase 2 Task Practice tutorial, as well as eighteen example worlds for practice. These example worlds are representative of the task configurations and environmental conditions specified in the task documentation.
For the evaluation of your Phase 2 submissions, the solutions will be run through each of these example worlds, along with some new configurations that are not released prior to the Phase 2 deadline. The objective of this approach is to allow teams to practice with some of the actual scenarios they will see in the Phase 2 competition, so that they can compare their local performance with the results from the phase. We also introduce new, previously-unseen scenarios, to exercise the solutions' ability to deal with new configurations and conditions.
Some features of docker require extra command line arguments. If your team would like to specify an extra command line argument when executing your submitted docker image, please let the VRX Technical Team know as soon as possible by creating a new issue in the issue tracker. We will deal with these requests on a case by case basis. We will make every effort to be as flexible as we can with the team solutions while providing a fair competition for all.
We do not expect many teams to want or need the flexibility to add additional command line arguments, but in cases where it can allow teams to innovate without giving them an unfair competitive advantage, we want to work to make that happen.
To submit an entry, you must do the following:
- Create a solution to one or more tasks.
- Create a Docker image containing your solution upload it to Docker Hub.
- Create the required files described in the submission format section (below);
- Follow the instructions in the submission process tutorial to submit these files.
We expect to receive three files from each competitor prior to this event:
-
dockerhub_image.txt: Contains the name of the image to be pulled from DockerHub. -
thruster_config.yaml: Defines the WAM-V thruster configuration (see tutorial for examples). -
component_config.yaml: Defines the WAM-V component configuration (see tutorial for examples)..
Important: If you're using a private DockerHub repository, please grant access to the virtualrobotx DockerHub user. Otherwise we will not be able to evaluate your submission.
Before submitting your pull request we recommend that you validate your submission and evaluate it to verify that it works as expected. See the following tutorials for a walk through of this process:
Once you submit the pull request, the VRX technical team will do two things before merging (accepting) the submission:
- Check that the WAM-V thruster and component configuration complies with the configuration constraints described in the VRX Technical Guide
- Check that the DockerHub image is accessible by the virtualrobotx DockerHub user.
Once those two requirements are met, the pull request will be merged and your submission is considered ready.
| Date | Description |
|---|---|
| 2022, March 15, 23:59 PDT | Deadline for submitting your solution |
| 2022, March 16, 23:59 PDT | Deadline for submitting a correction |
| 2022, March 22 | Results published |
note on submitting a correction: we allocate an extra day in the schedule for fixing any problems related with the content of the submission files (e.g.: a noncompliant component or thruster configuration, a typo pointing to the dockerhub image, a problem with dockerhub permissions not letting open robotics to download your solution image). teams are not allowed to modify the dockerhub image after the deadline for submitting a solution.
From the VRX Competition and Task Descriptions, "To be eligible for Phase 3, teams must submit their solution for automatic evaluation as described in the VRX Technical Guide." All teams that provide the three files will move on to Phase 3.
The submissions will be scored by the technical team and the scores will be made available to the teams, but the dress rehearsal score is not counted toward the final score. It is only for helping the teams get ready for the final.