The page provides information on the infrastructure setup will be using for final evaluation. It also includes details on team solution submission.

Final evaluation will be run on cloud machines on AWS EC2. Here is the set-up:

• The simulator will be run in a container in its own dedicated machine instance.
• Other separate machines instances will be launched to run the submitted team solutions.
  • Multiple team solutions will be run on one machine instance. For example, UAV1 and UAV2 may run on the same machine instance.
• The team solution communicates to the simulator over a bridge container which will be co-located in the same machine instance as the team solution container.

We will use g3.4xlarge instance type for all machine instances. Simulation will run headless (no GUI) on the cloud machines.

For practice before final evaluation, you can either reproduce our infrastructure setup locally or on AWS EC2. Take a look at the Simulation setup using Docker Compose page on how to reproduce the simulation setup on a local machine. For testing on AWS EC2, we recommend testing with the same AWS instance type as we can not guarantee that the simulation will run properly on a different cloud provider / instance type.

Submission should include:

• One solution docker image
  • Only one solution image should be submitted. The same solution will be used for all vehicles. An argument will be passed to the container to indicate the vehicle it is associated with.
• One yaml configuration file containing all vehicles and sensor payloads to be spawned into simulation.

Please see the Preparing Solution Docker Image page for information on how to prepare your Docker image for submission.

1. Install the AWS CLI tool and configure it with your teams credentials. Unique AWS credentials will be provided to you. To configure aws to use your keys, you can either:

   1. Run the command:

      aws configure

      Enter the keys and keep other settings the default. Default region should be us-east-1.

   2. Use named profiles. Open ~/.aws/credentials and add a new profile:

      [mbzirc_user]
      aws_access_key_id=<your_aws_access_key_id>
      aws_secret_access_key=<your_aws_secrete_access_key>

      Then set the env variable:

      export AWS_PROFILE=mbzirc_user

2. Use the mbzirc_submit.bash script to tag and upload your docker image. Usage:

   bash mbzirc_submit.bash <team_name> <local_docker_image> <local_tag> <upstream_tag> <config_file>

   • team_name: Please use the team name in the credentials provided to you. Must be lowercase letters. This is also the name of your Docker repository.
   • local_docker_image: This is the name of your Docker image. If you type docker image list, you should use the appropriate name under the REPOSITORY column.
   • local_tag: This is the tag associated with your docker image. A common tag is 'latest'.
   • upstream_tag: This is the tag you want associated with your image when uploaded to the repository. This will let you version your docker images. Please specify either, test, dry_run or final_evaluation
   • config_file: Path to your yaml configuration file containing all vehicles and sensor payloads to be spawned into simulation

   For example, if your team name is foo, you have built the example solution docker image called mbzirc_seed:latest, you want to submit this image for dry run, and there is a team.yaml in the current path:

   bash mbzirc_submit.bash foo mbzirc_seed latest dry_run ./team.yaml

   As for submitting your solution for final evaluation:

   bash mbzirc_submit.bash foo mbzirc_seed latest final_evaluation ./team.yaml

   This script will log you into docker, tag your image, push the image to your repository on AWS, and upload your config file to your s3 bucket on AWS.

   Here is an example output of successful submission:

   Logging into docker
   WARNING! Your password will be stored unencrypted in /home/developer/.docker/config.json.
   Configure a credential helper to remove this warning. See
   https://docs.docker.com/engine/reference/commandline/login/#credentials-store
   
   Login Succeeded
   Tagging image mbzirc_seed:latest with final_evaluation
   Pushing to docker repository
   The push refers to repository [200670743174.dkr.ecr.us-east-1.amazonaws.com/mbzirc/foo]
   d55f14d8ed82: Pushed 
   9b61fd5d061f: Pushed 
   6832188e7901: Pushed 
   a58f9c58c302: Pushed 
   5c846eef36c0: Pushed 
   83a234b1bc63: Pushed 
   0f61b971b629: Pushed 
   1d638b9e62f4: Pushed 
   d706a4a928cf: Pushed 
   3097b3556115: Pushed 
   c11f93922563: Pushed 
   ad62eb903a80: Pushed 
   2ef9f14570a9: Pushed 
   c7fa1bd01201: Pushed 
   62db52b6d5e9: Pushed 
   69560be64149: Pushed 
   c5cc56868f26: Pushed 
   ccda132558f7: Pushed 
   267339c207c6: Pushed 
   8e2231cfe414: Pushed 
   1962167633ed: Pushed 
   047aa3d1de06: Pushed 
   df2e8c86e495: Pushed 
   20026237d960: Pushed 
   c3bc5ba26848: Pushed 
   bf8cedc62fb3: Pushed 
   final_evaluation: digest: sha256:af98add961de2c311cfa634f041b999b4f26f3e112a7c6435bf584b8950b9971 size: 5779
   Docker image URL: 200670743174.dkr.ecr.us-east-1.amazonaws.com/mbzirc/foo:final_evaluation
   Uploading config file to s3
   upload: ./team.yaml to s3://cloudsim-mbzirc-logs/foo/config/config.yaml