Submissions - simula/biomedia-2020 GitHub Wiki

To submit please zip the submission files and send the zip archive via regular e-mail to the task organizers Pål Halvorsen (paalh at simula.no), Steven Hicks (steven at simula.no) and Michael Riegler (Michael at simula.no).

The subtasks are the following:

  • Prediction of motility - id: motility. The predictions must be made using the dataset supplied with the task (visual features, videos, participant meta-data). The goal is to maximize the algorithm's performance in terms of prediction accuracy. The predictions should be made over three-fold cross-validation using the split supplied in the test dataset. For more information about cross-validation, you can read about it here.

  • Prediction of morphology - id: morphology. The predictions must be made using the dataset supplied with the task (visual features, videos, participant meta-data). The goal is to maximize the algorithm's performance in terms of prediction accuracy. The predictions should be made over three-fold cross-validation using the split supplied in the test dataset. For more information about cross-validation, you can read about it here.

  • Unsupervised sperm tracking - id: tracking. Submissions to this task are quite open. We leave it to you, the participant, to deliver the information in the way that you see fit.

  • Sperm tracking support tool - id: tool. For this task, we ask the participants to submit a GitHub repository which contains their developed piece of software. A link to the repository should suffice.

Submission format

Submissions should be submitted in separated files with the following names, depending on the subtask:

 gc20bio_GroupName_SubmissionName_SubmissionId.csv

GroupName: the name of the participating group, maximum 12 symbols (example: spermwizards)

SubmissionName: the id of the submission, maximum 9 symbols, one of the following ids: motility, morphology, tracking or tool (example: motility).

SubmissionId: the id of the submission, maximum 15 symbols (example: ver1)

Examples: gc20bio_spermwizards_motility_ver1_2000.csv

For all submissions regarding the prediction, the submission file must be generated by evaluating the algorithm over three-fold cross-validation. The submission file should consist of predictions made on each of the three folds.

Motility subtask (semi-colon-separated)

For the motility task, the .csv file should contain the video ID, percentage of progressive sperm, percentage of non-progressive sperm, and percentage of immotile sperm. The file should be ordered as shown below.

video id;% progressive sperm; % non-progressive sperm; % immotile sperm

Example for the motility subtask:

1;0.9;0.09;0.01

12;0.2;0.41;0.39

...

Morphology subtask (semi-colon-separated)

For the morphology task, the .csv file should contain the video ID, percentage of sperm with head defects, percentage of sperm with midpiece defects, and percentage of sperm with tail defects. The file should be ordered as shown below.

video id;% sperm with head defects; % sperm with midpiece defects; % sperm with tail defects

Example for the efficient morphology subtask:

12;0.87;0.23;0.64

25;0.79;0.23;0.79

...

You can find sample submissions under the sample_submissions directory in the official GitHub repository.