MLPerf Training - AshokBhat/ml GitHub Wiki
About
- Training Benchmark
Timeline
| Version | Date |
|---|---|
| MLPerf Training v0.5 | Dec, 2018 |
| MLPerf Training v0.6 | Jul, 2019 |
| MLPerf Training v0.7 | Jul, 2020 |
| MLPerf Training v1.0 | Jun, 2021 |
| MLPerf Training v1.1 | Dec, 2021 |
| MLPerf Training v2.0 | Jun, 2022 |
| MLPerf Training v2.1 | Nov, 2022 |
| MLPerf Training v3.0 | Jun, 2023 |
Further information
- MLPerf Training Benchmark Paper - https://arxiv.org/pdf/1910.01500.pdf
Benchmarks (MLPerf v3.0)
| Area | Benchmark | Dataset | Quality Target | Model |
|---|---|---|---|---|
| Vision | Image classification | ImageNet | 75.90% classification | ResNet-50 v1.5 |
| Vision | Image segmentation (medical) | KiTS19 | 0.908 Mean DICE score | 3D U-Net |
| Vision | Object detection (light weight) | Open Images | 34.0% mAP | RetinaNet |
| Vision | Object detection (heavy weight) | COCO | 0.377 Box min AP and 0.339 Mask min AP | Mask R-CNN |
| Language | Speech recognition | LibriSpeech | 0.058 Word Error Rate | RNN-T |
| Language | NLP | Wikipedia 2020/01/01 | 0.72 Mask-LM accuracy | BERT-large |
| Language | LLM | C4 | 2.69 log perplexity | GPT3 |
| Commerce | Recommendation | Criteo 4TB multi-hot | 0.8032 AUC | DLRM-dcnv2 |
See also
- MLPerf Inference
- [ResNet-50]] ](/AshokBhat/ml/wiki/[MobileNet) | [SSD]]