Meeting 20.08.20 - TobiasSchmidtDE/DeepL-MedicalImaging GitHub Wiki

BigBlueButton Meeting Room

What happened so far

Distribution Chexpert Validation / "Test" Set

Our improvements

Last Week: DenseNet121_Chexpert_BCE_E3_B32_C1_N12 !!!DIFFERENT TEST SET THAN THIS WEEK!!!

Test auc:  0.7229323983192444
Test precision:  0.6811800599098206
Test recall:  0.29407998919487
Test f2_score:  0.3317897915840149
Test auc_enlarged_cardiomediastinum:  0.6274955868721008
Test auc_cardiomegaly:  0.827772855758667
Test auc_lung_opacity:  0.70615553855896
Test auc_lung_lesion:  0.6908576488494873
Test auc_edema:  0.7949528694152832
Test auc_consolidation:  0.7065780758857727
Test auc_pneumonia:  0.6720331907272339
Test auc_atelectasis:  0.6417831778526306
Test auc_pneumothorax:  0.7809439301490784
Test auc_pleural_effusion:  0.8473460078239441
Test auc_pleural_other:  0.7237358093261719
Test auc_fracture:  0.6555337309837341

This week:

DenseNet121_Chexpert_BCE_E5_B32_C0_N5_D256_DS9505_LR4

Val auc:  0.7821
Val precision:  0.6381
Val recall:  0.5018
Val f2_score:  0.5282
Val auc_enlarged_cardiomediastinum:  0.674
Val auc_cardiomegaly:  0.8719
Val auc_lung_opacity:  0.7486
Val auc_lung_lesion:  0.7723
Val auc_edema:  0.8416
Val auc_consolidation:  0.7561
Val auc_pneumonia:  0.7319
Val auc_atelectasis:  0.7165
Val auc_pneumothorax:  0.8517
Val auc_pleural_effusion:  0.8765
Val auc_pleural_other:  0.7678
Val auc_fracture:  0.7759

Test auc:  0.7031866908073425
Test precision:  0.8262910842895508
Test recall:  0.3223443329334259
Test f2_score:  0.36712557077407837
Test auc_enlarged_cardiomediastinum:  0.6314862966537476
Test auc_cardiomegaly:  0.7961995601654053
Test auc_lung_opacity:  0.9129188656806946
Test auc_lung_lesion:  0.1287553608417511
Test auc_edema:  0.8984127044677734
Test auc_consolidation:  0.9107493162155151
Test auc_pneumonia:  0.752765417098999
Test auc_atelectasis:  0.7597808241844177
Test auc_pneumothorax:  0.7903760671615601
Test auc_pleural_effusion:  0.9190276265144348
Test auc_pleural_other:  0.9377682209014893
Test auc_fracture:  0.0

DenseNet121_Chexpert_BCE_E3_B32_C0_N12_D256_DS9505_2LR1_LF5_SGD

Test auc:  0.8351618051528931
Test precision:  0.662162184715271
Test recall:  0.3344709873199463
Test f2_score:  0.37121209502220154
Test auc_cardiomegaly:  0.7818036675453186
Test auc_edema:  0.877366304397583
Test auc_consolidation:  0.8826322555541992
Test auc_atelectasis:  0.7445617318153381

Exploration JF Health Github (Chexpert Leadership Board Top 4)

More detailed Wiki papge

Pathologies	Paper	N5 Aug	N5 NoAug	N12 Aug	N12 NoAug	Our Best N5	Our Best N12
Enlarged Cardiomediastinum	-	-	-	0.9184	0.9016	-	0.6314
Cardiomegaly	0.8703	0.8401	0.8704	0.5015	0.6231	0.7818	0.7961
Lung Opacity	-	-	-	0.8101	0.7872	-	0.9129
Lung Lesion	-	-	-	0.9020	0.9226	-	0.1287
Edema	0.9436	0.9031	0.9448	0.5376	0.8592	0.8773	0.8984
Consolidation	0.9334	0.9094	0.9205	0.9127	0.9177	0.8826	0.9107
Pneumonia	-	-	-	0.8921	0.8580	-	0.7527
Atelectasis	0.9029	0.8229	0.7947	0.8404	0.7792	0.7445	0.7597
Pneumothorax	-	-	-	0.7917	0.8265	-	0.7903
Pleural Effusion	0.9166	0.9214	0.9258	0.8316	0.8857	0.8894	0.9190
Pleural Other	-	-	-	0.9203	0.9199	-	0.9377
Fracture	-	-	-	0.9095	0.9497	-	0.0

Other Work

Implementation

• Data Augmentation:
  • Affine Transformations
  • Equalize Histogramm (in progress)
• Recalculation of Metrics to test different thresholds
• Model Saving & Loading
• Classed based uncertainty encoding
• More Learning Rate Scheduler

Discussion Points

• Very odd bugs in our training

Next steps

• figure out whats causing the training bug
• fix it
• train again on augmented data
• train with class based uncertainity encoding
• try upsampling dataset to overcome class imbalance
• try weighted BCE to overcome class imbalance
• find best threshold for precision/recall (plot ROC)
• more visualization GradCAM, PCAM, etc.

Next meeting:

28.08.2020 13:00 Uhr