⚠️WARNING⚠️, 🔴IMPORTANT❗🔴 There seems to be an issue with GPU allocation for all notebooks using Tensorflow below the 2.5 version. We are investigating the issue.

ZeroCostDL4Mic is a toolbox for the training and implementation of common Deep Learning approaches to microscopy imaging. It exploits the ease of use and access to GPU provided by Google Colab.

Training data can be uploaded to Google Drive, which can be used to train models using the provided Colab notebooks in a web browser. Inference (predictions) on unseen data can then also be performed within the same notebook, therefore not requiring any local hardware or software set-up.

Running a ZeroCostDL4Mic notebook	Example data in ZeroCostDL4Mic	Romain's talk @ Aurox conference	Talk @ SPAOM	NEUBIAS webminar

ZeroCostDL4Mic provides fully annotated Google Colab optimized Jupyter Notebooks for popular pre-existing networks. These cover a range of important image analysis tasks (e.g., segmentation, denoising, restoration, label-free prediction). There are 3 types of implemented networks:

• Fully supported - considered mature and considerably tested by our team.
• Under beta-testing - an early prototype of networks that may not be stable yet.

We welcome network contributions from the research community. If you wish to contribute, please read our guidelines first.

Both fully supported and beta-testing versions of the individual notebooks can be directly opened from GitHub into Colab by clicking one of the respective links in the table below. You will need to create a local copy to your Google Drive in order to save and modify the notebooks. Once opened in Colab, follow the instructions described in the specific notebook that you selected to install the relevant packages, load the training dataset, train, check on test datasets and perform inference and predictions on unseen data.

With the exception of the U-net training data, we provide training and test datasets that were generated by our labs. These can be downloaded from Zenodo using the various links below. The U-net data was obtained from the ISBI segmentation contest.

⚠️WARNING⚠️, 🔴IMPORTANT❗🔴 There seems to be an issue with GPU allocation for all notebooks using Tensorflow below the 2.5 version. We are investigating the issue.

Network	Paper(s)	Tasks	Status	Last test	Link to example training and test dataset	Direct link to the notebook in Colab
U-Net (2D)	here and here	Binary segmentation	Fully supported	27/07/23 ✅ working (IH)	here
U-Net (3D)	here	Binary segmentation	Fully supported	27/07/23 ✅ working (IH)	EPFL dataset
U-Net (2D) multilabel	here and here	Semantic segmentation	Under beta-testing	16/07/23 ✅ working (IH)	here
DenoiSeg	here	Joint denoising and binary segmentation	Fully supported	⚠️ broken (no GPU) (GJ)	Available soon
StarDist (2D)	here and here	Instance segmentation	Fully supported	19/05/23 ✅ working (EGM)	here
StarDist (3D)	here and here	Instance segmentation	Fully supported	07/10/22 ✅ working (GJ)	from Stardist github
Cellpose (2D and 3D)	here	Instance segmentation (Cells or Nuclei)	Fully supported	05/09/23 ✅ working (IH)	here
SplineDist (2D)	here	Instance segmentation	Fully supported	07/10/22 ✅ working (GJ)	here
EmbedSeg (2D)	here	Instance segmentation	Under beta-testing	01/01/23 ✅ working (AR)	here and here
MaskRCNN (2D)	here	Panoptic segmentation	Under beta-testing		Coming soon!
Interactive Segmentation - Kaibu (2D)	here	Interactive instance segmentation	Under beta-testing		Coming soon!

Network	Paper(s)	Tasks	Status	Last test	Link to example training and test dataset	Direct link to the notebook in Colab
Noise2Void (2D)	here	Self-supervised denoising	Fully supported	14/07/23 ✅ working (IH)	here or here
Noise2Void (3D)	here	Self-supervised denoising	Fully supported	14/07/23 ✅ working (IH)	here
CARE (2D)	here	Supervised denoising	Fully supported	31/07/23 ✅ working (IH)	here or here
CARE (3D)	here	Supervised denoising	Fully supported	31/07/23 ✅ working (IH)	here
3D-RCAN	here	Supervised denoising	Under beta-testing	⚠️ broken (no GPU)	here
DecoNoising (2D)	here	Self-supervised denoising	Under beta-testing	07/10/22 ✅ working (GJ)	here or here

Network	Paper(s)	Tasks	Status	Last test	Link to example training and test dataset	Direct link to the notebook in Colab
Deep-STORM	here	Single Molecule Localization Microscopy (SMLM) image reconstruction from high-density emitter data	Fully supported	27/07/23 ✅ working (IH)	Training data simulated in the notebook or available from here
DFCAN	here	image upsampling	Under beta-testing	08/10/22 ✅ working (GJ)	here
WGAN	here	image upsampling	Under beta-testing	22/09/22 ✅ working (IvanHidalgo & EGM)	here

Network	Paper(s)	Tasks	Status	Last test	Link to example training and test dataset	Direct link to the notebook in Colab
YOLOv2	here	Object detection (bounding boxes)	Fully supported	🔴 Broken, not compatible with TF 2	here
Detectron2	here	Object detection (bounding boxes)	Under beta-testing		here
RetinaNet	here	Object detection (bounding boxes)	Under beta-testing		here

Network	Paper(s)	Tasks	Status	Last test	Link to example training and test dataset	Direct link to the notebook in Colab
Label-free prediction (fnet) 2D	here	Artificial labelling	Under beta-testing	🔴 Broken, not compatible with TF 2	Coming soon
Label-free prediction (fnet) 3D	here	Artificial labelling	Fully supported		here
CycleGAN	here	Unpaired Image-to-Image Translation	Fully supported	14/07/23 ✅ working (IH)	here
pix2pix	here	Paired Image-to-Image Translation	Fully supported	12/12/23 ✅ working (EGM)	here

Network	Paper(s)	Tasks	Status	Last test	Link to example training and test dataset	Direct link to the notebook in Colab
DRMIME	here	Affine or perspective image registration	Under beta-testing	12/08/22 ✅ working (GJ)	Coming soon!

Network	Paper(s)	Tasks	Status	Last test	Link to example training and test dataset	Direct link to the notebook in Colab
Diffusion Model for SMLM	here	SMLM image generation	Fully supported	12/02/24 ✅ working (EGM)	here and here

Networks that are compatible with BioImage.IO and can be used in ImageJ via deepImageJ or Ilastik. The trained models in these notebooks are also exported in the BioImage.IO format and can be uploaded to the BioImage Model Zoo. Check our user guide to learn how to use the resources in the BioImage Model Zoo with ZeroCostDL4Mic.

Network	Paper(s)	Tasks	Last test	Link to example training and test dataset	Direct link to the notebook in Colab
StarDist (2D)	StarDist: here and here, and DeepImageJ and BioImage Model Zoo	Nuclei segmentation	09/08/22 ✅ working (EGM)	here
Deep-STORM with DeepImageJ export	Deep-STORM and DeepImageJ	Single Molecule Localization Microscopy (SMLM) image reconstruction from high-density emitter data	10/08/22 ✅ working (EGM)	Training data simulated in the notebook or available from here
U-Net (2D)	U-Net and DeepImageJ and BioImage Model Zoo	Segmentation	16/07/23 ✅ working (IH)	ISBI challenge or here
U-Net (2D) multilabel	here and here and DeepImageJ and BioImage Model Zoo	Semantic segmentation	16/07/23 ✅ working (IH)	here
U-Net (3D)	3D U-Net and DeepImageJ and BioImage Model Zoo	Segmentation	27/07/23 ✅ working (IH)	EPFL dataset
pix2pix	here	Paired Image-to-Image Translation	12/12/23 ✅ working (EGM)	here

Network	Paper(s)	Tasks	Status	Last test	Link to example training and test dataset	Direct link to the notebook in Colab
Augmentor	here	Image augmentation	Fully supported	12/08/22 ✅ working (GJ)	None
Quality Control	here	Error mapping and quality metrics estimation	Fully supported	12/08/22 ✅ working (GJ)	None
Mounting DropBox or MEGA in Google Colab	None	Tutorial explaining how to mount a Dropbox or MEGA account in Google Colab using Rsync	Under beta-testing		None

Network	Paper(s)	Aim	Link to the project	Direct link to the notebook in Colab
DeepBacs	here	Toolbox to use Deep Learning to analyse microscopy images of bacteria	here	None
CAFI - DAIN	here	Content-aware frame interpolation using DAIN	here
CAFI - ZoomingSlowMo	here	Content-aware frame interpolation using ZoomingSlowMo	here
DECODE	here	Single Molecule Localization Microscopy (SMLM) image reconstruction	here
EM-stellar	here	Electron microscopy image segmentation	here

• Lucas von Chamier
• Johanna Jukkala
• Christoph Spahn
• Martina Lerche
• Sara Hernández-Pérez
• Pieta K. Mattila
• Eleni Karinou
• Seamus Holden
• Ahmet Can Solak
• Alexander Krull
• Tim-Oliver Buchholz
• Florian Jug
• Loïc A Royer
• Mike Heilemann
• Romain F. Laine
• Guillaume Jacquemet
• Ricardo Henriques

• Elias Nehme
• Lucien Weiss
• Yoav Shechtman
• Christophe Leterrier
• Daniel Krentzel
• Martin Jones
• Wei Ouyang
• Estibaliz Gómez de Mariscal
• Erlantz Calvo
• Ignacio Arganda-Carreras
• Amin Rezaei
• Ainhoa Serrano
• Seyed M. Hosseini
• Iván Hidalgo
• Alon Saguy

• Martin Weigert
• Uwe Schmidt