Run CDeep3M training and prediction - CRBS/cdeep3m GitHub Wiki

You did Demorun 1 and Demorun 2, but still not entirely sure how to run this for your own data now? Here are detailed step-by-step instructions for training your own model now:

Steps:

  • Upload training (images and labels)

  • Pre process training data

  • Run training

  • Upload images to segment

  • Pre process images

  • Run Segmentation

  • Ensemble Models

  • Download Results

Upload training (images and labels)

Collect your training data locally in a folder called training, with sub-directories called images and labels, containing a matching number of images (.png format, typically 8bit) and labels (.png) and have the same size in x/y.

Use the SCP command to transfer the entire training folder recursively:

scp -i ~/.ssh/your-sshkey.pma -r ./training_directory ubuntu@ec2-**-***-**-**.us-west-2.compute.amazonaws.com:~/

Augment Training images

Login remotely to your cloud stack:

ssh -i ~/.ssh/your-sshkey.pma ubuntu@ec2-**-***-**-**.us-west-2.compute.amazonaws.com

Pre process training data

PreprocessTrainingData.m ~/training_directory/images/ ~/training_directory/labels/ ~/augmentedtraining/

This will create the data variations needed to train the network.

Run training

runtraining.sh --numiterations 50000 ~/augmentedtraining/ ~/trained_network

This will start the actual training of the CNN of CDeep3M. Setting a reasonable number of iterations (e.g. 20000-50000) is important here. All three models are trained sequentially. Hitting Ctrl-c a single time will stop training of the model currently training early, and is useful to test the performance at an earlier iteration. Training can be resumed at the current iteration easily later on. By default every 2000 are saved, additionally when the training is stopped early the current iteration is saved. Therefore no data will be lost.

To train for 2D data only (no 3D component) add the --models 1fm flag:

runtraining.sh --models 1fm --numiterations 50000 ~/augmentedtraining/ ~/trained_network

Upload images to segment

scp -i ~/.ssh/your-sshkey.pma -r ./my_images ubuntu@ec2-**-***-**-**.us-west-2.compute.amazonaws.com:~/

Run Segmentation

runprediction.sh ~/my_trained_model ~/my_images ~/predictout

This will run the prediction for all three models of CDeep3M. Each image package will be segmented in 16 variations, those will be de-augmented. Then all data-packages will be merged in 3D into the original image size. The results of all three models will be stored as .png images in the ~/predictout folder in 1fm, 3fm and 5fm respectively.

To segment 2D data, use 1fm only:

runprediction.sh --models 1fm ~/my_trained_model ~/my_images ~/predictout

Working while CDeep3M is processing?

If you wish to work on the cloud or check the status while you are processing data, use the screen command before starting your processing. Most important commands to

Open new screen: screen

To leave the screen running (processing) but stop watching: Ctrl-A Ctrl-D

Resume your screen: screen -r

Done Processing? Close down your screen: exit

If you are completely new to Linux and command line tools, please see Linux cheatsheets for basic commands.