Convenience utilities - duaneloh/Dragonfly GitHub Wiki

powder.py

By default this utility operates within a reconstruction directory: it reads the out_photons_file in the config.ini file in this directory and adds all the patterns together into a virtual two-dimensional powder pattern. However, we do not average the detector readout in-plane, thus do not produce a one-dimensional powder pattern. The output of this utility is a binary two-dimensional array that is saved to data/powder.bin within the reconstruction directory.

For more help on customizing this utility, type:

./powder.py -h

frameviewer.py

This GUI utility can be used to go through frames in an emc file (or file list). This is useful in order to ensure that the conversion is performed correctly and to browse through just the final selected hits. Currently, one requires a rectangular detector geometry, and this is not very well optimized for complicated tiled detectors.

frameviewer.py Screenshot

run_emc.py

This Python utility helps you start, manage, and extend EMC reconstructions. This utility was initially developed as a convenience for the developers of this package, but it not necessary for users who are more comfortable with the command line interface and would like finer control of their EMC reconstructions (go to this page to learn how to do this).

The output of

./run_emc.py -h

are shown below:

usage: Starts EMC reconstruction [-h] [-c CONFIG_FILE] [-r] [-R] [-q QUAT_ADD]
                             [-m NUM_MPI] [-i NUM_ITER] [-t NUM_THREADS]
                             [--kahuna] [--bayes] [--tukey] [--slac]
                             [--dry_run]

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG_FILE, --config_file CONFIG_FILE
  -r                    resume reconstruction from last output
  -R                    same as -r, except we increase quaternion refinement level by one
  -q QUAT_ADD           increase quaternion sampling by an integer (default=0)
  -m NUM_MPI            number of mpi processes (default=0)
  -i NUM_ITER           number of iterations (default=10)
  -t NUM_THREADS        number of openMP thread(defaults to $OMP_NUM_THREADS on machine
  --dry_run             print commands to screen but we won't actually run them

A typical use case to start a reconstruction with 15 MPI processes, each with 6 threads, for 20 EMC iterations:

./run_emc.py -m 15 -t 6 -i 20

Then you can continue for another 10 iterations but now with a higher quaternion refinement:

./run_emc.py -m 15 -t 6 -i 10 -R

This last procedure reads the current quaternion refinement level in the config.ini file, then increases that by one, and makes this quaternion by calling [make_quaternion](EMC algorithm#make_quaternion).

After this, you can just continue for another 5 iterations at the same refinement.

./run_emc.py -m 15 -t 6 -i 5 -r

Note that ./run_emc.py basically calls the binary ./emc, which in turn reads config.ini to start. Therefore modifying the config file will only be reflected the next time ./emc is called.

autoplot.py

This utility is meant to process the intermediate output that the EMC algorithm writes to disk. These output are saved after every EMC iteration and include the 3D diffraction models, the RMS change between these consecutive models, the most likely quaternion of every pattern, and the visitation weights of all the voxels in the 3D diffraction volume.

Here are some of the elements implemented in this graphical user interface.

https://github.com/duaneloh/Dragonfly/blob/master/images/Screenshot_autoplot.png

Element	Description
Log Filename	This is the log file that we will use to plot these figures. By default this is the EMC.log in the reconstruction directory.
PlotMax	The maximum plot value for the central sections.
Filename	The central sections of this file is shown in the upper row. If in auto-check mode, this will be the 3D intensities from the latest iteration. For example, you could change this to either `data/intensities.bin` (true solution from simulation) or `data/weights/weights_nnn.bin` (visitation frequency of 3D volume).
Image name	The top row can be saved to an image of this name. We currently support these output formats: eps, jpeg, jpg, pdf, pgf, png, ps, raw, rgba, svg, svgz, tif, tiff.
Log image name	The bottom row can be saved to an image of this name. We currently support these output formats: eps, jpeg, jpg, pdf, pgf, png, ps, raw, rgba, svg, svgz, tif, tiff.
Layer no. (slider)	Scans through layers perpendicular to a particular axis of the volume. Click on the "-" and "+" buttons if the slider is unresponsive.
Iteration (slider)	Scans through ONLY the intensity sections of the reconstruction. Click on the "-" and "+" buttons if the slider is unresponsive.
Keep Checking	Checking this box will force this utility to scan the reconstruction directory every few seconds for updates on EMC.log (record of the reconstruction). The central sections of the reconstructed volume will be updated to those from the latest iteration whenever it becomes available. These updates occur for the diagnostic plots in the lower row as well.

Need more information?

We direct the interested reader to our paper for details on how to interpret these diagnostics.