FAQs - StochasticAnalytics/emClarity GitHub Wiki
I am running on a newer card (gpu architecture) and emClarity seems to be slooooow, especiall at startup
Matlab pre-compiles all of their gpu code specifically for each arch, and links to this whenever emClarity is compiled. They have not yet caught up to the newest arch which means that
ALLthe gpu code is compiled at run-time
for every call to emClarity. Fortunately setting a couple of environmental variables to increase the size of the default code cache, allows this to be done just once, instead of every time emClarity is run.
Here are the lines I've added to my own ~/.bashrc
# The default size is 256 MB max is 4GB - 2GB is probably sufficient
export CUDA_CACHE_MAXSIZE='2147483648'
# If the cache is disabled, then the size doesn't matter
export CUDA_CACHE_DISABLE=0`
# You can put this anywhere, but if working on a cluster or other network device, make sure it is accessible.
export CUDA_CACHE_PATH='${pathToHome}/.matlab/cudaCache
Warning: The CUDA driver must recompile the GPU libraries because your device is more recent than the libraries. Recompiling can take several minutes.
See previous answer
Is there a limit to the size of particle that may be processed?
Yes, there are two answers:
Template matching (> 500 KDa): This is more sensitive because the search space is much larger and therefore the noise floor is much higher, requiring a larger particle to get strong peaks. For the gag-hexamer for example, including neighboring hexamers in the reference substantially improves the search results.
Sub-tomogram alignment (> 200 KDa): This of course depends on a number of factors, but in general the limit will be higher than single-particle because [ Contrary to some rather strong opinions ] a tomogram as they are currently made, do not have the same SNR as a 2d projection of equivalent dose.
When are two particles considered "duplicates"
When running "removeDuplicates" a fraction of the particleRadius is used to determine if two sub-tomograms are overlapping such that the represent the same true particle.
What exactly is a "cycle" (aka the chicken and the egg)
- It is easiest to answer this from a Bayesian viewpoint, even though we only keep the top N likely parameters. A minimal cycle involves forming a weighted expectation value (average) based on your current beliefs about the world (alignment parameters, CTF parameters, and resolution parameters.) This new world view is then compared to the data, and you update your beliefs based on your findings.
emClarity avg paramN.m N RawAlignment
emClarity alignRaw paramN.m N
- Unlike any other sub-tomogram software, emClarity goes back to the tilt-series alignment. Any time this is updated, including the initial coarse alignment in IMOD, a cycle must start with two additional steps. Creating a transformed tilt-series, and then making your 3d reconstructions (tomograms.)
emClarity ctf update paramN.m
[ prior to sub-tomogram work emClarity ctf estimate paramN.m tiltM
]
emClarity ctf 3d paramN.m
emClarity avg paramN.m N RawAlignment
emClarity alignRaw paramN.m N
emClarity removeDuplicates paramN.m N
[ Generally run each time I step down the binning ]
emClarity tomoCPR paramN.m N
[ Generally run each time I step down the binning ]
- If you are running a cycle where you are doing classification. (set parameter flgClassify=1)
- update and 3d ctf if required, but generally it is advisable to run at least one round of alignment following a tilt-series geometry update.
emClarity avg paramN.m N RawAlignment
emClarity pca paramN.m N 0
emClarity cluster paramN.m N
emClarity avg paramN.m N Cluster_cls
emClarity geometry paramN.m N Cluster RemoveClasses selectedClasses.mod STD
emClarity skip paramN.m N
I'm having trouble with template matching, can I import coordinates from some other system?
emClarity is focused on getting high resolution results and the chances are good that if you can't get decent results from template matching, you also won't get very good results from the rest of the pipeline.
That being said, I realize there are good reasons to import coordinates from other software. I am currently working on solutions for:
- Relion
- Protomo/i3
If you have other software you'd like to import from, you might prepare the file with the output (positions and angles) a tilt-series, and the information required to reconstruct the tomogram from which your coordinates are aligned. With these, we can work to extend the import function to your software of choice.
Please note, this is the lowest priority for me. It is probably faster for you to try to figure out WHY template matching isn't working so well, as this will also likely improve your ability to process the data later in the pipeline.
Why alternate the out-of-plane and in-plane angles?
If you are starting from a reasonably close alignment (30A resolution or so) it has been my experience that you can decouple the out-of-plane and in-plane searches. If you search 100 out of plane angles and 30 in-plane, this would reduce your number of searches by 30x (i.e. much faster.)
Where are the parameters for the translational search?
The translational variables are refined 2x for each rotation that is searched. These are limited by a fraction of the particle radius, and do not need to be set by the user.
One exception would be helical filaments, where it you may want to have a particle that is larger than one repeat. In that case, the translational search may be further restricted by setting the parameters.
Tmp_eraseMaskType=sphere (may be sphere, cylinder, or rectangle) Tmp_eraseMaskRadius=[radius_x, radius_y, radius_Z] ( in angstrom )
I don't see an option for limiting resolution during refinement?
All references are filtered using my adaptation of the Single-particle wiener filter, which limits the resolution based on the SSNR estimated in a directional manner via the "gold-standard" FSC. The entire curve is used for this, however beyond a certain point (~0.33, really the "one-bit curve" intersection) the amplitudes are forced to zero.
I would like to use an external reference in the sub-tomogram alignment
No.
Can I remove particles based on CCC score?
Currently no but I will re-enable this feature once I can add an additional test that weights scores based on orientation.