CrystFEL 0.12 - biochem-fan/cheetah GitHub Wiki
CrystFEL 0.12
This is not part of the tutorial; this is my notes on installation and results of initial evaluation.
Installation
Installation was performed in the same way as CrystFEL-0.11#crystfel-011-1.
Testing:
Can we refine the detector distance?
Let's repeat the test in CrystFEL-0.11#can-we-refine-the-detector-distance.
The index rate:
$ index_rate *.stream
xg-198167-1-492.stream 476 269 56.5126
xg-198167-1-496.stream 476 375 78.7815
xg-198167-1-500.stream 476 394 82.7731
xg-198167-1-504.stream 476 387 81.3025
xg-198167-1-508.stream 476 367 77.1008
xg-198167-1-512.stream 476 149 31.3025
Unit cell parameters:
$ for f in *.stream; do echo -n $f" "; sh ~/scripts/cell.sh $f; done
xg-198167-1-492.stream 77.99 77.99 37.34 90.00 90.00 90.00 sigma 0.29 0.29 0.32 0.00 0.00 0.00
xg-198167-1-496.stream 78.50 78.50 37.61 90.00 90.00 90.00 sigma 0.32 0.32 0.34 0.00 0.00 0.00
xg-198167-1-500.stream 79.05 79.05 37.89 90.00 90.00 90.00 sigma 0.33 0.33 0.36 0.00 0.00 0.00
xg-198167-1-504.stream 79.61 79.61 38.18 90.00 90.00 90.00 sigma 0.34 0.34 0.36 0.00 0.00 0.00
xg-198167-1-508.stream 80.12 80.12 38.46 90.00 90.00 90.00 sigma 0.34 0.34 0.38 0.00 0.00 0.00
xg-198167-1-512.stream 80.59 80.59 38.70 90.00 90.00 90.00 sigma 0.34 0.34 0.39 0.00 0.00 0.00
Now that Bravais lattice constraints are applied, we cannot use the mismatch between a and b and between angles and 90 as selection criteria. Or we can use a P1 cell file.
We cannot use detector-shift to quickly adjust the beam center.
Let's try the new show-residual command:
$ for z in `seq 492 4 512`; do echo ==DISTANCE $z mm ==; ~/packages/crystfel-0.12.0/build/show_residuals -g sacla-24feb-shimada-$z.geom mille-xg-198167-1-$z/mille-data.bin 2>&1 | grep "Group all" -A3; done
==DISTANCE 492 mm ==
Group all:
Mean spot deviation in x-direction (5271 measurements) = +34.853792 µm
Mean spot deviation in y-direction (5271 measurements) = +38.676323 µm
Mean absolute reflection excitation error (5271 measurements) = +0.008141 nm^-1
==DISTANCE 496 mm ==
Group all:
Mean spot deviation in x-direction (7317 measurements) = +36.235735 µm
Mean spot deviation in y-direction (7317 measurements) = +39.397538 µm
Mean absolute reflection excitation error (7317 measurements) = +0.005599 nm^-1
==DISTANCE 500 mm ==
Group all:
Mean spot deviation in x-direction (7792 measurements) = +38.074937 µm
Mean spot deviation in y-direction (7792 measurements) = +38.721158 µm
Mean absolute reflection excitation error (7792 measurements) = +0.004037 nm^-1
==DISTANCE 504 mm ==
Group all:
Mean spot deviation in x-direction (7918 measurements) = +38.375901 µm
Mean spot deviation in y-direction (7918 measurements) = +37.628737 µm
Mean absolute reflection excitation error (7918 measurements) = +0.004266 nm^-1
==DISTANCE 508 mm ==
Group all:
Mean spot deviation in x-direction (7907 measurements) = +39.560330 µm
Mean spot deviation in y-direction (7907 measurements) = +38.403526 µm
Mean absolute reflection excitation error (7907 measurements) = +0.005872 nm^-1
==DISTANCE 512 mm ==
Group all:
Mean spot deviation in x-direction (3190 measurements) = +39.639796 µm
Mean spot deviation in y-direction (3190 measurements) = +35.278825 µm
Mean absolute reflection excitation error (3190 measurements) = +0.007525 nm^-1
Again, the residual is not very useful, but the number of "measurements" seems highest at the right distance.
Let's refine at the -l0 --camera-length level.
for z in `seq 492 4 512`; do
~/packages/crystfel-0.12.0/build/align_detector -g sacla-24feb-shimada-$z.geom \
-o sacla-24feb-shimada-$z-mille.geom mille-xg-198167-1-$z/mille-data.bin \
-l0 --camera-length --panel-totals 2>&1 > sacla-24feb-shimada-$z-mille.log
done
The result:
$ grep q1/coffset *mille.geom # z + coffset added manually
sacla-24feb-shimada-492-mille.geom:q1/coffset = 0.000881 # 50.081 mm
sacla-24feb-shimada-496-mille.geom:q1/coffset = 0.000544 # 50.144 mm
sacla-24feb-shimada-500-mille.geom:q1/coffset = 0.000191 # 50.191 mm
sacla-24feb-shimada-504-mille.geom:q1/coffset = -0.000188 # 50.212 mm
sacla-24feb-shimada-508-mille.geom:q1/coffset = -0.000553 # 50.247 mm
sacla-24feb-shimada-512-mille.geom:q1/coffset = -0.000909 # 50.291 mm
This suggests that the true value is around 50.2 mm. The initial value dependency is weaker than CrystFEL 0.11, which is a good news.
Let's examine the beam center.
$ grep q1 sacla-24feb-shimada-*-mille.log # The unit is mm.
sacla-24feb-shimada-492-mille.log: q1: 0.044 0.048 0.881 0.000° 0.000° 0.000°
sacla-24feb-shimada-496-mille.log: q1: 0.045 0.048 0.544 0.000° 0.000° 0.000°
sacla-24feb-shimada-500-mille.log: q1: 0.047 0.048 0.191 0.000° 0.000° 0.000°
sacla-24feb-shimada-504-mille.log: q1: 0.048 0.047 -0.188 0.000° 0.000° 0.000°
sacla-24feb-shimada-508-mille.log: q1: 0.049 0.048 -0.553 0.000° 0.000° 0.000°
sacla-24feb-shimada-512-mille.log: q1: 0.051 0.047 -0.909 0.000° 0.000° 0.000°
The shift seems consistent across all Z.
q2/corner changed from (36.444, 497.856) to (37.395, 498.794) px. When this dataset was initially processed with CrystFEL 0.10.2, detector-shift moved it to (36.709, 498.136) px.
I took the refined 50.4 mm geometry and varied the camera distances again.
$ ln -s ../center-test/sacla-24feb-shimada-504-mille.geom orig.geom
$ for len in `seq 498 1 506`; do sed 's/clen.*/clen = 0.0'$len'/' orig.geom > sacla-24feb-shimada-$len.geom; done
The result of the second round:
$ index_rate *.stream
xg-198167-1-498.stream 476 403 84.6639
xg-198167-1-500.stream 476 410 86.1345
xg-198167-1-501.stream 476 408 85.7143
xg-198167-1-502.stream 476 413 86.7647
xg-198167-1-503.stream 476 413 86.7647
xg-198167-1-504.stream 476 416 87.395
xg-198167-1-505.stream 476 414 86.9748
xg-198167-1-506.stream 476 422 88.6555
$ for f in *.stream; do echo -n $f" "; sh ~/scripts/cell.sh $f; done
xg-198167-1-498.stream 78.52 78.52 37.63 90.00 90.00 90.00 sigma 0.31 0.31 0.31 0.00 0.00 0.00
xg-198167-1-500.stream 78.79 78.79 37.77 90.00 90.00 90.00 sigma 0.31 0.31 0.32 0.00 0.00 0.00
xg-198167-1-501.stream 78.94 78.94 37.84 90.00 90.00 90.00 sigma 0.30 0.30 0.31 0.00 0.00 0.00
xg-198167-1-502.stream 79.09 79.09 37.89 90.00 90.00 90.00 sigma 0.31 0.31 0.33 0.00 0.00 0.00
xg-198167-1-503.stream 79.22 79.22 37.97 90.00 90.00 90.00 sigma 0.31 0.31 0.34 0.00 0.00 0.00
xg-198167-1-504.stream 79.35 79.35 38.04 90.00 90.00 90.00 sigma 0.32 0.32 0.33 0.00 0.00 0.00
xg-198167-1-505.stream 79.49 79.49 38.12 90.00 90.00 90.00 sigma 0.32 0.32 0.34 0.00 0.00 0.00
xg-198167-1-506.stream 79.63 79.63 38.19 90.00 90.00 90.00 sigma 0.33 0.33 0.33 0.00 0.00 0.00
$ for z in 498 `seq 500 1 506`; do echo ==DISTANCE $z mm ==; ~/packages/crystfel-0.12.0/build/show_residuals -g sacla-24feb-shimada-$z.geom mille-xg-198167-1-$z/mille-data.bin 2>&1 | grep "Group all" -A3; done
==DISTANCE 498 mm ==
Group all:
Mean spot deviation in x-direction (7717 measurements) = +6.707212 µm
Mean spot deviation in y-direction (7717 measurements) = +1.759375 µm
Mean absolute reflection excitation error (7717 measurements) = +0.005294 nm^-1
==DISTANCE 500 mm ==
Group all:
Mean spot deviation in x-direction (7990 measurements) = +5.962829 µm
Mean spot deviation in y-direction (7990 measurements) = +1.401687 µm
Mean absolute reflection excitation error (7990 measurements) = +0.004252 nm^-1
==DISTANCE 501 mm ==
Group all:
Mean spot deviation in x-direction (8103 measurements) = +7.866548 µm
Mean spot deviation in y-direction (8103 measurements) = +2.028132 µm
Mean absolute reflection excitation error (8103 measurements) = +0.003852 nm^-1
==DISTANCE 502 mm ==
Group all:
Mean spot deviation in x-direction (8255 measurements) = +7.467344 µm
Mean spot deviation in y-direction (8255 measurements) = +2.022598 µm
Mean absolute reflection excitation error (8255 measurements) = +0.003659 nm^-1
==DISTANCE 503 mm ==
Group all:
Mean spot deviation in x-direction (8251 measurements) = +9.551692 µm
Mean spot deviation in y-direction (8251 measurements) = +1.840597 µm
Mean absolute reflection excitation error (8251 measurements) = +0.003457 nm^-1
==DISTANCE 504 mm ==
Group all:
Mean spot deviation in x-direction (8345 measurements) = +10.106625 µm
Mean spot deviation in y-direction (8345 measurements) = +1.462461 µm
Mean absolute reflection excitation error (8345 measurements) = +0.003464 nm^-1
==DISTANCE 505 mm ==
Group all:
Mean spot deviation in x-direction (8440 measurements) = +8.617606 µm
Mean spot deviation in y-direction (8440 measurements) = +1.287564 µm
Mean absolute reflection excitation error (8440 measurements) = +0.003610 nm^-1
==DISTANCE 506 mm ==
Group all:
Mean spot deviation in x-direction (8454 measurements) = +8.581401 µm
Mean spot deviation in y-direction (8454 measurements) = +0.901949 µm
Mean absolute reflection excitation error (8454 measurements) = +0.003847 nm^-1
Both the index rate and the residual improved. Note that show_residual prints mean residuals, not RMSDs! Thus, the numbers cannot be compared with the outputs from geoptimiser or dials.refine. I wrote a script to calculate RMSDs.
But the number of measurements is not the highest at the right distance. Perhaps we have over-predictions when the distance is off?
$ for z in 498 `seq 500 1 506`; do
~/packages/crystfel-0.12.0/build/align_detector -g sacla-24feb-shimada-$z.geom \
-o sacla-24feb-shimada-$z-mille.geom mille-xg-198167-1-$z/mille-data.bin \
-l0 --camera-length --panel-totals 2>&1 > sacla-24feb-shimada-$z-mille.log
done
$ grep q1 sacla-24feb-shimada-*-mille.log # The unit is mm.
sacla-24feb-shimada-498-mille.log: q1: 0.008 0.001 0.541 0.000° 0.000° 0.000°
sacla-24feb-shimada-500-mille.log: q1: 0.007 0.002 0.349 0.000° 0.000° 0.000°
sacla-24feb-shimada-501-mille.log: q1: 0.010 0.002 0.270 0.000° 0.000° 0.000°
sacla-24feb-shimada-502-mille.log: q1: 0.009 0.002 0.171 0.000° 0.000° 0.000°
sacla-24feb-shimada-503-mille.log: q1: 0.012 0.002 0.094 0.000° 0.000° 0.000°
sacla-24feb-shimada-504-mille.log: q1: 0.012 0.002 -0.013 0.000° 0.000° 0.000°
sacla-24feb-shimada-505-mille.log: q1: 0.011 0.002 -0.103 0.000° 0.000° 0.000°
sacla-24feb-shimada-506-mille.log: q1: 0.011 0.001 -0.192 0.000° 0.000° 0.000°
$ grep q1/coffset *mille.geom # z + coffset added manually
sacla-24feb-shimada-498-mille.geom:q1/coffset = 0.000353 # 50.153 mm
sacla-24feb-shimada-500-mille.geom:q1/coffset = 0.000161 # 50.161 mm
sacla-24feb-shimada-501-mille.geom:q1/coffset = 0.000082 # 50.182 mm
sacla-24feb-shimada-502-mille.geom:q1/coffset = -0.000017 # 50.183 mm
sacla-24feb-shimada-503-mille.geom:q1/coffset = -0.000094 # 50.206 mm
sacla-24feb-shimada-504-mille.geom:q1/coffset = -0.000201 # 50.199 mm
sacla-24feb-shimada-505-mille.geom:q1/coffset = -0.000291 # 50.209 mm
sacla-24feb-shimada-506-mille.geom:q1/coffset = -0.000380 # 50.220 mm
Thus, when the beam center is better, the convergence radius of the distance is larger.
The beam center also looks converged to a subpixel level.