Using Smart Scan - derangedhk417/MicroscopeControl GitHub Wiki

Scan.py is the single most useful part of this repository for most people. The program is designed to scan a user defined region of the microscope stage. It does this by first taking images of a sample at a low zoom and determining which regions of the sample contain interesting material. It then zooms in on those regions and takes closeup images of them. Scan.py determines whether or not a region contains interesting materials by subtracting any non-uniform background from an image, calculating the optical contrast of everything in the image and then determining the percentage of each image that has optical contrast within a desired range.

What follows is a step by step outline of the operation of Scan.py during a typical scan.

1. Process command line arguments, ensure that the destination folder is empty and estimate the number of images that will be taken.
2. Move the zoom motor to the specified lower zoom value.
3. Move the stage to each of the "focus points" specified at the command line and determine the ideal position of the focus motor at each point.
   • See src\hardware_code\MicroscopeControl.py > MicroscopeController > autoFocus for the code that performs this focusing.
   • The focal points determined in this step are fit to a plane of the form z = a*x + b*y + c. This plane is then used to continuously adjust the focus throughout a scan.
4. Determine the background at this zoom level so it can be subtracted.
   • This process involves taking several images at different places on the sample and processing them to determine the background.
5. Move the stage to every location on the sample, take several images at each location and average them together to reduce noise.
6. Calculate the optical contrast of each image. Subdivide the image into several smaller images and determine what percentage of each of these images has optical contrast within the range specified by the user.
   • The size of these sub-images is determined by the size of each image that will be taken when zoomed in later.
7. Each region with a sufficient quantity of "interesting" material is added to a list for later imaging at a higher zoom level.
8. The zoom motor is set to the higher zoom specified by the user.
9. The automatic focusing mentioned above is repeated at the higher zoom level.
10. An image is taken of each region that was deemed "interesting" by the process mentioned above.
    • These images are saved in the specified output directory.
11. To Be Implemented These images are fed into an image processing program that creates a sqlite database of flakes that can be easily searched through later. The image processing program has already been implemented, but at the moment it needs to be run separately.

In order to operate correctly, Scan.py needs to be given information about the zoom levels to use, exposure time at these zoom levels, points to automatically focus on and the location on the stage to scan. The amount of exposure time necessary to produce good images varies based on the strength of the source lamp, the zoom and the objective lens attached to the system. Scan.py also needs to know the physical dimensions of the images being taken at each zoom level.

0. Place a white object (like a notecard) under the microscope, focus the microscope on it and run the automatic white balancing feature of the camera.
   • This can be found in the PixeLINK capture software under the Controls tab.
   • In the Controls tab of the software, also ensure that Gain is zero, Saturation is 100, Color Temperature is off and Gamma is off.
1. Determine the zoom levels that you want to use for the scan. Scans will run faster when the lower zoom level is as low as possible. Please note that lower zoom levels can increase the likelihood that a relevant piece of material will be missed.
   • Determine the physical dimensions of an image taken at each of the two zoom levels.
   • This can be done by placing a calibration slide under the microscope and using the Measurement tab of the pixeLINK software.
2. Set the position of the focus motor to half of it's maximum value (nominally 1495).
   • This can be done in the PixeLINK capture software in the lens tab.
3. Determine the exposure time that needs to be used at each zoom level. The zoom level can also be set in the lens tab of the PixeLINK capture software.
   1. Manually focus the camera using the micrometer stage it is mounted on.
   2. Open the histogram view in the PixeLINK software.
      • This is located between the button with the "Stop" icon and the button with the Camera icon below the live view.
   3. Adjust the exposure time until the mean of each channel is as far to the right (higher pixel value) as possible, without oversaturating the image.
      • Oversaturation has occurred when the rightmost part of the x-axis of the histogram (pixel value 255) is greater than zero.
      • When attempting to image materials with very poor optical contrast, it may be necessary to oversaturate the image.
      • Write down the exposure time that you determine. This will be supplied to Scan.py as an argument later.
   4. Repeat this process at the higher zoom level.
4. Determine the range of coordinates on the sample that you want to scan. These coordinates must be specified as a minimum and maximum x value followed by a minimum and maximum y value (a rectangle).
   • Ensure that images taken at both zoom levels within this range will image only the substrate. If images are taken that contain regions not on the substrate, they will interfere with the optical contrast calculation.
5. Determine the points on the sample that should be used for focus calibration. If no points are specified at the command line, the program will use the four corners of the scan region and the center of the scan region. In cases where these locations do not contain anything to focus on, this can interfere with the automatic focus. This is only relevant for substrates that are uniform enough to contain no features to focus on.
6. Determine the range of focus values to use when autofocusing. This is generally something like [0.45, 0.55]. This will cause the autofocus to iterate over values in this range when determining the ideal focus.
7. Construct a set of command line arguments and pass them to Scan.py at the command line.

In this example, images are output to a folder called test_scan_001. A 25 square millimeter rectangle is scanned with x bounds of 0 to 5 mm and y bounds of 0 to 5 mm. The higher zoom (fine zoom) used by the system is 1.0 (the maximum zoom). Images taken at this zoom have dimensions 0.394711 by 0.330216 mm. An exposure time of 4.8 ms is used at this zoom level. The lower zoom (coarse zoom) used by the system is 0.4235 (motor step 8629 of 20375). At this zoom, images have dimensions 1.183180 by 0.989850 mm. The exposure time used at this zoom is 1 ms. The focal range swept by the program is 0.45 to 0.55. Contrast values between 0.04 and 0.2 are considered interesting. When auto focusing, 25 steps are used, a single pass of the autofocus is performed and 3 images are averaged together during each step to reduce noise.

python Scan.py -o test_scan_001 -b 0 5 0 5 -f 1.0 0.394711 0.330216 0.0048 -c 0.4235 1.183180 0.989850 0.001 -fr 0.45 0.55 -cr 0.04 0.2 -ap 25 1 3

In this example, a file containing some commonly used command line arguments is used. Arguments specified in this file will be processed like regular command line arguments. If an argument is specified in both the command line and in the file, the value specified at the command line will take precedence.

python Scan.py -A _default_args.json -b 1.0 23 -11.5 -1 -o 2021-11-22_HOPG_90nm_SiO2

Here, _default_args.json contains the following:

{
	"fine_zoom": [
		1.0, 
		0.3947113834247017, 
		0.3302160593356982, 
		0.005
	],
	"coarse_zoom" : [
		0.4235, 
		1.183180280328661, 
		0.989850169163847, 
		0.001
	],
	"material_file" : "calibration_data/_graphene_on_90nmSiO2_Silicon.json",
	"n_processes"   : 8,
	"layer_range"   : [1, 2]
}

The layer_range parameter is used by Scan.py to calculate a range of contrast values that should be considered valid. This feature requires that a material_file be specified. See Material Files for details on this.

If you see something like the following when running the program

Connecting to focus controller . . . FAIL
Traceback (most recent call last):
  File "Scan.py", line 399, in <module>
    microscope = MicroscopeController(verbose=True)
  File "C:\Users\LAB\Desktop\MicroscopeControl\src\MicroscopeControl.py", line 51, in __init__

then the program could not connect to the focus controller. This usually means that the PixeLINK Capture software is turned on and that a lens controller is selected in the Lens tab. Navigate to the Lens Controller Select dropdown in the Lens tab and change it to No Controller Selected. If this doesn't solve the problem then there is most likely a problem with the controller hardware or the connection to the computer.

If you see something like the following

The focal points have been fit to a function of the form z = ax + by + c.
RMSE of fit: 0.011773
Fit too poor to continue.
Please ensure that the sample is flat and level.

then the automatic focus mechanism of the program could not be calibrated. This could mean that the substrate is too irregular to be fit to a function of the form z = ax + by + c. It could also mean that the sample contains very tall objects at the focus points and that they are resulting in each location having multiple focal points. This could also mean that the focus range parameter -fr/--focus-range is either too narrow or too wide. In the former case, the system may not sweep past the focal point at each location. In the later case, the program may move so far away from the correct focus that it focuses on the fiber cable being used to supply light to the system or on another part of the optics. It may then incorrectly determine that to be a good focal point.

When picking a sample up off of the stage and placing it back down, the set of stage coordinates occupied by the sample will change. This would normally necessitate updating the -b/--bounds command line parameter. You can avoid this by hitting the ZERO/HALT button on the stage controller with the camera centered on the top left of the sample. This should allow you to use the same coordinates, assuming the sample hasn't been rotated.

Be mindful of how well the sample is adhered to the stage when performing a scan. The stage will make thousands of small jerking motions when performing a scan. If the sample is not attached to the stage very well, it may move slightly during a scan. This will make it hard or impossible to relocate interesting flakes after a scan is done. Putting a piece of electrical tape on the bottom of a slide seems to prevent this sliding from occuring.