3D scanning - Cuddington-Lab/documents GitHub Wiki

Here, an outline for successful capture of the 3D structure of a plant is presented. The scanner used is a Creaform REVscan portable scanner. The software VX-Elements is used for data capture. The resolution of the scanner is theoretically high enough to enable at least partial capture of any plant structures, though in reality very fine structures like tendrils are extremely difficult to capture. Similarly, the scanner will usually not be able to detect two surfaces for the bottom and top of a leaf, it will likely detect whichever it sees first, and be unable to assign any sort of dimension to the leaf. The end result of this is that leaves will be represented purely as surfaces (as opposed to volumes).

Scanning Setup and Protocol Overview

To begin, open VXElements on the computer that the scanner is connected to (connect the scanner with the firewire cable). Scanning works like this: first, positioning targets (the small black/white stickers) are scanned, and then a surface is scanned relative to those targets. So, the first step in scanning a plant is to assemble your scanning stage. The small black box with positioning targets all over it and around it has produced the best results. The optimal placement of positioning targets will depend on the object being scanned, so different objects may warrant different setups.

Once the plant/object to be scanned has been placed in your scanning stage (Fig 1), you can begin to capture the positioning targets. Click the dropdown arrow beside the "Scan" button in VXelements, select "Scan positioning targets", and do so. Once confident all targets have been captured, click the dropdown once again, select "Scan Surface" and do so.

The scanner seems to operate best at a distance of about 6-10 inches from the target. Looking carefully, a flashing red 'X' of light can be seen, the apex of which seems to be the scanners focus. Watch the screen of VXElements as you scan, and adjust technique as needed. This can take some time to get used to. Capture from as many different angles as possible.

Scanning Protocol

Connect the scanner to the computer using the firewire cable. Connect the firewire cable to an outlet BEFORE connecting the scanner.
Open VXElements software.
Place the plant you'd like to scan into whatever enclosure you are using. ADVICE: Use flowerwire to support long stems. They may look stable but the do move slightly via changes in turgidity.
Ensure that you can see positioning targets from all angles around the plant. Add more if needed.
Start scanning: from the dropdown menu beside "Scan" select "Scan Positioning Targets". Then, click the "Scan" button, which should now have a picture of a bent blue surface with a positioning target on in.
Pull the trigger on the scanner to scan. Point the scanner at the scanning stage (with plant in it), and move around the stage to make sure the scanner has seen all of the positioning targets.
Once you have scanned all of the targets, put the scanner down and press the "Scan" button again which should now have a "Pause" (||) indicator.
From the same dropdown as before, now select "Scan Surface".
Before starting the scan, it may be helpful to change the resolution of surface you are after. In the left menu, under "New Project" there are a series of options. Select the "Surface" option. In the "Surface Parameters" box, change the resolution to 0.3-0.5 mm. This will make it easier to see how your scan is coming through in real time.
With the resolution changed, begin scanning by pressing the "Scan" button.
Again, pull the trigger on the scanner to begin scanning. Use the VXElements display as an indicator for your position. You should be able to see parts of the plant being scanned in real time if you are positioned well. Move around the stage, scanning from all angles. In addition to changing angle, alter depth, one position may capture one set of leaf tissue while missing leaf tissue further from the scanner.
Once your scan is taking shape, it may be wise to save your progress. Press the "Pause" button, it may take a couple of minutes for the computer to sort through all the raw data you've collected. Once the scanned surface appears on the screen, save the session.
You may notice that you have not yet captured a leaf or stem, you can restart scanning from this point to get more data. As you scan, the indicator at the bottom right of the VXElements window will tell you how much RAM your scan is currently using. The higher this % goes, the more sluggish the computer becomes. To cut back on how much memory in use, extraneous data can be deleted. The enclosure surrounding your plant will inevitably get scanned along with the plant. Do delete this data, enter editing mode by clicking "Edit Facets". In this mode you have a number of selection tools available, and then some simple operations to perform on selected data. Select areas corresponding to the enclosure, moving to get good angles by alternating between "Move Object" and "Rectangle Selection". Note that the selection will select everything along an axis, so you must zoom/rotate the image to get good angles for selection. Once you've selected a bunch of data you are sure can be deleted, delete it by selecting "Delete Facets" from the operator dropdown menu. This also may take some time. With extraneous data removed it will be much easier to see which parts of your plant need further scanning. Repeat this step (13) as necessary, until you are satisfied with the quality of your scan.
Before saving and exporting, do one final cleanup of the data. VXelements has an number of options available for automatically cleaning up your scan. The most helpful of these are the options under the "facets" tab of the left menu. Under this menu, scroll down to the bottom of the set of sliders, and set the "remove isolated patches" slider to around the first or second tick. Click apply to remove most of the dusty artifacts picked up during scanning. Feel free to play around with other settings, but be careful to save your session before you tweak,, some changes are quite catastrophic (and irreversible) for the scale of plants. Once you are happy with your scan and do not have unreasonable amounts of extraneous data, save your session. The session is a VXElements file format. To use the 3D image we've just created in other software, there are a number of more universal formats available. "*.stl" files seem to be the most flexible and commonly used. Exporting an STL file from VXElements is very simple. Simply "Save Facets", and an stl file will be created. Now move on to post-hoc editing and analysis.

IMPORTANT CONSIDERATION: Any movement at all of the plant during scanning will ruin the scan. Unfortunately, pea plants move enough naturally to cause problems. Because of this, scans that take longer than 25-30 minutes tend to start producing doubled structures. Around 15 minutes of scanning seems like the best compromise between completeness and movement errors. The plant must be in it's "usual" pot. Any transplant immediately prior to scanning is bound to cause significant and problematic wilting.

Significant post hoc work will need to be done to produce a complete scan, focus on capturing leaf structures as these are the hardest to recreate. As long as the "trajectory" of each stem is captured, they can be filled in confidently after the fact.

Post-hoc Manipulation

The open source and free software package Blender has been more than sufficient for the type of work needed for pea plants. The set of tools used specifically here is fairly concise, but there is still a significant learning curve associated with navigating the software/3D space. Youtube has good instructional tutorials for this.

The primary manipulation of the data needed for pea plant scans seems to be filling in of stem structures. To do this, a cylinder or series of cylinders can be positioned and transformed so that they match the trajectory of the stem. These can then be merged with the complete structure. Because of the way that these files are represented in MATLAB (at a lower resolution), the cylinder-fill in method seems reasonable and not too intrusive w.r.t. the data's original character. Obvious gaps in leaf surfaces can also be filled in with blender fairly easily.

Once the 3D structure has been manipulated to your heart's content, export it once again as an STL file for import into MATLAB.

Perhaps I will prepare a short Blender intro document to explain how to do all of this.