Vision Compositing a.k.a. Multi-Shot Bottom Vision can take more than one vision shot to determine the alignment of a part.

The following functions are provided:

• Align packages that are larger than the camera view.
• Align packages that are not rectangular in their overall shape.
• Improve accuracy by centering features in the camera view, avoiding errors from parallax, viewing & lighting side angles, and residual lens distortion.
• Align packages based on pad corners facing inwards, i.e. parts that are asymmetric on the outside, or so large you can't capture their outside corners.
• Support the Pipeline Editor to cycle through multiple shots.

Between tiny 0201/0402 passives and enormous LQFP-256 packages, it is quite hard to get the bottom camera view right. One has to trade resolution against view size. Attempting to escape this dilemma by using a higher resolution camera comes at a significant additional cost in processing power, more lighting required, more compression artifacts, or (worse) reduced frame rate (fps) and additional camera lag.

Wanting a large camera view also adds constraints to the machine build. The ideal camera has a long focal length ("telephoto") lens to minimize parallax errors, which means it has to be far away from the subject. However, this usually means building a taller machine, especially for table-top designs. Consequently, designers often use (very) wide-angle lenses, which are detrimental to alignment accuracy (see the next section).

Ironically, the very large parts are often quite rare in projects, just the one MCU, for example. The overwhelming majority of parts are rather small. It hurts to make poor tradeoffs for the few exceptions.

Vision Compositing helps with all these problems.

Cameras can now be optimized to the brunt of the work with medium and small sized parts. You can use medium resolution cameras with high fps, longish focal length lenses and still get a reasonable compact table-top design. A narrower view means more accuracy, and faster convergence on multi-pass alignment.

The few large parts can then still be aligned with multiple shots. Obviously, there is an extra cost in alignment time, which may or may not be compensated by avoided tradeoffs elsewehere.

As an independent benefit, multi-shot alignment increases accuracy, particularly for very large parts. In some cases it might be the key to successful placement, regardless of the package actually being too large for the camera view.

The following illustration (exaggerated) shows how a slightly tilted nozzle might result in large placements errors (red) due to large parallax errors in (very) wide angle lenses:

Detecting the same corners using two shots reduces these errors to nothing, as the parallax is negligible when looking straight up from the camera center, and even what little remains, is symmetric left and right, and cancels itself out.

Errors from viewing pins from the side, from having inconsistent light reflection across viewing angles, or from residual lens distortions, can also be reduced by always centering the relevant features in the camera view.

Multi-shot bottom vision also enables alignment of some non-rectangular (hull) parts (best see the video for examples).

For an easy overview, please watch the video.

A Camera Roaming Radius of zero (default) effectively switches off Vision Compositing.

A Camera Roaming Radius of more than zero enables Vision Compositing. The nozzle is allowed to hold the part anywhere within this radius of the bottom camera location.

Within the Camera Roaming Radius, the nozzle is allowed to move freely in X, Y, i.e. without going to Safe Z first. This is also used to optimize in-camera drag-jogging. However, this is not a soft-limit, you still have to be careful not to bump your nozzle and/or part into any obstacles near your camera.

The radius must accommodate both the nozzle distance from the camera center, and the part protruding from the nozzle. This is automatically observed during Vision Compositing, taking the the hull of the package footprint into consideration. Because the part is typically held in its center, and because the corners are the package's obvious multi-shot targets, this means that part sizes are more or less constrained to those with diameters smaller than the Camera Roaming Radius.

The following illustrates an LQFP-144 package within a 30mm camera roaming radius:

CAUTION: Your camera "pit" must physically provide this freedom of movement, when the nozzle tip and part is at Camera Z (focal plane), and with the worst pick tolerances and other deviations to spare.

It is recommended to enable the Align with Part? option on the nozzles:

It offsets the nozzle rotation coordinate to match that of the part, once the part is aligned. This means that the cross-hairs and the DRO are nicely aligned with the part. This is especially useful in the multi-shot scenario, where you often only see a fraction of the whole part, and the reticle (cross-hairs, grid etc.) provides valuable visual feedback.

Furthermore, revisit the Nozzle Tip Configuration, Max. Part Diameter and Max. Pick Tolerance are important inputs for the multi-shot operation.

In order for Multi-Shot Bottom Vision to work, the footprint must be defined. Most common footprints can easily be generated from very few datasheet parameters, right inside OpenPnP:

Note: the Eagle E-CAD importer can also import the pads along with the packages.

Basic operation should be automatic. Given the Camera Roaming Radius, the multi-shot is enabled. It will automatically be employed with parts nearly as large as the camera view.

The package Vision Compositing tab can give a quick preview of the footprint and its treatment by bottom vision:

You can hover your mouse over the shots, indicated by corner marker and masking radius. It then indicates the camera view rectangle according to your camera size. The thick red/dashed circle around the package indicates the Camera Roaming Radius, i.e. where the nozzle and part can be.

You can press the mouse to see how the pads are fused together, where they are too close to isolate. Only the outside corners of these blocks can be considered as shot locations. But even these are subject to many more constraints.

Method determines how the footprint is treated:

• None: the part is aligned with one shot, even if its size or other properties of the footprint would suggest otherwise. This can be used to basically enforce the old behavior. It might be a way to tell OpenPnP to still align select packages with one shot, even if the footprint (plus tolerances) are actually overlapping the camera central circle. Or when pre-rotate is switched off and the part just happens to fit inside the rectangular camera view.
• Restricted: the part is always aligned with one shot if it fits inside the camera view (central circle), regardless of other properties of the footprint, like convex pad patterns. Only if the footprint is larger than the camera view, will a multi-shot be performed. This is the default method, intended to provide continuity for the majority of packages.
• Automatic: the part is aligned with one shot if it fits inside the camera view (central circle), and if its corners are understood to be symmetric and convex, to be capture as one bounding rectangle. If this is not the case, a multi-shot is automatically performed. It might combine pairs of corners into "bracket" shots.
• SingleCorners: every isolated corner is shot individually. This can be used for the best accuracy, for the following reasons:
  • Only the camera center , i.e. a perpendicular "ray" up Z will provide positional information. Parallax as well as view and light side angle errors are minimized.
  • More shots reduce random errors, staticstically.

Extra Shots determines how many extra shots are taken for this package. The minimum is automatically computed from the geometry of the pads. For example, a square package shape can be aligned with just two corners shots, but the opposing two corners are made available as extra shots, to improve accuracy.

Max. Pick Tolerance: determines by how much a package, or rather its corners, can deviate after the pick. When set to zero, the analog setting on the nozzle tip is taken. If non-zero, this value overrides the nozzle tip's. The larger this tolerance, the more the corners must be isolated from other, obstructing corners, or from the limits of the camera view. Finding a solution can therefore fail, if tolerance are too large.

Min. Angle Leverage: determines how far away two corners have to be, in order to define the detected angle of the package, relative to the dimension of the package (the lesser of width and height). A setting of 0.5 means that the corners must be half-way apart across the package.

Allow inside corner?: determines if inside corners are allowed. For very large parts, where even the Camera Roaming Radius is not enough to position the part to its corners, the alignment might still be gleaned from the inside corners of large pads.

If a Vision Compositing pipeline has multiple shots, you can cycle through them using the Step button:

Note, the pipeline is the same for all the shots, i.e. any modifications to stages and stage properties will affect all shots, and must work with all of them. It is only the input image (camera subject position) and some externally controlled pipeline and stage properties that will change between shots.