서론

하단 비전은 구성 요소를 보다 정확하게 배치할 수 있게 해주는 OpenPnP의 기능입니다. 간단히 말해서 OpenPnP는 위로 보이는 카메라를 사용하여 부품 픽업시 오프셋 또는 회전 오류가 있는지 식별하고 해당 오류가 무엇인지 확인한 다음 배치하기 전에 수정을 적용할 수 있습니다. 하단 비전은 픽업 실패가 발생했는지 확인하는 데에도 사용할 수 있습니다.

작동 이론

하단 비전에 대한 기본적인 아이디어는 다음과 같습니다:

1. 피더에서 부품을 픽업합니다.
2. 상향 카메라 위에 노즐을 위치시키고 이미지를 찍습니다.
3. CvPipeline을 사용하여 부품의 중심 오프셋과 회전 여부를 결정합니다.
4. 배치 중에 오류를 수정할 수 있도록 결과 X, Y 및 회전 좌표를 JobProcessor에 제공합니다.

Bottom Vision 작업의 핵심은 CvPipeline입니다. 파이프라인은 입력 이미지를 'RotatedRect'로 변환하기 위해 수행되는 일련의 컴퓨터 비전 작업을 설명합니다. 'RotatedRect'는 회전 구성 요소가 있는 사각형(너비, 높이, X, Y)에 대해 말하는 컴퓨터 비전입니다. 이 RotatedRect를 사용하여 오류를 계산하고 수정합니다.

하드웨어 설치

• 상향 카메라를 연결하고 환경설정 해줍니다. 헤드에 장착되어서는 안됩니다.

• 노즐이 카메라 중앙에 있을 때 사용하려는 가장 큰 부품이 프레임 안에 들어오도록 렌즈를 설정해야 합니다. 어안 렌즈를 사용하거나 왜곡이 심한 렌즈를 사용하는 경우 카메라 렌즈 교정을 고려하십시오.

• 노즐이 중앙에 있고 카메라 위에 초점이 맞춰지도록 카메라의 위치(설정 패널에서)를 설정합니다.

포괄적 환경설정

하단 비전을 구성하시려면 Machine Setup -> Vision -> Bottom Vision으로 이동하십시오.

Enabled?: 하단 비전의 활성화 여부입니다.

Pipeline: 부품을 찾는 데 사용할 기본 파이프라인을 보고 편집하려면 'Edit' 버튼을 누르십시오. 부품별 파이프라인이 구성되지 않은 경우 기본 파이프라인이 사용됩니다. 기본 제공 파이프라인은 아래에 설명되어 있습니다. 파이프라인을 OpenPnP 기본값으로 재설정하려면 'Reset to Default'을 누르세요. 현재 파이프라인을 모든 부품에 할당하려면 'Reset All Parts'를 누르십시오.

Rotate parts prior to vision?: 미리 노즐의 부품을 최종 배치 각도로 사전 회전합니다. 이것은 일반적으로 정밀도를 향상시킵니다. 이 옵션은 적응형 다중 패스 비전도 가능하게 합니다. 이 옵션은 굴절이 제한된(예: 360° 미만) 노즐의 경우 필수입니다. PlacementAngle 최적화 Rotation Mode가 노즐에 설정된 경우에도 활성화해야 합니다. 노즐 회전 모드를 참조하십시오.

Multi-pass Vision

The part will first be positioned and rotated as picked from the feeder, which means it might be slightly offset both in location and angle, due to play in the feeder etc. The vision operation then determines these offsets, but these may be inaccurate as the part is seen slightly from the side, which may create parallax errors and slight changes in light relection angles which might affect how beveled/angled surfaces are lighted. Furthermore, the scale in the camera view (units per pixel) might be slightly in error as the part or some of its features (angled pins etc.) might be slightly outside the focal plane.

In order to further improve precision, the part is then centered in the camera view, according to the preliminary offsets and the process is repeated. By centering the part (multiple times), the mentioned errors can be cancelled out by symmetry.

Max vision passes: Determines the maximum number of passes used to pinpoint the part (only applies when pre-rotate is active).

Max linear offset: As long as preliminary linear offsets are larger than this, further passes are made, up until Max. vision passes is reached. Linear offsets are center offsets and corner offsets, see the illustration below. (only applies when pre-rotate is active).

Max angular offset: As long as preliminary angular offsets are larger than this, further passes are made, up until Max. vision passes is reached (only applies when pre-rotate is active).

You will probably need to customize the pipeline a bit for your machine. See the Tips section below. There is likely to be a lot of discussion and learning happening on the mailing list. That should be your first stop for help.

Part Configuration

Each Part in your Parts library can have it's own custom pipeline. In most cases the default pipeline will work but this allows you tweak the pipeline for troublesome parts or create entirely new pipelines when the default won't work.

You can also enable or disable bottom vision on for each part.

To access the bottom vision part settings go to the Parts tab in OpenPnP, select a part and look for the Alignment tab on the right.

Enabled?: Switches on bottom vision for this part. This is checked in addition to the global Enabled? switch (see above).

Pre-rotate: This can either inherit the global setting or override it as Always On or Always Off.

Test: Press the Test Alignment button to perform an alignment of that part. It must already be picked and held on the nozzle.

Center After Test: Centers the part after the vision alignment test.

Pipeline: Press the Edit button to view and edit the part specific pipeline that will be used to locate parts. Press Reset to Default to reset the pipeline to the global default (see Global Configuration).

Rotation: Use the option Adjust for all standard pipelines. Only an adjustment of ±45° can be detected and applied, which is usually more than sufficient for parts picked from normal feeders. Use the option Full only for special pipelines that specifically support full 360° orientation detection (pin 1 detection). CAUTION: You may get very strange and intermittent errors, if you set Full and your pipeline is not specifically made for it.

Part Size Check: You can add a part size check, where the vision result is compared against a known good part size. This can serve as an alternative to vacuum sensing tests, to detect whether a pick was successful.

• Use BodySize if your pipeline looks for the body of the part (e.g. the black body of lead free packages).
• Use PadExtents if your pipeline looks for the contacts (which the default pipeline does). Note, you need to define the footprint in the Package for this to work.

Size tolerance (%): Determines by how much (relatively speaking) the detected part size may deviate from the footprint defined size. If the tolerance is exceeded, the aligment fails. Most likely the part was then not properly picked.

Multi-Shot Bottom Vision

Vision Compositing a.k.a. Multi-Shot Bottom Vision can take more than one vision shot to determine the alignment of a part. See the extra Vision Compositing page.

Usage

Note: This section will be expanded soon.

See https://www.youtube.com/watch?v=pRYQaFKhsuw for a short demonstration of how to pick, test and discard a part for bottom vision.

When bottom vision is enabled in Machine Setup and for a specific part it will be used automatically during a job run. If the system is able to determine the offsets they will be applied. If the operation fails the placement will continue with no offset correction. This will be improved in the future to handle retry and discard.

Default Pipeline

OpenPnP comes with a default pipeline. The pipeline was developed for one particular machine design but using the CvPipeline tools it is possible to customize the pipeline for any type of machine. In general, the changes should be minimal if certain rules are followed. If you have not read CvPipeline it's worth taking a moment to do so as it will help you understand the rest of this.

The default pipeline is described below:

1. ImageCapture: Waits for the camera to settle and captures an image.
2. ImageWriteDebug: Writes the input image to a file on disk to help with debugging.
3. BlurGaussian: Performs minor blurring on the input image. This is used to reduce noise in the image.
4. MaskCircle: Blacks out everything outside of a circle of a given diameter. On the development machine this circle represents a "safe" area in the image where nothing is visible except the nozzle.
5. ConvertColor: Convert from RGB color to HSV color, which is required for the next stage.
6. MaskHsv: Searches the image for any pixels that match a certain hue (the H in HSV) and turns them black. The purpose of this is to remove green and "greenish" pixels from the image. Green is the color of the nozzle holder. This is similar to the concept of "green screening". See also the Nozzle Tip Background Calibration.
7. ConvertColor: Convert back from HSV to RGB. This is required by the next stage.
8. ConvertColor: Convert from RGB to grayscale.
9. Threshold: Turns the image into a binary image - meaning that it has only two colors: white and black. Any gray pixels that are darker than the threshold value turn black and any lighter turn white.
10. FindContours: Find connected contours in the image. Contours are a way to describe simple features in an image such as lines and curves.
11. FilterContours: Removes any contours from the previous stage that are smaller than a specified value. This helps remove noise and features that don't pertain to the main contour around the part.
12. SetColor: Sets the entire image to black. This simply provides a blank canvas for the next stage to draw on.
13. DrawContours: Draw all the of the remaining contours in white on the black background. At this point we hope that we're simply drawing the shape of the part only.
14. MinAreaRect: This is where the magic happens! MinAreaRect creates a RotatedRect that fits around any non-black pixels in the image. Since we drew contours representing the part this now finds the bounds and rotation of the part.
15. ImageRecall: Recalls the original input image so that we can show the user the results of all this work.
16. DrawRotatedRects: Draw the RotatedRect in red overtop the recalled original image. If all went well we should now see the original input image with a red rectangle surrounding the part.
17. ImageWriteDebug: Writes the resulting image out to a file for help with debugging.

Tips

• Much of the purpose of the vision pipeline is to filter the image so that the only thing that is visible is the part you are interested in. The various Mask stages and Thresholds can help with this.

• Switch off the camera's auto-exposure. Otherwise, this will never be stable and repeatable. If your camera does not allow that, chose a different model. Seriously.

• You should set your Camera's exposure, so you the dynamic range includes the brightest parts of the image without clipping. We need to be able to distinguish the bright tones. Use the shiniest part to set this up. The image will be rather dark for humans, but it is fine for machine vision.

• The following animation quickly shows you how to initially set or tune a threshold, using the mouse to probe image pixels. Look at the third channel (the V) of the HSV(full) indicator on the status line (this indicator for brightness will work both for color and grayscale images). Probe the pixels that should be excluded and probe the pixels that should be included, then set the threshold to a value in between:

  While holding down the Shift key, use the context menu to display the image separately. This will restart the animation from the beginning.

  

• If you have green color Juki nozzle tips you can use the MaskHSV stage to mask the green parts reliably. Make yourself familiar with the HSV model (from the Wikipedia).

  

• The following animation quickly shows you how to initially set or tune an HSV mask, using the mouse to probe image pixels. First we probe green pixels and look at the first channel (the H as in "hue") of the HSV(full) indicator on the status line to include what is green. Then we exclude parts that are too bright (the pins) looking at the third value (the V as in "value" i.e. brightness) of the HSV(full) indicator on the status line. Finally, we exclude the parts that are too dark (background stuff, the IC body), using the same method.

  While holding down the Shift key, use the context menu to display the image separately. This will restart the animation from the beginning.

  

FAQ

How do I see debug images?

Enable DEBUG or TRACE level logging. See this FAQ for more information. Bottom vision will now produce a pair of debug images in your .openpnp/org.openpnp.vision.pipeline.stages.ImageWriteDebug directory.