03.03 preset projects - advantech-EdgeAI/edge_agent GitHub Wiki

The Factory Outfit Detection project is designed to automatically monitor personnel in an industrial or factory setting to ensure compliance with required safety outfits or work attire. Using advanced object detection models, this system can identify specific items of clothing or Personal Protective Equipment (PPE) and alert supervisors or individuals if any required items are missing. This project helps enhance workplace safety and adherence to company policies.

To run this project, load the preset named:

• ATTIRE_DET_WEBRTC_ADVAN

Note:

• Ensure the Edge Agent is running and accessible via your web browser.
• The demonstration video file (Attire_det_advan.avi) must be located in the /ssd/jetson-containers/data/videos/ directory on your Jetson device.
• The OpenWord_detector configuration JSON file (e.g., owl.json) should be correctly placed, typically at /opt/NanoLLM/nanoowl/owl.json.

Figure 3.1 — Pipeline overview

This project utilizes the following key nodes connected in a pipeline:

• VideoSource: Provides the video input (e.g., from the Attire_det_advan.avi file).
• RateLimit_1: Controls the frame processing rate for the detection model.
• OpenWord_detector: The core node that performs the outfit detection based on a customizable configuration.
• VideoOutput: Displays the video feed with visual overlays of the detection results (e.g., bounding boxes, status messages).
• PiperTTS module (Preset): A pre-configured set of nodes (PiperTTS, RateLimit, WebAudioOut) that provides text-to-speech voice alerts for missing items.

Data Flow: The VideoSource sends frames to RateLimit_1. The limited frames go to OpenWord_detector for analysis. The OpenWord_detector then sends the processed image (with visual detection info) to VideoOutput and any textual alert messages (e.g., "Missing face shields") to the PiperTTS module for audible alarms.

The primary customization for this project lies within the OpenWord_detector node and its associated JSON configuration file (owl.json located at /opt/NanoLLM/nanoowl/owl.json).

• OpenWord_detector Node Settings (in Edge Agent UI):

  • Model: Select the detection model, e.g., owlv2.
  • Json Path: Set to /opt/NanoLLM/nanoowl/owl.json.
  • Activate: Ensure this is set to True for detection to occur.
  • Other parameters like Drop inputs and Return Copy should be set as per the preset (typically True).

• Customizing Detected Outfits (via owl.json file): This JSON file defines what items to detect, detection sensitivity, and logic. You will need to edit this file on your Jetson device to change detection parameters.

  • MainObject: Defines the primary object to focus on (e.g., "person").
    • conf: Minimum confidence score for the main object.
    • minArea: Minimum pixel area for the main object.
  • Prompts: A dictionary where each key is an outfit item to detect (e.g., "face shields", "white glove", "aluminized heat protection apron").
    • abbr: An abbreviation or alternative name.
    • conf: Minimum confidence score for this specific item.
    • minArea: Minimum pixel area for this item.
    • pos: "LR": Add this if the item needs to be detected on both Left and Right sides (e.g., for gloves), relative to the MainObject's center.
  • Compl: A list of items that are often misclassified. Adding items here can help the model differentiate better (e.g., adding "plam" if it's confused with "glove").
  • Detection Logic Parameters:
    • CheckPeriod, ValidChunkSize, ValidRatio: Control how consistently an item must be detected to be considered present (e.g., worn).
    • OKWaiting, NGWaiting: Determine how long "OK" or "NG" (Not Good / Missing) states persist after an assessment.
  • ROI: Region of Interest [x_min, y_min, x_max, y_max] (normalized 0.0-1.0) to focus detection if needed.

  Example Prompts entry for a "safety helmet":

  "safety helmet": {
      "abbr": "helmet",
      "conf": 0.25,
      "minArea": 1500
  }

  You would add this within the "Prompts": {} block of the owl.json file.

1. Launch the Edge Agent UI in your browser.
2. Load the ATTIRE_DET_WEBRTC_ADVAN preset:
   • Click the "Agent" menu in the top-right corner.
   • Select "Load."
   • Choose ATTIRE_DET_WEBRTC_ADVAN.json from the list of presets.
3. The pipeline will appear in the Node Editor.
4. Ensure the VideoSource node's "Input" parameter is correctly set to /data/videos/Attire_det_advan.avi.
5. The project should start running automatically.
6. Observe the VideoOutput panel:
   • You will see the video playing.
   • Detected personnel (as per MainObject) and outfit items (as per Prompts) will be highlighted with bounding boxes.
   • An overlay text will indicate the detection status (e.g., "NG status") and list any "Missing" items that were defined in your owl.json but not detected.
7. Listen for audio alerts: If any required items are detected as missing, the PiperTTS module will announce them (e.g., "Missing face shields, white glove left").

• Visual Output: The VideoOutput will show the processed video stream. Compliant individuals will have their detected outfit items outlined (often in green). If items are missing based on the JSON configuration and detection logic, the status will indicate "NG," and missing items will be listed.
• Audio Output: Voice alerts will announce the names of any outfit items that are required (as per Prompts in owl.json) but not detected on the person.
• JSON Output (for integration): The OpenWord_detector node also generates a JSON output (Json and AllDetRes ports) containing detailed detection results, which can be published via MQTT or used by other custom nodes if further integration is needed.

• No Detection:
  • Verify the Activate switch in the OpenWord_detector node is set to True.
  • Check the owl.json path in the OpenWord_detector settings.
  • Ensure the conf and minArea thresholds in owl.json are appropriate for your video and items. If thresholds are too high, items might not be detected.
• Incorrect Detection:
  • Adjust conf and minArea for specific items.
  • Utilize the Compl list in owl.json to help differentiate commonly confused items.
  • Consider using the OpenWord_FineTune project to improve model accuracy for specific, hard-to-detect items.
• No Audio Alerts:
  • Check if the PiperTTS module is correctly connected to the message output of the OpenWord_detector.
  • Ensure your system's audio is working and unmuted.
• Customizing for Different Scenarios: The primary method of adaptation is by carefully editing the owl.json file to define the MainObject, the desired Prompts (outfit items), and the detection logic parameters.

⬆️ Top

---