03 preset projects - advantech-EdgeAI/edge_agent GitHub Wiki
To help you get started quickly with the Edge Agent, several preset projects are available. These presets are ready-to-run examples demonstrating key functionalities like object detection, visual language model applications, and automated alerts. Loading a preset allows you to immediately explore a working pipeline and understand its configuration before building your own.
The Edge Agent includes several preset projects to demonstrate its capabilities across various applications. These pre-configured pipelines offer a practical way to understand how different nodes and AI models can be combined to create powerful edge AI solutions. The table below provides a comparison of the key features of these projects.
Project Name | Primary AI Model(s) Used | Key Task | Key Features |
---|---|---|---|
Factory Outfit Detection | OpenWord (e.g., owlv2) | Detect if personnel are wearing required attire/PPE. | Customizable detection items via JSON; L(left)/R(right) specific detection; voice alerts. |
Smoke and Fire Detection | VLM (e.g., Llama-3-VILA-1.5-8B) | Detect the presence of smoke or fire in video footage. | Immediate alarm upon detection; uses prompt-engineered VLM for classification. |
Clearance Space Detection | VLM (e.g., VILA-1.5-13B) | Monitor a restricted area and alert if items are placed there. | Utilizes In-Context Learning (ICL) with a reference image; Region of Interest (ROI). |
Opened Door Detection | VLM (e.g., VILA1.5-3b), NanoDB (CLIP) | Monitor door status; alert if open too long; identify attire of passersby. | Two-step alert logic; image capture on event; multimodal database search. |
Adaptive Door Status Detection | VLM (e.g., VILA-1.5-13B), NanoDB (CLIP) | Enhance Visual Question Answering (VQA) model accuracy using RAG. | Multi-step process (error tagging, DB learning, intelligent retrieval); dynamic context enhancement. |
OpenWord FineTune | OpenWord (e.g., YOLOWorldV2_V8X) | Improve detection accuracy of an OpenWord model via training. | Utilizes user-annotated data (LabelMe); involves a model training cycle. |
Loading and running the preset projects in the Edge Agent is straightforward. These presets provide a quick way to see the agent's capabilities in action.
-
Ensure prerequisites are met:
- The Edge Agent is installed and running on your Jetson device.
- You can access the Edge Agent UI via your web browser at
https://IP_ADDRESS:8050
. - Essential data like demo videos and images required by the presets have been downloaded and placed in the correct directories (e.g.,
/ssd/jetson-containers/data/videos/
,/ssd/jetson-containers/data/images/
,/ssd/jetson-containers/data/datasets/
).
Note
If you follow the installation guide from this repository's README.md
, then all the data will be properly downloaded and placed.
-
Loading a preset project: Each example project typically has a corresponding preset name (e.g.,
DOOR_ALERT_WEBRTC_ADVAN
,ATTIRE_DET_WEBRTC_ADVAN
).There are multiple ways to load a preset:-
Using the "Agent" menu (recommended for full pipelines):
- In the Edge Agent UI, click on the "Agent" button in the top-right corner.
- Select "Load" from the dropdown menu.
- A list of available saved presets (JSON files from the
jetson-containers/data/nano_llm/presets directory
) will appear. Select the desired preset name corresponding to the project you want to run.
-
Right-clicking in the node editor (often for subgraphs):
- Right-click anywhere in the main Node Editor canvas.
- A context menu should appear. Look for options like "Load," "Insert," or a submenu for presets/subgraphs. Select the desired preset or subgraph (e.g., a pre-configured ASR or TTS module).
-
Using the
--load
flag at startup (For advanced users):- When starting the Edge Agent from the command line, you can use the
--load
option followed by the path to the preset JSON file to pre-load a specific pipeline.
- When starting the Edge Agent from the command line, you can use the
-
Using the "Agent" menu (recommended for full pipelines):
-
Running the project:
- Once a preset is loaded, the corresponding pipeline of nodes and their connections will automatically appear in the Node Editor.
- The project will typically start running automatically. Data will begin to flow from the
VideoSource
(if configured with a video file or camera) through the connected nodes. -
Observe the output:
- Check the
VideoOutput
panel in the UI. This will display the video feed, often with visual overlays like bounding boxes for object detection, text from VLM analysis, or status messages. - Listen for audio alerts if the project uses a Text-to-Speech (TTS) module like PiperTTS.
- Monitor the Terminal panel for log messages, detailed text outputs, or error messages.
- Interact with node-specific UI elements if the project uses them.
- Check the