ROS2 and AI Experiments - slgrobotics/articubot_one GitHub Wiki

                         ┌───────────────────┐
                         │   Strategy API    │
                         │  model-agnostic   │
                         └─────────┬─────────┘
                                   │
                     ┌─────────────┼──────────────┐
                     │             │              │
                     ▼             ▼              ▼
                  GPT/Claude    Ollama        Gemini
                   cloud        local          cloud
                                 RTX 3060
                     │             │              │
                     └─────────────┼──────────────┘
                                   │
                                  MCP
                                   │
                         ┌─────────▼─────────┐
                         │  ros-mcp-server   │
                         │      Seggy        │
                         └─────────┬─────────┘
                                   │
                                  ROS2
                                   │
              ┌────────────────────┼────────────────────┐
              │                    │                    │
             Nav2              perception          behaviors
              │                    │                    │
              └────────────────────┼────────────────────┘
                                   │
                              robot hardware

                    AI SERVICES
                        │
          ┌─────────────┼─────────────┐
          │             │             │
       Strategy      Vision        Future AI
         LLM           VLM           services
          │             │             │
          └─────────────┼─────────────┘
                        │
                       MCP
                        │
             ┌──────────┴──────────┐
             │                     │
        ROS-MCP                Vision-MCP
             │                     │
             ▼                     ▼
           ROS2                HuskyLens
             │
             ▼
           Seggy

Connected successfully. It’s `ros-mcp-server` v2.13.0, with 31 tools for ROS/ROS 2 robot inspection and control.

You can:

- Discover connected robots, test ROSbridge connectivity, and detect the ROS version.
- Inspect ROS nodes, topics, services, actions, message types, and parameters.
- Read camera/image-topic data and retrieve saved images.
- Subscribe to topic messages once or over a time window.
- Inspect action state, send action goals, and cancel running goals.
- Call ROS services.
- Change or delete parameters.
- Publish individual messages or timed command sequences—for example, send velocity commands to `/cmd_vel`.

The server marks robot-affecting operations as destructive, including publishing commands,
 action goals, service calls, and parameter changes. 
Read-only discovery and monitoring tools are separately labeled.

The server currently sees four ROS 2 topics:

     Topic         |    Message type
------------------ | ------------------------------------
/client_count      | std_msgs/msg/Int32
/connected_clients | rosbridge_msgs/msg/ConnectedClients
/parameter_events  | rcl_interfaces/msg/ParameterEvent
/rosout            | rcl_interfaces/msg/Log

These are ROS/rosbridge system topics; no robot sensor or command topics are visible yet.

Refresh succeeded: the ROS graph now exposes **133 topics**—a Nav2 mobile-robot stack is active.

Key topics include:

- Motion commands: `/cmd_vel`, `/cmd_vel_nav`, `/cmd_vel_teleop`, `/cmd_vel_smoothed`
- Localization/navigation: `/amcl_pose`, `/map`, `/scan`, `/goal_pose`, `/plan`, `/waypoints`
- Robot state: `/odometry/local`, `/odometry/global`, `/joint_states`, `/imu/data`, `/battery/battery_state`
- Navigation costmaps: `/global_costmap/*`, `/local_costmap/*`
- Transforms: `/tf`, `/tf_static`
- Vision/interaction: `/face_gesture_detections`, `/fgs/*`, `/joy`
- Docking and safety: `/dock_pose`, `/docking_trajectory`, `/collision_monitor_state`, `/geofence/*`

The change from four system topics means the robot/navigation system has since come online.

Moved forward at 0.1 m/s for 2 seconds, then sent a zero-velocity stop.
The ROS server confirmed 21 messages published to `/cmd_vel` with no errors.

Nav2 planned a valid 40-pose path and began executing it.
The robot moved approximately **1.00 m** from its starting odometry position.

It stopped about 11 cm short of the exact Nav2 target while the original goal remained active,
 so I cancelled that lingering goal. Nav2 confirms that original goal is now **aborted**.

import time
from pyhuskylens import HuskyLens
from pyhuskylens import (
    HuskyLens,
    ALGORITHM_OBJECT_RECOGNITION,
    ALGORITHM_OBJECT_CLASSIFICATION,
)

print("Initializing HuskyLens 2 on I2C...")
# Pass 1 for /dev/i2c-1. The library automatically looks for the 0x50 address.
try:
    hl = HuskyLens(1)
    print("Success! Connected to HuskyLens 2.")
except Exception as e:
    print(f"Connection Failed: {e}")
    print("Double check that your port light is BLUE and i2cdetect shows 0x50.")
    exit()

print("IP: setting OBJECT_RECOGNITION algorithm...")
hl.set_alg(ALGORITHM_OBJECT_RECOGNITION)

# For "learned" objects:
# print("IP: setting ALGORITHM_OBJECT_CLASSIFICATION algorithm...")
# hl.set_alg(ALGORITHM_OBJECT_CLASSIFICATION)

print("Streaming object data... Press Ctrl+C to stop.")
try:
    while True:
        # Grab any active bounding boxes/blocks from the screen
        blocks = hl.get_blocks()
        if blocks:
            for block in blocks:
                # name       → semantic class ("chair")
                # ID         → learned-object/category ID
                # classID    → classification class ID (where supported)
                # confidence → model confidence, but not necessarily retrievable
                print(f"Name: {getattr(block, 'name', None)}  \t" \
                      f"Center: ({block.x}, {block.y})  " \
                      f"Size: {block.width} x {block.height}     " \
                      f"Type: {getattr(block, 'type', None)}  " \
                      f"ID: {getattr(block, 'ID', None)}  " \
                      f"Learned: {getattr(block, 'learned', None)}  " \
                      f"Confidence: {getattr(block, 'confidence', None)}  " \
                      f"Content: {getattr(block, 'content', None)}")
                # print("------------------------------")
        time.sleep(0.2)
except KeyboardInterrupt:
    print("\nExiting script.")

person
bicycle
car
motorcycle
airplane
bus
train
truck
boat
traffic light
fire hydrant
stop sign
parking meter
bench
bird
cat
dog
horse
sheep
cow
elephant
bear
zebra
giraffe
backpack
umbrella
handbag
tie
suitcase
frisbee
skis
snowboard
sports ball
kite
baseball bat
baseball glove
skateboard
surfboard
tennis racket
bottle
wine glass
cup
fork
knife
spoon
bowl
banana
apple
sandwich
orange
broccoli
carrot
hot dog
pizza
donut
cake
chair
couch
potted plant
bed
dining table
toilet
tv
laptop
mouse
remote
keyboard
cell phone
microwave
oven
toaster
sink
refrigerator
book
clock
vase
scissors
teddy bear
hair drier
toothbrush