Service: Browserhand - EyevinnOSC/community GitHub Wiki

Getting Started

Browserhand is a reliable, deterministic web-automation API for AI agents. It runs managed headless browser sessions and exposes them through a small, high-signal HTTP and WebSocket API. Agents perceive a page as a selector-free accessibility snapshot with stable element refs, act on it by ref, read content, capture screenshots, and reuse session state. There is no LLM inside the service — the calling agent does the reasoning; Browserhand reliably executes and reports back.

Available as an open web service in Eyevinn Open Source Cloud.

Prerequisites

Step 1: Create a Browserhand instance

Navigate to the Browserhand service in the Eyevinn OSC web console. Click "Create browserhand" and enter a name for your instance.

Optional configuration:

Option Default Description
MaxConcurrentSessions 5 Hard cap on simultaneous sessions. Requests beyond the cap receive a session_limit_reached (429) error.
SessionTimeoutMs 300000 Idle timeout in milliseconds (5 minutes) before a session is auto-closed.

Once the instance card shows status "running", click it to see the instance URL — this is your Browserhand endpoint.

Authentication

If BROWSERHAND_API_KEY is set on the instance, every request to /v1/* must include:

Authorization: Bearer <your-api-key>

Health check endpoints (/healthz, /readyz) remain open regardless.

Step 2: Session lifecycle

Browserhand uses a session-based model. The typical agent loop:

POST /v1/sessions                          -> { sessionId, connectUrl, status }
POST /v1/sessions/:id/navigate { url }
loop:
  GET  /v1/sessions/:id/snapshot           -> nodes:[{ref, role, name, value, ...}], outline
  # the agent decides what to do from the snapshot
  POST /v1/sessions/:id/click  { ref }
  POST /v1/sessions/:id/type   { ref, text, submit? }
  GET  /v1/sessions/:id/content            -> read result / extract data
DELETE /v1/sessions/:id

A ref is a stable element identifier returned by the snapshot. All action endpoints address elements by ref, never by CSS selector or XPath.

Step 3: API reference

Method and path Purpose
POST /v1/sessions Open a session. Returns { sessionId, connectUrl, status }. Optional body: viewport, timeoutMs, context.
GET /v1/sessions List all open sessions.
GET /v1/sessions/:id Get status of a session.
DELETE /v1/sessions/:id Close a session.
GET /v1/sessions/:id/snapshot Accessibility snapshot: elements with stable refs plus a readable outline.
GET /v1/sessions/:id/content Readable page text. Includes a truncated flag when the page is long.
GET /v1/sessions/:id/screenshot PNG screenshot. Query params: ?fullPage=true, ?ref=eN for a single element.
POST /v1/sessions/:id/navigate Navigate to a URL. Body: { url }. Also supports /back, /forward, /reload sub-paths.
POST /v1/sessions/:id/click Click an element by ref. Body: { ref }.
POST /v1/sessions/:id/type Fill a field. Body: { ref, text, submit? }. Set submit: true to press Enter after typing.
POST /v1/sessions/:id/select Select option(s) in a select element. Body: { ref, values }.
POST /v1/sessions/:id/hover Hover over an element. Body: { ref }.
POST /v1/sessions/:id/press Press a key. Body: { key }.
POST /v1/sessions/:id/scroll Scroll. Body: { direction } or { ref }.
POST /v1/sessions/:id/wait Wait for a condition. Body: { text?, ref?, timeoutMs? }.
POST /v1/sessions/:id/upload Set files on a file input. Body: { ref, files[] } (base64 contents).
GET /v1/sessions/:id/logs Captured browser logs. Query: ?type=network or ?type=console.
GET /v1/sessions/:id/downloads List captured downloads.
GET /v1/sessions/:id/downloads/:downloadId Fetch a specific download.
GET /v1/sessions/:id/context Export cookies and localStorage for session reuse (pass back as context on create).
GET /v1/connect/:id (WebSocket) Remote Chrome DevTools Protocol endpoint for power users.
GET /healthz Liveness check.
GET /readyz Readiness check.

The interactive API docs (Swagger UI) are available at <instance-url>/documentation.

Error handling

All errors share one shape with a stable, machine-readable code:

{
  "error": {
    "code": "stale_ref",
    "message": "Element ref \"e5\" is no longer on the page. Call GET /v1/sessions/:id/snapshot to get fresh refs, then retry.",
    "details": { "ref": "e5" }
  }
}

Common error codes:

Code HTTP status Meaning and action
session_not_found 404 Session ID does not exist or has been closed. Open a new session.
session_limit_reached 429 MAX_CONCURRENT_SESSIONS cap hit. Wait for a session to close or raise the limit.
stale_ref 409 The element ref is no longer on the page. Call GET /snapshot again to get fresh refs, then retry the action with the updated ref.
element_not_actionable 409 The element exists but cannot be interacted with (e.g. hidden). Re-snapshot and reassess.
navigation_failed 502 The browser could not reach the URL.
timeout 504 The action timed out.
invalid_request 400 Malformed request body.
unauthorized 401 Missing or invalid Authorization: Bearer token.

Stale ref pattern: Whenever you receive a stale_ref error, call GET /v1/sessions/:id/snapshot to retrieve current refs, then retry the action with the fresh ref. Do not cache refs across navigations.

How OSC agent tasks authenticate

Inside an agent task (for example, inside the Agentic SDLC or My Agent Tasks), authenticate to your Browserhand instance as follows:

  1. Call describe-service-instance with the Browserhand service ID and your instance name to get the instance URL.
  2. Retrieve the BROWSERHAND_API_KEY value from the parameter store using get-parameter.
  3. Include Authorization: Bearer <api-key> on every /v1/* request.
# Example from an agent task shell step
INSTANCE_URL="<url from describe-service-instance>"
API_KEY="<value from parameter store>"

SESSION_ID=$(curl -s -X POST "$INSTANCE_URL/v1/sessions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' | jq -r '.sessionId')

curl -s -X POST "$INSTANCE_URL/v1/sessions/$SESSION_ID/navigate" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com"}'

curl -s "$INSTANCE_URL/v1/sessions/$SESSION_ID/screenshot" \
  -H "Authorization: Bearer $API_KEY" \
  --output screenshot.png

curl -s -X DELETE "$INSTANCE_URL/v1/sessions/$SESSION_ID" \
  -H "Authorization: Bearer $API_KEY"

Remote browser (CDP)

For advanced use cases, connect using the Chrome DevTools Protocol via the connectUrl returned on session creation:

import { chromium } from 'playwright';
const browser = await chromium.connectOverCDP('wss://<instance-url>/v1/connect/<sessionId>');
const page = browser.contexts()[0].pages()[0];

Notes

  • Autoplay: Browserhand uses headless Chromium. Standard browser autoplay policies apply — autoplay of video or audio may be blocked unless the page has prior user gesture state.
  • Resource sizing: Chromium is memory-heavy. Budget approximately 300-500 MB per concurrent session and size MaxConcurrentSessions accordingly.
  • Stateless: No persistent disk is required. Downloads and other artifacts are ephemeral and live only for the duration of the session.
  • Graceful shutdown: The service drains all sessions on SIGTERM.

CLI usage

osc create birme-browserhand myautomation \
  -o MaxConcurrentSessions="5" \
  -o SessionTimeoutMs="300000"

Resources

⚠️ **GitHub.com Fallback** ⚠️