The harness (crates/tinyagents-harness/) is the model/tool orchestration layer: the agent loop, provider-neutral model calls, typed tools, middleware, structured output, streaming, usage/cost accounting, and observability. It does not depend on the graph runtime — you can run a harness agent on its own, without building a graph.

Message, model-request/response, provider, and embedding types are not owned by this crate. They live in the vendored tinyinference workspace (vendor/tinyinference/crates/): chat types in tinyinference-llm and embeddings in tinyinference-embeddings. They are used directly as tinyinference_llm::message::Message, tinyinference_llm::model::ModelRequest, and so on — tinyagents-harness is not a facade over them, it depends on them.

Cargo features (crates/tinyagents-harness/Cargo.toml): claude-code and langfuse are on by default. Opt-in: sqlite (bundled SQLite via rusqlite), storage-drivers, builtin-tools (pulls in chrono-tz for the built-in tool set; tools is a deprecated alias for it), multimodal, png-optimize, media, and tracing (a no-op: tracing instrumentation is always compiled in). Sub-agents, teams and workflows are not in this crate; they live in tinyagents-orchestration (see Sub-agents).

The loop is implemented as async methods on AgentHarness<State, Ctx> (crates/tinyagents-harness/src/agent_loop/entry.rs):

• invoke(state, ctx_data, config, input) — full control over run identity and context data.
• invoke_default(state, input) — convenience wrapper that builds a default RunConfig.
• invoke_with_status / invoke_in_context / invoke_in_context_with_status — variants that run inside an existing RunContext, which is how a sub-agent call inherits the parent's identity.
• invoke_streaming, invoke_streaming_default, invoke_streaming_in_context[_with_status] — the same loop, emitting ModelStreamItems.
• invoke_collecting_partial / invoke_in_context_collecting_partial — streaming variants that also collect the final AgentRun.

The lifecycle (model -> tool -> model):

input messages
  -> build RunContext, emit RunStarted
  -> before_agent middleware
  -> loop:
       enforce model-call cap + wall-clock deadline (fail-closed)
       build ModelRequest (messages + tool schemas + response format)
       before_model middleware; emit ModelStarted
       resolve + invoke model (with retry + fallback)
       after_model middleware; emit ModelCompleted; fold Usage into AgentRun
       append assistant message
       if tool calls -> enforce tool cap, before_tool, run tools,
                        after_tool, append tool results, continue
       else -> extract structured output (if configured) and break
  -> after_agent middleware; emit RunCompleted

On error the loop emits RunFailed, runs on_error middleware, and returns the error. Retry backoff is computed via RetryPolicy::backoff_for_attempt without the loop itself sleeping, which keeps tests deterministic.

AgentRun (from middleware::AgentRun) holds the final messages, folded Usage, and any extracted structured response.

Model invocation goes through tinyinference_llm::model::ChatModel<State>, with invoke (unary) and stream (incremental). A call is a ModelRequest, answered by a ModelResponse carrying the assistant message, Usage, finish reason, and a ResolvedModel recording which provider/model actually served the call. tinyagents_harness::model_registry::ModelRegistry resolves a named model registration to a concrete ChatModel handle; ModelSelection and ResolvedModelBinding record what was chosen and why.

tinyinference_llm::providers holds the adapters: MockModel for offline tests, and providers::openai::OpenAiModel, which speaks the OpenAI Chat Completions wire format and also serves every OpenAI-compatible endpoint through preset constructors (deepseek, anthropic, groq, xai, openrouter, together, mistral, ollama, plus compatible(base_url, model) for anything else). See Providers.

Tools implement the shared tinytools::Tool trait (the harness depends on it rather than defining its own) with name, description, parameters_schema() -> serde_json::Value, and an async execute(args) -> anyhow::Result<ToolResult> (plus execute_with_context and an optional policy()). ToolCall (from tinyinference_llm::tool) carries an id, name, and JSON arguments; tinytools::ToolResult is built with ToolResult::success(...) / ToolResult::error(...). The harness's tool::ToolRegistry registers Arc<dyn tinytools::Tool> and keys tools by name and is consulted by the loop when the model requests one.

An entire agent can be exposed as a tool — see Sub-agents.

A tool can also call another tool through the harness with ToolExecutionContext::call_tool; the nested call goes through the same admission and execution path as a model-issued call (allow-list, argument validation, timeouts, run budget), and never defers its parent. Nested calls are off by default (RunLimits::max_nested_depth is 0); opt in with RunLimits::with_max_nested_depth. A middleware that enforces policy only in before_tool must also implement check_nested_tool to bind nested calls. See tool/README.md.

Middleware<State, Ctx> exposes hooks the loop calls into: before_agent/after_agent, before_model/on_model_delta/after_model, before_tool/on_tool_delta/after_tool, and on_error. They run through a MiddlewareStack (before_* in registration order, after_* in reverse). Built-ins under middleware::library include logging, message trimming, context compression, prompt-cache guarding, and usage accounting middleware, alongside the resilience, tool-policy, budget, and memory-protocol middleware in middleware::library.

StructuredExtractor extracts a typed value from the final ModelResponse according to a StructuredStrategy. response_format_for_strategy(...) maps a strategy onto a ResponseFormat. The parsed value lands in StructuredOutput, exposing as_value() and parse::<T>(). Set RunPolicy::default_response_format to attach a schema to every model request in a run; the loop runs extraction automatically before completing. See the openai_structured example.

At the provider edge, ChatModel::stream yields a ModelStream of ModelStreamItems. At the harness edge, stream::StreamSink exposes StreamChunks with selectable StreamModes. The invoke_streaming* agent-loop methods surface model deltas as the loop runs.

Usage (from tinyinference_llm::usage) records input/output tokens, cached-token bookkeeping, and reasoning tokens when a provider reports them. The loop folds each call's Usage into the AgentRun. cost::estimate_cost(pricing, usage) turns a tinyagents_harness::cost::ModelPricing and Usage into CostTotals. Because a sub-agent call executes through the parent's RunContext, child usage and cost roll up into the parent's totals.

RunLimits (with_max_model_calls, with_max_tool_calls, with_max_wall_clock_ms, with_max_retries_per_call, with_max_tool_concurrency, with_max_depth) is enforced fail-closed. Reaching a cap returns TinyAgentsError::LimitExceeded; the deadline returns TinyAgentsError::Timeout. max_depth bounds how deep nested sub-agents may recurse — exceeding it fails with TinyAgentsError::SubAgentDepth.

retry provides RetryPolicy (attempts, backoff, multiplier, jitter), an is_retryable(err) classifier, FallbackPolicy (an ordered model fallback chain), and a token-bucket RateLimiter. Failover is decided by reason: FailoverReason::classify names why a call failed and decide chooses retry-same, fall back, or surface (for example an auth error falls back at once rather than retrying). See retry/README.md.

Bound per-tool timeouts by installing ToolTimeoutSettings on the harness:

use tinyagents_harness::runtime::AgentHarness;
use tinyagents_harness::tool::ToolTimeoutSettings;

let mut harness: AgentHarness<()> = AgentHarness::new();
harness.with_tool_timeout_settings(ToolTimeoutSettings::new(
    120_000, // inherited default
    1_000,   // minimum explicit budget
    3_600_000, // maximum explicit budget
    5_000,   // scheduling grace for explicit budgets
));

Tools return ToolTimeout::Inherit by default; a tool may opt out with Unbounded or request a clamped, explicit Millis budget. A per-tool deadline produces a recoverable tool-error message and the agent loop continues so the model can retry or choose another tool; the run's wall-clock deadline stays the outer hard abort.

Two distinct ideas:

• Local response cache — ResponseCache (with InMemoryResponseCache), keyed by cache_key(request). Attach it with AgentHarness::with_response_cache; the loop checks it before each provider call and stores successful responses.
• Provider prompt/KV-cache layout — PromptCacheLayout makes the stable prompt prefix explicit so middleware editing model-visible prompt segments can be detected. Keep thread_id stable across parent agents, sub-agents, and nested harness calls so a provider's prompt cache sees one logical conversation.

The harness has no memory module and no chat-history type. Long-term memory is a host capability: a host implements host::AgentMemory (recall, remember, thread_summary) and supplies it as HostCapabilities::memory (an Option; None means the deployment has no memory store). host::InMemoryAgentMemory is the bundled in-memory implementation. The runtime injects recalled items as the host returned them and does not re-rank or re-filter them. The event stream has MemoryLoaded and MemorySaved events, and middleware::library ships MemoryProtocolMiddleware, which enforces a read-index, write, update-index cycle over the host's memory tools. Durable conversation history lives in tinyagents-session (transcripts and the run ledger), not in the harness.

Retrieval is a separate seam: retriever::Retriever is the harness contract, while EmbeddingModel, VectorStore and cosine_similarity live in the tinyinference-embeddings crate (tinyinference_embeddings).

Sub-agents live in the tinyagents-orchestration crate (tinyagents_orchestration::subagent), not in the harness; the harness provides the primitives they compose (run contexts, depth limits, steering, events). See the orchestration README and the subagent README.

• SubAgent<State, Ctx> wraps an AgentHarness as a child run with inherited lineage, cancellation, events, host authority and recursion limits. invoke_in_parent threads the live parent RunContext, so the child runs at parent.depth() + 1 and its usage and cost roll up to the parent.
• SubAgentTool adapts a sub-agent into a Tool. Delegation is background by default: the tool returns a stable job id immediately, and mode: "inline" waits for the child's result in the same call.
• SubAgentJobRegistry records queued, running, completed, failed and cancelled jobs; it is backed by tinyagents_tasks::DetachedTaskRegistry. SubAgentJobsTool and SubAgentMessageTool query jobs and send messages to live children.
• SubAgentSession keeps one child and its transcript alive across turns.
• SpawnPolicy / SpawnAdmission bound fan-out (per-parent live cap, per-root total budget, target allowlist) with atomic reservation.
• SubagentDriver is the single engine for an agent step: it coordinates durable resume, preparation, execution, pause and terminal persistence through host-supplied SubagentPlanner / SubagentExecutor / SubagentPersistence seams. Team members (teams::run_member_graph / run_member_graph_with) and workflow agent children (WorkflowEngine::with_step_config) run through it via subagent::run_agent_step, so they get the same spawn admission, SubAgentPolicy (timeout/retry/budget), ResultPolicy, SubagentRole and typed SubagentOutcomeKind.
• A finished child can be pushed to its parent through the durable CompletionRouter from tinyagents-tasks (SubagentDriver::with_completion_router).

Deprecated: graph sub-agent nodes. tinyagents_graph::SubAgentNode and subagent_node (a graph node that embeds a harness agent) are #[deprecated(note = "use tinyagents_orchestration SubagentDriver")]; graph cannot depend on orchestration, so the node does not adapt the driver. Use SubagentDriver for new work. See Graph Runtime. The orchestrator_subagents example lives in crates/tinyagents-orchestration/examples/.

Steering is typed runtime control of an already-running agent, distinct from sub-agent session reuse (which acts between runs). A SteeringHandle sends a SteeringCommand — Pause, PauseWith { reason }, Resume, Cancel, InjectMessage(Message), Redirect { instruction }, or SetMetadata — and the loop drains pending commands at a safe checkpoint (before each model call). A SteeringPolicy allowlist checks each command's SteeringCommandKind (SteeringPolicy::new permits nothing by default; allow_all is for tests); a disallowed command is rejected with TinyAgentsError::Steering. Applying a batch yields a SteeringOutcome (Continue / Pause / Cancel). SwitchModel { model } changes the model mid-run; it is not part of SteeringPolicy::allow_all and needs an explicit opt-in (allow_all_with_model_switch or allow(SteeringCommandKind::SwitchModel)).

estimate_tokens(text) and TokenEstimate provide a cheap budget; trim_messages(messages, strategy) applies a TrimStrategy; and SummarizationPolicy decides when to compact, optionally derived from a model's own profile (from_profile, with_context_window). should_summarize(messages) / plan(messages) split the transcript, the Summarizer trait (with ConcatSummarizer) produces the summary, and SummaryRecord / CompressionProvenance record what was compressed.

events::AgentEvent enumerates lifecycle boundaries — run, model, tool, middleware, sub-agent, cache hit/miss, retry/rate-limit/fallback, usage/cost, limit-reached, memory, compression. Events flow through an EventSink to listeners and are journaled in an EventJournal; a run's overall state is HarnessRunStatus. Because a child run shares the parent's EventSink, one stream shows the whole call tree with correct depth annotations.

observability is the durable side: AgentObservation wraps each event with correlation metadata (event_id, run_id, parent_run_id/root_run_id, stream offset, timestamp). HarnessEventJournal (InMemoryEventJournal, StoreEventJournal<A>) durably records observations; HarnessStatusStore (InMemoryStatusStore) snapshots run status. Sinks compose: FanOutSink (multiplex), RedactingSink (scrub sensitive fields), JournalSink (write through to a journal), JsonlSink (append JSONL). AgentLatencyMetrics::from_record(...) summarizes a run's timing.

Two additions to the event stream: TurnStarted / TurnCompleted bracket each model turn and MessageAppended mirrors the working transcript, so a consumer can rebuild it from events alone; and a running tool reports live progress through ToolRunContext::report_progress, which the loop emits as ToolProgressDetail (wire kind tool.progress; the older ToolProgress variant is kept but no longer emitted). Run boundary events carry a typed TerminalOutcome (terminal module: a TerminalReason such as Completed, Timeout, Cancelled, Halted or ProviderFailed, plus a TerminalClass) instead of only a free-form string. See events/README.md.

Export observations to Langfuse:

use tinyagents_harness::{LangfuseClient, LangfuseTraceConfig};

let client = LangfuseClient::proxy("https://api.tinyhumans.ai", backend_jwt)?;
client
    .send_observations(
        LangfuseTraceConfig {
            user_id: Some("user_123".to_string()),
            session_id: Some("thread_abc".to_string()),
            ..Default::default()
        },
        &observations,
    )
    .await?;

LangfuseClient::proxy sends through a backend /telemetry/langfuse/ingestion endpoint with bearer auth; LangfuseClient::direct(langfuse_url, public_key, secret_key) talks to Langfuse directly. tinyagents_graph::GraphLangfuseExporter reuses the same transport for graph runs (see Graph Runtime).

PromptTemplate renders {name} substitutions ({{ and }} escape literal braces) with a TemplateRole, and MessagesTemplate renders an ordered multi-message conversation. PromptBuilder assembles a request out of named, ordered segments — push_system, push_tools_segment, push_instructions, push_history, push_volatile — then build(tail) emits a ModelRequest. Stable segments before volatile ones let fingerprint() identify the cacheable prefix, used by the prompt-cache layout in cache.

ScriptedModel / StreamingMock / SlowModel fake providers, FakeTool, DeterministicClock, DeterministicIds, EventRecorder, and a Trajectory assertion helper (from_events(...).assert_tool_called(...), assert_model_called_times(n), assert_order(&[...]), assert_completed()) make loop behavior deterministically testable.

A stuck turn is caught in stages rather than by a single cap. Repeated identical failures escalate through an ordered ladder: keep going, nudge the model to change approach, then halt. Repeated successful no-op calls (or a short repeating cycle) escalate too: the first threshold warns, a call that keeps repeating is blocked before it executes, and a second block of the same call halts. See no_progress/README.md.

When a model calls an unregistered tool, RunPolicy::unknown_tool: UnknownToolPolicy decides what happens: Fail (default) aborts with TinyAgentsError::ToolNotFound; ReturnToolError injects a tool-error result and continues so the model can retry; Rewrite { tool_name } retargets the call to a fixed compatibility tool once, falling back to ReturnToolError if that target is also missing. Every recovery consumes a tool-call slot and emits AgentEvent::UnknownToolCall.

A WorkspaceDescriptor (root, trusted roots, policy_id, SandboxMode) tells a tool which paths it may touch; RunContext::with_workspace(descriptor) threads it into every tool call context. WorkspaceIsolation providers prepare and tear down per-agent environments (SharedRootWorkspace is the built-in). workspace::enforce_workspace_path(&descriptor, path, &events) is a fail-closed gate: an out-of-root path emits WorkspaceViolation and returns TinyAgentsError::Validation.

Each tool advertises a serializable ToolPolicy (side_effects, runtime, access, classified) via Tool::policy(). ToolPolicyMiddleware enforces it both at exposure (before_model hides the tool) and at execution (before_tool rejects with Validation).

ContextualToolSelectionMiddleware filters model-visible tools per call using a predicate over ToolSelectionContext { run_id, depth, tags, requested_model }, emitting AgentEvent::ToolsFiltered whenever it withholds tools. ContextualToolSelectionMiddleware::inheriting(parent_allow, parent_deny, child_allow, child_deny) composes a child policy against an inherited parent one so a sub-agent can only narrow (deny is additive, allow is intersective).

A middleware can steer the loop out-of-band with RunContext::request_control(MiddlewareControl): StopWithFinal(text) ends the run with a final answer, Interrupt { node, message } pauses at the next safe checkpoint. The highest-precedence pending request wins (Interrupt > StopWithFinal), so a pause is never downgraded to a stop.

BudgetMiddleware enforces BudgetLimits across a run, or a whole sub-agent tree when a shared BudgetTracker is handed to every sub-agent. Beyond token/cost limits it adds max_cached_input_tokens and a preflight reservation: before_model estimates the upcoming call's input tokens and blocks before dispatch if it would breach budget; after_model reconciles the estimate against provider-reported usage.

• Graph Runtime — durable typed state graphs and subgraphs.
• Providers — configuring hosted providers.
• Examples — annotated, runnable catalog.