ZMF4 AI and Zephyr Security - zephyrproject-rtos/zmf2026 GitHub Wiki

AI and Zephyr Security Discussion

← Agenda · ZMF#4 · 2026-10-06 · 9:00 AM - 10:45 AM

Key Speakers, Leads, and Required Attendees

  • David Brown, Linaro (Lead Organizer & Moderator)
  • Benjamin Cabé, Zephyr Project (Documentation, co-organizer)
  • Jukka Rissanen, networking maintainer

Decision or Outcome Sought

Make sure Zephyr gets the most out of AI without paying too high a toll for it: establish rules that keep a human accountable for every AI-assisted change, and keep the cost of verifying it off maintainers and the PSIRT.

By the end of this session, participants should have:

  • Agreed on:
    • Up to 3 additions to the AI Coding Assistants guidelines (D1).
    • A recommendation to the TSC on AI-assisted review (D2).
    • The bar for AI-assisted vulnerability reports in reporting.rst, and criteria for routing bulk AI-found bugs (D3).
    • Where should skills that developers in the project create reside (D4).
    • What level of review should skills have before they are published to the project level (D4).
    • What categories of skills should the project host (onboarding, security, etc.) (D4).
  • Identified:
    • Which review-load metrics to track (time to first review, review rounds, reverts; AI vs non-AI, per area).
    • PSIRT capacity gap and what it takes to close it (people, tooling).
  • Assigned (owners named in the room):
    • D1 guidelines PR: [owner]
    • D2 AI review recommendation to TSC: [owner]
    • D3 reporting.rst PR and sweep-routing criteria: [owner]
    • D4 skills location, review rules and first skills; RFC #108612 updated: [owner]
    • Review-load metric, first readout at the next TSC: [owner]

Problem Statement

Problem statement:

  • AI is already mainstream. Assisted-by: commits on main went from 1% (Mar 2026) to 36% (Sep 2026), from 155 authors. The tag is optional, so this is a lower bound.
  • The ecosystem isn't waiting for us. Third parties already publish Zephyr skills (beriberikix, Ac6) and vendors ship MCP servers (Nordic), none reviewed by maintainers or versioned with the tree. We have AGENTS.md and a docs MCP server, but no reference skills; RFC #108612 is stalled.
  • Review cost is unmeasured. No data on time to first review, review rounds or reverts, AI vs non-AI.
  • Security hurts most. 245 CVEs assigned in 2026 so far vs 14 in all of 2025. Linux and curl are hitting the same wall; OpenSSF/CNCF published guidance in May.
  • Our rules are thin. DCO, licensing and a "should" Assisted-by: tag. Nothing on explaining the change, review replies, autonomous agents, large AI series or AI-assisted vulnerability reports.

Scope

In scope

  • Contributor-side rules for AI-assisted PRs (guidelines.rst, alignment with AGENTS.md).
  • AI-assisted review: what works today (opt-in Copilot, ad-hoc LLM reviews), what a pilot looks like.
  • Intake of AI-found bugs and AI-assisted vulnerability reports.
  • Agent-facing files: AGENTS.md and project-hosted skills.

Out of scope

  • CRA reporting obligations (ZMF#2; D3 outputs handed over there).
  • Static analysis, code scanning, scheduled AI checks in CI (ZMF#5).
  • Backport policy (ZMF#2).
  • Copyright beyond DCO (LF legal).
  • AI/ML features in Zephyr itself.
  • PSIRT deep dive: "AI in Zephyr PSIRT: Lessons From the Front Line" (Flavio Ceolin, David Brown, OSS EU).

Questions Requiring Agreement

Decision 1: Closing the gaps in the guidelines

Decision required: Pick at most 3 rules to add.

Already covered today: the human submitter takes full responsibility and signs the DCO; Assisted-by: disclosure ("should"); reviewers vet AI output and never paste raw LLM output; low-quality PRs may get changes requested without detailed feedback.

Possible outcomes:

  • Explain and defend: the submitter can explain every change without the tool (Linux).
  • Replies to review are the submitter's own; translation tools are fine (LLVM, RIOT).
  • No autonomous agents opening PRs/issues or commenting without per-action human approval; CI bots excluded (LLVM, RIOT).
  • Large AI-generated series or tree-wide sweeps: issue first, area maintainer buy-in (QEMU proposal).
  • Disclosure becomes "must"; keep tool:model or move to product-neutral Assisted-by: LLM (NuttX, Linux).
  • Fold AGENTS.md-only rules into guidelines.rst (one trailer, state what was not built/tested).

Decision 2: AI-assisted review

Decision required: How far do we go beyond per-reviewer opt-in?

Possible outcomes:

  • Status quo: per-reviewer opt-in Copilot, advisory (#89343).
  • Pilot a review system with automated first-pass review. Maybe pilot in 2-3 volunteer areas. Need to measure impact against TBD review-load metric.
  • Project-wide first pass on every PR, with per-area opt-out.

Decision 3: Security intake

Decision required: What do we require from AI-assisted reports, and when do AI-found bugs go to PSIRT first?

Possible outcomes:

  • Report bar in reporting.rst: AI use disclosed, working reproducer, human-written impact analysis, fix proposal welcome; reports below the bar may be closed without triage (OpenSSF/CNCF guide, curl).
  • Threat model clarification so out-of-model findings don't become CVEs (e.g. malicious bus peripheral, unsupported configs).
  • Criteria for when bulk AI-found bugs go through PSIRT instead of public PRs (e.g. userspace boundary, network/BT input parsing, crypto).

Decision 4: Project skills

Decision required: Where do project skills live, what review do they get before they are published, and which categories do we host first? RFC #108612 / PR #113542 have stalled on exactly this. SKILL.md is an open format read by Claude Code, Codex, Copilot, Cursor and Gemini CLI.

Possible outcomes:

  • Where:
    • Nowhere; improve the human docs instead.
    • Separate repo (no code dependency on the tree).
    • In-tree under .agents/skills/.
  • Review level:
    • Same as code: owned by the area's maintainers in MAINTAINERS.yml.
    • Docs-style: curated by the Doc WG or Process WG.
    • Either way: skills point to docs instead of restating them, CI checks referenced paths exist, each skill ships a reference task to check it still works (e.g. re-add an existing board).
  • Categories:
    • Contribution tasks: add a board, add a sensor driver, add a test...
    • Onboarding and app development: upgrade an app across releases (#113542).
    • Security: route a suspected vulnerability to the right channel; secure-coding checks for syscall verifiers and input parsing.
    • Not a skill: commit and PR rules stay in AGENTS.md, which agents always load.

Proposed Session Agenda

  • 9:00 Framing and data (5 min)
  • 9:05 Case study: QUIC, written and reviewed with AI (14 min)
    • Jukka Rissanen, author side: building the QUIC stack with AI, and the TTCN-3 conformance suites as the guardrail (7 min)
    • David Brown, reviewer side: the Claude deep analysis of #103932, merged as experimental, follow-ups fixed the concurrency and flow-control findings; then PSIRT under AI load (7 min)
  • 9:19 D1 guidelines (22 min), facilitated by a Process WG member
  • 9:41 D2 AI-assisted review (12 min)
  • 9:53 D3 security intake (20 min)
  • 10:13 D4 project skills (18 min)
  • 10:31 Close: confirm decisions, owners, dates (14 min)

Pre-Session Preparation

Session owner

Participants

  • Read the document before the event.
  • Add one useful and one useless AI review comment you have seen on a Zephyr PR.
  • Comment on the D1 shortlist and D4 options; flag strong objections before the session.
  • Area maintainers: say if you'd volunteer your area for the D2 review pilot or a D4 skill.