2026 09 16 package restructure - kubed-io/selenium-flow GitHub Wiki
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Reorganise 29 flat modules into subpackages with the app on top, collapse four request wrappers into one, move the OpenAPI schema data out of Python, and make every description and document true and surface-neutral โ with no change to behaviour.
Architecture: server.py, main.py, routes.py and errors.py stay at the package root as the app itself. Independent components move into core/, session/, flows/, http/, mcp/ and spec/. One http/answer.py owns authorise โ read body โ name session โ run โ shape failure; errors.as_response becomes the single failure-to-JSON site; http/mount.py mounts the declared route tables that already exist.
Tech Stack: Python 3.10โ3.14, FastMCP 4.0.3, Starlette, pytest, setuptools + setuptools_scm, redocly (spec lint), ruff (E501 at 88 chars).
Spec: docs/superpowers/specs/2026-09-16-package-restructure-design.md
- Refactor only. No endpoint, tool signature, result shape or behaviour changes. Nothing gained, nothing lost.
- Existing test assertions are frozen. Import lines change; expectations do not. If an assertion seems to need editing, the move is wrong.
- The suite is 1,150 passed, 18 skipped and must stay so, except where a task explicitly adds a test.
- Run tests with:
PYTHONPATH=$DEPS:. python3 -m pytest -q -p no:cacheprovider tests --ignore=tests/integrationwhere$DEPSis the scratchpad deps directory (no venv in this pod). - Ruff line limit is 88.
git diff -U0 -- '*.py' | grep '^+' | awk 'length > 89'must print nothing before any commit. - Every new test is proved by breaking the thing it guards, watching it fail, then restoring.
-
js.pyandprompts.pyeach load files from a sibling directory of the same name, mapped by[tool.setuptools.package-dir]. Moving either module means moving its mapping key in the same commit. - Baselines to beat, measured 2026-09-16 on
3a57900: 28 tools / 19,257 description bytes, spec 96,071 bytes with 16,238 bytes of operation descriptions,INSTRUCTIONS1,379 bytes.
Nothing today asserts that pyproject's hand-written packages list covers the source tree. A subpackage missing from it ships a wheel with a module silently absent and CI green. This guard comes first because every later task adds a package.
Files:
- Modify:
tests/test_packaging.py
Interfaces:
-
Produces:
declared_packages()andsource_packages()helpers used by later tasks' verification. -
Step 1: Write the failing test
def declared_packages() -> set[str]:
"""Every package name pyproject says ships."""
data = tomllib.loads(PYPROJECT.read_text())
return set(data["tool"]["setuptools"]["packages"])
def source_packages() -> set[str]:
"""Every importable package in the source tree, as a dotted name."""
found = set()
for init in (REPO / "kubed").rglob("__init__.py"):
parts = init.relative_to(REPO).parent.parts
found.add(".".join(parts))
return found
def test_every_source_package_is_declared():
"""setuptools takes an explicit list here, so a package missing from it is
simply absent from the wheel โ no error at build, no error at import until
something reaches for it in an installed copy."""
missing = source_packages() - declared_packages()
assert not missing, (
f"add to [tool.setuptools] packages in pyproject.toml: {sorted(missing)}"
)- Step 2: Prove it is not vacuous
mkdir -p kubed/selenium_flow/_guardcheck && touch kubed/selenium_flow/_guardcheck/__init__.py
PYTHONPATH=$DEPS:. python3 -m pytest -q tests/test_packaging.py -k every_source_package
# Expected: FAIL naming kubed.selenium_flow._guardcheck
rm -rf kubed/selenium_flow/_guardcheck- Step 3: Run it clean
Run: PYTHONPATH=$DEPS:. python3 -m pytest -q tests/test_packaging.py
Expected: PASS
- Step 4: Commit
git add tests/test_packaging.py
git commit -m "A package missing from pyproject's list ships an absent module"Files:
- Create:
kubed/selenium_flow/core/__init__.py - Move:
browser.py,actions.py,probe.py,pointer.py,js.pyโcore/ - Modify:
pyproject.toml(packages list, package-dir key forjs) - Modify: every importer of those modules
Interfaces:
-
Produces:
from .core import browser, actions, probe, pointer, jsfor all later tasks. -
js.pykeepsPath(__file__).parent / "js", which is why its package-dir key moves with it. -
Step 1: Move the modules
mkdir -p kubed/selenium_flow/core && touch kubed/selenium_flow/core/__init__.py
git mv kubed/selenium_flow/{browser,actions,probe,pointer,js}.py kubed/selenium_flow/core/- Step 2: Update pyproject
In [tool.setuptools] packages add "kubed.selenium_flow.core" and change "kubed.selenium_flow.js" to "kubed.selenium_flow.core.js". In [tool.setuptools.package-dir] and [tool.setuptools.package-data] rename the kubed.selenium_flow.js key to kubed.selenium_flow.core.js, value unchanged ("js").
- Step 3: Fix imports in the package
grep -rln "from \.\(browser\|actions\|probe\|pointer\|js\) import\|from \. import.*\(browser\|actions\|probe\|pointer\|js\)" kubed/selenium_flow/Inside core/, siblings import each other as from . import browser. Outside it, from .core import browser. The four string-form patch targets in tests that name kubed.selenium_flow.browser / .actions become kubed.selenium_flow.core.browser / .core.actions.
- Step 4: Fix imports in tests
grep -rl "kubed.selenium_flow.\(browser\|actions\|probe\|pointer\|js\)\|from kubed.selenium_flow import.*\(browser\|actions\|probe\|pointer\|js\)" tests/Change the import path only. No assertion may change.
- Step 5: Run the suite and the packaging guard
Run: PYTHONPATH=$DEPS:. python3 -m pytest -q -p no:cacheprovider tests --ignore=tests/integration
Expected: 1,150 passed, 18 skipped.
- Step 6: Prove the wheel still carries the JavaScript
python3 -m build --wheel --outdir /tmp/w . 2>/dev/null || pip wheel --no-deps -w /tmp/w .
unzip -l /tmp/w/*.whl | grep -E "core/js/.*\.js|core/browser.py" | headExpected: kubed/selenium_flow/core/js/outline.js and friends are present.
- Step 7: Commit
git add -A && git commit -m "core/: the browser, the page, and the JavaScript it runs"Files:
- Create:
kubed/selenium_flow/session/__init__.py - Move:
sessions.py,store.py,settings.pyโsession/ - Modify:
pyproject.tomlpackages list; importers;tests/import lines
Interfaces:
-
Produces:
from .session import sessions, store, settings. -
The string-form patch target
kubed.selenium_flow.sessions.http_requestbecomeskubed.selenium_flow.session.sessions.http_request. -
Step 1: Move
mkdir -p kubed/selenium_flow/session && touch kubed/selenium_flow/session/__init__.py
git mv kubed/selenium_flow/{sessions,store,settings}.py kubed/selenium_flow/session/-
Step 2: Declare the package โ add
"kubed.selenium_flow.session"to the packages list. -
Step 3: Update importers and test import lines (assertions frozen).
-
Step 4: Run the suite โ Expected: 1,150 passed, 18 skipped.
-
Step 5: Commit
git add -A && git commit -m "session/: the caller's name, the record, the settings"Names lose the flow prefix they only carried to stay unique while flat.
Files:
- Create:
kubed/selenium_flow/flows/__init__.py - Move:
flowdoc.pyโflows/document.py,flows.pyโflows/library.py,flowrun.pyโflows/run.py,flowapi.pyโflows/api.py - Modify:
pyproject.toml; importers; test import lines
Interfaces:
-
Produces:
from .flows import document, library, run, api. -
flows.librarykeeps every public name it has today (valid_name,valid_session_name,RESERVED_SESSIONS,GLOBAL_SESSION,InvalidName). -
Step 1: Move with renames
mkdir -p kubed/selenium_flow/flows && touch kubed/selenium_flow/flows/__init__.py
git mv kubed/selenium_flow/flowdoc.py kubed/selenium_flow/flows/document.py
git mv kubed/selenium_flow/flows.py kubed/selenium_flow/flows/library.py
git mv kubed/selenium_flow/flowrun.py kubed/selenium_flow/flows/run.py
git mv kubed/selenium_flow/flowapi.py kubed/selenium_flow/flows/api.py-
Step 2: Declare the package โ add
"kubed.selenium_flow.flows". -
Step 3: Update importers and tests โ
tests/test_flows.py,tests/test_flowdoc.py,tests/test_flowapi.py,tests/test_flowrun.pychange import lines only. -
Step 4: Run the suite โ Expected: 1,150 passed, 18 skipped.
-
Step 5: Commit
git add -A && git commit -m "flows/: document, library, run, api"Files:
- Create:
kubed/selenium_flow/http/__init__.py - Move:
auth.py,links.py,files.py,admin.pyโhttp/ - Modify:
pyproject.toml; importers; test import lines
Interfaces:
-
Produces:
from .http import auth, links, files, admin. -
routes.pystays at the package root and importsfrom .http import auth. -
Step 1: Move
mkdir -p kubed/selenium_flow/http && touch kubed/selenium_flow/http/__init__.py
git mv kubed/selenium_flow/{auth,links,files,admin}.py kubed/selenium_flow/http/-
Step 2: Declare the package โ add
"kubed.selenium_flow.http". -
Step 3:
admin.pyfindsstatic/viaPath(__file__)โ checkstatic_path()still resolves after the move; the packaged copy iskubed/selenium_flow/static, now one level up fromhttp/. Fix the parent count and keep the source-checkout fallback. -
Step 4: Run the suite โ Expected: 1,150 passed, 18 skipped.
tests/test_files_and_admin.pyexercises the served page, which is what catches a wrong parent count. -
Step 5: Commit
git add -A && git commit -m "http/: auth, links, files, admin"Files:
- Create:
kubed/selenium_flow/mcp/__init__.py - Move:
tools.py,resources.py,skill.py,apps.py,prompts.pyโmcp/ - Move + rename:
hints.pyโmcp/annotations.py - Modify:
pyproject.toml(packages;kubed.selenium_flow.promptsโkubed.selenium_flow.mcp.promptsin package-dir and package-data)
Interfaces:
-
Produces:
from .mcp import tools, resources, skill, apps, prompts, annotations. -
annotations.hints()andannotations.reads()keep their names and signatures โ only the module name changes. -
prompts.pykeepsPath(__file__).parent / "prompts", so its package-dir key moves with it. -
Step 1: Move and rename
mkdir -p kubed/selenium_flow/mcp && touch kubed/selenium_flow/mcp/__init__.py
git mv kubed/selenium_flow/{tools,resources,skill,apps,prompts}.py kubed/selenium_flow/mcp/
git mv kubed/selenium_flow/hints.py kubed/selenium_flow/mcp/annotations.py-
Step 2: Update pyproject โ add
"kubed.selenium_flow.mcp", rename thekubed.selenium_flow.promptsandkubed.selenium_flow.skillskeys tokubed.selenium_flow.mcp.promptsandkubed.selenium_flow.mcp.skills, values unchanged. -
Step 3: Update importers โ five modules import
hints; they becomefrom .mcp import annotationsand callannotations.hints(...)/annotations.reads(...). -
Step 4: Run the suite โ Expected: 1,150 passed, 18 skipped.
-
Step 5: Prove the wheel carries the skill and prompts
pip wheel --no-deps -w /tmp/w . && unzip -l /tmp/w/*.whl | grep -E "mcp/skills/selenium-flow/SKILL.md|mcp/prompts/.*\.md"Expected: both present. This is the failure mode that is invisible until an installed copy serves no skill.
- Step 6: Commit
git add -A && git commit -m "mcp/: tools, resources, skill, prompts, apps โ and hints is annotations"Files:
- Create:
kubed/selenium_flow/spec/__init__.py,spec/builder.py,spec/schemas.py - Move:
openapi.pyโ split betweenspec/builder.py(the fourteen helpers andbuild_spec) andspec/schemas.py(the fifteen literal blobs:PAGE_STATE,RESPONSES,ERROR,HEALTH,STARTED,READY,INFO,SESSION_PARAMETERS,FLOW_STEP,FLOW_SCHEMAS,_SESSION,_WRITE_SESSION,_FLOW_OPERATIONS,FILE_SCHEMAS,_FILE_OPERATIONS) - Modify:
scripts/generate_openapi.py,routes.py,tests/test_openapi.pyimport lines
Interfaces:
-
Produces:
from .spec.builder import build_spec, PLACEHOLDER_VERSION. -
DESCRIPTIONstays with the builder: it is prose about the API, not a schema. -
Step 1: Split โ move the fifteen constants verbatim into
spec/schemas.py;builder.pyimports them. No text changes in this task. -
Step 2: Run the suite โ Expected: 1,150 passed, 18 skipped.
test_openapi.pycompares the built document against the live tools, so an accidental edit shows up here. -
Step 3: Regenerate and lint the spec
PYTHONPATH=$DEPS:. python3 scripts/generate_openapi.py
npx --yes @redocly/cli@2 lint openapi.yamlExpected: same path and schema counts as the baseline (26 paths, 52 schemas), lint clean.
- Step 4: Commit
git add -A && git commit -m "spec/: the builder, and the schema data it assembles"Files:
- Create:
kubed/selenium_flow/http/answer.py,kubed/selenium_flow/http/mount.py - Modify:
errors.py(addas_response),routes.py,flows/api.py,http/files.py,http/admin.py - Test:
tests/test_http_answer.py(new)
Interfaces:
-
Produces:
async def answer(request, token, what, call, *, body: bool = True) -> JSONResponse-
def as_response(exc: Exception, what: str) -> JSONResponseinerrors -
def mount(mcp, prefix: str, table: dict, handler) -> Noneinhttp/mount.py
-
Consumes:
auth.authorized,sessions.name_from,errors.status_for,errors.message. -
Step 1: Write the failing test
async def test_one_answer_shapes_every_surface_the_same(server):
"""Four wrappers used to decide this separately: unauthorised is 401, a
non-object body is 400, and a failure carries errors.message โ whichever
tree answered."""
from kubed.selenium_flow.http import answer as answer_module
assert answer_module.answer.__module__ == "kubed.selenium_flow.http.answer"
client = TestClient(server.mcp.http_app())
for path in ("/browser", "/files", "/flows/x"):
assert client.post(path, json={}).status_code == 401- Step 2: Run it to verify it fails
Run: PYTHONPATH=$DEPS:. python3 -m pytest -q tests/test_http_answer.py -v
Expected: FAIL โ kubed.selenium_flow.http.answer does not exist.
-
Step 3: Implement
answer.pyโ liftroutes._answerverbatim, then add thebody=Falsebranch thatfiles.answerneeds for GETs. Keep the comment explaining why a 503's traceback is suppressed. -
Step 4: Move the three other wrappers onto it โ
flows/api.pyandhttp/files.pydelegate;http/admin.pykeepsguardedas a thin decorator overauth.authorizedbecause its handlers return HTML and event streams, and says so in one line. -
Step 5: Add
errors.as_responseand use it in all five shaping sites. -
Step 6: Run the suite โ Expected: 1,150 passed + the new test, 18 skipped.
-
Step 7: Prove the new test is not vacuous โ delete the
auth.authorizedcheck inanswer.py, watch the 401 assertions fail, restore. -
Step 8: Commit
git add -A && git commit -m "One answer(): authorise, read, name, run, shape the failure"Files:
- Create:
kubed/selenium_flow/mcp/guidance.py - Modify:
flows/run.py,session/sessions.py,core/browser.py,mcp/tools.py(INSTRUCTIONS) - Test:
tests/test_guidance.py(new)
Interfaces:
-
Produces:
def pointer(page: str, section: str = "", prompt: dict | None = None) -> dictreturning{"read": "skill://selenium-flow/references/<page>", "section": ..., "prompt": ...}with empty keys omitted. -
Step 1: Write the failing test
def test_every_pointer_names_a_reference_that_exists():
"""A skill:// URI that 404s is worse than no URI: it teaches the agent the
manual is broken."""
from kubed.selenium_flow.mcp import guidance, skill
root = skill.skill_path()
for page in guidance.PAGES:
assert (root / "references" / page).is_file(), page
def test_the_instructions_name_the_skill():
"""INSTRUCTIONS is the one blurb every client reads at connect. Before this,
only failures pointed at the skill โ the manual arrived after the mistake."""
from kubed.selenium_flow.mcp.tools import INSTRUCTIONS
assert "skill://selenium-flow/SKILL.md" in INSTRUCTIONS- Step 2: Run it to verify it fails
Run: PYTHONPATH=$DEPS:. python3 -m pytest -q tests/test_guidance.py -v
Expected: FAIL โ no guidance module, and INSTRUCTIONS does not mention the skill.
-
Step 3: Implement
guidance.pointerwith aPAGEStuple of the reference filenames it may name. -
Step 4: Move the three emitters onto it โ
flows/run.py'shint_for,session/sessions.py's bareguidancestring,core/browser.py's dialog hint. The result keys stay exactly as they are today forhint_for;sessionsgains the same object shape in place of a bare string. This is the one place a result shape changes, sotests/test_sessions.pyand theopenapischema forcurrent_sessionare updated together, and the CHANGELOG records it. -
Step 5: Add one sentence to
INSTRUCTIONSnamingskill://selenium-flow/SKILL.mdas where the how-to lives. -
Step 6: Run the suite, prove the new tests break โ point
PAGESat a file that does not exist, watch it fail, restore. -
Step 7: Commit
git add -A && git commit -m "One guidance shape, and the instructions finally name the skill"Files:
- Modify:
kubed/selenium_flow/mcp/tools.py(every tool docstring),skills/selenium-flow/SKILL.mdandreferences/*.md(receive the agent coaching) - Test:
tests/test_surfaces.py(add)
Interfaces:
-
Consumes: the baseline numbers in Global Constraints.
-
Step 1: Write the failing test
MCP_ONLY = ("keep_file", "session_files", "list_secrets", "execute_script",
"open_session", "end_browser", "run_flow", "save_flow")
async def test_the_http_spec_speaks_http(server):
"""The spec's prose is the tool's prose, so a tool description written at an
agent โ 'do not paste the image back into your reply' โ is published to
every HTTP caller as if it were API documentation."""
spec = await build_spec(server.mcp, ENDPOINTS, "", authenticated=True)
for path, ops in spec["paths"].items():
for method, op in ops.items():
if method not in ("get", "post", "put", "delete"):
continue
text = op.get("description") or ""
found = [w for w in MCP_ONLY if w in text]
assert not found, f"{method.upper()} {path} names MCP-only {found}"- Step 2: Run it to verify it fails
Expected: FAIL naming nine operations (/browser/write, /browser/press-key, /browser/script, /browser/assert, /browser/screenshot, /browser/upload, POST /browser, DELETE /browser, /schemas/flow).
-
Step 3: Rewrite the offending descriptions โ say what the call does, what it returns, and the common mistake. Move every sentence addressed at a model ("do not paste the image back into your reply", "give them the absolute_url") into
skills/selenium-flow/references/READING_PAGES.mdandSKILL.md, where agents read it. -
Step 4: Re-measure
PYTHONPATH=$DEPS:. python3 scripts/measure_descriptions.py # written in this stepRecord tools/description bytes, spec bytes and operation-description bytes against the baseline. Expect a material drop; record the real number even if it is small.
-
Step 5: Run the suite and regenerate the spec โ
test_skill.py's selector guard andtest_wiki.pyboth read these files, so they fail loudly on a broken example. -
Step 6: Commit
git add -A && git commit -m "Descriptions say what the call does; coaching moves to the skill"Files:
-
Modify:
scripts/generate_openapi.py(docstring),saga/Chapter_1_The_Flight_Plan.md(E5 table row),README.md,AGENTS.md,CONTRIBUTING.md,.github/copilot-instructions.md,skills/selenium-flow/**,wiki/*.md(hand-written pages),CHANGELOG.md -
Modify:
http/admin.py(orphan session rows) -
Test:
tests/test_files_and_admin.py(add) -
Step 1: Fix the lie โ
scripts/generate_openapi.pysaysopenapi.yamlis "checked in so it can be reviewed in a pull request". It is gitignored and untracked; it is a build artifact regenerated and linted in CI. -
Step 2: Fix the saga table row โ Chapter 1's E5 row reads
$ROUTE_PREFIX/admin,/files,/flows, which reads as only/adminbeing mounted. All three are mounted. -
Step 3: Orphan session rows โ write the failing test
def test_the_listing_skips_keys_that_cannot_be_session_names(server, client):
"""Pre-#34 records are keyed `named:<name>`. A session name can never
contain a colon, so such a key is not a session and must not render as one."""
server.sessions.store.set("named:ghost", SessionRecord(session_id=""))
names = [row["key"] for row in client.get("/admin/sessions", headers=AUTH).json()["sessions"]]
assert "named:ghost" not in names-
Step 4: Implement โ the listing filters keys that
flows.library.valid_session_namerejects. No migration: the live keys carry TTLs and expire. -
Step 5: Read every document end to end against the shipped contract โ README, AGENTS, CONTRIBUTING, copilot-instructions, all seven skill files, and the hand-written wiki pages (
Home,Installing,Deployment,Administration,Sessions,Flows,Files,Secrets). Not a keyword skim: the #35 sweeps found stale wording three times, each after the previous sweep reported done. -
Step 6: Regenerate the wiki, push the submodule first
PYTHONPATH=$DEPS:. python3 scripts/generate_wiki.py
cd wiki && git add -A && git commit -m "..." && git push origin master && cd ..-
Step 7: CHANGELOG โ one short line per user-visible entry under
[Unreleased], in the existing Added/Changed/Fixed blocks. The restructure itself earns no line; thecurrent_sessionguidance shape does. -
Step 8: Commit
git add -A && git commit -m "Every document read end to end against what ships"Files:
-
Modify:
saga/Chapter_2_Pilot_Reports.md -
Verify: everything
-
Step 1: Record the refit in the saga โ what moved and why, the four-wrappers-into-one collapse, the measured before/after description bytes, the
x-mcpidea and why it was declined (FastMCP 4.0.3 reads no such extension), and the blocking-handlers item that remains open. -
Step 2: Full verification
PYTHONPATH=$DEPS:. python3 -m pytest -q -p no:cacheprovider tests --ignore=tests/integration
PYTHONPATH=$DEPS:. python3 scripts/generate_openapi.py && npx --yes @redocly/cli@2 lint openapi.yaml
PYTHONPATH=$DEPS:. python3 scripts/generate_wiki.py --check
pip wheel --no-deps -w /tmp/w . && unzip -l /tmp/w/*.whl | grep -cE "\.py$"
git diff -U0 origin/main -- '*.py' | grep '^+' | awk 'length > 89'Expected: suite green, lint clean, wiki current, wheel carries every module, no over-length line.
-
Step 3: Compare against
mainโgit diff origin/main --statshould show moves and prose, and no change to any test assertion. Spot-check three test files to confirm only import lines moved. -
Step 4: Commit and open the PR
git add -A && git commit -m "The saga records the refit"
git push -u origin chore/refit-and-docs-pass
gh pr create --title "The refit: rooms in the package, one way to answer a request" --body "..."Spec coverage: layout โ Tasks 2โ7; one answer โ Task 8; annotations rename โ Task 6; guidance + INSTRUCTIONS โ Task 9; neutral prose โ Task 10; docs findings โ Task 11; orphan rows โ Task 11; packages guard โ Task 1; saga โ Task 12. No spec section is unimplemented.
Placeholders: none. Every code step carries the code; every verification step carries the command and the expected result.
Type consistency: answer(request, token, what, call, *, body=True), as_response(exc, what), mount(mcp, prefix, table, handler) and guidance.pointer(page, section, prompt) are used under those exact names in Tasks 8, 9 and 11.
One deviation from the spec, recorded deliberately: the spec's tree put js/ inside core/. It goes there, but the move is only safe because Task 2 moves the package-dir key with it โ the module loads its files from the sibling directory of the same name, and a wheel that keeps the old mapping serves no JavaScript with no error anywhere. Task 2 Step 6 asserts the wheel contents for exactly this reason.