design phase 7.4 multi source - sinrimin/harness-nexus GitHub Wiki

Design: Phase 7.4 โ€” Multi-source skill search

Status: implemented in Phase 7.4 (โœ…). PRD: prd-phase-7-skills.md ยง7.4. Prior designs: phase-7.1-plugin-source.md (storage) + phase-7.2-marketplace-fetch.md (single-marketplace browse) + phase-7.3-hub-ui.md (the hub page this extends). Phase 7 complete.

Hermes-style multi-source skill search. A query dispatches to a set of SkillSource adapters in parallel; results are merged, deduped by identifier, ranked by trust, and surfaced in the existing /skills/hub page alongside the 7.2/7.3 marketplace browse. This is the last sub-phase of Phase 7.

Scope (ground-truth verified, with corrections)

Hermes ships 10 adapters. This phase ports the 4 worth porting and explicitly skips/defers the rest, with reasons:

Adapter Verdict Why
GitHubSource PORT Foundational. Taps + recursive-tree cache + trust tiers.
WellKnownSkillSource PORT Open /.well-known/skills/ protocol, no auth, ~220 lines.
UrlSource PORT Trivial single-file fetch. ~160 lines.
ClaudeMarketplaceSource PORT as MarketplaceSource Same .claude-plugin/marketplace.json mechanism as 7.2 โ€” wrapped, not reimplemented.
SkillsShSource DEFER (7.5+) Thin GitHub-redirect layer with sitemap-walking complexity. If ported, /api/search only.
BrowseShSource DEFER (7.5+) Clean API but niche (browser-automation skills only).
HermesIndexSource SKIP Hermes-specific artifact (hermes-agent.nousresearch.com). Borrow the schema later, not the source.
ClawHubSource SKIP Docstring itself confirms distrust post-ClawHavoc (341 malicious skills, Feb 2026).
LobeHubSource SKIP System-prompt templates, wrong artifact class.
OptionalSkillSource SKIP Hermes-bundled official skills (not a remote source).

Architecture

 GET /api/skills/search?q=<query>
        โ”‚
        โ–ผ
 SkillSearchRouter.search(q)
        โ”‚
        โ”‚ Promise.all(sources.map(s => race(s.search(q), timeout)))   โ† per-source timeout
        โ”‚   โ€ข marketplace source โ†’ SkillCatalogService.getCatalog (7.2, cached)
        โ”‚   โ€ข github source      โ†’ GitHub trees API (per-tap, tree-cached 1h)
        โ”‚   โ€ข well-known source  โ†’ GET <host>/.well-known/skills/index.json
        โ”‚   โ€ข url source         โ†’ no-op (fetch-only; contributes nothing to search)
        โ–ผ
 merge: dedupe by identifier, keep higher trust tier on collision
        โ”‚
        โ–ผ
 { results: SkillMeta[], timedOut: string[], errored: string[] }

Components (packages/server/src/infra/source-fetchers/)

github-source.ts

  • GitHubSource { fetcher, token?, taps, logger }. taps default = the 4 TRUSTED_REPOS (openai/skills, anthropics/skills, huggingface/skills, NVIDIA/skills).
  • search(q): for each tap, GET /repos/{repo}/git/trees/HEAD?recursive=1 (the recursive tree API โ€” one call per tap, vs the naive per-directory walk that blows the rate limit). Collect every dir containing a SKILL.md, filter by name โЇ q, return SkillMeta (identifier = owner/repo/dir, trust via resolveTrustTier).
  • Tree cache: Map<repo, {dirs, expiresAt}>, 1h TTL. Trees change rarely.
  • Rate-limit handling: 403/429 โ†’ log warn + return [] for that tap. The source never throws โ€” partial results from other sources still surface.
  • Auth: optional GITHUB_TOKEN (5000/hr authenticated vs 60/hr anonymous). Hermes's gh auth token subprocess + GitHub App fallbacks deliberately not ported โ€” too heavy; a user with a token sets the env.
  • inspect/fetch return null (7.4 ships search only; install-time fetch is Phase 3.3).

well-known-source.ts

  • WellKnownSource { fetcher, logger }. No auth.
  • search(q): only handles URL-form queries (http(s)://host). GETs <origin>/.well-known/skills/index.json โ†’ {skills:[{name,description}]}, returns SkillMeta (identifier well-known:<origin>/<name>, always community). Non-URL queries return empty (no host auto-discovery).
  • inspect/fetch return null.

url-source.ts

  • UrlSource { fetcher, logger }. Single-file SKILL.md at a direct URL.
  • search(): no-op (Hermes is too; this source is fetch-only โ€” it has no catalog). It contributes when a user pastes a URL into the search box and the hub resolves it via inspect (not yet wired in the UI โ€” Phase 3.3 territory).
  • inspect(url): GET url, parse YAML frontmatter for name/description.

marketplace-source.ts

  • MarketplaceSource { catalog, allowlist }. Wraps 7.2's SkillCatalogService โ€” no reimplemented fetch, the catalog cache is reused.
  • search(q): for each allowlist entry, catalog.listPlugins(id, url, {q}), map MarketplacePlugin โ†’ SkillMeta (identifier {marketplaceId}:{name}, trust via resolveTrustTier). Catalog-fetch errors (502) are swallowed โ€” a failed marketplace contributes nothing.

search-router.ts

  • SkillSearchRouter { sources, timeoutMs, logger }.
  • search(q, limit=50): Promise<{results, timedOut, errored}>:
    • Per-source timeout via Promise.race([s.search(q), timeout]) โ€” a slow source can't block the response past timeoutMs (default 30s). Timed-out source ids go to timedOut; rejected ones to errored. The whole search never throws on a source failure (partial-results-first, matching Hermes).
    • Merge/dedupe: key = identifier (NOT name โ€” browse-sh skills across sites share task names). On collision, the higher-trust entry wins (TRUST_RANK = {builtin:2, trusted:1, community:0}); within a tier, insertion order (source priority) is preserved.
  • Decorated as app.skillSearch (co-located Fastify augmentation).

Two Hermes gotchas carried over

  1. Timeout must not block. Hermes deliberately avoids Python's with ThreadPoolExecutor(...) because its __exit__ calls shutdown(wait=True), making the overall timeout a no-op. The JS equivalent of its shutdown(wait=False, cancel_futures=True) is Promise.race + a per-source timeout promise โ€” a plain await Promise.all(...) would let a slow source block past the budget. The in-flight promises settle in the background after the timeout; their results are discarded.
  2. Dedupe by identifier, not name. Same skill name from two sources (or two repos) are distinct results. The collision rule keeps the higher-trust copy.

Config (4 new env keys)

Env Default Purpose
GITHUB_TOKEN unset optional PAT for GitHubSource (5000/hr vs 60/hr)
SKILL_GITHUB_TAPS openai/skills,anthropics/skills,huggingface/skills,NVIDIA/skills comma-separated owner/repo taps
SKILL_SEARCH_TIMEOUT_MS 30000 per-source timeout (overall budget)
SKILL_DISABLED_SOURCES unset comma-separated source ids to disable (test mode)

API + SDK

  • GET /api/skills/search?q=<query>&limit=50 โ†’ {results: SkillMeta[], timedOut: string[], errored: string[]}. q required (400 if absent). Browsing without a query stays on 7.2's /api/skills/marketplaces/:id/plugins.
  • SDK: api.searchSkills(q, limit?).

Web (/skills/hub)

The existing hub page (7.3) gains a dual mode:

  • Search box empty โ†’ 7.2/7.3 marketplace browse (marketplace Select + category Select).
  • Search box non-empty โ†’ multi-source search. The marketplace/category Selects hide; the table shows results from all sources with a per-row sourceKind badge (marketplace / github / well-known / url).

Both modes project into a unified HubRow (key/name/description/sourceKind/ category/homepage/tier/pluginSource/hasPin) so the table and the save dialog render one way. pluginSource is precomputed by each adapter and carried in SkillMeta.extra.pluginSource โ€” the save dialog stores it verbatim via skillMetaToResourceSource(meta), no identifier reverse-engineering.

If timedOut.length > 0, a text-muted-foreground notice appears above the table ("Some sources timed out (โ€ฆ); showing partial results").

Testing

The smoke suite verifies the search pipeline with SKILL_DISABLED_SOURCES= github,well-known,url โ€” so only the marketplace source runs against the 7.2 fixture. The [7.4] block asserts: q required (400), the marketplace result surfaces (jfrog), every result carries identifier + trustLevel, extra.pluginSource is the plugin variant, and timedOut/errored are arrays. 133 passed / 0 failed.

The per-adapter fetch logic (GitHub tree parsing, well-known index shape, URL frontmatter) is covered by typecheck + manual verification (like 7.3's UI): real GitHub/well-known behavior requires a GITHUB_TOKEN and live network, which the smoke suite deliberately avoids.

Runbook

Production (real multi-source search):

JWT_SECRET=โ€ฆ pnpm dev:server
# GITHUB_TOKEN unset โ†’ github source runs anonymous (60 req/hr); set it for 5000/hr

Test (marketplace-only search against a fixture):

JWT_SECRET=โ€ฆ STORAGE_DRIVER=memory \
  MARKETPLACE_FIXTURE_PATH=$PWD/scripts/fixtures/marketplace.json \
  SKILL_DISABLED_SOURCES=github,well-known,url \
  pnpm dev:server

Add a custom GitHub tap:

SKILL_GITHUB_TAPS="openai/skills,anthropics/skills,myorg/my-skills"

Out of scope (deferred / skipped)

  • fetch() (materialize bytes) โ€” Phase 3.3's install writers. 7.4 adapters implement search (+ inspect for url); fetch returns null.
  • skills.sh / browse.sh adapters โ€” DEFER (7.5+).
  • clawhub / lobehub / hermes-index โ€” SKIP (verified).
  • Taps management UI/API โ€” env config is enough for 7.4; a CRUD surface is a follow-on.
  • Content scanning โ€” Out of Scope (Harness Nexus stores references; the target tool executes).
  • A pre-built merged index (the Hermes hermes-index pattern with resolved_github_id shortcuts). Worth revisiting if multi-source search proves slow against live GitHub; for now the per-tap tree cache suffices.
โš ๏ธ **GitHub.com Fallback** โš ๏ธ