research phase 4.4 skills - sinrimin/harness-nexus GitHub Wiki
Status: research complete. PRD:
prd-phase-4-web-ui.md(Β§4.4). Design: to be written (design-phase-4.4-skills.md) before implementation.
This answers the open questions that gate the Skill management sub-phase: how
Harness Nexus should represent and resolve an external skill reference, whether
the existing ResourceSource variants cover it, and how a stored reference maps
onto the three install targets (Claude Code, ZCode, Hermes).
Skills are never distributed alone β they are bundled inside plugins in
the Claude Code / ZCode world. "Install a skill from a hub" is, in that world,
"install a plugin (from a marketplace) whose skills/<name>/SKILL.md the tool
then exposes." And there is no live-reference model: installing a plugin
clones and caches it locally (~/.claude/plugins/cache/β¦,
~/.zcode/cli/plugins/cache/β¦). A "reference" is materialized, not dereferenced
at runtime.
But this is only half the picture. Hermes (installed and running on this
machine, source read-only at ~/.hermes/hermes-agent/) implements a far richer
multi-source model where skills are first-class, addressable across many
heterogeneous sources β not just one plugin marketplace. That model is the more
complete answer to "how does an agent find skills from multiple sources," and it
is the model we should learn from for 4.4. See the dedicated section
"Hermes: a multi-source skill hub (verified on this machine)" below.
This reshapes 4.4: Harness Nexus isn't storing a live skill reference β it's storing a source spec (plugin entry for CC/ZCode; one of several adapter kinds for Hermes) that, at install time, gets resolved and materialized. The "search the skill hub" feature is really search across configured sources.
Verified against official Claude Code docs, the live claude-plugins-official
catalog, GitHub issues, and on-disk ZCode evidence on this machine.
-
Three Anthropic-maintained plugin directories (none is literally named
"Skill Hub"):
claude-plugins-official(auto-available by default),anthropics/skills(skills-specific),claude-plugins-community(validated third-party, SHA-pinned). Browse UI atclaude.com/plugins. -
ZCode mirrors this: its own
zcode-plugins-officialmarketplace + tracksanthropics/claude-plugins-officialas agithub-sourced known marketplace (confirmed in~/.zcode/cli/plugins/known_marketplaces.jsonon this machine). -
Addressing:
/plugin install <plugin-name>@<marketplace-name>. A marketplace is added once via/plugin marketplace add <owner/repo | git-url | local-dir | url-to-marketplace.json>.marketplace add owner/repois the dominant form (git clone).
A marketplace entry points at a plugin via a source. The verified kinds:
| Source kind | Fetch | Pinning |
|---|---|---|
github (owner/repo, optional ref/sha) |
git clone GitHub repo |
sha is effective pin |
url (any git URL; ref/sha/path) |
git clone any host |
sha is effective pin |
git-subdir (url/path/ref/sha) |
sparse/partial clone of a subdirectory |
sha is effective pin |
relative path (./β¦) |
co-located in the marketplace's own repo | resolved against .claude-plugin/
|
npm (package/version/registry) |
npm install |
version field |
Notes:
-
filesystemis NOT a Claude source kind. It only appears in ZCode's local cache metadata (cachePath), describing an already-installed plugin β not a fetchable source. (The PRD's earlier "skill hub / npx" enumeration was approximate; this is the corrected list.) -
npmis a real, documented source kind (consumer-side schema in the official docs; triple-corroborated by issues #37093, #33253, #37670) but is not used in the current official catalog β the liveclaude-plugins-officialmarketplace.jsonuses onlygithub/url/git-subdir/relative-path.git-subdiris the common choice for skill-heavy plugins from big vendors (cloudflare, mongodb, databricks, stripe, airtable). -
git-subdirhas client-compatibility bugs (issues #585, #768) β preferurlfor broad compatibility.
-
The native flow for an npm-distributed plugin: a marketplace entry with
source: { source: 'npm', package, version, registry? }; the user runsplugin install <name>@<marketplace>; the tool runsnpm installunder the hood. So "npm skill" = "plugin whose source kind isnpm". -
npx @org/skill-nameas a one-liner is NOT native β it's a third-party CLI pattern (e.g. Vercel'sadd-skill,skills.sh). We should not model our reference on it; if a user has an npm-packaged skill, the right representation is thenpmmarketplace source kind, not annpxinvocation string. - Concrete published npm-sourced skill example: not found. Schema and bug reports confirm npm works; we could not point to a canonical published one. If a user asks for an npm skill, we store the npm source spec.
Resolution order (from the plugin-marketplaces doc):
-
versionin the plugin'splugin.json. -
versionin the marketplace entry. - the git commit SHA.
Git-based sources may omit version entirely β "every new commit is treated as
a new version." No evidence that a version field in SKILL.md frontmatter
drives resolution; versioning is plugin/marketplace-entry level, not per-skill.
Once installed, a skill is addressed plugin:skill-style
(e.g. document-skills:docx, skill-creator:skill-creator) β confirmed by the
skills loaded in this very environment. The plugin namespace disambiguates
collisions.
Source read-only at ~/.hermes/hermes-agent/tools/skills_hub.py (4073 lines) +
tools/skills_guard.py + hermes_cli/skills_hub.py. All read-only; Hermes is
running in production on this host and was not modified. This is the most
complete real-world design for "find skills from multiple sources" we have
access to, and it diverges meaningfully from the CC/ZCode single-marketplace
model β skills here are first-class across heterogeneous sources.
class SkillSource(ABC):
def search(self, query, limit=10) -> List[SkillMeta] # find
def fetch(self, identifier) -> Optional[SkillBundle] # download files
def inspect(self, identifier) -> Optional[SkillMeta] # metadata only
def source_id(self) -> str # 'github', 'url', β¦
def trust_level_for(self, identifier) -> str # trust tierEvery source is a plug-in implementing the same 4-method interface. The runtime holds a list of sources and queries them in parallel (thread pool, per-source timeout, 30 s overall). This is the key architectural lesson: the source layer is an open set behind a uniform interface, not a hardcoded marketplace.
From create_source_router(), in priority order:
| source_id | Class | What it fetches | Trust | Notes |
|---|---|---|---|---|
official |
OptionalSkillSource |
skills shipped in the repo's optional-skills/
|
builtin |
Nous-maintained, not activated by default |
hermes-index |
HermesIndexSource |
a pre-built merged index (cached, 1 h TTL) | per-skill | When available, all external API sources are skipped (avoids ~70 GitHub calls/search) |
skills-sh |
SkillsShSource |
skills.sh catalog (~20k+, via sitemap) β underlying GitHub repo | community |
This is the real npx skills add source β its regex parses npx skills add <repo> [--skill <name>]
|
well-known |
WellKnownSkillSource |
a domain's /.well-known/skills/index.json
|
community | standard discovery protocol |
url |
UrlSource |
a single SKILL.md at a direct HTTP(S) URL |
community | identifier IS the URL; single-file only |
github |
GitHubSource |
GitHub repos via Contents API + configurable taps |
trusted for 4 named repos, else community |
default taps: openai/skills, anthropics/skills, huggingface/skills, NVIDIA/skills, garrytan/gstack |
clawhub |
ClawHubSource |
clawhub.ai HTTP API (50k+) | community | explicitly distrusted β "ClawHavoc" incident (341 malicious skills, Feb 2026) |
claude-marketplace |
ClaudeMarketplaceSource |
CC marketplace repos (.claude-plugin/marketplace.json) |
community | known: anthropics/skills, aiskillstore/marketplace |
lobehub |
LobeHubSource |
LobeHub agent marketplace (14.5k) β converts prompt templates to SKILL.md | community | data in lobehub/lobe-chat-agents |
browse-sh |
BrowseShSource |
browse.sh catalog (200+ site-automation SKILL.md) | community | Browserbase-hosted |
This directly answers the "npx skill" question: npx skills add is the
install command surfaced by skills.sh (SkillsShSource parses it). It is
not a source kind of its own β it's a CLI wrapper around a GitHub repo, where
skills.sh is the discovery index. The underlying fetch is still GitHub.
Three tiers, resolved per-source and enforced by a security scan before
install (tools/skills_guard.py):
TRUSTED_REPOS = {"openai/skills", "anthropics/skills",
"huggingface/skills", "NVIDIA/skills"}
INSTALL_POLICY = {
# safe caution dangerous
"builtin": ("allow", "allow", "allow"),
"trusted": ("allow", "allow", "block"),
"community": ("allow", "block", "block"),
}-
builtinβ ships with Hermes, never scanned. -
trustedβ the 4 named GitHub repos; caution findings allowed, dangerous blocked. -
communityβ everything else; any non-safe finding blocks (unless--force, and never fordangerous).
Every fetched skill goes through quarantine β scan β install: the bundle is
written to skills/.hub/quarantine/, scanned against a regex threat-DB
(exfiltration / injection / destructive / persistence / network / obfuscation
patterns), and only installed to ~/.hermes/skills/ if the verdict Γ trust tier
permits. This is a meaningful security boundary the CC/ZCode model lacks.
Installed skills are tracked in skills/.hub/lock.json (full provenance per
skill):
-
taps.jsonβ user-added GitHub repo sources ({repo, path}); extendGitHubSource's default taps at runtime. This is the user-facing "add a source" mechanism. -
audit.logβ append-only record of every install/scan/uninstall with verdict. - content_hash β the integrity pin (analogous to CC's SHA pin, but per-skill and hash-based, not git-SHA-based).
Search results from all sources are merged (parallel_search_sources), deduped
by identifier (keeping the highest-trust copy), then sorted by
(-trust_rank, source != "official", name). So builtin > trusted >
community, official first within a tier, then alphabetical. This is the
ranking policy that makes a multi-source hub usable.
The CC/ZCode-only recommendation was a single plugin variant. Hermes shows
the real shape: the source layer is open and heterogeneous. The "plugin"
concept is just one adapter kind among many (CC marketplace, GitHub repo,
skills.sh, url, well-known, vendor hubs). The recommendation in the next
section is updated to reflect this β ResourceSource should model a source
kind (open set), not a single plugin variant.
Our domain (packages/core/src/domain/resource.ts):
type ResourceSource =
| { type: 'git'; url: string; ref?: string; path?: string }
| { type: 'tarball'; url: string; checksum?: string }
| { type: 'local'; path: string }
| { type: 'inline'; content: string };The existing four don't cleanly express "this skill comes from plugin foo in
marketplace bar, pinned to this SHA / npm version, living at this subpath" β
nor the richer "this skill comes from skills.sh / a direct URL / a well-known
index" shapes that Hermes surfaces.
-
gitis close but lacks the plugin/marketplace indirection (the namespace, the skill subpath inside the plugin, the entry version pin). A bare git ref loses "which plugin in this repo, which skill in that plugin." -
tarball/local/inlinedon't apply to marketplace distribution. - Nothing models the non-plugin external sources (direct URL, skills.sh, well-known index) that Hermes treats as first-class.
Recommended new variant β a plugin source for the CC/ZCode marketplace
shape, validated against the marketplace source-kind table above:
| {
type: 'plugin';
// A marketplace plugin entry β mirrors marketplace.json source kinds.
source:
| { source: 'github'; repo: string; ref?: string; sha?: string; path?: string }
| { source: 'url'; url: string; ref?: string; sha?: string; path?: string }
| { source: 'git-subdir'; url: string; path: string; ref?: string; sha?: string }
| { source: 'npm'; package: string; version: string; registry?: string };
// Which plugin this entry resolves to, and which skill(s) it exposes.
plugin: string;
// Optional: pin a marketplace-entry version (falls back to sha/git-commit).
version?: string;
}This is a deliberate choice over overloading git, because:
- It preserves the plugin namespace (
pluginfield) so emitted bundles and profile entries can addressplugin:skillcorrectly. - It distinguishes
git-subdir(sparse clone) fromgithub/url(full clone), which matters for install correctness β not all clone strategies are equal. - It keeps
npmfirst-class, matching the documented source kind.
inline still covers a user-authored markdown skill (no external source) β the
simple case from the PRD. The existing url/tarball/local variants already
cover the Hermes url / single-file / local-directory cases, so no new variant
is needed for those β only plugin is genuinely missing for the marketplace
world. (skills.sh is ultimately a GitHub fetch behind an index, so a plugin
github entry or git covers it once the index resolves it.)
Skill source
|
Install-time behavior (Phase 3 writer) |
|---|---|
inline |
Emit skills/<key>/SKILL.md directly into the bundle (content is in the row). |
plugin |
Emit a marketplace-add + plugin-install instruction (or pre-resolve by fetching the plugin and copying its skills/<name>/ into the bundle). Pin via sha/version. |
git / tarball / local
|
Resolve the bytes and copy SKILL.md into the bundle's skills/. |
The "search the skill hub" UI feature = fetch a known marketplace's
marketplace.json, list its plugin entries (filtered to those exposing skills),
let the user pick one, and store it as a plugin-source resource. Do not
attempt to fetch/index individual SKILL.md files across repos β the marketplace
entry is the unit. (Hermes's parallel_search_sources + merged index is a more
ambitious version of this; for our first cut a single-marketplace browse is the
pragmatic scope, with the SkillSource-adapter pattern as the documented
extension path if we later want multi-source.)
-
New
ResourceSourcevariant vs. reusegit. Research recommends a newpluginvariant (better fidelity). Confirm before the design doc, since it touchescore/shared/storage/SDK. -
Marketplace registry: which to search? Start with
claude-plugins-official(auto-available, broadest). Whether to also surfaceanthropics/skills, community marketplaces, or a user-configured custom marketplace URL is a product-scoping decision. -
Resolution at store-time vs install-time. When a user "saves a skill
reference," do we (a) store only the spec and resolve at install, or (b)
fetch + pin the SHA immediately so the reference is immutable? Recommend (a)
store-the-spec, with an optional explicit
sha/versionpin for (b). Pinning avoids supply-chain drift; floatingreftracks upstream. -
Fetching marketplace JSON server-side. Searching a marketplace requires
Harness Nexus to fetch a remote
marketplace.json(or a git repo) β this is a new outbound-network path (the server is otherwise fetch-light; only the proxy dials configured upstreams). Decision: a dedicated, allowlisted marketplace-fetch endpoint, or have the CLI/web fetch client-side. Affects the security model. -
Skill within a multi-skill plugin. A plugin can expose many skills. Does
one stored resource = one plugin (all its skills) or one plugin + one skill
name? Recommend plugin-level (matches how tools address them as
plugin:skilland how marketplaces list them), with the skill nameε―ι. -
npm registry for
npmsources. Default to the public registry; private registries need credential handling (reuse the${cred:NAME}model from 2.1). -
Trust tiers + security scan (from Hermes). Hermes blocks community skills
with any non-safe scan finding, and its
clawhubsource is explicitly distrusted after a real malicious-skills incident. For 4.4's first cut we likely don't run a content scanner (Harness Nexus stores references, it doesn't execute them β the target tool does). But the trust tier concept (builtin/trusted/community, visible in the UI) and provenance pinning (sha/content_hash, recorded like Hermes'slock.json) are worth adopting from the start, since they're cheap and inform the install-warning UX. A scan gate is a later hardening if we ever materialize/execute skill bytes. -
Multi-source extensibility (from Hermes). Hermes's
SkillSourceABC + 9 adapters + parallel search + merged index is the mature end-state. Our first cut is one marketplace; but designingResourceSourceand the (future) search endpoint so a second source kind (skills.sh, direct URL, custom tap) can be added without schema churn is worth it. Concretely: keep the source spec as a discriminated union (open set), not a single shape.
- No concrete published npm-sourced skill/plugin to point to. The schema is verified; adoption is not. Model it anyway (it's documented), but don't build npm-search until there's demand.
-
Skill-frontmatter
versionfor resolution: not evidenced. Treat versioning as plugin/marketplace-entry-level. -
commitvsshafield semantics in some livegithubentries: both appear in the wild but the docs only explainsha. Treat them as aliases for now; verify in the design doc.
- Claude Code: plugin-marketplaces (source kinds, version resolution), discover-plugins, skills.
-
anthropics/claude-plugins-official
- its live
marketplace.json.
- its live
- anthropics/skills.
- npm source evidence: issues #37093, #33253, #37670.
-
git-subdircompat: issues #585, #768. - ZCode (on-disk, this machine):
~/.zcode/cli/plugins/known_marketplaces.json,~/.zcode/cli/plugins/marketplaces/zcode-plugins-official/marketplace.json,~/.zcode/cli/plugins/cache/zcode-plugins-official/*/.zcode-plugin-seed.json. - Earlier Phase 3 target research:
research-phase-3-plugin-targets.md(plugin/skill format per target β this doc is about sourcing).
{ "version": 1, "installed": { "<skill-name>": { "source": "github", // source_id "identifier": "anthropics/skills/skill-creator", "trust_level": "trusted", "scan_verdict": "safe", "content_hash": "...", // integrity pin "install_path": "~/.hermes/skills/skill-creator", "files": ["SKILL.md", "..."], "metadata": {}, "installed_at": "...", "updated_at": "...", }, }, }