2026 04 13_storage_system_comparison_for_verse - mark-ik/graphshell GitHub Wiki
Date: 2026-04-13 Status: Research / synthesis Scope: Compares Syncthing, Tahoe-LAFS, Storj, and Filecoin against Verse's decentralized storage-bank direction. Clarifies which ideas Verse should adopt for bilateral/private storage, opaque encrypted third-party hosting, community durability, optional incentive layers, and the fit of specific mechanism and crate choices for a Rust implementation.
Related docs:
- ../technical_architecture/VERSE_AS_NETWORK.md - canonical Verse boundary and storage-bank positioning
- ../technical_architecture/2026-03-05_verse_economic_model.md - three-track economic model and anti-plutocracy rules
- ../implementation_strategy/2026-03-28_decentralized_storage_bank_spec.md - canonical storage-bank operational model
- ../implementation_strategy/proof_of_access_ledger_spec.md - receipt and accounting model
- ../../verso_docs/technical_architecture/VERSO_AS_PEER.md - bilateral storage visibility and named-peer trust boundary
- 2026-02-23_storage_economy_and_indices.md - earlier speculative storage-economy framing
Verse is not trying to become "Dropbox on a blockchain" or a generic storage coin. The more precise goal is:
- users can privately replicate data across their own devices or trusted peers
- communities can pool storage for shared durable services such as rooms, workspaces, files, capsule content, and similar hosted state
- providers can help store encrypted or fragmented data without necessarily gaining permission to read it
- public or community service should be auditable and optionally incentivized
- private self-backup should remain visible and accountable but should not automatically mint public credit
This note compares four relevant systems because each one solves a different slice of that problem.
The strongest hybrid for Verse is:
- Syncthing-like for bilateral and personal device replication
- Tahoe-LAFS-like for encrypted opaque hosting and capability-oriented access
- Storj-like for audits, accounting, and untrusted-host durability mechanics
- much lighter than Filecoin for the first serious implementation, while preserving the option to borrow stronger proof or settlement ideas later
The key design conclusion is that Verse should treat storage as three distinct trust zones:
| Trust zone | Examples | Audit | Credit |
|---|---|---|---|
| Personal / bilateral | my phone, my laptop, my friend hosting my backup | yes | no by default |
| Community service | room state, shared workspace, capsule content, shared files | yes | yes, if receipt policy allows |
| Open public market | anonymous or pseudonymous third-party providers | yes, stronger | optional later |
Trying to collapse those into one model creates bad incentives and unnecessary complexity.
Syncthing is a local-first file replication system for devices and explicitly trusted peers. It focuses on correctness, visibility, and user consent rather than markets or public service incentives.
- clear device identity and trust establishment
- direct replication without central coordination
- strong visibility into what is present, missing, out of sync, or stale
- practical offline-first behavior
- no attempt to force an economy where one is unnecessary
- opaque third-party hosting at scale
- untrusted public providers
- community storage pools for shared services
- receipts, payouts, or shared durability economics
This is the best prior art for Verso bilateral storage visibility. The right lesson is not "copy Syncthing's protocol" but "keep the bilateral tier boring, inspectable, and non-financial."
If a user backs up their own devices, or a small trusted cluster shares storage informally, the system should track:
- bytes held
- what is replicated where
- challenge/verification status
- imbalance and withdrawal
But it should not automatically create public storage credit from those relationships.
Private should mean accountable but not necessarily incentivized.
Tahoe-LAFS is a secure distributed filesystem built around client-side encryption, capability-based access, and splitting data across multiple storage nodes.
- data is encrypted before storage
- storage providers can hold opaque shares without needing plaintext access
- capability-bearing references control access
- erasure coding and share distribution are first-class
- strong alignment with least-authority principles
- broad incentive design
- public community accounting
- modern app/service durability economics
- integrated social or governance layer
This is the strongest precedent for your intuition that "the data you host should be encrypted and sharded without the provider needing the owner's permission to read it." That is a real and proven pattern.
Verse should adopt the following posture from Tahoe-LAFS:
- store ciphertext or encrypted fragments by default for non-public service objects
- verify by hash/CID, not by semantic inspection
- separate storage authority from read authority
- keep reconstruction and decryption at the edge where the holder has the right capability or membership proof
Opaque encrypted hosting is realistic and should be the default model for shared private/community data.
Storj is a decentralized object-storage network where encrypted, erasure-coded fragments are stored on independent nodes while coordinating satellites handle metadata, audits, and payouts.
- encrypted client-side storage on untrusted hosts
- practical erasure-coded fragment distribution
- audit and payment mechanisms tied to availability and service
- clear distinction between storage nodes and coordination/control functions
- community sovereignty; satellites are a major coordinating dependency
- room/workspace/community-governed service semantics
- peer-local or small-community simplicity
Storj is useful because it shows that the combination of:
- encrypted fragments
- audits/challenges
- provider accounting
- payout eligibility
can work in practice.
But Verse should avoid adopting Storj's architectural center of gravity. Verse wants the coordination layer to be reconstructable from receipts, announcements, heartbeats, and community policy rather than from a privileged satellite role.
Borrow Storj's audit/accounting discipline, not its central coordination assumptions.
Filecoin is a storage market with on-chain commitments and cryptographic proofs designed for large-scale, adversarial, long-term storage.
- serious durability incentives
- explicit collateral and slashing logic
- strong proof culture around storage commitments
- credible public-market behavior under adversarial assumptions
- small trusted groups
- low-friction self-hosting
- simple room/workspace/community service backing
- comprehensibility for users who just want to pool storage for shared state
Filecoin matters as a reference point for:
- bond/collateral thinking
- service-level reliability expectations
- the distinction between accounting and settlement
But it is too heavy to be the default mental model for Verse. The Verse storage bank should remain:
- off-chain-first
- receipt-ledger-first
- payout-second
- optional for communities that want incentives at all
Do not import full public-market complexity before the community-scale service model is proven useful.
| Property | Syncthing | Tahoe-LAFS | Storj | Filecoin | Verse target |
|---|---|---|---|---|---|
| Trusted-device sync | strong | weak | weak | weak | strong via Verso |
| Opaque encrypted hosting | low | strong | strong | mixed | strong |
| Erasure-coding-ready | low | strong | strong | strong | strong, staged |
| Public incentive layer | none | none | yes | yes | optional |
| Community-governed shared services | low | low | low | low | core requirement |
| Audit/accounting | moderate | moderate | strong | very strong | strong |
| Simplicity for users | high | moderate | moderate | low | moderate |
| Good first implementation model | yes for bilateral | yes for opaque storage | partial | no | hybrid |
For my devices and trusted peers, Verse/Verso should be Syncthing-like:
- tracked
- challengeable
- inspectable
- imbalance-aware
- no automatic public credit
This tier can still support explicit private agreements and even local bookkeeping, but there is no reputation stake, so there should be no default public reward.
For shared rooms, shared workspaces, files, capsule bundles, or similar durable objects, Verse should be Tahoe-like plus Storj-like:
- encrypted or encrypted-fragment payloads by default unless intentionally public
- CID-addressed storage objects and fragment manifests
- availability challenges and repair incentives
- receipts for storage service and retrieval service
- service-class policy to decide what deserves priority
This is the tier where the storage bank actually matters.
For large public communities or anonymous providers, Verse may later borrow more from Filecoin-style collateral and stronger proof semantics. That should remain optional until the lower-friction community model is actually useful in practice.
If a provider cannot read the payload, how can the network know the data is worth storing?
The answer is: the storage layer should not attempt to infer semantic usefulness from ciphertext. Instead it should use policy and observed service value.
Useful signals include:
- declared service class
- community pinning
- allocation priority
- retrieval receipts
- hold duration
- under-replication status
- governance policy for retention and pruning
The system does not need to know "this room transcript is philosophically important" at the storage layer. It only needs to know:
- this object backs a shared room
- the community marked it
StandardorCritical - it is under-replicated
- it is being retrieved and depended on
That is enough for a storage bank.
Verse should keep two different storage subjects separate:
- host capability package
- shared service instance
Examples:
- Matrix applet package: host/runtime capability, more like software distribution
- Matrix room state/history: persistent shared service object
- Gemini renderer capability: host/runtime capability
- Gemini capsule bundle: shared or public content object
The storage bank should focus primarily on persistent service objects, not on automatically turning every installed applet into a streamed network artifact.
That means the important storage-bank question is usually not "how do we ship Matrix?" It is "how do we durably back this particular room, workspace, or hosted object?"
- bilateral/private storage: tracked, auditable, no credit by default
- community storage: auditable, optionally credit-bearing, community policy controlled
- payout: disabled by default; receipt and reputation accounting still active
- encryption: default for non-public service objects
- fragmentation: start with full-copy replication, keep interfaces erasure-coding-ready
- transferable storage tokens if a real coordination problem emerges
- stronger collateral/slashing for anonymous public providers
- more formal proof-of-storage machinery if open-market durability becomes central
- no mandatory token economy
- no assumption that all useful hosting is public-market hosting
- no need to expose plaintext to providers for the network to decide it is worth retaining
- no conflation of installed host capability with the durable shared object that capability operates on
The right reading of the comparison is:
- Syncthing explains the personal and trusted-peer tier.
- Tahoe-LAFS explains how opaque encrypted community storage can work.
- Storj explains how audit and payout mechanics can work around opaque fragments.
- Filecoin explains what a full public market looks like, and why Verse should not start there.
Verse should therefore behave less like a storage coin and more like a community durability fabric with optional incentives layered on top.
That framing fits the rest of the architecture:
- Graphshell remains the host.
- Verso remains the bilateral named-peer and personal-device layer.
- Verse remains the community-scale durability, discovery, and optional incentive layer.
The result is simpler and more realistic than a one-size-fits-all storage economy, while still leaving room for a stronger market model if the project ever truly needs it.
A useful refinement is to describe Verse as a three-zone storage architecture rather than a single decentralized storage network turning disk into one commodity class.
The proposed blueprint gets several things right:
- encrypted, opaque fragments are the correct model for community storage
- signed receipts and audit trails are more realistic than immediate on-chain settlement
- a lighter-weight accountability model than Filecoin is a better fit for Verse
But four corrections matter.
Calling the whole system a market or commodity layer makes the private and bilateral tier look more economic than it should be.
Verse needs three different postures:
- personal / bilateral replication
- shared-service durability
- optional open public hosting
Only the middle and later the third tier need a genuine storage-bank or market vocabulary.
Full PoR / PoRep language implies stronger cryptographic guarantees than the current Verse design actually requires.
The more accurate first-implementation model is:
- content-addressed integrity
- challenge-based availability verification
- real retrieval receipts
- signed append-only accounting
Merkle commitments can help organize proofs of blob/chunk layout, but by themselves they do not prove that a provider is still storing the data over time.
The current architecture already has the correct transport split:
-
irohfor bilateral named-peer trust and device sync -
libp2pfor community-scale discovery and swarm behavior
Using libp2p as the default private/bilateral layer would blur a boundary that is currently clean in VERSO_AS_PEER.md.
Receipts are not web bearer tokens. They are signed evidence records.
The better conceptual split is:
- signed domain receipts for accounting and audit
- UCAN for delegation and capability transfer
- optional VC-like attestations for portable provenance
JWT would import unnecessary web-auth assumptions into a system that already has a cleaner cryptographic boundary.
The improved blueprint is:
Use iroh plus explicit trust store semantics.
- own devices and trusted peers
- tracked, inspectable, challengeable
- no public credit by default
- strong fit with the existing Verso layer
Use encrypted content-addressed blobs or fragments to back durable objects.
- room state/history
- workspaces and graph projections
- capsule bundles and feed snapshots
- shared files and attachments
- applet state checkpoints where applicable
This is the real storage-bank layer:
- availability challenges
- retrieval receipts
- repair incentives
- service-class policy
- allocation and retention controls
Only later, if needed:
- anonymous or pseudonymous providers
- stronger collateral and slash conditions
- more formal proof machinery
- DHT-heavy placement and provider discovery
This is where Filecoin-like lessons become more relevant, but it should not be the first operational target.
The right Rust stack should follow the architectural boundaries already present in Verse and Verso rather than flattening them.
Recommended fit:
-
irohfor bilateral/private transport and direct trusted-peer replication -
rust-libp2pfor community-scale provider discovery, pubsub, and swarm participation when the community layer is actually activated
This preserves the current split described in VERSO_AS_PEER.md and related Verse docs.
Recommended fit:
-
aes-gcmorchacha20poly1305for opaque encrypted payloads or chunks -
rs-ucanfor delegation/capability expression -
crypto_box-style key wrapping for encrypted object/key handoff, matching patterns already used in the session capsule plan
This aligns better with Verse's existing capability language than JWT-style bearer tokens.
Recommended fit:
-
cidplus multihash-compatible content addressing for durable object identity -
redborfjallfor local durable state, matching existing documentation posture more closely than a pivot tosled
The primitive is right; the crate should be chosen only after a focused dependency audit.
Recommended posture:
- design Verse around a
FragmentManifestand aCodingSchemeabstraction first - start with full-copy replication as already documented
- leave room for Reed-Solomon / Zfec-compatible erasure coding later
- avoid prematurely hard-coding the architecture around one crate name before validating maintenance posture, portability, and WASM implications
Recommended fit:
- Merkle or manifest commitment structures are useful for chunk layout and integrity proofs
- challenge-response availability checks are still required
- signed retrieval receipts remain the strongest practical signal of actual usefulness and service value
In other words, Merkle proofs help; they do not replace availability and retrieval evidence.
| Concern | Recommended fit | Why |
|---|---|---|
| Bilateral/private transport | iroh |
Matches Verso's existing trust and sync boundary |
| Community transport | rust-libp2p |
Useful for Kad/GossipSub once community swarms matter |
| Opaque payload encryption |
aes-gcm or chacha20poly1305
|
Good AEAD choices for encrypted blobs/fragments |
| Capability delegation | rs-ucan |
Better semantic fit than JWT for access rights |
| Key wrapping / object sharing |
crypto_box-style flow |
Already aligned with existing capsule-sharing patterns |
| Local durable storage |
redb or fjall
|
Closer to existing Graphshell/Verse documentation |
| Content identity | cid |
Fits VerseBlob and content-addressed storage |
| Erasure coding | abstract behind CodingScheme
|
Keeps architecture flexible while v1 stays full-copy |
| Accounting | signed domain receipts | Receipts are evidence, not auth tokens |
The right implementation posture is:
- do not market the whole thing as one storage commodity system
- do not overclaim PoR/PoRep properties before the protocol actually has them
- do not replace
irohwith libp2p in the bilateral tier - do not use JWT as the core receipt primitive
- do use encrypted opaque fragments, capability-oriented access, signed receipts, and content-addressed durable objects
That refined blueprint is both more realistic and more compatible with the Verse architecture already described in the active docs.