Roadmap

Most targets are pursuable in parallel. For instance, the design of the core filesystem abstractions must be informed by the design of actual virtual filesystems, and no code is necessary to drive that feedback loop.

Short-term targets

• Abstract specification: a language-agnostic characterization of the services provided by suffuse and what guarantees will be offered to higher layers.
• Reference APIs: abstract interfaces coordinated with the abstract specification. Initial language targets include java, scala, rust, and the mock DSL.
• Mock implementation: a direct lifting of filesystem operations into the command line, allowing prototyping suffuse filesystems via the mock DSL.

The reference APIs can be thought of as concrete specification as well as abstract implementation.

Once these three exist in a provisional form, we will develop a variety of filesystems and allow the process to feed back into the specification and APIs before we allow anything to harden. Stability is an absolute requirement so it's critical we take pains to foresee possible weaknesses in the architecture.

Medium-term targets

• Native Packaging: it must require near-zero effort to install/update suffuse, and it should be invisible afterward. I'm leaning toward an entire subproject to address this aspect somewhat more generally.
• Java Daemon: filesystems written in jvm languages won't be viable without a rock-solid long-running jvm. Another requirement which has broader applicability than suffuse. Initially this will be built on mechanically derived BridJ FUSE bindings.
• Scala Daemon: a scala-specific interface above the low-level java bindings.
• Rust Daemon: this is where rust ought to shine.
• Confuse: AKA configuration-based suffuse, because most applications of suffuse won't require custom code, only custom parameters. You will want to be confused whenever possible.
• Core Filesystems: see below

Long-term targets

• Shuffle: the suffuse shell, including typed pipelines
• Infinite storage: integration with IPFS or another DHT

Core Filesystems

Readonly

• Implied files: define transformations between file types (or simply extensions) such that for a transformation A => B any file of type A implies a peer file of type B to be created on demand.
• Lazy files: define Path => URI mapping so as to map an unbounded non-local set of resources into local namespace, with data obtained on demand and cached by policy.
• Metadata-driven files: remap a set of files (such as images called IMG_.) to new paths derived from their metadata.
• Union: Combining any number of filesystems into a common hierarchy, with hooks for manageing collisions.
• Observer: Any subset of filesystem events triggers further specified activity.

Read/write

• Container transparency: every known container and compression format to be bidirectionally mapped to its exploded contents.
• Immutable files: input is a read-only filesystem. All writes to virtual filesystem are translated into repository operations on a suffuse-private git repository. All previous revisions of any given file are available, to the limits of resource constraints.
• Typed files: extends immutable files with constraints. Writes which conform to type are committed to the repository, others are left as dirty writes to the index to be committed once the file contents are type-conformant again.
• SQL filesystems: A SQLite backend which can produce filesystem hierarchies based on datalog queries.

Cross-cutting subjects

• Caching: critical both in API and implementation
• Portability: FUSE vs. OSXFUSE, Windows, consider bypassing FUSE altogether
• Metadata: how much, how eager, distinction from data, mapping to xattrs
• Robustness: graceful degradation is a requirement
• Deployment: loading code into a running system, bundling logic with filesystems
• Composition: the usual composition challenges plus a few new ones

(Cross-cutting subjects each needs its own page.)