Particles - breadboard-ai/breadboard GitHub Wiki

Particles is a UI toolkit designed to answer the following question: "Now that LLMs can generate UI code, what's the right way to do this?" The word "right" is load-bearing here and contains various intuitions and practices that guide the overall design—also known as "The Opinion." This document outlines the guiding principles, core architecture, and key mechanisms of the Particles toolkit.

The following assertions comprise the foundational principles of the Particles toolkit design:

• Sandbox the LLM -- The code that is generated by LLMs will have to be sandboxed to run safely. Specifically for UI, the code that drives the UI:
  • must be able to do so asynchronously (though with minimal latency) relative to the code that renders the UI;
  • must not be able to directly manipulate or access the UI rendering layer.
• Separate rendering from semantics -- The oldie-but-goodie from HTML, but now with feeling.
  • The code that renders the UI must have ultimate control over how the UI is presented.
  • The code that drives the UI must provide the semantics of what it intends to convey, but not dictate the exact rendering.
• Lean on signals -- Recognizing signals as a promising direction for Web UI, Particles embraces them as a core reactive primitive.
• Prefer concrete use cases -- Practical implementations and challenges (e.g., from the Breadboard project) should inform priorities and the overall design, favoring practical solutions over purely theoretical explorations.

The sandboxing requirement is a primary driver of the Particles architecture. It necessitates a clear separation between the environment where LLM-generated code runs and the environment where the UI is rendered. This leads to a layered approach:

• The Emitter (Model + Controller): Located inside the sandbox. This component is driven by the LLM-generated code and is responsible for producing the semantic information that will eventually be displayed in the UI.

• The Receiver (View): Located outside the sandbox. This component consumes the information produced by the Emitter and is responsible for rendering the actual UI. It has control over the presentation.

• The Pipe: This layer serves as the communication channel between the Emitter and the Receiver. It handles the secure and structured shuttling of information (Particles) from the sandboxed environment to the rendering environment.

The fundamental unit of information exchanged between the Emitter and Receiver is a Particle.

flowchart BT
    u["UI code"]
    v["Receiver (View)"]
    t["Pipe"]
    mc["Emitter (Model + Controller)"]
    l["LLM-generated code"]
    subgraph Sandbox
      l -- "Calls Emitter to produce/update particles" --> mc
    end
    mc -- "Sends particles" --> t
    t -- "Delivers particles" --> v
    subgraph Outside of sandbox
      v -- "Presents signal-backed structure for rendering" --> u
    end

The communication also goes in the other direction, facilitating events, sent by the UI back to code that drives the UI.

// TODO: Invent this.

Particles are lightweight data structures representing a distinct piece of information to be presented in the UI. Each particle carries semantic meaning. There are three primary types of particles:

• Text particle represents textual information. This can include plain text, Markdown, HTML, JSON, or other text-based formats.
• Data particle represents binary or rich media information that typically requires specific rendering handling. Examples include images, audio, video, PDF documents, or specialized objects like Google Drive files.
• Group particle represents a logical grouping of particles (think div). It allows for the creation of structured hierarchies.

Here are their type definitions:

type TextParticle = {
  /**
   * Content of the particle.
   */
  text: string;
  /**
   * The type of the content. If omitted, "text/markdown" is assumed.
   */
  mimeType?: string;
} 

type DataParticle = {
  /**
   * A URL that points to the data.
   */
  data: string;
  /**
   * The type of the data.
   */
  mimeType: string;
}

type GroupParticle = {
  /**
   * The sub-particles that are part of this group.
   * The Map structure is key for reactive updates.
   */
  group: Map<ParticleIdentifier, Particle>;
  /**
   * The type of a group. Allows the particle to be bound to a particular
   * UI element. Optional. If not specified, the group particle doesn't have
   * an opinion about its type (think "generic grouping").
   * If specified, can be used to identify semantics. For example, can be used
   * to bind to the right custom element. 
   */
  type?: string;
};

type Particle =
  | TextParticle
  | DataParticle
  | GroupParticle;

type ParticleIdentifier = string;

The Group particle is fundamental to how the Receiver organizes and presents information. The group property being a Map is designed to be implemented using a reactive data structure like SignalMap. This allows the UI rendering layer to:

• React efficiently to changes within the group (additions, removals, updates to child particles).
• Maintain rendering stability for the elements of the group by using the ParticleIdentifier (the Map's keys) for identity (e.g., with Lit repeat directive).

Since Group particles can contain other particles, including other Group particles, they naturally form a Particle Tree. This tree is a hierarchical representation of the information that the Emitter intends to convey. The Receiver layer ultimately presents one or more such particle trees to the UI rendering code, with a Group particle serving as the root of each tree.

A key requirement is the ability for the Emitter to dynamically update the UI over time. The LLM-generated logic might produce information incrementally or react to new inputs. Particles facilitate this through a streaming mechanism:

1. The Emitter sends a stream of changes (individual particle creations, updates, or deletions) to the Pipe layer.

2. The Pipe transmits these changes to the Receiver.

3. The Receiver applies these changes to its internal representation of the particle tree(s).

This means the Emitter's API for sending particles feels like writing to a stream, while the Receiver's API (and the structure it exposes to the UI rendering code) looks like a reactive, readable tree.

To manage the creation and manipulation of these particle streams, the Emitter layer will feature an API centered around Particle Beams. A Particle Beam represents an instance of a stream that connects to the Pipe layer, typically corresponding to a root Group particle in the particle tree.

This API aims to provide a DOM-like manipulation experience for the Emitter, but operating on an abstract, semantic representation (the particle stream) rather than directly on UI elements.

Here's an API sketch (TBD):

type BeamIdentifier = string;

type Emitter = {
  /**
   * Creates or gets existing beam with specified id.
   */
  beam(id: BeamIdentifier): Promise<Outcome<Beam>>;
};

type Beam = {
  /**
   * Closes the beam, flushing and pending updates.
   */
  close(): Promise<Outcome<void>>;

  /**
   * Provides a way to tell the operations to automatically create
   * a unique index for a newly appended/inserted Particle.
   */
  readonly autoIndex: symbol;

  // Operations

  /**
   * Appends Particle at the end of the Map. If "id" specified as an array,
   * uses this array as a path to the GroupParticle in the current tree. 
   */
  append(
    id: ParticleIdentifier | ParticleIdentifier[] | symbol,
    particle: Particle,
  ): Promise<Outcome<ParticleIdentifier>>;

  /**
   * Inserts Particle before specified index. If "id" specified as an array,
   * uses this array as a path to the GroupParticle in the current tree. 
   */
  insert(
    id: ParticleIdentifier | ParticleIdentifier[] | symbol,
    particle: Particle,
    before: ParticleIdentifier | undefined,
  ): Promise<Outcome<ParticleIdentifier>>;

  /**
   * Removes specified Particle. If "id" specified as an array,
   * uses this array as a path to the GroupParticle in the current tree. 
   */
  remove(
    id: ParticleIdentifier | ParticleIdentifier[]
  ): Promise<Outcome<Particle>>;
};

Here's sample of how it might be used. In this particular case, the sandbox is calling Gemini API and wants to provide updates to the user.

// in sandbox
async function callGemini(emitter: Emitter, inputs: GeminiInputs) {
  // Here, the beam is either created or acquired.
  // Beam -> generic GrouParticle.
  // This allows having a  single group being shared across different
  // callsites. All the rules of stomping over ids apply, so the
  // consumer needs to be careful.
  const beam = await emitter.beam("console");
  if (!ok(beam)) return beam;

  // Create a group particle that represents an update.
  const update = new Map<ParticleIdentifier, Particle>();

  // The renderer of the update expects four properties:
  // - config -- contains icon and title to show in the header of the update
  // - input -- the JSON of the input, submitted to the API
  // - progress -- the ephemeral progress indicator (markdown)
  // - output - a GroupParticle containing the output

  // Set "config", "input", and "progress" at the beginning of the Gemini call.
  update.set("config", fromJson({ title: "Call Gemini API", icon: "spark" }));
  update.set("input", fromJson(inputs));
  update.set("progress", { text: "Waiting for response..." });

  // The type = "update" is defines the meaning of this group particle.
  // We let the append auto-index, so that it picks a unique index value
  // without us having to worry about stomping over other updates.
  const id = await beam.append(beam.autoIndex, {
    group: update,
    type: "update",
  });
  if (!ok(id)) return id;

  // ... invoke gemini API.
  const output = await gemini(inputs);

  // Now set "output". The value is an LLMContent, so we create another
  // GroupParticle for it, using a helper function (implemented elsewhere)
  const outputGroup = fromLLMContent(output);

  // Now, we need to update the update group` with `outputs` and
  // remove `progress`.
  // Since the group isn't reactive on the Emitter end, instead of doing this:
  //    info.set("output", outputGroup);
  //    info.delete("progress");
  // We need to do this:
  await beam.append([id, "output"], outputGroup);
  await beam.remove([id, "progress"]);

  // Finally, close this beam. We're done with the update.
  await beam.close();
}