Screens - breadboard-ai/breadboard GitHub Wiki

This is a proposal to revamp the app view.

The app consists of screens.

Each step produces zero or more screens.

A typical step can produce two types of screens, loosely mapping to the state of the step.

  • Progress screen -- when the step wants to show intermediate results to the user.
  • Input screen -- when the step is requesting information from user.

Showing Progress

The Progress screen matches to an output event that is labeled (TBD how) as "progress". The "output" shows some intermediate results. This screen has a "Continue" button on it, which allows the user to pause to examine what is on screen before moving on.

When the Progress screen is presented to the user, the run does not stop, which means that multiple screens may be created by the step (or the following steps) while one screen is presented. This allows the author of the flow to create stacks of screens that are shown to the user while the run continues.

This is particularly useful in situations when the steps that follow take a long time (a trick similar to those in early phone startup screens -- entertain the user while frantically doing the work in the background).

By default, all outputs of steps are presented as progress screens. The author should have a choice to disable that per step.

For steps that invoke tools, tools can present progress screens as well. When the invoking step has its screen-showing disabled, only the steps of tools show up. The author should have an option to disable progress screens for all invoked tools.

The Progress screen has two states:

  • Interactive -- when in this state, the "Continue" button is shown
  • Complete -- when in this state, no continue button is shown

When first presented to the user, a Progress screen is interactive. Once the user clicks the "Continue" button, the screen becomes Complete for the duration of the run.

In situations where the user clicks "Continue" on all the progress screens (or all steps prior had progress screens disabled), the Default Progress screen is shown. This screen will likely have some simple animation (maybe a variant of the graphic from the home page) that tells the user that work is happening, with no buttons or choices beyond that. This screen is shown until the next screen comes up.

Asking for Input

The Input screen matches to a pair of output + input events from the step that are labeled (TBD how) as "input". The output event is an opportunity to show some intermediate results and the input event pauses the run to ask user for input.

The "input" event may request the following kinds of information from the user:

  • multimodal content (aka LLMContent). The author should be able to specify what modalities are preferred/supported and how many (for example, "three images" or "one text")
  • choose one or many from a list of choices. The author provides a list of choices, and specifies the mode ("one" or "many"). Then, the user picks one of them, and the input returns the chosen items back to the step. A single choice is perfectly acceptable.
  • a combination of the two above -- like a form.

Forms allow presenting inputs like "enter content or select Love it": the first input is the multimodal to enter content and the second is the single-choice Love it option with a single "Continue" button to submit both.

Like with the Progress screen, tools invoked by the step can present their own input screens.

Like the Progress screen, the Input screen has two states:

  • Interactive -- shows the user the input fields and allows the user to make choices and enter content.

  • Complete -- show the user what was entered, but does not allow changing it.

When presented to the user for the first time, the Input screen is Interactive. Once the user submits the their inputs, the screen becomes Complete and can't be changed back.

Navigating screens

Throughout the run, the user should be able to navigate across the screens, like pages in a browser. This allows them do things like going back to game instructions on the first screen or just flipping through the various artifacts. All past Input and Progress screens show up as Complete.