What's next - pokey/cursorless-vscode Wiki

The big picture

Cursorless's major direction can be summed up as follows:

Cursorless will become a spoken programming language enabling large-scale, structurally aware voice-controlled code generation and refactoring.

The goal is to enable levels of efficiency that aren't possible using today's keyboard-driven interfaces. Let's break down each component of the above statement.

"spoken programming language"

There's two aspects here: one is "programming language", and the second is "spoken ... language". Let's take each in turn

"programming language"

Today's cursorless grammar will become a full-fledged programming language inspired by jq. In the same way that jq enables powerful one-liners, which can easily grow into longer programs, we'd like cursorless commands to function like powerful "spoken one-liners", but also be chained into larger scale "spoken programs" that can transform your code in semantically rich ways by leveraging cursorless's deep integration with tree-sitter and vscode. To get there, we need to make two major changes to the very core of cursorless.

Today, cursorless targets are flat, enabling only a single modifier to be applied to a given mark, eg "funk air". We intend to make cursorless modifiers fully compositional, so that they can be joined into long chains. Examples include:

In addition, cursorless targets need to be unified and enriched with additional semantic information. These intermediate forms will enable short utterances to describe powerful modifier chains. When the needs of a transformation extend beyond the limits of what the user can say in a single phrase, the intermediate forms can be persisted across phrase boundaries, either by using an enhanced that mark for impromptu chains, or by bookmarking intermediate steps of a chain, or even by bookmarking the chain itself as a reusable pipeline component.

See the tracker project for more info.

"spoken ... language"

Because cursorless is spoken, there is the opportunity for users to speak in long continuous sentences, rather than short, choppy commands. Although it is possible to issue fairly long phrases today, there are still some limitations. A great example is "bring"ing a piece of code while modifying it in the same breath. See the tracker project for more on how we intend to improve the fluency with which users can operate Cursorless.

Beyond just removing technical limitations, an important aspect of attaining fluency is education. The learning curve for cursorless can be daunting, so we have several efforts underway to enable new users to get up to speed much faster, and for existing users to rapidly improve fluency. In particular,

See the education tracker project for more.

"large-scale, structurally aware"

There are two main directions here:

Cross-file operations

We'd like to extend the powers of cursorless beyond operations in a single file. One of our inspirations is Plop, a "micro-generator framework" that allows parametrized operations across multiple files. In Cursorless, the bookmarks described above could be stitched together into a combined bookmark that spans multiple files. Each component of the combined bookmark can indicate how new items are inserted using a stable snippet with smart holes.

For example, notice how much boilerplate is needed to add a cursorless hat color. By getting users comfortable with applying syntactic transformations and constructing lightweight snippets as part as their daily workflows, building large-scale, semantically rich transformations will become second nature, rather than a chore.

See the tracker project for more about this and other new modes of interaction we have planned for cursorless.

Smart multiple cursors

Ironically, cursorless becomes most powerful when used with multiple cursors. Multiple cursors without cursorless can feel like herding cats, because the operations supported by your editor are often too low level to work correctly at each cursor. For example, say I have three strings on my clipboard, and I want them to replace each of the following strings:

a = "hello world";
b = "high";
c = "I am a big old string";

Assuming I have a cursor in each string, with cursorless I can just say "paste to string".

We'd like to expand this power further by introducing new scope types and actions, as well as adding new operations to precisely control large numbers of cursors by voice.

Check out the tracker project for more details.

What else are we working on?

In addition to the big picture efforts above, we have several other projects underway to continue to improve cursorless.

Cursorless everywhere

Today, cursorless is restricted to editing text in VSCode. We'd like to expand the places where the cursorless grammar can be used, both within VSCode and without.

A bit of polish

The cursorless code base is quite stable, due to its extensive suite of >1000 tests. However, it is still a bit rough around the edges in a few places. We'd like to remove the surprises, and get Cursorless to the point where it always does exactly what you'd expect, and you never have to clean up after it (which is rare, but still happens here and there 😊).

Improved contributor experience

We'd like to make it easy for anyone to roll up their sleeves and hack on cursorless. The cursorless code base is strongly typed, modular, and fairly well documented, but can still be a bit daunting for new contributors. We'd like to create a proper documentation hub, including an internal API reference, videos, architecture overviews, etc.