Design Goals - mr-martian/quest-editor GitHub Wiki

The contributors to this project all have somewhat different opinions of text editors and different reasons for being involved. The following is our thoughts on what we hope to achieve.

For me this starts with the thought that how computers process text is broken at most levels but that this is probably not impossible to fix:

Fonts are too constrained by the starting assumption that a sequence of discrete rectangles is a good representation of human writing. That does a decent job of displaying printed English, but you don't even have to leave the Latin alphabet before you start running into issues (for example, any diacritics that don't a pre-composed form in Unicode that the font creator included).

Once you start messing with something as basic as text rendering, it seems to me that making a new editor isn't necessarily all that much more work than fitting it into an old one, plus this gives an opportunity to fix other annoyances.

Of course, in the short run, we'll almost certainly want to support existing fonts for the sake of usability.

The following are things that seem really cool:

• Plugin system set up from the beginning to enable code from different places to communicate with more than just text streams
• Spellchecking on code
• Queuing several edits and then applying them to a whole file

Context in the first paragraph, but you can skip to the second paragraph if you want.

The First text editor I used extensively was Atom. It worked pretty well. I never got into its hackability. Ultimately, its memory and power-hungriness irritated me enough to find an alternative. Emacs was my next thing. Hearing some gurus rave about it on Quora led me to believe it would solve nearly all my problems! They were right, but not it would be a rocky road. I went through three cycles of love and disappointment before settling on Emacs as the only possible editor to suit my needs. Ultimately, the discovery of using the side of your hand to push control is what saved me. No Emacs pinky ever again. Thanks, Xah! <- Applicable for everyone not just Emacs users! What keeps in Emacs is Org mode. There just isn't anything else out there like it. It truly does as Neil Stevenson writes (paraphrasing): it is as the sun to the stars.

Emacs is like a religion almost. At first, you're resentful of everything being so different. Why is paste called yank? Why is cut called kill? Why is Ctrl-F forward char and not find? Why is Ctrl-s find and not save? Why is Ctrl-x everything and not cut? And the list goes on. But then whether it's stockholm syndrome or being baptized in the Church of Emacs, you come to appreciate these differences, and have some empathy for why they exist. However, much like Martin Luther against the Pope, I'm having some doubts about the Doctrines of Saint Ignucius. At some point I might try compiling my own 95 theses in Disputation on the Power and Efficacy of Emacs.

• Very steep learning curve
• Unpredictable "easy customization interface"
• Difficult to configure
• Elisp (Decent programming language, but difficult for beginners and not very resource efficient)
• Better (proportional) font support (but admittedly, Emacs handles this better than a lot of editors)
• Single-threaded: One misbehaving extension can cause Emacs to crash.
• Requires an awakening.

• More intuitive defaults for whoever wants to learn it (options for MS-DOS vs. Emacs vs. Vim style standard shortcuts)
• Keyboard layout agnosticism if people want to use ergonomic vim bindings with Dvorak or something like that.
• Drastically improved GUI and mouse support (while maintaining 100% keyboard driven-ness), like following conventional GUI design principles.
• Better official tutorial and documentation geared towards beginners. (But SystemCrafters might have spurred a revival).
• Include by default a number of convenience features/plugins: something similar to Treemacs, Helm and which-key, while also including Dired and Org.
• Provide a better customization interface that behaves more intuitively.
  • Font face priority more explicit, better warnings for conflicting styles
• Use an extension language with more procedural syntax.
• Easy access to the syntax highlighting markup AST (so you can see what the parser is actually doing, so you can set variable width faces for Markdown and Org properly)
• Support for multithreading and multiprocessing.
• Built-in use-package support
  • use-package is an Emacs extension to the default package management system that checks for installed packages, installs them automatically if they're missing, and prevents them from being loaded before you use them, which can speed up start time by quite a bit. but getting it set up is confusing for something that should just work. When you migrate to a new Android or iOS phone, it asks you if you want to reinstall all of the apps associated with your account. Instead, just have installed plugins be in your key-value config app, and it can automatically configure your instance to be exactly the way you want it regardless of underlying operating system.

There are many things that Emacs got very right (and I would love to include):

• graduated complexity
  • use as much or as little as you want. You can learn the basic editing commands and let all the major/minor mode hooks be handled by the default configuration. The experience isn't great, but it's better than Notepad or TextEdit.
  • No limits to extensibility. Why not create a window manager using Emacs as a backend? If it's the interface you're used to, and it makes you more efficient, no one can form a pragmatic argument against that (except for some technical limitations of emacs itself, which this project aims to fix).
• Package management: Even the default package manager for emacs isn't a pain. A lot of the newer packages encourage compiling from source and setting your own this, that, and the other thing, but for the vast majority of people ELPA + MELPA is very accessible, reliable (because it's curated), and pretty user friendly, all things considered. (User ratings and reviews would be nice though.)
• The flexibility to display in a comprehensible format many kinds of data
  • you can view and edit binaries with hexl-mode
  • you can view pictures and bitmaps inline with text (extremely useful when writing papers with figures).
  • you can do file-management tasks with dired (mainly editing directories as plaintext)

Other (maybe) intriguing ideas:

• Arbitrarily nestable tab or window groups
• 1-to-1 command to GUI event correspondence (might be difficult with mouse and trackpad events).
• Design Core like a mini-game engine, but limit execution to 1 frame/16 ms to conserve computer time.
  • Use Entity-Component-System to keep things modular.

My ultimate goal, however, is to use this editor as the framework for several other programs I would otherwise write as terminal tools with ncurses (A world-building/novel outlining tool, conlanging toolkit). But a lot of these are things that would be great to interact with each other. So rather than write some obtuse ipc protocol, why not just work on an extensible text editor with a plugin interface that lets me do just that?

My primary interest in a new text editor is to provide an excellent editing experience, with simple and easy customization, usable by people with absolutely no programming experience, without compromising on extensibility and flexibility, and especially accessibility. I believe that with a robust core that provides a base level of functionality, the experience of using, configuring, and extending the editor can be made simple and expressive. I apologize that this letter is so long; I lacked the time to write a shorter one. (I'll edit this down later.)

I currently use Qt Creator for writing C++, and find it to be a generally quite good experience, but it is much less useful for any other language. So for most other tasks, I use Notepad++, which is easy to use (all of its commands use common design language and generally standard keyboard shortcuts), has autosaving so that loss of work is very rare (I have several scratch files I've never saved (and named) open for years at this point), and has the only GUI-based syntax highlighting definition tool I have ever seen in a text editor, which is helpful for me because I often use unusual or ad-hoc formats which I am usually not interested in re-implementing in an extension language I don't otherwise use. However, it is limited in what it can express compared to the builtin language highlighters. Notepad++ also allows workspace/project organization using folders that are parallel to the on-disk folders, which is lacking in many other editors. When I began using Linux, I tried a number of GUI text editors such as Geany and Kate and none of them seemed better to me than Notepad++, which is why I've kept using it, despite the fact that I have to run it under Wine, which is not perfectly stable. For quick terminal editing I use Nano, which is basic and unwieldly, but more user-friendly than most other CLI editors because it displays the most critical shortcuts on screen. I would prefer not to use two separate text editors in conjunction this way, especially since I often need both in a single project. However, not using an IDE would cause me to lose too much productivity, but I can't use the IDE for everything because Qt Creator's support for other languages is nearly nonexistent.

I tried both VS Code and Atom, and disliked both for extremely similar reasons (they were unnecessarily difficult to configure due to using raw JSON configuration, any significant use required downloading several extensions just to get basic support, because the out-of-the-box feature set was too minimal, and their use of Electron made them bloated, awkward, and unstable compared to ordinary GTK+/Qt applications). I have tried Emacs, and found it to be a very frustrating experience, and ultimately decided after a couple hours that no matter how powerful it is, it can't be worth its learning curve for me. I have not yet tried any form of Vim, which is in at least a small part because it is fractured among several branches and forks which I don't know how to choose between.

Additionally, sometimes I just want to have some basic WYSIWYG formatting in my text, such as could be accomplished with a simple text backend like Markdown or Enriched Text, but in order to do that I have to use a completely different type of program from a standard text editor. This is an artificial and restrictive separation that usually means I end up just making do with ASCII approximations and abusing a close-enough syntax highlighting mode in Notepad++ (YAML is what I typically use for structured notes) because I don't want to use a third editor in a single project.

• Poor ergonomics. Often, some or most keyboard shortcuts are hardcoded, some features may have no keyboard shortcuts at all, common operations may take several clicks and/or keystrokes to perform, and edge cases will be awkward to deal with.
• Poor accessibility and flexibility. Most developers develop software primarily for themselves. This fact is largely unavoidable, especially in open-source software. However, when a user comes along who wants (or needs) to use software in a way that is different from what the designer had in mind, this conflict is brought into focus, and different developers handle the situation with varying degrees of grace. Additionally, sometimes these issues can't be fixed by the application developer, because the underlying issue is with library code. This may go several levels deep. Many well-used libraries, such as native system GUI frameworks like GTK+, have over time acquired a large number of accessibility features. But this history often brings with it complexity and technical debt, or it may limit theming by making all applications look similar. Thus, many developers to turn to newer libraries, with simpler or more customizable designs, either not recognizing or not caring about the inherent trade-off they are making. Libraries like Electron have become popular because they allow every single aspect of the GUI to be styled however the designer wants, but this comes at the expense of consistency (it is becoming impossible to have consistent UI theming across disparate applications as more and more devs choose 'branding' instead) and accessibility, as their custom widgets don't interface with screen readers or other accessibility tools, or their custom themes have contrast or colorblindness issues. Full Unicode support, including for directional text or unusual characters, is also highly spotty. In Notepad++ for example, some text areas use a different font from the main editing area, which means that Unicode characters such as ů or ṙ to appear as boxes when typed into the search and replace menu, for instance.
• Idiosyncratic design. This overlaps with the above problems to an extent. Some editors force you to learn not only some new keybindings, but an entire new paradigm of human-computer interaction, such as Emacs (see below). Emacs is not the only offender; any editor originally designed for the command-line will necessarily have some of these issues, such as how Vim uses a completely different paradigm of keyboard control than any other program, in some ways even more unique than Emacs, but Emacs is the most egregious in that this same complaint applies to almost the entire program.
• Wasteful design. Text is one of the smallest and lightest resources computers have to deal with, to the extent that entire novels take less memory than a single high-resolution photo. (The average book on Amazon Kindle, according to one website, is about 2.5 MiB. A single photo from my cell phone, even when stored with JPEG compression, typically takes 3-4 MiB.) However, many modern text editors use incredible amounts of memory, and often suffer from performance issues with large files. There is no good reason for a text editor to ever struggle to keep up with a fast typist—even if that editor is running on a 15-year-old computer. It is an unfortunate fact of life, however, that almost all of us have encountered software that struggles to keep up even with fairly slow typists on high-end modern hardware, which execute upwards of 20,000,000 clock cycles per keystroke, as an extremely rough approximation. Some people even turn to very old programs on equally old computers just because they are more responsive.
• Poor adaptability to syntax. At minimum, every text editor should allow for setting custom indentation settings per language. Some languages, such as Make, have a semantic distinction between tabs and spaces, while others, such as YAML, ban tabs outright. Even in languages that don't have such restrictions, different indent widths make sense for different languages. Additionally, many editors only support highlighting a limited set of languages, or have idiosyncratic or overly complex ways of specifying new highlight definitions, such as long XML files or making the user manually write a tokenizer in a scripting language. EBNF and regular expressions are much lighter and easier to read and write, and the technology to dynamically build parsers from such specifications is decades-old and not particularly difficult to use.
• Configuration being either too limited and simplistic, or being far more complex than necessary.
  • Prime examples of the former include most basic text editors (Notepad, Featherpad, Leafpad, etc). In these editors, there are typically only a handful of settings in total, and many things are simply not configurable at all. Either the application works and does what you need, or it does not and can not.
  • The latter extreme is exemplified first and foremost by Emacs, but to a lesser extent by editors such as VS Code and Atom. In Emacs, the only way to configure the editor at all is to write a configuration script in Emacs Lisp. For most purposes, this is unnecessary complexity that worsens the experience of the user for no real gain. Emacs Lisp is a language unique to Emacs, and Lisp as a family of languages has become ever more obscure over time, such that most people will be encountering it for the first time. It also contributes to the common complaint that Emacs takes too long to start up, because it has to interpret a large amount of code rather than simply parse data.
    • Fundamentally, using a programming language to express configuration suffers from severe and unavoidable problems as a result of the halting problem. It is often desirable to write programs that adapt to and modify their own configurations, such as through a GUI, or simply for persistence of runtime-toggled modes; however, it is difficult (and in the general case, impossible) to write a program which takes as an input another program and modifies it optimally to change its output in a specific way. This is possible (and not even especially hard) if the input program is known to follow a trivial structure, and often programs can be written that will automatically generate from scratch such trivial scripts, but this is no counterargument at all, because if you know that all configuration scripts will be trivial, then the fact that they are scripts, and the fact that more complex programs are expressible at all is unnecessary complexity which is only begging to cause problems down the line. Therefore, using a configuration script is fundamentally at odds with providing an accessible and fully-featured configuration experience.
    • This argument suggests applying a version of the 'Rule of Least Power' to configuration and plugins. Limiting the capabilities of parts of a program and the ways that they can interact with each other results in a simpler and more comprehensible overall design, which can be used in more flexible and open-ended ways, and often can have benefits to performance as well.
  • In VS Code and Atom, as well as some other editors, most configuration is simple data, in a language such as JSON, which is manipulated by plugins written in a full programming language, like JavaScript. This structure works quite well in theory, however, the specific implementation chosen by these editors sacrifices accessibility and discoverability in the name of implementer laziness. Forcing end-users to modify structured data, especially in languages like XML or JSON which are not designed with ergonomics as a primary consideration (JSON is one of extremely few text formats for any purpose that does not have standard comment support, which I consider unacceptable for human interface use), still poses a worse experience for users. A GUI allows for users to easily discover what settings are available, what values are valid for each, and what their defaults are, using well-known design patterns going back decades. A user editing a text configuration file simply has to hope that the documentation is well-written or that someone else will have posted their question online already. We can do better.
  • This is not to say that the GUI should be the only way to access settings. Textual settings files have some clear advantages. However, using a GUI is in no way incompatible with backing it up with a textual configuration system, and in fact many programs already do this, using (in order of ease of editing) INI files or ad-hoc settings.txt files; simple structured data languages like TOML or YAML; or less ergonomic languages like the aforementioned JSON or XML.
• Poor or non-standard support for important editing commands. These include regular expression-based search and replace, which vary widely between programs. The most common omission is multiline regex support. Also, some regex engines have pathological worst-case time complexity on some inputs, despite the fact that this is a well-known problem that can be easily avoided.

All of these problems are solvable, and in most cases solutions are well-known, but I haven't encountered any single editor which solves all of them at once. Ideally, I want an editor which has IDE-level support for every language, instead of just a few, while still being as efficient, stable, and ergonomic as a basic text editor. That way, I could have a single editor capable of all my editing needs, instead of using (and having to remember keybinds for) 3 separate ones with different feature sets.

Since I've found I have so much to say about it, I figure it is better to collect my thoughts about Emacs in one place instead of just bringing it up as an example so much. It has many problems that are largely unique to it, anyway.

• In Emacs, every single feature has a bespoke implementation that differs from comparable features in various ways. Sometimes, these differences are useful, such as the different undo system it uses, while others are just inconsistency for no reason of modern relevance, like the way that every single UI element or editing action in Emacs is named in a way that runs counter to the standard names used for literally every other kind of software. Instead of cutting and pasting text, you 'kill' and 'yank' it. Instead of 'frames', there are 'windows'. Instead of the backspace key, you type <DEL> (not to be confused with <Delete>). Instead of plugins, you have 'modes'. The list goes on. This means that if you start from knowing nothing about Emacs, there is almost no way for you to learn anything about it other than just reading the documentation as if it were a book, because you won't even know the basic terms to look up otherwise. Unlike most other programs, there's no chance to just start using it and learn it on the fly, because you have to go out of your way just to learn the most basic interactions.
• Emacs' philosophy is to be a single program which does everything. More than just editing text in it, it's also a terminal multiplexer, a shell, a tiling graphical window manager, and more. A common joke is "Emacs is a fine operating system in need of a good text editor." This has the effect of making Emacs into a bubble with its own ecosystem, whose parts cannot be used outside of it even if they have general utility.
• Whenever Emacs has a problem, instead of changing the design in any way, the solution is always just to do more Emacs to it. Emacs is a memory-hungry application that takes a long time to load? Run it as a daemon on startup so you don't have to wait later, and now as a neat bonus it's now a persistent service, and you can let it do even more than it did before. The problem isn't solved—in fact it's allowed to get worse—it's just pushed further away.

• "The plugin model should be written with the assumption that the end-user does not need to learn the plugin language just in order to configure the editor."
• "I wanted to make simple things simple and to ensure that complex things are not impossible or unnecessarily hard."—Bjarne Stroustrup, on the design of C++.
• "Dealing with large files is rare, but when you do it, you very quickly find all the faults in your editor's handling of them."
• As little code should be run on startup as possible. Plugins should ideally just load their initial state from the configuration files.
• Program state should be persistent by default, in order to prevent data loss due to crashes or other events. Non-persistence should be the exception, and justified accordingly.

I have hopped around with my editors quite a bit. The first text editor I gained substantial experience with was emacs. Vanilla emacs, to be precise. I appreciated the that it allowed me to easily and quickly perform complex text manipulations that I would have otherwise thought too difficult to do in one go. When I decided I wanted to customize emacs to be a Python IDE, I was again impressed with its capabilities. When I started writing my own macros, I found myself rather impressed with elisp.

I admire elisp for a lot of things. I like how its extensibility allows emacs to become a frontend for a wide variety of different applications, even though this power can be corrupted at times. I think that the way it thins the line between editor functionality and the API used to access it is brilliant (i.e. each keybind actually just runs an elisp function). This makes writing extensions for emacs very nice, and it allows you to bridge emacs knowledge with elisp knowledge, allowing for a smoother transition between using the editor and hacking the editor.

I eventually fell out of favor with emacs. Sure, I could do fancy things with text, but it was rarely satisfying to do so. The keybindings are unweildy and actually force me to look at my keyboard as I'm typing, which is basically a sin. I've actually heard some emacs defenders admit that remapping the control key is the only way to somewhat sanely use vanilla emacs. That is ridiculous. Common complaint, I know. But it's so common because it's so relevant.

When I loaded Void onto my laptop at the start of my first semester of college, I committed myself to learning vim. I only had basic knowledge of vi editing from instances when I needed to edit something quick and easy from the command line. Once I started learning the power of vim I loved it. It could do everything emacs could, but it ran better in my terminal, loaded up faster, and was far more intuitive. Compare C-x SPC, C-x r t to C-v, c. Indeed, vim proved itself to be a much more sane way of doing basic text editing tasks. Across the board, I started to improve my productivity by orders of magnitude using vim's text interactions. I learn more things about vim every day that make me fall in love with its text editing more and more.

But I'm not in love with vim itself.

Indeed, although I love writing text in vim, I don't especially love how vim frames that power. In emacs, I can have many buffers open in the background easily and swap between them whenever it behooves me, even when that buffer may not be visible on the screen. I did learn that vim also supports this. I looked it up in the vim manual once but nothing stuck and I also remember the entire mode of interaction being very convoluted for something that should be obvious. Half of the buffers are hidden for no reason, you have to type ! or else it won't leave a buffer even though it's already loaded into memory and will not stop being loaded into memory were you to switch to another buffer. Maybe these are features for some people. I find them simply annoying, and it actually demotivated me from even trying to make it work. That's never a good sign.

I also found that I ended up butting heads with vim when trying to customize my keybinds. There is no way to get a list of all current keybindings. You can look up a specific default keybinding in the vim manual and you can get a list of what you've changed, but it's never actually possible to get the full picture. Additionally, if you press a key by mistake and want to know what you've done, you cannot view the lossage like you can in emacs. Small inelegancies, but inelegancies nonetheless.

So if I like emacs' backend and vim's editing, why not use Spacemacs or Doom Emacs? Well, I tried both of those. I found that, even though emacs itself is off in its own world, these layers over emacs both take that to the extreme. Very little knowledge of how vanilla emacs functions is all that applicable to Spacemacs or Doom Emacs. Both layers are very heavy and require you to have a butt ton of very specialized knowledge to be proficient in them. This knowledge, unlike vim knowledge or, to an extent, vanilla emacs knowledge, will likely not be transferrable to any other program you use. So why bother? You bother because evil + emacs isn't fun to set up by yourself by any means. You need to have a layer to take care of some of that work for you. The only layers available, however, are large and memory-intensive and definitely not sane. At this point, I went back to vim.

Then there's the things neither of them do right.

Why can't I make a set of variables to apply to every buffer of a specific type? Why do I have to use hooks in emacs and autocmds in vim? Both are great if you want to do something more complicated, but to set a variable? Come on! Going further, I have never found a convenient way to apply these hooks or autocmds to already-open buffers. Sometimes I want to set things up on the fly. Is that so hard to let me do that?

In all likelihood this will never really matter that much speed-wise or performance-wise. I, however, don't care. It's an inelegant feature of both programs that grinds my gears.

These and more flaws are what I want to see fixed. Also I like having a programming project for the summer that will require me to learn about a lot of different things that I've been itching to know more about.