How Memory Works - Inktomi93/OrbWeaver GitHub Wiki

A language model can only read so much at once. In a long chat, the oldest messages eventually stop fitting, and the story forgets how it started. Memory is what lets a chat keep its thread past that point: Orbweaver writes short summaries of older messages as the chat grows, and when an old moment becomes relevant again, it finds the right summary and puts it back in front of the model.

It recalls summaries rather than raw excerpts. Raw semantic retrieval over a long chat was ass when I tried it, so this builds tiered digests that form an arc instead. The rest of this page is how, so you can tell what memory will and won't do for you. To set up the connections it needs, see Connect a model.

What memory is for

Memory covers the stretch of a chat that no longer fits in the prompt. The most recent messages go to the model as they are, word for word. Memory is about everything before them.

You don't write it by hand, and it isn't a world book. You don't pin notes or pick what to keep. It's built from what happened in the chat, in the background, and it only brings back what looks relevant to the moment you're in. If you want facts that are always true about your world, that's a job for world books.

Memory has two halves:

Writing: after replies, Orbweaver turns older messages into summaries and rolls those up into broader ones.
Recalling: before each reply, it looks for summaries that match what's happening now and adds them to the prompt.

How summaries are written

After a reply finishes, Orbweaver checks whether there is a new stretch of older messages to summarize. It runs in the background. On a hosted Utility model it never delays your reply; on one local server it shares the GPU with your chat.

Messages are summarized in blocks of eight. A block is only summarized once it is complete.
The newest eight messages are never summarized. They are still fresh and go into the prompt as they are. In practice, the first summary appears once a chat passes about sixteen messages.
Each summary has three parts: a heading line saying who is involved and what the scene was, the facts from that stretch that are likely to matter later (small talk is dropped), and a list of distinctive keywords such as names, places and objects.
A cheap background model writes them. That's your Utility model. In a room, it is the host's.

If you later edit an older message, the summary of that stretch is rewritten from the new text. Swiping or editing the newest messages doesn't touch the summaries, because those messages haven't been summarized yet.

How summaries roll up

Summaries are small, but a chat that runs for thousands of messages still produces a lot of them. So they are rolled up. Every four summaries are combined into one broader summary, and every four of those into a broader one still, up to three roll-ups deep:

Summary	Covers (by default)	Tier
The base summary	8 messages	0
First roll-up	32 messages	1
Second roll-up	128 messages	2
Third roll-up	512 messages	3

A roll-up isn't a copy-paste of its parts. It's asked to describe the arc across them. The finer summaries it was built from are kept, so nothing is thrown away. Past the third roll-up there are no further levels, so a very long chat just accumulates broad summaries.

The numbers are defaults. An admin can change them under Memory tuning in the admin settings.

When you read the recall readout (see below), memories are labeled with a tier and a block. The tier is the last column of the table above, and the block number counts up within its tier from the start of the chat.

Why it's built this way

I said raw retrieval was ass. The research agrees:

Retrieval over a tree of recursive summaries beats plain chunk retrieval: RAPTOR, Sarthi et al. 2024, arXiv 2401.18059.
Recursive summaries are what keep a long dialogue consistent: "Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models", Wang et al., arXiv 2308.15022.
Models miss facts buried in the middle of a long context: "Lost in the Middle", Liu et al., arXiv 2307.03172. A bigger window doesn't fix it.
Old information degrades retrieval of new information across every transformer size tested, and a longer context window doesn't fix it: "Unable to Forget", Wang and Sun, ICML 2025, arXiv 2506.08184.
The layered-memory survey, Fofadiya and Tiwari, arXiv 2603.29194, lays out working, episodic and semantic layers, which is roughly the shape of the tiers above.

The papers back the reasoning. The mechanics are the ones on this page.

How recall picks memories

Before each reply, Orbweaver decides which summaries to add to the prompt. You see this as a short "recalling" moment before the model starts writing. By default it works like this:

It drops anything the model can already read. If a stretch is still in the prompt as ordinary messages, its summary is skipped. Memory only brings back what has scrolled out. So a chat short enough to fit entirely in your model's context recalls nothing. Correct.
It builds a compact view of the story from what's left. Distant history is represented by the broad roll-ups, and the most recent stretch by the finer summaries. That keeps the pool small and the "story so far" about the same size however long the chat gets.
It looks at what's being said right now. The search uses the last couple of messages, including the one you just sent, as the question.
It searches by meaning and by keyword. A summary qualifies if it is similar enough in meaning to the recent conversation, or if its keywords appear, as whole words, in the recent messages. That keyword route lets a name or an object qualify even when the wording is far from the summary's.
It reranks. The best handful of candidates (eight, by default) are handed to a second model whose job is to judge relevance more carefully, and the top few (three, by default) are kept. If the rerank model can't be reached, you get a notice and recall falls back to the order from the meaning search.
It puts the winners back in order. The chosen summaries are laid out oldest to newest so they read as a story.

That default is called Sharper semantic recall in the admin settings. The recall readout shows its short code, mixC. The admin settings offer other modes, such as recalling in time order with no search at all, but most people never need them.

The "meaning" part uses your Text embedding model, and the reranking uses your Rerank model. Orbweaver sets up built-in versions of both that run on your own device, so you don't have to connect anything for them. They download once, the first time they're needed.

What goes into the prompt

The text of the recalled summaries, and nothing else, goes into the prompt, under a heading that says "Past events" by default. They sit in the Memory section of your preset, after the chat history, near the end of the prompt. If nothing is recalled, the section adds nothing at all.

The model reads the summaries, never the original lines, so a summary is only as good as the model that wrote it. Dumb models write dumb summaries. A small Utility model can drop a detail, or phrase something a little off. If you need an exact quote from an old scene, memory is the wrong tool; search for the scene yourself (see below).

The host can see exactly what went into a turn, including this section, in the Preview. How a turn is built covers the whole prompt.

The recall indicator

A small brain icon labeled Memory shows what memory is doing:

Dimmed: memory is off, or no reply has gone out yet since you opened the chat.
Recalling: Orbweaver is searching. It's also a handy tell that your request isn't hung; the model hasn't started yet because recall happens first.
A number: that many memories were recalled for the reply. A 0 means recall ran and nothing qualified.

Everyone in the room sees the icon and the count. Clicking it opens a short summary.

Once a reply has gone out, the popover goes further for the host. It shows the question the search used, how many summaries were in the pool, which ones made it into the prompt and how well each matched (as a percentage), and a sample of the ones that didn't, with why. The reasons are plain: below the score floor, covered by a higher tier, still in the live history, not eligible in this mode, or not witnessed by this speaker.

The memory recall popover

Two limits to keep in mind:

It shows which memories and how they scored. To read their text, use the Preview.
The detail is a fresh check. It shows what recall would do with the room's current settings, which is usually what happened but isn't a recording of the last turn. The number on the icon is the real count from that turn.

The detail is host-only. A member sees the count and the summary line.

Personal memory in group chats

By default, a group chat shares one memory: every character recalls the same things. Orbweaver can also give each character a personal memory, where a character only recalls what happened while they were in the room.

That needs three things together:

Memory is on.
The group replies Per-speaker, so each character answers in their own message. With Narrator, one message voices everyone and there is nobody to give a separate memory to.
Each character sees only their own card is on, in the group's advanced settings. Left off, the group stays on the shared memory even with per-speaker replies.

With those on, when a character speaks, recall searches the shared summaries and that character's own, and then drops any stretch they weren't seated for. Someone you add halfway through doesn't recall what came before they arrived. Someone you remove for a while misses that stretch even after you add them back.

This is about room presence. Memory only knows when a character was seated in the room and when they were taken out. It has no idea whether a character was "in the scene" in the fiction. A character sitting in the room but written as being elsewhere still remembers everything. Taking their seat away is the only way to make them miss something.

It works in blocks, too. Since memory is built from blocks of eight messages by default, a character who was present for any part of a block counts as having been there for all of it.

Presence only affects recalled memories. The recent messages in the prompt aren't filtered by presence. Rooms and group chats covers the group settings.

Turning memory on

The switch, the Utility model it needs, and the backfill for chats you already have are on Your first chat, under "Turn on Memory".

Which models it runs on

Summaries run on the host's Utility model. Recall uses the Text embedding and Rerank models, which are built in and run locally by default. A group keeps one shared set of summaries plus one set for each seated character, since each character's set is what personal memory reads from. If you change your Text embedding model, your whole library is indexed again.

In a shared room, the host's memory switch and model roles apply, not yours. Summaries run on the host's Utility model; recall uses the host's Text embedding and Rerank models.

Searching past scenes yourself

Memory also makes your history searchable by you. In the Corpus, Explore has two targets that read from it:

Scenes searches the text of your past scenes. Describe the moment, like "the night she lied about the key", and it finds it even if you don't remember which chat it was in.
Memories searches the summaries instead, which is handy for "what did we ever decide about the harbor?"

A result opens a window of the original transcript around that moment, and for a memory it also shows the summary. Open in chat at this moment takes you to that exact point in the story, or Open chat if the exact spot can't be found. Search always returns the nearest matches it has, and when nothing is really close the list says so.

A few conditions:

Scenes are indexed as part of memory, so a chat needs memory to have been on when its messages aged out, or the backfill run for older chats. Memories also need a Utility model, since they are the summaries.
It only covers chats you host.
It uses the same Text embedding model as recall, so on a new install the built-in one needs to finish downloading and your library needs to be indexed first.

When recall seems to do nothing

If the icon never shows a count, it's usually one of these:

Memory is off (or an admin has set the retrieval mode to Off), or you haven't reached about sixteen messages yet.
There's no Utility model, so summaries are paused.
The chat still fits in the model's context, so everything is already in the prompt.
The chat is older than your memory switch and hasn't had a reply or a backfill since you turned memory on. Imported chats count too, unless you took the import's Build memory for imported chats offer.
In a group with personal memory, the character wasn't seated for the stretch in question.

If a notice says your library couldn't be searched, check the embedding connection in your connections. The reply went out without recalled memory, but nothing was lost.