Connect a Model - Inktomi93/OrbWeaver GitHub Wiki

Orbweaver doesn't come with a language model, so nothing will answer you until you connect one. You have three ways to do it: paste a key for a hosted service, point Orbweaver at a model server you run yourself, or (at your own risk) use a Claude subscription.

A connection is one provider plus one model, with its own key or server URL. "OpenRouter with this model" is a connection. "My Ollama server with that model" is another. You can keep as many as you like, for instance a big model for the story, a cheap one for background work, and a local one for testing.

Saving a connection doesn't make Orbweaver use it. You then choose which connection does which job under Model roles, which is covered below. Until you pick one for Chat, Send stays off and the Next reply: line says why.

1. Open Settings, go to Connections, and press Add connection.
2. Pick a Provider. A server you run yourself is listed under Your own server (URL).
3. For a hosted service, paste your API key. For your own server, type its address into Server URL. The key field there is API key (optional), which you fill in only if your server asks for one.
4. Pick a Model. Press List models to read the list from the provider (after you've pasted the key) or from your server (after you've typed its URL), then pick yours. If no list comes back, or your model isn't in it, type the model's id the way the provider or server spells it.
5. Give it a Label if you want one. Turn on Allow background work only if this connection should run background jobs such as the Utility model. Press Add connection.

The first connection you add that can chat doesn't just close the dialog. It becomes your Chat model, and the dialog turns into Set up your first model, which asks what background work (memory summaries, image captions and extraction) runs on. That's the Utility model, and a cheaper model is fine for it. Pick one:

• Use the same one, (name) runs background work on this connection too. It's offered when this connection can do that job.
• Use another connection picks one you already have.
• Add a different model for it swaps the button for Add a model, which opens the add form for a second model. Save it and you're back on this step with it picked.
• Not now leaves it unset. Memory summaries, captions and extraction wait until you pick one under Model roles.

Press Finish. Whichever connection you picked gets Allow background work turned on, and the step tells you so before you press it. You can change both later under Model roles. If that first connection is a Claude subscription, the step starts at Not now instead, and says why: background work on a subscription draws on your plan's limits. More under Picking a cheap one.

For any other connection, open Model roles in the same place and pick the roles it should do. If a connection should handle more than chat, its menu has Use this connection for everything it can serve. It sets every role the connection can do to this connection in one go, replacing what those roles were set to, and the menu lists the roles first. If that includes Text embedding, your library gets indexed again. Roles that run in the background are skipped unless the connection has Allow background work on. You can change any of them afterwards.

Once Chat is set, the line under the message box goes away. It only comes back to say why a reply can't be sent. The model that wrote a reply can show with that message's actions.

About keys and models:

• Your key is encrypted where Orbweaver stores its data and never shown again. It's only sent to the provider or server you picked.
• A key is its own saved item. To use a second model from the same provider, choose Add another model on this key from a connection's menu instead of pasting the key again. For a server saved without a key it says Add another model on this server. You can replace or revoke a key under Saved keys, and every connection that uses it follows.
• The list is a hint, not a gate. If the list loads and your id isn't in it, you can still use it. The model field says "This model id wasn't in <host>'s list. It'll be sent as-is; if the server doesn't have it, turns will fail." The connection saves with a Model not in list badge. If the list can't be read at all, you get "Couldn't list <host>'s models" with the reason and Try the list again, and the field becomes a plain box for the id.
• Where a provider's list says what each model can do (OpenRouter's does), models are marked vision or tools, and you can filter the list by Vision and Tools. A list from your own server has only ids, plus a context size when the server reports one, so it has no marks and no filter. Game mode needs a model that supports tools to update its own state, and Context size and tools below says how that works on your own server.

Provider	You give it
Google Gemini	An API key
OpenRouter	An API key
Anthropic	An API key
OpenAI	An API key
vLLM	The server's URL (a key if you set one)
LM Studio	The server's URL (a key if it needs one)
Ollama	The server's URL (a key if it needs one)
llama.cpp server	The server's URL (a key if it needs one)
KoboldCpp	The server's URL (a key if it needs one)
Custom OpenAI-compatible	The server's URL (a key if it needs one)
Claude subscription	A setup token. At your own risk, see below
Built-in (this device)	Nothing to enter. It runs the built-in search and rerank models and can't chat

There's no text-completion mode: replies are written through a chat API (OpenAI-compatible, Anthropic, Google or a Claude subscription). Ollama uses its native chat endpoint; the other named local servers use their OpenAI-compatible endpoint. There's no built-in NovelAI or AI Horde connection. For other hosted services, such as Mistral, DeepSeek or Cohere, use OpenRouter if it carries the model you want, or Custom OpenAI-compatible with the service's own base URL and key. A plugin can also add a provider of its own. It shows up in the same list, marked with the plugin's name.

The OpenAI connection uses Chat Completions. That route has different tool support from OpenRouter.

• With GPT-6 Sol or GPT-6 Luna, turn off the preset's Reasoning switch when using tools. OpenAI accepts tool calls on this route only with reasoning disabled. See the Sol and Luna model pages.
• GPT-6.1 Sol supports prose and native structured JSON on this route, but not tool calls. Choose another connection for features that require tools. See the model page.

If a Refinery schema check refuses a tool-based fallback, check the preset selected as Utility preset under Model roles, then try the schema check again. See Refinery.

Ollama, LM Studio, vLLM, llama.cpp's server and KoboldCpp each have their own entry under Your own server (URL). Anything else that serves an OpenAI-style chat endpoint, like TabbyAPI, goes through Custom OpenAI-compatible. All of them take the same two fields, Server URL and an optional key. Pick the named entry when there is one, because it carries that server's known settings. If a server needs something unusual, a saved connection's Diagnostics has Request & response shaping: add your own request headers, stop sending fields the server rejects, and point Orbweaver at where the reply comes back.

1. Start your model server and note its address and port.
2. In the Add a connection dialog, pick the provider and type the address into Server URL, starting with http:// or https://. Ollama's is typically http://127.0.0.1:11434, LM Studio's http://127.0.0.1:1234, llama.cpp's server http://127.0.0.1:8080 and KoboldCpp's http://127.0.0.1:5001.
3. Leave API key (optional) empty unless your server was started with a key. If it was, paste it, and Orbweaver sends it to your server as a bearer token.
4. Press List models and pick yours, or type the id your server expects.
5. If the Server URL field shows a private-address message, admit the address. If Orbweaver runs in Docker, read Orbweaver in Docker first.

Orbweaver sends its requests to <Server URL>/v1/...: the model list, chat and embeddings. Typing the /v1 is optional. Orbweaver adds it unless the URL's path already names a version, and it ignores a trailing slash, so http://127.0.0.1:11434 and http://127.0.0.1:11434/v1/ reach the same place. A service whose API root names another version, like /v4 or /v1beta, is used exactly as you typed it.

Ollama is the chat exception: Orbweaver sends native /api/chat requests at the server root. Its Max context tokens preset setting controls the requested num_ctx, up to the model's trained limit. Custom body fields can override that request, so don't set a conflicting options.num_ctx unless you intend to replace it.

What breaks:

• No http:// or https://. The field says "<what you typed>" is not an http(s) URL.
• A whole endpoint pasted, like http://127.0.0.1:11434/v1/chat/completions. Orbweaver adds /models after it, the server answers 404, and List models reports "Couldn't list <host>'s models" with the HTTP 404. Stop the URL at /v1.
• The wrong port, or a server that isn't running. List models reports "Couldn't list <host>'s models" with "transport failure".

To see the exact address Orbweaver calls, open the connection, expand Diagnostics and press Send a test request. It shows the request line.

• vLLM sleep mode: start the server with VLLM_SERVER_DEV_MODE=1 and --enable-sleep-mode if you want Orbweaver to wake a sleeping model before a request. Keep the development endpoints private. See vLLM's sleep-mode guide.
• Qwen on vLLM: use a chat template and tool parser matched to your checkpoint. The template must honor the thinking controls you send. Our local Qwen testing uses a corrected server template; that is not an OpenRouter configuration or a promise that every stock Qwen template behaves the same. Orbweaver does not replace your server's template.
• llama.cpp reranking: launch a dedicated reranker with --reranking, then choose it for Rerank. The ordinary chat model is not a reranker.
• llama.cpp game replies: when using In the reply state delivery, start with --no-reasoning-preserve so old reasoning is not kept throughout the history. The app also controls template reasoning where the server supports it. See llama.cpp's server options.

Under Advanced, Endpoint quirks has a template thinking switch fact for servers that expose one. Orbweaver uses it to spell the resolved thinking choice in the server's chat template. A model that requires reasoning is not silently told to stop thinking.

Diagnostics → Request & response shaping → Fields to add or replace is the final custom body override. It accepts JSON, YAML or key=value lines. These fields replace the app's computed values, including template kwargs; Don't send these fields removes a field even if you set it here. Use the test request or wire trace to check the actual body, not just the preset controls.

If Orbweaver runs in Docker, 127.0.0.1 and localhost mean the container, not your computer, and nothing is listening there. Use host.docker.internal instead, like http://host.docker.internal:11434. A server on another machine uses that machine's address. Either way, the owner then has to admit the address, as described next. The address a "just me" install admits ahead of time is 127.0.0.1, which in Docker is the container.

• On Linux, a server that listens only on the loopback address, which is Ollama's default, refuses connections that come from the container. Make it listen on all interfaces. For Ollama that is OLLAMA_HOST=0.0.0.0.
• Docker Engine on Linux and Podman can skip all of this with the host-network overlay: docker compose -f docker-compose.yaml -f docker/compose.build.yaml -f docker/compose.host-network.yaml up -d --build. The container then shares your computer's network, so http://127.0.0.1:11434 works as written and is already admitted.

Orbweaver won't reach out to addresses on your own computer or network unless the owner has allowed them. That's so a connection can't be used to poke at things it shouldn't.

When you press List models or Add connection with an address that isn't admitted, the Server URL field says so, and the owner gets an Admit button under it naming the host and port. Press it, then try again.

• An IP address or localhost, such as http://192.168.1.50:1234, reads "192.168.1.50:1234" is a private address this deployment does not admit., with an Admit 192.168.1.50:1234 button.
• A name, such as host.docker.internal or ollama.lan, is looked up first. If it lands on a private address, the field reads "host.docker.internal:11434" resolves to a private address this deployment does not admit., with an Admit host.docker.internal:11434 button. A name that doesn't resolve yet isn't refused here; it's checked when Orbweaver connects.

You can also admit an address ahead of time under Settings, Admin, Multi-user, by adding a line like host.docker.internal:11434 to Allowed private endpoints, or press Admit under Reachability in a saved connection's Diagnostics.

The rules:

• An entry with a port admits that host and that port only, and a server on a different port needs its own entry. An entry without a port admits every port on that host.
• Only the owner can admit an address. An admin or a member sees no button, so ask the owner.
• On a "just me" install, 127.0.0.1 and localhost are already admitted, so a server running on this computer works straight away. Nothing else is: a LAN address needs admitting, and in Docker so does host.docker.internal. On an install where people sign in, the list starts empty, so even a server on the same machine needs the owner to admit it.
• Allowed private endpoints takes one entry per line: a host, an IP address, a CIDR range, or host:port. A CIDR range can't carry a port. You can add and remove entries there at any time.

Open a connection and expand Diagnostics. Press Check again under Reachability. It says "Reached it." when the server answered. If the server answered with an error, it says "<host> answered HTTP 404, so it is running but refused this request." (with the real status), followed by the reason, which usually means a wrong URL path. If nothing answered at all, it says "Can't reach <host>" and "the server may be down." A key the server rejects shows the server's own reason instead. Send a test request, in the block above it, shows what the server actually answered. The usual causes are that the server isn't running, the URL has a typo, or you're in Docker and used 127.0.0.1. The owner also gets the Admit button here.

Some servers tell Orbweaver what a model can do, and some don't. It uses three things, and later ones win: what the server reports (a context size from vLLM and LM Studio's model lists, and from the model info of Ollama, llama.cpp's server and KoboldCpp), what Orbweaver already knows about the model from its name, and anything you set yourself. All of it is under the connection's Advanced, in What this server accepts, one row per fact, each with an Override button.

• Context window. On Ollama, Orbweaver reads the window Ollama actually cuts the chat at: the model's num_ctx if its Modelfile sets one, or the window of the loaded model. With no figure from the server, Orbweaver assumes 8,192 tokens, trims the chat history to fit, and the row reads (assumed). If your model has more, press Override on context window. max output sits next to it, starts at 4,096 and is marked the same way when it's a guess.
• Tool calls, image input and structured output. Ollama, llama.cpp's server and KoboldCpp tell Orbweaver what a model can do, and the rows read as reported. On Ollama, tool calls and takes come from the model's own capabilities, and an embedding model is picked up as one. On llama.cpp's server, tool calls come from the chat template and image input from the loaded projector. On KoboldCpp, image input comes from the loaded projector; tool calls are not reported, so declare them with Override if your model makes them. Structured output is on for each of these servers from the release that added it. When a server says nothing about a fact, the row reads not stated or (assumed), and Override sets it. A connection you made earlier as Custom OpenAI-compatible keeps working as it was; to let it read the server, set model info API under Endpoint quirks, or add the server again under its own entry.
  • llama.cpp's server started without --jinja reports tool support it then refuses. The turn fails with a message that says so; restart the server with --jinja.
  • A server you restart with a different model setup, such as without its image projector, keeps its earlier facts until Orbweaver asks it again: change any field of the connection, or press Send a test request under its Diagnostics, and the next turn reads what the server says now.
• Tools. Tool calls are how Game mode changes state. When the server doesn't report them, Orbweaver recognizes some models by name, Qwen models and OpenAI's gpt models among them. For any other model the tool calls row reads not stated, which counts as no. If your server and model handle tool calls, press Override on that row and pick yes, one at a time or yes, several at once. Without tools the game panel shows Read-only: trackers still steer the story, you update them by hand, and nothing the story says changes them.
• Structured JSON. The Utility model's Needs: line asks for it, and it is reported or recognized the same way. Otherwise the model's structured output row reads not stated; override it if your server supports it. Where it's missing, structured extraction skips.
• Vision. When the server doesn't say what a model accepts, Orbweaver assumes images may go through, and takes reads (assumed). It won't stop you sending one to a text-only model. Correct the takes row under Advanced if yours is text only.

• Chat and Utility model work on any own-server connection. The Utility model also needs Allow background work on that connection.
• Text embedding works when the server serves /v1/embeddings. Ollama says which of its models make embeddings, so those are picked up as embedding models. For any other, open the connection and use the change menu beside "This looks like a chat & writing model" under Purpose to pick Search vectors. Turn on Allow background work. Each embedder has its own vector width under Advanced; there is no universal 1024-wide requirement. A truncatable model can use a supported shorter width; other models must return the width they state. Before a change moves your index, Orbweaver probes the model. A mismatched width, rejected key or unreachable server refuses the change and keeps the old binding and index.
• Changing an embedder or its width can delete and rebuild the affected search index. When stored items need rebuilding, Rebuild your search index? says what will be rebuilt before you confirm. Search pauses while it runs. Follow the work under Jobs; rebuilding chat memory also needs the Utility model.
• Rerank works on vLLM and on a llama.cpp server started with --reranking, using a model set to Reranking search results and Allow background work on. Ollama, LM Studio and KoboldCpp do not have a built-in rerank route. An OpenRouter connection can serve Rerank too, or keep the built-in model.
• Image generation isn't available on an own-server connection today.

If you ran a local backend through SillyTavern's text-completion mode, its presets don't carry over, because Orbweaver has no text-completion mode. Coming from SillyTavern has the details.

Open the saved connection and review Prompt caching. Use the controls offered for that model, and read the applied policy and any warning before changing the preset. A route with automatic caching may have no manual controls.

For OpenRouter complete-answer reuse, open your preset's Params view and review Response replay. Check Editing for Chat role and Connection in view first. Leave replay off when you want every ordinary request to reach the model. Fresh generation actions, including swipes, bypass replay.

Caching is a request, not a guaranteed discount. Prompt reuse and complete-answer replay are different. See How a turn is built and Stats and Insights.

Open Settings, then Connections, then Model roles. Each role is a job, and each job runs on one of your connections.

Role	What it's for	Out of the box
Chat	Writing the replies in your chats	Not set. Send stays off until you pick one
Utility model	Quiet background work, covered below	Whatever you picked in Set up your first model, or not set
Image generation	Making pictures with /imagine. Optional	Not set. /imagine tells you so
Text embedding	Turning text into something searchable, for search and memory recall	A built-in model on your computer
Image embedding	Searching pictures directly. Optional	The same built-in model as Text embedding. If you clear it, picture search uses image captions instead
Rerank	Putting search results in a better order	A built-in model on your computer

Text embedding and Rerank already work without you doing anything, so most people never touch them. Changing the text embedding model can rebuild its index, which can take a while on a big library. The built-in search model uses about 2 GB of RAM once loaded, on the CPU. The default Ettin reranker 32M adds about 150 MB while idle and can use roughly another 0.6 GB while ranking a long turn. Its served window is 2,048 tokens per query/document pair, not the whole transcript. Install covers the downloads and timing. They load only when a role that uses them runs, so if no role is set to a built-in model, they never load.

A role with requirements shows a Needs: line for what the model has to do, and which feature skips if the model you picked can't do it. A connection that can't do a role's job isn't offered for it, and one that doesn't allow background work is greyed out for it, with the reason.

The Utility model is a second, cheaper model for work that happens behind the scenes. These run on it:

• Memory summaries, which is how long chats keep their thread. Memory is the Remember earlier in long chats switch under Chat behavior. It's off until you turn it on, it asks before it turns on, and it needs this model. → How memory works
• Captions for images.
• Writing an image prompt from the scene when you use /imagine without words of your own. → Images
• Choosing who speaks next when a Smart group chat has Use the Utility model instead on. Smart normally uses the Rerank model, not Utility. → Rooms and group chats
• The Refinery and the analysis of your character library.

With no Utility model set, memory summaries stay paused and captions are skipped. Smart's default reranker still works; only its opt-in Utility picker falls back to Natural when it needs a model pick. Features that can't work without Utility tell you so.

Your first connection asks for it in Set up your first model (above). To change it, or to set it if you chose Not now, use the Utility model row in Model roles.

Choose a small, fast model from a provider you already use. Most providers have a small, inexpensive tier. Or point it at a small model on your own server. It runs in the background, so the cheaper the better.

• It needs to write prose and return structured JSON. Image input is only needed for captions. A text-only model still does the summaries, and only captions skip.
• The connection must have Allow background work on. It's off by default, and you can turn it on when you add the connection. A connection without it is greyed out in the Utility model menu with the reason, and the role offers the switch right there.
• If you already have a key saved, Add another model on this key is the quickest way to add the small model beside your main one.
• If cost isn't a worry, you can set Utility to the same connection as Chat. Use the same one in Set up your first model does that and turns on Allow background work for you. Later, under Model roles, turn on Allow background work for it first.
• Keep a Claude subscription for chat. Background work runs unattended and would spend your plan's allowance.

If you pay for a Claude subscription, you can chat on it instead of paying per call. It's the riskiest way to connect a model, so read this part first.

At your own risk.

• Orbweaver doesn't sign in to Claude by itself. It runs Anthropic's official Claude Code runtime on your server with your sign-in, so every request counts against your plan's usage limits, the same pool as Claude Code.
• Past those limits, Anthropic may bill requests as usage credits at API rates if you have usage credits turned on. Check Settings > Usage on claude.ai before long sessions.
• Anthropic's terms reserve subscription sign-ins for its own apps and ordinary individual use, and don't permit third-party developers to route requests through Free, Pro or Max plan credentials. Using your subscription here may fall outside those terms, and their rules can change. The app shows the same warning when you add the connection.

If you'd rather not take that risk, use an API key with Anthropic, which is the route Anthropic's documentation points third-party tools to. OpenRouter is another way to reach Claude models on a key.

To connect it:

1. On the computer where you use Claude Code, run claude setup-token. The Add a connection dialog shows the command with a copy button.
2. In Add a connection, pick Claude subscription (it's listed under Subscription) and paste what the command printed into Setup token.
3. Type the model id and press Add connection.

Orbweaver checks the sign-in once after you save and tells you how it went. Check sign-in under the connection's Diagnostics runs it again. Each check sends a one-word request, so it spends a little of your allowance.

Each person adds their own subscription connection. There's no server-wide switch. In a room you host, your connection answers everyone's turns (see below), so a subscription connection would be serving your friends too. Bigger step than using it yourself. Check Anthropic's current terms first.

If the entry is greyed out with "The Claude runtime isn't installed on this server," that server can't run it, so use another provider.

Orbweaver has no per-chat or per-room model picker. A room never has a model of its own.

• In your own chats the answer comes from your Chat connection. To use a different model, change your Chat role. That takes effect on the very next reply.
• In a room with friends, the host's connection answers every turn, including the ones your friends send. Background work for that room, like memory summaries, also runs on the host's connection. Hand the room to someone else and their connection pays from then on.
• If the host hasn't set a Chat connection, everyone else sees "The host has no chat connection set" and can't send until the host picks one. → Rooms and group chats, Share with friends
• Automation rules and plugins that call a model can each be given a connection of their own for the jobs they run. If you don't give them one, they use the Model roles connections. → Automation, Plugins

• "<host:port> is a private address this deployment does not admit", or "resolves to a private address" for a name. The owner presses Admit under the field, as above.
• "Can't reach <host>", followed by "the server may be down." Nothing answered: the server isn't running, or the URL or port is wrong. In Docker, use host.docker.internal.
• "<host> answered HTTP 404, so it is running but refused this request." The server is up, and the URL has a path it doesn't have. Check what you typed in Server URL.
• "Couldn't list <host>'s models" with "transport failure." Nothing answered. Check the server is running and the URL, and see Orbweaver in Docker if you used 127.0.0.1. With "HTTP 404", the URL has a path the server doesn't have.
• "No chat connection is set." Choose one for Chat under Model roles. A saved connection isn't used until you do.
• "Your connection doesn't allow background work." Turn on Allow background work for that connection.

More: Troubleshooting. When you're connected, say hello in Your first chat.