Token Usage - supercog-ai/community GitHub Wiki

Whenever you run a Supercog agent it consumes tokens on the underlying Large Language Model. The companies that host LLMs charge money per token for using their models. Generally speaking, the latest and smartest models cost the most money.

LLMs work by generating "text completion". This means you give the LLM some input, like:

Who was the first person on the moon?

and the LLM generates an answer (called a "completion"):

The first person on the moon was Neil Armstrong.

The LLM vendors therefore charge for both input tokens (your text) and output tokens (the text generated by the model).

When you run AI agents, the LLM is called repeatedly whenever the agent needs to "think" or "plan", and these calls consume tokens.

Supercog keeps track of the number of tokens used in each session with your agent, and displays this at the bottom of the chat window, as "input tokens count/output tokens count":

Since the raw counts don't mean much, you can hover over the numbers to see the actual cost of those tokens:

In this case those 12k tokens, from the Claude 3.5 model, cost 4 cents to process.

See the LLM Models page for more info on the costs of different models.