OpenRouter

Example output of Eisenhower Matrix project

📊 API Usage:
Tokens: 383 (Prompt: 321, Completion: 62)

💰 API Key Balance:
Credits Used: 0.000223344
Credits Remaining: 19.999776656
Credit Limit: 20
Free Tier: No

OpenRouter API Usage Guide

This document explains how to query the OpenRouter API for key balance information and track usage costs in the Eisenhower Matrix application.

Table of Contents

1. Introduction
2. Querying API Key Balance
3. Tracking Token Usage
4. Displaying Usage Information
5. Testing API Usage

Introduction

The Eisenhower Matrix application uses the OpenRouter API to access various language models for task categorization. To help users monitor their API usage, we've implemented functionality to:

1. Query the remaining balance of an OpenRouter API key
2. Track token usage for each API call
3. Display this information in the user interface

Querying API Key Balance

OpenRouter provides an endpoint to query information about your API key, including the remaining balance. Here's how we implement this in our application:

import os
import requests
from typing import Dict, Any

def get_key_balance(api_key: str = None, base_url: str = "https://openrouter.ai/api/v1") -> Dict[str, Any]:
    """Get the remaining balance and information for the API key.
    
    Args:
        api_key: OpenRouter API key (if None, will try to get from environment)
        base_url: OpenRouter API base URL
        
    Returns:
        Dictionary with key balance information
    """
    # Use provided API key or get from environment
    api_key = api_key or os.getenv("OPENROUTER_API_KEY")
    if not api_key:
        return {"error": "API key not provided"}
    
    url = f"{base_url}/key"
    headers = {
        "Authorization": f"Bearer {api_key}"
    }
    
    try:
        response = requests.get(url, headers=headers)
        if response.status_code == 200:
            # The response has a 'data' object with key information
            data = response.json().get("data", {})
            return {
                "label": data.get("label", ""),
                "limit": data.get("limit", 0),
                "usage": data.get("usage", 0),
                "limit_remaining": data.get("limit_remaining", 0),
                "is_free_tier": data.get("is_free_tier", False)
            }
        else:
            return {
                "error": f"Failed to get key balance: {response.status_code}",
                "message": response.text
            }
    except Exception as e:
        return {"error": f"Error querying key balance: {str(e)}"}

Example Response

When successful, the /key endpoint returns information like this:

{
  "data": {
    "label": "sk-or-v1-...",
    "limit": 120,
    "usage": 109.84,
    "is_provisioning_key": false,
    "limit_remaining": 10.16,
    "is_free_tier": false,
    "rate_limit": {
      "requests": 110,
      "interval": "10s"
    }
  }
}

Tracking Token Usage

The OpenAI client library used by OpenRouter provides usage information after each API call. Here's how we track this information:

from openai import OpenAI
import json

def categorize_task(task_description: str, model: str = "google/gemini-2.0-flash-001"):
    """Categorize a task using LLM and track usage.
    
    Args:
        task_description: Description of the task to categorize
        model: Model to use for categorization
        
    Returns:
        Tuple of (category, usage_info)
    """
    client = OpenAI(
        api_key=os.getenv("OPENROUTER_API_KEY"),
        base_url="https://openrouter.ai/api/v1"
    )
    
    messages = [
        {"role": "system", "content": "You are a task categorizer."},
        {"role": "user", "content": f"Categorize this task: {task_description}"}
    ]
    
    try:
        # Make the API call
        response = client.chat.completions.create(
            model=model,
            messages=messages,
            max_tokens=200,
            temperature=0.1,
            response_format={"type": "json_object"}
            # Note: OpenRouter doesn't support the 'usage' parameter
        )
        
        # Extract usage information
        usage_info = None
        if hasattr(response, 'usage'):
            usage_info = {
                "prompt_tokens": getattr(response.usage, "prompt_tokens", 0),
                "completion_tokens": getattr(response.usage, "completion_tokens", 0),
                "total_tokens": getattr(response.usage, "total_tokens", 0)
            }
        
        # Parse the response
        result = json.loads(response.choices[0].message.content)
        category = result.get("category", "")
        
        return category, usage_info
        
    except Exception as e:
        raise RuntimeError(f"Failed to categorize task: {e}")

Important Note: Unlike the OpenAI API, OpenRouter does not include a cost attribute in the usage information. The cost must be calculated separately based on the token usage and the model's pricing.

Displaying Usage Information

In our Gradio application, we display the API usage information in the status output after each task categorization:

def categorize_task(self, task_description: str, selected_model: str):
    """Categorize a task and save it to history."""
    # ... (task categorization code) ...
    
    # Get API usage information
    usage_info = self.categorizer.get_last_usage_info()
    key_balance = self.categorizer.get_key_balance()
    
    # Format status message with task categorization and API usage information
    status_message = f"✅ Task categorized as: {task.category}\nJustification: {task.justification}"
    
    # Add usage information if available
    if usage_info:
        status_message += f"\n\n📊 API Usage:\n"
        status_message += f"Tokens: {usage_info.get('total_tokens', 0)} "
        status_message += f"(Prompt: {usage_info.get('prompt_tokens', 0)}, "
        status_message += f"Completion: {usage_info.get('completion_tokens', 0)})\n"
        # Note: Cost information is not provided by OpenRouter API
    
    # Add key balance information if available
    if key_balance and 'error' not in key_balance:
        status_message += f"\n💰 API Key Balance:\n"
        status_message += f"Credits Used: {key_balance.get('usage', 'N/A')}\n"
        
        # Show remaining credits if available
        if 'limit_remaining' in key_balance:
            status_message += f"Credits Remaining: {key_balance.get('limit_remaining', 'N/A')}\n"
        
        # Show credit limit
        limit = key_balance.get('limit', 'Unlimited')
        status_message += f"Credit Limit: {limit}\n"
        
        # Show if it's a free tier key
        if 'is_free_tier' in key_balance:
            status_message += f"Free Tier: {'Yes' if key_balance.get('is_free_tier') else 'No'}\n"
    
    return category_result, status_message, self.get_task_history_df()

Testing API Usage

We've created a test script to verify the behavior of the OpenRouter API:

import os
import requests
from openai import OpenAI
import json

def test_key_endpoint():
    """Test if the OpenRouter API supports the /key endpoint."""
    api_key = os.getenv("OPENROUTER_API_KEY")
    if not api_key:
        print("OpenRouter API key not found in environment variables.")
        return
    
    base_url = "https://openrouter.ai/api/v1"
    url = f"{base_url}/key"
    headers = {
        "Authorization": f"Bearer {api_key}"
    }
    
    try:
        print("\n--- Testing /key endpoint ---")
        response = requests.get(url, headers=headers)
        print(f"Status code: {response.status_code}")
        
        if response.status_code == 200:
            print("Response JSON:")
            print(json.dumps(response.json(), indent=2))
        else:
            print(f"Error response: {response.text}")
    except Exception as e:
        print(f"Error: {str(e)}")

def test_usage_parameter():
    """Test if the OpenRouter API supports the usage parameter."""
    api_key = os.getenv("OPENROUTER_API_KEY")
    if not api_key:
        print("OpenRouter API key not found in environment variables.")
        return
    
    base_url = "https://openrouter.ai/api/v1"
    client = OpenAI(
        api_key=api_key,
        base_url=base_url
    )
    
    try:
        print("\n--- Testing usage parameter ---")
        response = client.chat.completions.create(
            model="google/gemini-2.0-flash-001",
            messages=[
                {"role": "user", "content": "Hello, how are you?"}
            ],
            max_tokens=50,
            temperature=0.1
        )
        
        print("Response attributes:")
        print(dir(response))
        
        if hasattr(response, 'usage'):
            print("\nUsage information:")
            usage = response.usage
            print(f"Usage attributes: {dir(usage)}")
            
            # Try to access common usage attributes
            print(f"Prompt tokens: {getattr(usage, 'prompt_tokens', 'Not available')}")
            print(f"Completion tokens: {getattr(usage, 'completion_tokens', 'Not available')}")
            print(f"Total tokens: {getattr(usage, 'total_tokens', 'Not available')}")
        else:
            print("No usage information available in the response.")
    except Exception as e:
        print(f"Error: {str(e)}")

if __name__ == "__main__":
    test_key_endpoint()
    test_usage_parameter()

This test script helped us discover that:

1. The OpenRouter API supports the /key endpoint for querying API key balance
2. The OpenRouter API does not support the usage parameter in the completions request
3. The OpenRouter API does not include a cost attribute in the usage information

Based on these findings, we've implemented our API usage tracking functionality to work with the actual behavior of the OpenRouter API.