Using AI Runner with RAG (Retrieval-Augmented Generation)

AI Runner provides a Retrieval-Augmented Generation (RAG) system that allows you to enhance your AI's responses with knowledge from your own documents.

How RAG Works in AI Runner

AI Runner uses a keyword-based retrieval system to find relevant information in your documents. When you ask a question, the system:

1. Extracts keywords from your query
2. Searches the index for documents containing those keywords
3. Retrieves the most relevant passages
4. Uses those passages to generate a more informed response

Performance Optimizations

RAG search in AI Runner has been optimized for performance:

• Caching of keyword extraction: Previously extracted keywords are cached to avoid redundant processing
• Throttled index refreshing: The index is only refreshed after a certain time interval (5 minutes) to avoid excessive processing during multiple searches
• Batched document processing: Documents are processed in batches for more efficient indexing
• Progress reporting: Long-running operations show progress indicators
• Performance timing: Key operations are timed to help identify bottlenecks

Using RAG with Your Documents

To use RAG with your own documents:

1. Add documents in the AI Runner interface
2. The system will automatically index them
3. When chatting with your AI, it will now use these documents to enhance its responses

Troubleshooting

If RAG search seems slow:

• Reduce the number of documents in your index
• Use smaller documents or break large documents into smaller chunks
• Ensure your documents are properly formatted
• Check the logs for any error messages related to indexing