Ollama OpenWebUI - spinningideas/resources GitHub Wiki

You can use a local llm chat setup without any limits using Ollama and Open WebUI and OpenAI API or any model running locally.

How to Video: https://www.youtube.com/watch?v=8J6OJzseYuo

Here is the step by step process:

Step 1: Install Ollama

• https://ollama.com/

Enable nvidia GPU

• https://hub.docker.com/r/ollama/ollama
• https://learn.microsoft.com/en-us/windows/ai/directml/gpu-cuda-in-wsl
• https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#prerequisites
• https://medium.com/@u.mele.coding/a-beginners-guide-to-nvidia-container-toolkit-on-docker-92b645f92006

Step 2: Copy and Paste Llama 3 install command using Terminal

https://github.com/ollama/ollama?tab=readme-ov-file#quickstart

ollama run llama3.2

Step 3: Add other LLM models (optional)

• https://ollama.com/library

Step 4: Install Docker

• https://www.docker.com/

Step 5: Install OpenWebUI via running docker container

• https://docs.openwebui.com/#installation-with-default-configuration

docker run -d -p 6000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main