25.0317 Doc URLs Stack URLs Lab notes (Gdrive), Git

This BE #322 demos local model (NO RAG).

Note:

• This demo does not include RAG (thats the next goal).
  • Something like demo'd in RAG #311 (had a very big delay, but ran all locally)
• Or: For production you might instead spend the money to custom train a model for the product?

FE integrations

1 FE #266 React

This is the basic workflow:

• 1.1 FE (3). On the frontend enter a prompt (asking something about the target product; the data is from the product docs which was added to the model using RAG (adding RAG content is the next demo I will be working on; this demo merely uses a model that I downloaded)).
• 1.2 Ngrok. The request goes through Ngrok to the backend running on my personal PC (this is usually turned off, so wont work if you try).
• 1.3 AI model (Mistral) (4.4). Running 100% on my PC (not fast, but free). Formulates the response. In a future setup I want to customize the model content (with RAG) to cover topics specific to the documented product..
• 1.4 BE (1). The backend sends the prompt to the AI model.

The response is then sent back to the FE.

1.1 FE (3)

Note: Posting with Postman

1.2 Ngrok

1.3 AI model (Mistral) (4.4)

See #322 for details.

Installing Ollama in WSL (Windows Linux).

Ollama pull Mistral.

1.4 BE (1)

Simple python program that calls the Mistral API.