AssistantController - applebiter/gnatwriter GitHub Wiki

AssistantController encapsulates functionality related to the ollama server.

update_models()

update_models() -> List[Type[OllamaModel]]

Returns a list of OllamaModel objects. They contain the name of a model, its instruction template, a default system message (priming) that is used if a custom primer is not included in the request, and a Modelfile that is unused in the app for now. This method requests the list of available models, and any models in that list that are not already stored in the database are added. Note that any models in the database that do not appear in the list provided by the ollama server will be deleted from the database. This method is automatically called as the first step in any requests of the AssistantController.

chat()

chat(
    prompt: str,  # The query or instruction to send to the LLM
    temperature: Optional[float] = 0.5,  # see Ollama docs  
    seed: Optional[int] = None,  # see Ollama docs 
    priming: str = None,  # Optional input used to supply a "system" role message to the LLM
    options: Optional[dict] = None,  # see Ollama docs 
    session_uuid: str = None,  # A new one is generated with every request unless one is supplied
    keep_alive: Optional[Union[float, str]] = None  # see Ollama docs 
) -> dict

This method and ALL of the methods involving an LLM, directly, can take a while to complete, depending on factors outside of my control.

It's using Ollama's Python library to make a REST request against an ollama server. The keep_alive option can optionally be set to 0 and the LLM will still have a log of the entire chat if the same session_uuid is used with each request in a chat.

rag_chat()

rag_chat(
    prompt: str,  # The query or instruction to send to the LLM
    documents: List[str],  # A list of absolute paths to unadorned text documents, only
    temperature: Optional[float] = 0.5,  # see Ollama docs
    seed: Optional[int] = None,  # see Ollama docs
    priming: str = None,  # Optional input used to supply a "system" role message to the LLM
    options: Optional[dict] = None,  # see Ollama docs
    session_uuid: str = None,  # A new one is generated with every request unless one is supplied
    keep_alive: Optional[Union[float, str]] = None  # see Ollama docs 
) -> dict

This is the same as the chat() method except it also allows the developer to supply a list of plain text documents, about which the user and LLM may chat.

describe_image()

describe_image(
        images: List[str],  # A list of absolute paths to images to send to the LLM
        prompt: Optional[str] = "Describe the image",  # Optionally overwrite the prompt.
        temperature: Optional[float] = 0.5,  # see Ollama docs
        seed: Optional[int] = None,  # see Ollama docs
        priming: Optional[str] = None,  # Optional input used to supply a "system" role message to the LLM
        return_format: Literal['', 'json'] = '',  # Lol, either way, you get a dict (to be removed)
        options: Optional[dict] = None,  # see Ollama docs
        session_uuid: str = None,  # A new one is generated with every request unless one is supplied
        keep_alive: Optional[Union[float, str]] = None  # see Ollama docs
) -> dict

This method could be used for OCR, in some limited capacity. I wonder if you sent embeddings of photos of a person's face using the RAG technique whether it could use those embeddings to recognize that face in another image. Facial recognition?

generate()

generate(
        prompt: str = None,  # The query or instruction to send to the LLM
        temperature: Optional[float] = 0.5,  # see Ollama docs
        seed: Optional[int] = None,  # see Ollama docs
        priming: Optional[str] = None,  # Optional input used to supply a "system" role message to the LLM
        return_format: Literal['', 'json'] = '',  # Lol, either way, you get a dict (to be removed)
        options: Optional[dict] = None,  # see Ollama docs
        session_uuid: str = None,  # A new one is generated with every request unless one is supplied
        keep_alive: Optional[Union[float, str]] = None  # see Ollama docs
) -> dict

This method is generally for one-shot tasks, rather than a round of back and forth as in chat.

list_models()

list_models() -> Mapping[str, Any]

This is a proxy method for the ollama list request, and returns a list of dictionaries representing each model in the ollama server registry.

show_model()

show_model(model: str) -> Mapping[str, Any]

This is a proxy for the ollama show request, and returns a dictionary containing more details about a given model in return for the model's model attribute as it appears in the list.