NeMo Guardrails - DrAlzahraniProjects/csusb_fall2024_cse6550_team4 GitHub Wiki
NeMo Guardrails
- NeMo Guardrails ensures AI chatbots follow predefined rules, providing safe, controlled, and predictable conversations
This wiki provides a guide for setting up and running Nemo Guardrails to docker container
Table of Contents
Installation
- To install NeMo Guardrails to the dockerfile provided, follow these steps:
- Update
requirements.txt
file with the following
Image 1: Added nemo-toolkit[asr,nlp]
to requirements
# Update requirements
RUN pip install -r requirements.txt
- Check base image and dependencies
Image 2: Dockerfile Setup with Python Base
- Install NeMo Guardrail dependencies
Image 3: Install NeMo and Dependencies in Dockerfile
- Modify the Dockerfile: To install NeMo Guardrails, you need to ensure that it's included in the environment setup.
Copy requirements into the docker by following command
Image 4: Copying requirements file
- Install NeMo Guardrail to docker by using following command
- NeMo and its guardrails module are installed via
pip
- The repository is cloned and installed in editable mode
Image 5: Install NeMo Guardrails
Configuration
-
- Environment path configuration: Ensure the environment created by Mamba (team4_env) is used for all processes
Image 6: Environment path setup
-
- Set Up the Guardrails in environment file (environment.yaml)
environment.yml
file should be updated to define the dependencies and configuration for the NeMo Guardrails chatbot
Image 7: Add Nemo toolkit to environment YAML dependencies
- Setting up NeMo environment
# `environment.yaml` - setting up NeMo environment
name: team4_env
dependencies:
- python=3.8
- pip
- nemo-guardrails
- To check if the environment is properly set up
conda env list | grep team4_env
Implementation
Image 8: Clone and install Guardrails
- This line ensures that NeMo Guardrails is implemented in the specified Conda environment
(team4_env)
Image 9: Start NGINX, Streamlit, Jupyter, and NeMo Guardrails app
- These steps collectively set up NeMo Guardrails within a conda environment in the docker container
Running NeMo Guardrails
To run NeMo Guardrails for controlling chatbot behavior, execute the following command inside the container
python -m nemo_guardrails.main --environment-file /app/environment.yaml
- Using NeMo Guardrails in the application In your Python application, load and apply the Guardrails using NeMo
import nemo_guardrails as guardrails
# Load the guardrails configuration for the research paper bot
guard = guardrails.RailManager.from_environment_file('/app/environment.yaml')
# Example: Running the guardrails for a research paper chatbot interaction
user_input = "Can you summarize the findings of this research paper?"
response = guard.run(input_text=user_input)
print(f"Bot Response: {response}")
- Interacting with NeMo Guardrails
You can now interact with the chatbot by sending POST requests to the Flask API
curl -X POST http://localhost:5004/chat -H "Content-Type: application/json" -d '{"input": "Can you summarize the findings of this research paper?"}'
- Running the NeMo Guardrails setup within the conda environment
conda activate team4_env
python -m nemo_guardrails.main --environment-file /app/environment.yaml
Usage
- Performing NeMo Guardrails inference Once NeMo Guardrails is set up, you can run inference on chatbot input data
# Example input related to a research paper
user_input = "Can you summarize the findings of this paper?"
# Run NeMo Guardrails to generate a response based on the defined rules
response = guard.run(input_text=user_input)
print(f"Bot Response: {response}")
- To run this Dockerfile and use NeMo Guardrails:
Build the docker image:
docker build -t team4-app .
- Run the Docker container to test if NeMo Guardrails are properly set up in the Docker file
docker run -p 5004:5004 nemo-guardrails-app
Troubleshooting
-
Make sure all dependencies in
requirements.txt
orenvironment.yml
are correctly listed and compatible. -
NeMo Guardrails configuration not loaded Ensure the configuration file
environment.yaml
is correctly copied to the container during the build process. Check the path in the Dockerfile:
COPY enivornment.yaml/app/environment.yaml
- Verify that the file path in your code matches the location in the docker container
guard = guardrails.RailManager.from_environment_file('/app/environment.yaml')
Image 10: Copy NeMo Guardrails files
- NeMo Guardrails fails to start
- The NeMo Guardrails service fails to start when running the docker container
- Make sure you are correctly activating the environment and running the right startup commands in the Dockerfile
Image 11: Start services and applications
- Common Errors and Checks: Provide specific commands to help diagnose common issue
# Verify environment.yaml is in the correct path
ls /app/environment.yaml
- Configuration Not Loaded: Provide examples of error messages related to misconfigured paths or missing dependencies and how to resolve them.
- Dependency Checks: Guide users on how to confirm that each dependency is installed correctly in the Docker container, such as using
pip
list to check for specific packages:
# Check for installed packages
pip list | grep nemo