NeMo Guardrails - DrAlzahraniProjects/csusb_fall2024_cse6550_team4 GitHub Wiki

NeMo Guardrails

  • NeMo Guardrails ensures AI chatbots follow predefined rules, providing safe, controlled, and predictable conversations

This wiki provides a guide for setting up and running Nemo Guardrails to docker container

Table of Contents

  1. Installation
  2. Configuration
  3. Implementation
  4. Usage
  5. Troubleshooting

Installation

  • To install NeMo Guardrails to the dockerfile provided, follow these steps:
  1. Update requirements.txt file with the following

image

Image 1: Added nemo-toolkit[asr,nlp] to requirements

# Update requirements
RUN pip install -r requirements.txt
  1. Check base image and dependencies

image

Image 2: Dockerfile Setup with Python Base

  1. Install NeMo Guardrail dependencies

image

Image 3: Install NeMo and Dependencies in Dockerfile

  1. Modify the Dockerfile: To install NeMo Guardrails, you need to ensure that it's included in the environment setup.

Copy requirements into the docker by following command

image

Image 4: Copying requirements file

  1. Install NeMo Guardrail to docker by using following command
  • NeMo and its guardrails module are installed via pip
  • The repository is cloned and installed in editable mode

image

Image 5: Install NeMo Guardrails

Configuration

    1. Environment path configuration: Ensure the environment created by Mamba (team4_env) is used for all processes

image

Image 6: Environment path setup

    1. Set Up the Guardrails in environment file (environment.yaml)

environment.yml file should be updated to define the dependencies and configuration for the NeMo Guardrails chatbot

image

Image 7: Add Nemo toolkit to environment YAML dependencies

  • Setting up NeMo environment
# `environment.yaml` - setting up NeMo environment
name: team4_env
dependencies:
  - python=3.8
  - pip
  - nemo-guardrails
  • To check if the environment is properly set up
conda env list | grep team4_env

Implementation

image

Image 8: Clone and install Guardrails

  • This line ensures that NeMo Guardrails is implemented in the specified Conda environment (team4_env)

image

Image 9: Start NGINX, Streamlit, Jupyter, and NeMo Guardrails app

  • These steps collectively set up NeMo Guardrails within a conda environment in the docker container

Running NeMo Guardrails

To run NeMo Guardrails for controlling chatbot behavior, execute the following command inside the container

python -m nemo_guardrails.main --environment-file /app/environment.yaml
  • Using NeMo Guardrails in the application In your Python application, load and apply the Guardrails using NeMo
import nemo_guardrails as guardrails

# Load the guardrails configuration for the research paper bot
guard = guardrails.RailManager.from_environment_file('/app/environment.yaml')

# Example: Running the guardrails for a research paper chatbot interaction
user_input = "Can you summarize the findings of this research paper?"

response = guard.run(input_text=user_input)
print(f"Bot Response: {response}")
  • Interacting with NeMo Guardrails

You can now interact with the chatbot by sending POST requests to the Flask API

curl -X POST http://localhost:5004/chat -H "Content-Type: application/json" -d '{"input": "Can you summarize the findings of this research paper?"}'

  • Running the NeMo Guardrails setup within the conda environment
conda activate team4_env
python -m nemo_guardrails.main --environment-file /app/environment.yaml

Usage

  • Performing NeMo Guardrails inference Once NeMo Guardrails is set up, you can run inference on chatbot input data
# Example input related to a research paper
user_input = "Can you summarize the findings of this paper?"

# Run NeMo Guardrails to generate a response based on the defined rules
response = guard.run(input_text=user_input)

print(f"Bot Response: {response}")
  • To run this Dockerfile and use NeMo Guardrails:

Build the docker image:

docker build -t team4-app .
  • Run the Docker container to test if NeMo Guardrails are properly set up in the Docker file
docker run -p 5004:5004 nemo-guardrails-app

Troubleshooting

  1. Make sure all dependencies in requirements.txt or environment.yml are correctly listed and compatible.

  2. NeMo Guardrails configuration not loaded Ensure the configuration file environment.yaml is correctly copied to the container during the build process. Check the path in the Dockerfile:

COPY enivornment.yaml/app/environment.yaml
  1. Verify that the file path in your code matches the location in the docker container
guard = guardrails.RailManager.from_environment_file('/app/environment.yaml')

image

Image 10: Copy NeMo Guardrails files

  1. NeMo Guardrails fails to start
  • The NeMo Guardrails service fails to start when running the docker container
  • Make sure you are correctly activating the environment and running the right startup commands in the Dockerfile

image

Image 11: Start services and applications

  • Common Errors and Checks: Provide specific commands to help diagnose common issue
# Verify environment.yaml is in the correct path
ls /app/environment.yaml
  • Configuration Not Loaded: Provide examples of error messages related to misconfigured paths or missing dependencies and how to resolve them.
  • Dependency Checks: Guide users on how to confirm that each dependency is installed correctly in the Docker container, such as using pip list to check for specific packages:
# Check for installed packages
pip list | grep nemo