Nemo Guardrails - DrAlzahraniProjects/csusb_fall2024_cse6550_team2 GitHub Wiki

Overview

NeMo Guardrails is a feature within NVIDIA's NeMo framework that ensures AI-powered systems (e.g., conversational agents, models) follow specific, user-defined constraints or "guardrails" to make their outputs reliable, safe, and relevant. This documentation provides step-by-step instructions for implementing NeMo Guardrails in a Python 3.10 slim Dockerized environment.

Tables of content

  1. Installation
  2. Configuration
  3. Implementation
  4. Usage
  5. Troubleshooting

1. Installation

Step-1: Base Image

Start with the Python 3.9 slim image as the base for efficiency.

FROM python:3.9-slim

Python-L

Figure 1: The base image python:3.11-slim used for the Docker container.

Advantages of Using a Slim Base Image (Summary):

  • Efficiency: Faster image pull and build times, reducing CI/CD pipeline delays and storage costs.
  • Security: Smaller attack surface with fewer packages, making auditing and compliance easier.
  • Deployment Speed: Quicker container startup and resource provisioning, ideal for server less and microservices architectures.
  • Simplicity: Minimal components streamline development and maintenance, improving documentation and onboarding.
  • Consistency: Ensures uniform environments by limiting unnecessary dependencies, reducing deployment issues.

Step-2: Install System Dependencies

Install the essential dependencies:

RUN apt-get update && apt-get install -y wget

ss2-L

Figure 2: Installing system dependencies using apt-get in the Docker container.

apt-get update && apt-get install

  • Apt-get update: updates the system's package list from repositories, ensuring access to the latest versions, reducing dependency conflicts, and improving installation speed. Regular use helps maintain security and keep packages up-to-date.

  • Apt-get install -y wget: -y installs wget without user prompts, using the -y flag. wget is a versatile tool for downloading files via HTTP, HTTPS, and FTP, often used for retrieving datasets or resources during builds. It supports recursive downloads and resuming interrupted transfers.

Step-3: Install NeMo Toolkit and Nemo Guardrails

Install NeMo toolkit and guardrails using pip:

RUN pip install --no-cache-dir nemo_toolkit[all] nemo-guardrails

nemo-inst_L

Figure 3: Installing the NeMo toolkit and its dependencies using pip.

What is NeMo?

  • NVIDIA created the open-source NeMo (NVIDIA Neural Modules) toolbox, which offers parts for creating conversational AI models. It is appropriate for a number of applications, such as text-to-speech synthesis, natural language processing (NLP), and voice recognition, since it makes it simple for users to create, train, and implement models.

Why is NeMo Beneficial?

  • Because of NeMo's modular architecture, users may build sophisticated models without having to start from scratch by combining different parts, such encoders and decoders. Users may save a significant amount of time and computing resources by fine-tuning the pre-trained models included in the toolbox for their particular applications.

2. Configuration

Step-1: Set Environment Variables

Once the required software has been installed, we must set up our environment for best performance. Setting up environment variables that NeMo Guardrails will use during runtime is one of the initial tasks.

ENV GUARDRAILS_CONFIG=/app/guardrails_config.yaml

Importance of Environment Variables

  • NEMO_DATA_PATH: This variable defines the directory where NeMo stores its data. Setting it ensures the toolkit can easily access required datasets and files for model training and evaluation.

  • Other Environment Variables: Beyond NEMO_DATA_PATH, you can configure additional environment variables to further optimize and customize the setup.

Figure 4: Setting environment variables for NeMo, specifying the data path.

Step 2: Create and Add Configuration File

Example guardrails_config.yaml:

guardrails:
- type: "block"
condition: "intent == 'sensitive_topic'"
message: "Sorry, I cannot discuss this topic."
  • Copy the configuration file into the Docker container:
COPY guardrails_config.yaml /app/guardrails_config.yaml

Puypose of .yaml

  • Configuration File: The guardrails_config.yaml file likely contains essential settings or rules for NeMo Guardrails to manage conversational flow and enforce guidelines.
  • Container Access: By copying it into the Docker image, the application running inside the container can easily access and use the configuration during runtime.
  • Portability: Including the configuration in the image ensures consistent behavior across environments (development, staging, production).
  • Simplifies Deployment: Avoids the need to manually transfer the file to the container after deployment.

Figure 5: Creating and adding configuration file

3. Implementation

Step 1: Using NeMo Guardrails in Your Application

Example Python code to apply guardrails:

import nemo_guardrails as ng
  • This imports the nemo_guardrails library, which provides tools to enforce conversational rules, relevance, and safety in AI chatbot interactions.

Step 2: Load guardrails from configuration file

guardrails = ng.Guardrails.from_config("/app/guardrails_config.yaml")
  • The Guardrails.from_config() method loads a set of pre-defined rules or policies from the guardrails_config.yaml file.

Step 3: User query

user_input = "Can you discuss politics?"
  • Represents an example query from a user. In this case, the user asks about discussing politics.

Step 4: Apply guardrails to user input

response = guardrails.apply(user_input)

# Output the response
print(response)
  • The guardrails.apply() method evaluates the user input against the loaded rules in guardrails_config.yaml.
  • It decides whether to allow, restrict, or modify the input or its response based on the configured policies.
  • For example, if discussing politics is restricted, the method might return a controlled response like, "I'm sorry, but I cannot discuss political topics."
  • Prints the processed response generated by the guardrails.apply() method. This response adheres to the configured conversational rules.

4. Usage

Step-1: Defining Guardrails

Add multiple types of guardrails in the configuration file:

guardrails:
- type: "block"
condition: "intent == 'sensitive_topic'"
message: "This topic is not allowed."

- type: "redirect"
condition: "intent == 'offensive_language'"
redirect_to: "polite_response"
  • This configuration file defines two guardrails for chatbot behavior. The block guardrail prevents responses to sensitive topics by displaying a message like "This topic is not allowed." The redirect guardrail handles offensive language by redirecting the conversation to a predefined polite response.

Step 2: Expose Ports

  • Expose necessary ports in the Dockerfile:
EXPOSE 5002

image

Figure 6: Exposing necessary ports to make application accessible externally

  • The EXPOSE 5002 command in the Dockerfile informs Docker that the container will listen on port 5002 for incoming network connections. This is typically used to make the application (like a Streamlit app) accessible externally through the specified port.

Step 3: Running the Application

  • Run the application using Streamlit:
CMD ["sh", "-c", "streamlit run app/main.py --server.port=5002 --server.address=0.0.0.0 --server.baseUrlPath=/team2 & jupyter notebook --ip=0.0.0.0 --port=6002 --no-browser --allow-root --NotebookApp.base_url=/team2/jupyter --NotebookApp.token=''"]

cmd-L

Figure 7: Running the application using CDM.

  • The CMD instruction in the Dockerfile runs a shell command to start both a Streamlit app and a Jupyter notebook within the container. It starts the Streamlit app on port 5002 with a specific base URL path (/team2) and also runs Jupyter notebook on port 6002, with a custom base URL (/team2/jupyter), allowing root access and disabling the browser launch.

5. Troubleshooting

Step 1: Debugging Guardrails

  • Use Docker logs to debug errors in guardrails:
docker logs <container_id>
  • Using docker logs <container_id> allows you to view the output of a running container, helping to identify and debug errors related to guardrails. This command provides logs generated by the application, including any issues with configuration or execution, making it easier to troubleshoot.

logs_L

Figure 8: Checking docker logs for debugging purpose.

Step 2: File Access Issues

  • Ensure the configuration file is accessible in the container and matches the environment variable path.

Step 3: Rebuilding Docker Image

  • Stop and remove the existing Docker container:
docker stop <container_id>
docker rm <container_id>
  • The commands docker stop <container_id> and docker rm <container_id> are used to stop and remove a running Docker container, respectively. Stopping the container halts its execution, while removing it frees up system resources by deleting the container from the Docker host.

stop-rm-L

Figure 9: Removing existing docker image.

  • Rebuild the Docker image:
docker build -t team2_app .
  • The command docker build -t team2_app . is used to rebuild a Docker image from the current directory (denoted by .) and tag it with the name team2_app . This allows you to recreate the image, incorporating any changes made to the Dockerfile or application files.

run_L

Figure 10: Rebuilding new docker image.