Jupyter Notebook - DrAlzahraniProjects/csusb_fall2024_cse6550_team4 GitHub Wiki
Table of Contents
Installation
To install Jupyter in the Dockerfile provided, follow these steps:
1.Make sure to add Jupyter to your requirements.txt file so that it gets installed with the other Python packages:
jupyter
ipykernel
Image : Add Jupyter to requirements.txt
2.Check dependencies Specify the base image for the Docker container in your Dockerfile. Use the lightweight version of Python:
FROM python:3.11-slim
Set the working directory:
WORKDIR /app
3.Jupyter Installation in Docker Verify Docker Installation: Before setting up Jupyter in Docker, make sure Docker is installed by running:
docker --version
If Docker is not installed, refer to the https://github.com/DrAlzahraniProjects/csusb_fall2024_cse6550_team4/wiki/Docker
4.Modify the Dockerfile To install Jupyter, you need to ensure that it's included in the environment setup
5.Jupyter installation To install Jupyter, ensure you have Python 3.x installed on machine. You can install Jupyter using pip. Run the following command in your terminal:
pip install jupyter
Prerequisites:
Ensure you have Python 3.6 or later installed: If you don't have pip installed, you can refer to the official pip installation guide.
Install Jupyter to docker by using following commands
RUN /bin/bash -c "source ~/.bashrc && mamba install -c conda-forge jupyter"
Configuration
Steps to configure Jupyter in the Dockerfile:
1.Environment activation
ENV PATH /root/miniconda3/envs/team4_env/bin:$PATH
Set the environment path for your Conda environment:
The environment path is set to include team4_env
The activation command is appended to .bashrc to activate the environment
RUN echo "source activate team4_env" >> ~/.bashrc
2.Jupyter configuration Create a configuration directory and set configurations:
Create a configuration directory and set configurations:
RUN mkdir -p /root/.jupyter && \
echo "c.NotebookApp.allow_root = True" >> /root/.jupyter/jupyter_notebook_config.py && \
echo "c.NotebookApp.ip = '0.0.0.0'" >> /root/.jupyter/jupyter_notebook_config.py && \
echo "c.NotebookApp.port = 6004" >> /root/.jupyter/jupyter_notebook_config.py && \
echo "c.NotebookApp.open_browser = False" >> /root/.jupyter/jupyter_notebook_config.py && \
echo "c.NotebookApp.token = ''" >> /root/.jupyter/jupyter_notebook_config.py && \
echo "c.NotebookApp.password = ''" >> /root/.jupyter/jupyter_notebook_config.py && \
echo "c.NotebookApp.log_level = 'DEBUG'" >> /root/.jupyter/jupyter_notebook_config.py
Note: The above commands configure Jupyter Notebook to run without authentication and to listen on all network interfaces.
Creates a directory at /root/.jupyter. The -p option ensures that no error is thrown if the directory already exists, and it will also create any necessary parent directories
This appends a configuration line to the jupyter_notebook_config.py file that allows the Jupyter notebook to be run as the root user
Implementation
Jupyter implementation
1.Installing Jupyter and IPython Kernel
To install Jupyter and IPython kernel, add this to Dockerfile:
RUN /bin/bash -c "source ~/.bashrc && mamba install -c conda-forge jupyter ipykernel"
/bin/bash -c: Executes the command within a new bash shell.
source ~/.bashrc: Loads the user's Bash configuration to ensure that the Conda environment is activated properly
mamba install -c conda-forge jupyter ipykernel: Installs both Jupyter Notebook and the IPython kernel (needed to run Python code in Jupyter) from the conda-forge channel. Mamba is used for faster package management
2.Installing the kernel for the environment
RUN /root/miniconda3/envs/team3_env/bin/python -m ipykernel install --name team4_env --display-name "Python (team4_env)"
/root/miniconda3/envs/team4_env/bin/python: Specifies the Python interpreter from the team3_env Conda environment
Expose the necessary ports in Dockerfile:
EXPOSE 6004
Launching Jupyter:
jupyter notebook
Set the default command to run Jupyter:
CMD ["jupyter", "notebook", "--port=6004", "--no-browser", "--ip=0.0.0.0"]
3.Installing Nginx
RUN apt-get update && apt-get install -y nginx
RUN /bin/bash -c "source ~/.bashrc && mamba install -c conda-forge jupyter" \
&& apt-get update && apt-get install -y nginx
This line repeats the installation of Jupyter Notebook using Mamba and updates and installs Nginx again 5.Here’s a snippet to add Jupyter with security and port configuration:
# Install Jupyter
RUN pip install jupyter
#Set up Jupyter notebook with password and port
COPY jupyter_notebook_config.json /root/.jupyter/jupyter_notebook_config.json
EXPOSE 6004
<img width="713" alt="image" src="https://github.com/user-attachments/assets/a5ce64ef-6b81-46fb-871b-5a5635ac39de">
# Default command to run Jupyter Notebook
CMD ["jupyter", "notebook", "--port=6004", "--no-browser", "--ip=0.0.0.0"]
5.Port mapping: To make Jupyter accessible on a different port outside the container, use Docker’s -p flag
docker run -d -p 5004:5004 -p 6004:6004 shaik/team4-app
7.Using Jupyter in Docker: Launching the Jupyter Container: Run the Docker container with Jupyter using:
docker run -d -p 5004:5004 -p 6004:6004 shaik/team3-app
Usage
Usage of Jupyter for the documentation of Swebok chatbot:
Jupyter documentation is a comprehensive guide for the SWEBOK textbook chatbot, providing clear, structured documentation.
The notebook serves as both a development environment and a documentation resource, integrating code, explanatory markdown, and interactive elements to demonstrate the chatbot's workflow.
The documentation is organized into the following sections:
1.Introduction
Purpose: Provides a high-level overview of the chatbot, designed as an educational tool for querying SWEBOK content.
Details: Links to the textbook and highlights the use of Mistral 7B and LangChain libraries for a Retrieval-Augmented Generation (RAG) chatbot.
2.Setup
Environment Requirements: Specifies Python version and dependencies.
Library Setup: Details the tools and libraries used, including LangChain modules and embedding models.
3.Building the Chatbot
Document Loading: Explains how to load SWEBOK documents as the chatbot's corpus using Python scripts.
Embeddings: Outlines the process of generating vector embeddings using Alibaba’s HuggingFace embedding model.
FAISS Vector Store: Details the storage and retrieval system for embedding vectors to enhance query efficiency.
4.Improvement of chatbot with Inference
Helper Functions: Introduces similarity search and response generation methods for efficiently retrieving context-relevant information.
Prompt Engineering: Defines a structured system prompt for the chatbot to guide its behavior and response generation.
5.Testing the Chatbot
Demonstrates the integration of interactive widgets to test user queries directly within the Jupyter Notebook.
Highlights the ability to refine and debug chatbot interactions interactively.
6.Conclusion
Summarizes the steps taken to develop, test, and enhance the chatbot.
Troubleshooting
Common Issues and Troubleshooting Steps
If you encounter issues while using Jupyter, consider the following troubleshooting tips:
1.Jupyter notebook not starting
If Jupyter Notebook fails to start, there are several possible causes. Here are some troubleshooting steps:
Check logs: Review the terminal output when starting the container for any error messages related to Jupyter
Verify CMD: Ensure that the command to start Jupyter Notebook is correctly specified in the Dockerfile
2.Unable to access Jupyter in browser
If you cannot access Jupyter Notebook from your browser, try the following steps:
Check port exposure: Ensure that port 6003 is exposed and mapped correctly in your Docker run command
Firewall settings: Ensure that no firewall rules are blocking access to port 6003
IP address: Verify that you are using the correct IP address (or localhost if running locally)
3.Kernel not starting
Check installed kernels: Ensure that the necessary Python kernel is installed in the Conda environment. You can install it using:
mamba install ipykernel
Recreate kernel: If there are issues with the existing kernel, recreate it:
4.Package installation errors
Requirements file: Ensure the requirements.txt file is present and correctly specified. Any missing packages might lead to runtime errors.
Dependency conflicts: Check for any dependency conflicts in the installed packages. Consider specifying compatible versions in the requirements.txt
5.Configuration issues
Jupyter config file: If Jupyter fails to start with the desired configurations, check the generated configuration file at for any misconfiguration.
/root/.jupyter/jupyter_notebook_config.py