Step by Step : LLaMA - SoojungHong/Riding_LLaMA-and-Fine-Tuning GitHub Wiki

Welcome to the Riding_LLaMA wiki!

Instruction to set up LLaMA

https://github.com/facebookresearch/llama

After download the LLaMA files (e.g. parameters), run the script from the github above ./download.sh

The URL in the email is necessary while running the download.sh

How to set up environment

In the instruction,In a conda env with PyTorch / CUDA available clone and download this repository.

A conda environment is a directory that contains a specific collection of conda packages that you have installed.

1. How to set up conda?

The following URL contains how to set up conda .

https://saturncloud.io/blog/how-to-create-a-conda-environment-with-a-specific-python-version/

In summary,

conda create --name myenv python=3.7

conda activate myenv

python --version

my conda venv name in my ubuntu machine (big box) is conda_env

2. Installation of PyTorch and CUDA in conda virtual environment

conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

The following URL (https://www.cherryservers.com/blog/how-to-install-pytorch-ubuntu) has more explanation.

During this step, python version is need to be upgraded, do following command.

conda install python=3.10.6

The following URL (https://bobbyhadz.com/blog/syntax-error-future-feature-annotations-is-not-defined) has more explanation.

How to infer LLaMA

In the instruction, following command should be executed.

torchrun --nproc_per_node 1 example_chat_completion.py \
    --ckpt_dir llama-2-7b-chat/ \
    --tokenizer_path tokenizer.model \
    --max_seq_len 512 --max_batch_size 6

The LLaMA directories are following. (llama-2-7b, llama-2-7b-chat, tokenizer.model) The command should be ran like following.

(conda_env) soojung@soojung-X299-UD4:~/llama$ torchrun --nproc_per_node 1 example_chat_completion.py --ckpt_dir /home/soojung/llama-2-7b-chat/ --tokenizer_path /home/soojung/tokenizer.model --max_seq_len 512 --max_batch_size 6

Use this script to convert parameters to HuggingFace and run the test question

reference : https://ai.meta.com/blog/5-steps-to-getting-started-with-llama-2/

⚠️ **GitHub.com Fallback** ⚠️