Model Converter - Mungert69/GGUFModelBuilder GitHub Wiki

Welcome to the GGUF Model Builder project! This toolkit provides a robust, automated pipeline for downloading, converting, quantizing, uploading, and cataloging large language models in GGUF format, with full support for Hugging Face Hub integration and advanced metadata management.

---

Automates the quantization, chunking, and uploading of GGUF models to Hugging Face Hub.

• Loads quantization configs, authenticates with Hugging Face, and manages .imatrix files.
• Applies quantization with compatibility fallback, splits large files, uploads to Hugging Face, and updates README.md.
• Usage:
  python make_files.py <company/model_name> [--allow-requantize] [--is_moe]

Downloads a model from Hugging Face, converts it to BF16 GGUF format, adds metadata, and cleans up.

• Handles authentication, file download, conversion, and metadata injection.
• Usage:
  python download_convert.py <repo_id>

Uploads all GGUF model files from a specified directory to Hugging Face Hub, then cleans up local files and cache.

• Determines quantization type for each file and uploads to the correct folder.
• Usage:
  python upload-files.py <model_name>

Adds or overrides metadata in a GGUF file by invoking update_gguf.py.

• Supports direct key=type:value overrides and loading from a file.
• Usage:
  python add_metadata_gguf.py <input_gguf_file> [--override key=type:value ...] [--override-file overrides.txt]

Advanced script for copying GGUF files with new or overridden metadata.

• Supports batch metadata updates, removals, and chat template injection.
• Used internally by other scripts.

Monitors the llama.cpp GitHub repo for new model commits, uses a local LLM to detect new models, and updates the Redis-based model catalog.

• Runs in a loop, checking every 10 minutes.
• Usage:
  python auto_build_new_models.py

Core class and script for managing the end-to-end model conversion pipeline.

• Handles downloading, conversion, uploading, disk space checks, and Redis catalog management.
• Used by other scripts for orchestration.

Batch processes a list of models from a JSON file, ensuring each is present in the catalog and running the full conversion pipeline.

• Usage:
  python run_all_from_json.py <path_to_models_json>

Automates building and patching the llama.cpp backend, including patch application and binary copying.

---

• quant_configs.json / quant_rules.json: Configuration files for quantization types and rules.
• update_readme.py: Updates README.md files with quantization and model details.
• tensor_list_builder.py: Processes quantization and tensor information for models.
• redis_utils.py: Utilities for connecting to and managing the Redis model catalog.

---

• Requires Python 3.8+ and packages: huggingface_hub, dotenv, tqdm, llama_cpp, redis, etc.
• Hugging Face API token and Redis credentials must be set in a .env file.
• Some scripts require access to the llama.cpp repository and its binaries.

---

1. Clone the repository and install dependencies.
2. Set up your .env file with Hugging Face and Redis credentials.
3. Use the scripts above to download, convert, quantize, upload, and manage your GGUF models.

---

For detailed usage and advanced options, see the docstrings in each script or visit the individual wiki pages for each tool.