ML - CornellDataScience/MathSearch GitHub Wiki

ml-pipeline-sp24

Features

  • Object Detection with YOLOv8 model for accurate detection and cropping of equations from document images: Enhanced efficiency and precision in image processing, ensuring all equations are correctly identified and extracted.
  • Image to LaTeX conversion with integration of MathPix OCR: Streamlined extraction and conversion process, ensuring precise translation of equations into text.
  • Similarity Calculation using a fuzzy matching approach leveraging Python-Levenshtein: Enhanced text comparison and matching accuracy.

Important Files

  • yolov8m.pt: Final weights for fine-tuned YoloV8 model.
  • eqn-detect.yaml: YAML file with configuration for fine-tuning YoloV8 model (assumes binary labels, big-eqn i.e. LaTeX math display mode, and inline-eqn).
  • yolov8_training.py: Python script to load pre-trained model and fine-tune using YAML file configuration.
  • cropped_data: Contains cropped equations detected by fine-tuned YoloV8 model.
  • runs/detect/train: Data generated by YoloV8 during fine-tuning.
  • datasets/eqn-images-dataset/labels: Custom dataset of our hand-labelled equations (contains bounding box coordinates in YoloV8 formatting). Can be directly used for fine-tuning YoloV8 model.