PP OCR LFM CPU check - 100-hours-a-week/KTB4-3rd-wiki GitHub Wiki

PP-OCRΒ·LFM CPU μ‹€μΈ‘ λ³΄κ³ μ„œ

평가 κΈ°μ€€

  • μž₯치: Apple Silicon CPU, μ„œλ²„ 4 vCPU 쑰건을 κ°€μ •ν•œ μ΅œλŒ€ 4μŠ€λ ˆλ“œ μ œν•œ
  • μ œν•œ: 이미지별 μ΅œλŒ€ 360초
  • μ–‘μ„± 정확도 : μš”κΈˆμ΄ μžˆλŠ” 이미지(91μž₯)μ—μ„œ 정닡을 맞힌 λΉ„μœ¨
  • μŒμ„± κ±°λΆ€μœ¨ : μš”κΈˆμ΄ μ—†λŠ” 이미지(51μž₯)μ—μ„œ κΈˆμ•‘μ„ μ•ˆ λ§ν•œ λΉ„μœ¨ -> 정직함 νŒλ‹¨κ³Ό νŠœλ‹ μ—¬λΆ€ νŒλ‹¨
  • JSON μ€€μˆ˜μœ¨ : json ν˜•μ‹μ΄ λ§žλŠ”μ§€λ§Œ νŒλ‹¨
  • 엄격 JSON 정확도 : json ν˜•μ‹λ„ 맞고, 정닡을 맞힌 λΉ„μœ¨

단일 λͺ¨λΈ μ‹€μΈ‘

λͺ¨λΈ μ—­ν•  μ™„λ£Œ 의미 정확도 엄격 JSON 정확도 JSON μ€€μˆ˜μœ¨ μ–‘μ„± 정확도 μŒμ„± κ±°λΆ€μœ¨ νƒ€μž„μ•„μ›ƒ 평균 p95
PP-OCRv6 medium OCR 151/151 22.54% 22.54% N/A 13.19% 39.22% 0 2.774초 3.124초
PP-OCRv6 small OCR 151/151 23.24% 23.24% N/A 15.38% 37.25% 0 0.285초 0.627초
LFM2.5-VL-450M native VLM 151/151 71.83% 36.62% 48.34% 80.22% 56.86% 0 2.424초 3.696초
LFM2.5-VL-450M Q8_0 VLM 151/151 71.13% 40.14% 54.30% 79.12% 56.86% 0 2.449초 11.950초
LFM2.5-VL-3B Q4_K_M VLM 151/151 78.87% 73.24% 92.72% 93.41% 52.94% 0 17.028초 78.182초
LFM2.5-VL-1.6B native VLM 151/151 72.54% 57.75% 80.79% 90.11% 41.18% 0 6.590초 10.620초

PP-OCR β†’ LFM μ‘°ν•© μ‹€μΈ‘

8개 μ‘°ν•© λͺ¨λ‘ μ‹ κ·œ 151μž₯κ³Ό κΈ°μ‘΄ 10μž₯을 μ™„λ£Œν–ˆλ‹€.

OCR β†’ LFM μ™„λ£Œ 의미 정확도 엄격 JSON 정확도 JSON μ€€μˆ˜μœ¨ OCR 후보 μž¬ν˜„μœ¨ μ–‘μ„± 정확도 μŒμ„± κ±°λΆ€μœ¨ νƒ€μž„μ•„μ›ƒ 평균 p95 μ΅œλŒ€ CPU μ½”μ–΄ 평균
PP-OCRv6 medium β†’ LFM2.5-VL-450M native 151/151 42.96% 28.87% 49.01% 94.51% 53.85% 23.53% 0 5.411초 9.885초 179.561초 1.73
PP-OCRv6 medium β†’ LFM2.5-VL-450M Q8_0 151/151 52.11% 34.51% 56.95% 94.51% 67.03% 25.49% 0 4.638초 14.805초 137.251초 3.43
PP-OCRv6 medium β†’ LFM2.5-VL-3B Q4_K_M 151/151 72.54% 69.01% 93.38% 94.51% 93.41% 35.29% 0 20.896초 82.955초 147.038초 3.86
PP-OCRv6 medium β†’ LFM2.5-VL-1.6B native 151/151 71.83% 69.01% 89.40% 94.51% 93.41% 33.33% 0 8.053초 13.521초 172.842초 1.96
PP-OCRv6 small β†’ LFM2.5-VL-450M native 151/151 44.37% 28.17% 45.03% 91.21% 54.95% 25.49% 0 2.886초 4.705초 6.583초 1.74
PP-OCRv6 small β†’ LFM2.5-VL-450M Q8_0 151/151 53.52% 34.51% 56.29% 91.21% 65.93% 31.37% 0 2.848초 12.599초 15.568초 3.89
PP-OCRv6 small β†’ LFM2.5-VL-3B Q4_K_M 151/151 69.72% 66.90% 95.36% 91.21% 90.11% 33.33% 0 19.270초 81.548초 111.689초 3.97
PP-OCRv6 small β†’ LFM2.5-VL-1.6B native 151/151 69.01% 66.20% 89.40% 91.21% 87.91% 35.29% 0 6.497초 12.188초 14.460초 1.95

단계별 지연과 단일 VLM λŒ€λΉ„ λ³€ν™”

OCR β†’ LFM OCR 평균 LFM 평균 총 평균 단일 VLM λŒ€λΉ„ 정확도 λ³€ν™”
PP-OCRv6 medium β†’ LFM2.5-VL-450M native 2.348초 3.061초 5.411초 -28.87%p
PP-OCRv6 medium β†’ LFM2.5-VL-450M Q8_0 2.028초 2.607초 4.638초 -19.01%p
PP-OCRv6 medium β†’ LFM2.5-VL-3B Q4_K_M 2.081초 18.811초 20.896초 -6.34%p
PP-OCRv6 medium β†’ LFM2.5-VL-1.6B native 2.057초 5.993초 8.053초 -0.70%p
PP-OCRv6 small β†’ LFM2.5-VL-450M native 0.228초 2.657초 2.886초 -27.46%p
PP-OCRv6 small β†’ LFM2.5-VL-450M Q8_0 0.268초 2.578초 2.848초 -17.61%p
PP-OCRv6 small β†’ LFM2.5-VL-3B Q4_K_M 0.271초 18.998초 19.270초 -9.15%p
PP-OCRv6 small β†’ LFM2.5-VL-1.6B native 0.252초 6.244초 6.497초 -3.52%p

κΈ°μ‘΄ 10μž₯으둜 μ •ν™•ν•œ 정보인지 검증

OCR β†’ LFM μ™„λ£Œ 정확도 νƒ€μž„μ•„μ›ƒ 평균 p95
PP-OCRv6 medium β†’ LFM2.5-VL-450M native 10/10 50.00% 0 3.724초 6.235초
PP-OCRv6 medium β†’ LFM2.5-VL-450M Q8_0 10/10 90.00% 0 3.807초 10.439초
PP-OCRv6 medium β†’ LFM2.5-VL-3B Q4_K_M 10/10 100.00% 0 21.766초 64.128초
PP-OCRv6 medium β†’ LFM2.5-VL-1.6B native 10/10 90.00% 0 7.227초 10.373초
PP-OCRv6 small β†’ LFM2.5-VL-450M native 10/10 50.00% 0 2.875초 4.119초
PP-OCRv6 small β†’ LFM2.5-VL-450M Q8_0 10/10 60.00% 0 3.188초 9.713초
PP-OCRv6 small β†’ LFM2.5-VL-3B Q4_K_M 10/10 100.00% 0 25.272초 107.915초
PP-OCRv6 small β†’ LFM2.5-VL-1.6B native 10/10 100.00% 0 6.935초 10.282초