Super resolution benchmark - WolframRhodium/muvsfunc GitHub Wiki
Super-resolution benchmark
Experimental settings
Testing Dataset: 100 private anime images, 1920 x 1080
SR Algorithms: Super-Resolution-Zoo (b575c32), znedi3
Evaluation methods: PSNR, WaDIQaM (FR/NR, dataset="tid", top="weighted")
A RGB high-resolution (HR) image is first downscaled by a factor of 2 using Catmull-Rom (resize.Bicubic(filter_param_a=0, filter_param_b=0.5)), then the low-resolution (LR) image is upscaled by a factor of 2 using a super-resolution algorithm (patch_size=image.size), resulting in a super-resolved (SR) image.
The chroma of the SR image is upscaled by catmull-rom for super-resolution algorithms that operate on luma channel (denote as Y).
The WaDIQaM_FR and PSNR scores are evaluated between the cropped (left=16, right=16, top=12, bottom=12) HR-SR image pair. PSNR is evaluated in luma color space.
The WaDIQaM_NR score is evaluated on the cropped (left=16, right=16, top=12, bottom=12) SR image.
Results (Rank by PSNR)
| Algorithm | PSNR, higher is better (std) | WaDIQaM_FR, lower is better (std) | WaDIQaM_NR, lower is better (std) |
|---|---|---|---|
| Origin | 100.000 (0.000) | 11.915 (1.253) | 37.828 (11.788) |
| EDSR (baseline) | 48.481 (3.230) | 11.934 (1.875) | 38.609 (11.782) |
| MDSR (baseline) | 48.214 (3.241) | 12.050 (2.078) | 38.623 (11.771) |
| CARN | 47.825 (3.139) | 11.871 (2.185) | 38.593 (11.824) |
| MR-ESPCN (Y) | 47.717 (3.506) | 12.392 (2.090) | 38.217 (11.750) |
| MDSR (baseline, jpeg) | 47.704 (3.201) | 12.103 (1.866) | 39.026 (11.759) |
| CARN-Mobile | 47.025 (3.378) | 12.231 (2.222) | 38.706 (11.760) |
| waifu2x (upconv7, noise=-1) | 45.466 (3.397) | 14.745 (3.074) | 40.570 (11.637) |
| IDN (Y) | 45.250 (3.511) | 14.832 (2.479) | 40.242 (11.871) |
| waifu2x (upconv7, noise=0) | 44.301 (3.170) | 14.446 (3.723) | 41.632 (11.104) |
| Catmull-Rom (invks) | 44.110 (4.278) | 16.543 (3.336) | 40.357 (11.149) |
| nnedi3 (Y, nsize=0, nns=3, qual=2) | 43.881 (4.005) | 17.351 (3.366) | 42.119 (11.136) |
| Lanczos4 | 43.253 (4.286) | 18.649 (4.077) | 42.227 (11.135) |
| Lanczos4 (noring) | 43.013 (4.272) | 18.924 (4.251) | 42.485 (11.163) |
| Lanczos3 | 42.905 (4.315) | 19.206 (4.353) | 42.452 (10.946) |
| Lanczos3 (noring) | 42.756 (4.295) | 19.323 (4.427) | 42.633 (11.037) |
| Spline36 | 42.569 (4.330) | 19.843 (4.603) | 42.917 (10.904) |
| Catmull-Rom | 41.382 (4.344) | 21.855 (5.270) | 44.157 (10.887) |
Please note that the results may be biased (model conversion may be wrong, the experiment setting is ideal, etc).