Framework referred to A Unified Conditional Framework for Diffusion-based Image Restoration(Zhang et al., 2023)

• Related Work
  • Generative Image Restoration
  • Dynamic Networks in Image Restoration
• Method
  • Overview of Conditional Framework
  • Diffusion Model Block
  • High-resolution Image Inference
• Results
• References

• We propose a unified conditional framework for diffusion-based image restoration tasks. It leverages a UNet to predict the initial guidance and enable integrating the multi-sources conditional information to every block to better guide the generative model.

• To effectively incorporate conditional information into diffusion models, we design a basic module and an Adaptive Kernel Guidance Block (AKGB). It combines the spatial guidance and auxiliary scalar information to adaptively fuse the dynamic kernels in each diffusion model block.

• A simple yet effective inter-step patch-splitting strategy is proposed for handling high-resolution images in low-level vision tasks. This practical strategy enables diffusion models to generate consistent high-resolution images without grid artifacts.

• Through extensive experiments on extreme low-light denoising, image deblurring, and JPEG restoration tasks, we demonstrate that our method not only achieves a significantly higher perceptual quality than strong regression baselines and recent diffusion-based models but also show good generalization to various restoration tasks.

Zhang, Y., Shi, X., Li, D., Wang, X., Wang, J., & Li, H. (2023). A Unified Conditional Framework for Diffusion-based Image Restoration. Neural Information Processing Systems, 36, 49703–49714. https://proceedings.neurips.cc/paper_files/paper/2023/hash/9bf0810a4a1597a36d27ceea58667d92-Abstract-Conference.html

Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. Learning enriched features for real image restoration and enhancement. In ECCV, 2020.

Liangyu Chen, Xiaojie Chu, Xiangyu Zhang, and Jian Sun. Simple baselines for image restoration. arXiv preprint arXiv:2204.04676, 2022.

Sung-Jin Cho, Seo-Won Ji, Jun-Pyo Hong, Seung-Won Jung, and Sung-Jea Ko. Rethinking coarse-to-fine approach in single image deblurring. In ICCV, 2021.

Dasong Li, Yi Zhang, Ka Chun Cheung, Xiaogang Wang, Hongwei Qin, and Hongsheng Li. Learning degradation representations for image deblurring. In ECCV, 2022.

Dasong Li, Xiaoyu Shi, Yi Zhang, Xiaogang Wang, Hongwei Qin, and Hongsheng Li. No attention is needed: Grouped spatial-temporal shift for simple and efficient video restorers. CoRR, abs/2206.10810, 2022.

Abdelrahman Abdelhamed, Radu Timofte, and Michael S Brown. NTIRE 2019 challenge on real image denoising: Methods and results. In CVPR Workshops, 2019.

Jianrui Cai, Shuhang Gu, Radu Timofte, and Lei Zhang. NTIRE 2019 challenge on real image superresolution: Methods and results. In CVPR Workshops, 2019.

Seungjun Nah, Sanghyun Son, Suyoung Lee, Radu Timofte, and Kyoung Mu Lee. Ntire 2021 challenge on image deblurring. In CVPR Workshops, 2021.

Yi Zhang, Dasong Li, Xiaoyu Shi, Dailan He, Kangning Song, Xiaogang Wang, Hongwei Qin, and Hongsheng Li. Kbnet: Kernel basis network for image restoration. CoRR, abs/2303.02881, 2023.

Dasong Li, Yi Zhang, Ka Lung Law, Xiaogang Wang, Hongwei Qin, and Hongsheng Li. Efficient burst raw denoising with variance stabilization and multi-frequency denoising network. IJCV., 2022.

Xu Jia, Bert De Brabandere, Tinne Tuytelaars, and Luc Van Gool. Dynamic filter networks. In NIPS, pages 667–675, 2016.

Steve Bako, Thijs Vogels, Brian McWilliams, Mark Meyer, Jan Novák, Alex Harvill, Pradeep Sen, Tony DeRose, and Fabrice Rousselle. Kernel-predicting convolutional networks for denoising monte carlo renderings. ACM Trans. Graph., 36(4):97:1–97:14, 2017.

Ben Mildenhall, Jonathan T Barron, Jiawen Chen, Dillon Sharlet, Ren Ng, and Robert Carroll. Burst denoising with kernel prediction networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

Zhihao Xia, Federico Perazzi, Michaël Gharbi, Kalyan Sunkavalli, and Ayan Chakrabarti. Basis prediction networks for effective burst denoising with large kernels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11844–11853, 2020.

Yifan Jiang, Bartlomiej Wronski, Ben Mildenhall, Jonathan T. Barron, Zhangyang Wang, and Tianfan Xue. Fast and high quality image denoising via malleable convolution. In ECCV, 2022.