LoFormer: Local Frequency Transformer for Image Deblurring

🌈 Abstract

The paper introduces a novel approach called Local Frequency Transformer (LoFormer) for image deblurring. LoFormer aims to effectively model long-range dependencies without sacrificing fine-grained details by incorporating a Local Channel-wise Self-Attention (Freq-LC) in the frequency domain. Additionally, it introduces an MLP Gating mechanism (MGate) to filter out irrelevant features while enhancing global learning capabilities. The experiments demonstrate that LoFormer significantly improves performance on the image deblurring task.

🙋 Q&A

[01] Introduction

1. What are the key challenges in image deblurring that the paper aims to address?

• The paper notes that existing methods either adopt localized self-attention (SA) or employ coarse-grained global SA methods, both of which have drawbacks such as compromising global modeling or lacking fine-grained correlation.
• To address this issue, the paper proposes LoFormer, which aims to model long-range dependency without compromising fine-grained details.

2. What are the main contributions of the paper?

• The paper proposes Freq-LC to model long-range dependency without compromising fine-grained details, and introduces MGate to enhance global information learning.
• The paper proves that Spa-GC (global channel-wise SA in the spatial domain) is equivalent to Freq-GC, and verifies that Freq-LC has stronger capability in exploring divergent properties in the frequency domain than Spa-GC.
• Extensive experiments show that LoFormer achieves state-of-the-art results on image deblurring tasks.

[02] Method

1. What is the overall architecture of LoFormer?

• LoFormer employs a UNet architecture as the backbone, with the basic building block being the Local Frequency Transformer (LoFT) block.
• The LoFT block consists of a Local Frequency Network (LoFN) and a Feed-Forward Network (FFN).

2. What are the key components of the LoFN module?

• DCT-LN: Applies layer normalization after the discrete cosine transform (DCT) to ensure an equitable distribution of frequency tokens.
• Freq-LC: Performs local channel-wise self-attention in the frequency domain to capture cross-covariance within low- and high-frequency local windows.
• MGate: Applies an intra-window MLP gating mechanism in the frequency domain to filter out irrelevant features and enhance global learning capabilities.

3. How does Freq-LC differ from other self-attention methods?

• The paper shows that Spa-GC (global channel-wise SA in the spatial domain) is equivalent to Freq-GC, and that Freq-LC has stronger capability in exploring divergent properties in the frequency domain than Spa-GC.

[03] Experiments

1. What are the main findings from the experimental results?

• LoFormer outperforms other CNN-based, Transformer-based, and MLP-based methods on the GoPro, HIDE, RealBlur-R, RealBlur-J, and REDS datasets.
• LoFormer-L achieves a PSNR of 34.09 dB on the GoPro dataset with 126G FLOPs, significantly improving upon the performance of Restormer.
• The ablation studies demonstrate the effectiveness of the key components of the LoFT block, such as DCT-LN, Freq-LC, and MGate.

2. How does Freq-LC perform compared to Spa-GC in capturing high-frequency details?

• The paper shows that Freq-LC is more capable of capturing high-frequency details compared to Spa-GC, which tends to focus more on low-frequency components.
• Freq-LC performs better than Spa-GC when involving more high-frequency parts in the PSNR calculation, indicating its advantage in restoring fine details.

</output_format>