Neighbor2Neighbor is a method to train an effective image denoising model with only noisy images. A random neighbor sub-sampler is proposed for the generation of training image pairs. A regularizer is proposed as additional loss for better performance.
In the last few years, image denoising has benefited a lot from the fast
development of neural networks. However, the requirement of large amounts of
noisy-clean image pairs for supervision limits the wide use of these models.
Although there have been a few attempts in training an image denoising model
with only single noisy images, existing self-supervised denoising approaches
suffer from inefficient network training, loss of useful information, or
dependence on noise modeling. In this paper, we present a very simple yet
effective method named Neighbor2Neighbor to train an effective image denoising
model with only noisy images. Firstly, a random neighbor sub-sampler is
proposed for the generation of training image pairs. In detail, input and
target used to train a network are images sub-sampled from the same noisy
image, satisfying the requirement that paired pixels of paired images are
neighbors and have very similar appearance with each other. Secondly, a
denoising network is trained on sub-sampled training pairs generated in the
first stage, with a proposed regularizer as additional loss for better
performance. The proposed Neighbor2Neighbor framework is able to enjoy the
progress of state-of-the-art supervised denoising networks in network
architecture design. Moreover, it avoids heavy dependence on the assumption of
the noise distribution. We explain our approach from a theoretical perspective
and further validate it through extensive experiments, including synthetic
experiments with different noise distributions in sRGB space and real-world
experiments on a denoising benchmark dataset in raw-RGB space.