Decomposed Neural Architecture Search for image denoising

In practical applications of deep learning, as the demand for the modeling capability increases, the network size may need to be massively enlarged in response. This may form a significant challenge in practice, especially when facing the dilemma of limited computational resources, making model compression indispensable. It can be time-consuming and interminable to obtain an appropriate network architecture through manual compression. In this paper, we propose an automated method for searching decomposed network architectures, named DNAS (standing for Decomposed Neural Architecture Search). It integrates both tasks of neural architecture search and tensor decomposition based model compression within a unified framework. The method is able to efficiently find a compact network with high performance for image denoising, with respect to memory and runtime. Particularly, using one single V100 GPU, it only takes about 1.5 h to obtain a denoising network on the BSD500 dataset. Experimental results demonstrate that compared with models developed using existing methods, DNAS consumes significantly less inference time and employs much fewer trainable parameters, outperforming existing approaches on both synthetic and real-world denoising datasets.