Image inpainting algorithm based on convolutional neural network structure and improved Deep Image Prior

Junshu Wang, Yuxing Han

Abstract


Abstract: In previous studies, researchers believed that the reason for the excellent performance of convolutional neural networks was that they could learn hidden information from special-purpose datasets, and the emphasis was on learning.  Recently, the authors of Deep Image Prior proved that the generator structure itself (using convolutional neural network) could extract image prior information and be used for the image inpainting task.  In this paper, based on Deep Image Prior, four improvements (mix input, network noise, weight decay, and burning mean output) are proposed for preventing overfitting and improving output stability.  Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) in two stepwise comparative experiments showed that our image inpainting algorithm surpassed the original algorithm and state-of-the-art algorithms after adding the proposed improvements in sequence.  In large hole inpainting, the PSNR of our algorithm was 3.23 dB higher than in the original Deep Image Prior.  Then, in a binary Bernoulli inpainting experiment, our algorithm achieved better performance in most classical image inpainting, proving that the algorithm could use the same set of parameters for each image in the task.  In addition, this experiment also illustrated the performance of burning mean output in stabilizing the output and reducing the influence of meaningless noise in the early stage of the iteration on subsequent image inpainting.

Keywords: convolutional neural network; Deep Image Prior; image prior information; image inpainting; overfitting; large hole inpainting; binary Bernoulli inpainting

DOI: 10.33440/j.ijpaa.20200304.135

 

Citation: Wang J S, Han Y H.  An image inpainting algorithm based on convolutional neural network structure and improved Deep Image Prior.  Int J Precis Agric Aviat, 2020; 3(4): 65–73.


Full Text:

PDF

References


Xiang Y, Yu B, Yuan Q, et al. GPU acceleration of CFD algorithm: HSMAC and SIMPLE. Procedia Computer Science, 2017, 108: 1982–1989. doi: 10.1016/j.procs.2017.05.124

Pouyanfar S, Sadiq S, Yan Y, et al. A survey on deep learning: Algorithms, techniques, and applications. ACM Computing Surveys (CSUR), 2018, 51(5): 1–36. doi: 10.1145/3234150

Li Z, Yang W, Peng S, et al. A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects. arXiv preprint arXiv:2004.02806, 2020.

Li S, Song W, Fang L, et al. Deep learning for hyperspectral image classification: An overview. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9): 6690–6709. doi: 10.1109/TGRS.2019. 2907932

Zhao Z Q, Zheng P, Xu S, et al. Object detection with deep learning: A review. IEEE transactions on neural networks and learning systems, 2019, 30(11): 3212–3232. doi: 10.1109/TNNLS.2018.2876865

Liu L, Wang H, Li G, et al. Crowd counting using deep recurrent spatial-aware network. arXiv preprint arXiv:1807.00601, 2018.

Garcia-Garcia A, Orts-Escolano S, Oprea S, et al. A survey on deep learning techniques for image and video semantic segmentation. Applied Soft Computing, 2018, 70: 41–65. doi: 10.1016/j.asoc.2018.05.018

Gong K, Guan J, Kim K, et al. Iterative PET image reconstruction using convolutional neural network representation. IEEE transactions on medical imaging, 2018, 38(3): 675–685. doi: 10.1109/TMI.2018.2869871

Wei B, Sun X, Ren X, et al. Minimal effort back propagation for convolutional neural networks. arXiv preprint arXiv:1709.05804, 2017.

Ulyanov D, Vedaldi A, Lempitsky V. Deep image prior. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 9446–9454.

Kurrant D, Baran A, LoVetri J, et al. Integrating prior information into

microwave tomography Part 1: Impact of detail on image quality. Medical physics, 2017, 44(12): 6461–6481. doi: 10.1002/mp.12585

Ranjan R, Singh A, Rizvi A, et al. Classification of Chest Diseases Using Convolutional Neural Network. Proceedings of First International Conference on Computing, Communications, and Cyber-Security (IC4S 2019). Springer, Singapore, 2020: 235–246.

Zhang C, Bengio S, et al. Understanding deep learning requires rethinking generalization. In Proc. ICLR, 2017

Dubey A K, Jain V. Comparative Study of Convolution Neural Network’s ReLu and Leaky-ReLu Activation Functions. Applications of Computing, Automation and Wireless Systems in Electrical Engineering. Springer, Singapore, 2019: 873–880. doi: 10.1007/ 978-981-13-6772-4_76

Zhang X, Zou Y, Shi W. Dilated convolution neural network with LeakyReLU for environmental sound classification. 2017 22nd International Conference on Digital Signal Processing (DSP). IEEE, 2017: 1–5.

Liu N, Zhai G. Free energy adjusted peak signal to noise ratio (FEA-PSNR) for image quality assessment. Sensing and Imaging, 2017, 18(1): 11.

Hore A, Ziou D. Image quality metrics: PSNR vs. SSIM. 2010 20th international conference on pattern recognition. IEEE, 2010: 2366–2369.

Li L I. The inpainting model by Curvature-Driven Diffusions (CDD). Computer Knowledge and Technology, 2006.

Iizuka S, Simo-Serra E, Ishikawa H. Globally and locally consistent image completion. ACM Transactions on Graphics, 2017, 36(4): 1–14.

Papyan V, Romano Y, Elad M, et al. Convolutional Dictionary Learning via Local Processing. 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017.

Ren J, Xu L, Yan Q, Sun W. Shepard convolutional neural networks. In Proc. NIPS, pages 901–909, 2015.


Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.