류재훈

update result

1 +# Superresolution
2 +___
3 +## Introduction
4 +We propose a super resolution method based on Convolution Neural Network (CNN), which is similar to or superior to existing methods, while dramatically lowering the number of network parameters and computation cost compared to the conventional CNN method. Our proposed network has a much more efficient network structure than deconvolution operations or other methods of enhancing the instantaneous resolution of inputs by using sub-pixel cnn techniques to increase the resolution inside the network. In addition, by applying Modified Depth-Wise Separable CNN, which has been introduced in MobileNet, the number of parameters can be kept smaller while having a more complicated network structure in comparison with SRCNN. We have applied Depthwise Separable as described above to reduce the number of parameters. Finally, our network converts Floating point to Fixed Point operation, which makes it more efficient and faster. Ultimately, our network has a lightweight, simple structure that can be extended to hardware implementations such as FPGAs or ASICs, which makes it possible to extend to these areas.
5 +
6 +## Requirements
7 +
8 +- PyTorch
9 +- skimage
10 +- python3
11 +
12 +## Datasets
13 +
14 +### Train、Val Dataset
15 +- download dataset from this [link](https://drive.google.com/open?id=1-5eKvxDnIqrXE3ABSk6RcPwMrgsKeCsw) and put it in this project
16 +
17 +### Test Image Dataset
18 +The test image dataset are sampled from
19 +| **Set 5** | [Bevilacqua et al. BMVC 2012](http://people.rennes.inria.fr/Aline.Roumy/results/SR_BMVC12.html)
20 +| **Set 14** | [Zeyde et al. LNCS 2010](https://sites.google.com/site/romanzeyde/research-interests)
21 +| **BSD 100** | [Martin et al. ICCV 2001](https://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/)
22 +
23 +## [Proposal](images/딥러닝을 이용한 영상 화질 개선 알고리즘의 최적화.pdf)
24 +
25 +## TODO
26 +* Fix PSNR BUG
27 +
28 +## DONE
29 +* compare other model
30 +* apply Video
31 +* [Tensorboard](https://github.com/yunjey/pytorch-tutorial/tree/master/tutorials/04-utils/tensorboard)
32 +* make Inference script
33 +* addition metric ssim
34 +* implement psnr to skimage
35 +* inference 3 dataset (set5,set14,BSD100)
36 +* drawing info result
37 +* drawing background
38 +* Lossless compression bmp dataset
39 +* valid dataset
40 +
41 +> Image Results
42 +
43 +The left is high resolution image, the middle is Bicubic-interpolation image, and
44 +the right is super resolution image(output of the ESPCN).
45 +
46 + ** Upscale Factor is 2**
47 +
48 +- Set5
49 +<table>
50 + <tr>
51 + <td>
52 + <img src="images/Set5_img_001_SRF_2_HR.png"/>
53 + </td>
54 + <td>
55 + <img src="images/Set5_img_001_SRF_2_bicubic.png"/>
56 + </td>
57 + <td>
58 + <img src="images/Set5_img_001_SRF_2_superResolution.png"/>
59 + </td>
60 + </tr>
61 +</table>
62 +<table>
63 + <tr>
64 + <td>
65 + <img src="images/Set5_img_002_SRF_2_HR.png"/>
66 + </td>
67 + <td>
68 + <img src="images/Set5_img_002_SRF_2_bicubic.png"/>
69 + </td>
70 + <td>
71 + <img src="images/Set5_img_002_SRF_2_superResolution.png"/>
72 + </td>
73 + </tr>
74 +</table>
75 +
76 +<table>
77 + <tr>
78 + <td>
79 + <img src="images/Set5_img_003_SRF_2_HR.png"/>
80 + </td>
81 + <td>
82 + <img src="images/Set5_img_003_SRF_2_bicubic.png"/>
83 + </td>
84 + <td>
85 + <img src="images/Set5_img_003_SRF_2_superResolution.png"/>
86 + </td>
87 + </tr>
88 +</table>
89 +
90 +<table>
91 + <tr>
92 + <td>
93 + <img src="images/Set5_img_004_SRF_2_HR.png"/>
94 + </td>
95 + <td>
96 + <img src="images/Set5_img_004_SRF_2_bicubic.png"/>
97 + </td>
98 + <td>
99 + <img src="images/Set5_img_004_SRF_2_superResolution.png"/>
100 + </td>
101 + </tr>
102 +</table>
103 +<table>
104 + <tr>
105 + <td>
106 + <img src="images/Set5_img_005_SRF_2_HR.png"/>
107 + </td>
108 + <td>
109 + <img src="images/Set5_img_005_SRF_2_bicubic.png"/>
110 + </td>
111 + <td>
112 + <img src="images/Set5_img_005_SRF_2_superResolution.png"/>
113 + </td>
114 + </tr>
115 +</table>
116 +
117 +## Reference
118 +- Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias
119 +Weyand, Marco Andreetto, Hartwig Adam. MobileNets: Efficient Convolutional Neural
120 +Networks for Mobile Vision Applications. arXiv:1704.04861, 2017
121 +- Chao Dong, Chen Change Loy, Member, IEEE, Kaiming He, Member, IEEE, and Xiaoou
122 +Tang, Fellow, IEEE. Image Super-Resolution Using Deep Convolutional Networks. IEEE
123 +TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 38, NO.
124 +2, FEBRUARY 2016
125 +- Jiwon Kim, Jung Kwon Lee, Kyoung Mu Lee. Accurate Image Super-Resolution Using
126 +Very Deep Convolutional Networks. Computer Vision and Pattern Recognition (CVPR), 2016
127 +IEEE Conference on
128 +- Jiwon Kim, Jung Kwon Lee, Kyoung Mu Lee. deeply recursive convolutional network for
129 +image super resolution. 2016 IEEE Conference on Computer Vision and Pattern Recognition
130 +(CVPR)
131 +- Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P. Aitken, Rob
132 +Bishop, Daniel Rueckert, Zehan Wang. Real-Time Single Image and Video Super-Resolution
133 +Using an Efficient Sub-Pixel Convolutional Neural Network. 2016 IEEE Conference on
134 +Computer Vision and Pattern Recognition (CVPR)
135 +- Sajid Anwar, Kyuyeon Hwang, Wonyong Sung. Structured Pruning of Deep Convolutional
136 +Neural Networks. ​ ACM Journal on Emerging Technologies in ComputingSystems.Article No. 32
137 +- Darryl D. Lin, Sachin S. Talathi, V. Sreekanth Annapureddy.Fixed point
138 +quantization of deep convolutional networks. ICML'16 Proceedings of the 33rd
139 +International Conference on International Conference on Machine Learning -
140 +Volume 48 Pages 2849-2858