Showing
22 changed files
with
140 additions
and
0 deletions
README.md
0 → 100644
1 | +# Superresolution | ||
2 | +___ | ||
3 | +## Introduction | ||
4 | +We propose a super resolution method based on Convolution Neural Network (CNN), which is similar to or superior to existing methods, while dramatically lowering the number of network parameters and computation cost compared to the conventional CNN method. Our proposed network has a much more efficient network structure than deconvolution operations or other methods of enhancing the instantaneous resolution of inputs by using sub-pixel cnn techniques to increase the resolution inside the network. In addition, by applying Modified Depth-Wise Separable CNN, which has been introduced in MobileNet, the number of parameters can be kept smaller while having a more complicated network structure in comparison with SRCNN. We have applied Depthwise Separable as described above to reduce the number of parameters. Finally, our network converts Floating point to Fixed Point operation, which makes it more efficient and faster. Ultimately, our network has a lightweight, simple structure that can be extended to hardware implementations such as FPGAs or ASICs, which makes it possible to extend to these areas. | ||
5 | + | ||
6 | +## Requirements | ||
7 | + | ||
8 | +- PyTorch | ||
9 | +- skimage | ||
10 | +- python3 | ||
11 | + | ||
12 | +## Datasets | ||
13 | + | ||
14 | +### Train、Val Dataset | ||
15 | +- download dataset from this [link](https://drive.google.com/open?id=1-5eKvxDnIqrXE3ABSk6RcPwMrgsKeCsw) and put it in this project | ||
16 | + | ||
17 | +### Test Image Dataset | ||
18 | +The test image dataset are sampled from | ||
19 | +| **Set 5** | [Bevilacqua et al. BMVC 2012](http://people.rennes.inria.fr/Aline.Roumy/results/SR_BMVC12.html) | ||
20 | +| **Set 14** | [Zeyde et al. LNCS 2010](https://sites.google.com/site/romanzeyde/research-interests) | ||
21 | +| **BSD 100** | [Martin et al. ICCV 2001](https://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/) | ||
22 | + | ||
23 | +## [Proposal](images/딥러닝을 이용한 영상 화질 개선 알고리즘의 최적화.pdf) | ||
24 | + | ||
25 | +## TODO | ||
26 | +* Fix PSNR BUG | ||
27 | + | ||
28 | +## DONE | ||
29 | +* compare other model | ||
30 | +* apply Video | ||
31 | +* [Tensorboard](https://github.com/yunjey/pytorch-tutorial/tree/master/tutorials/04-utils/tensorboard) | ||
32 | +* make Inference script | ||
33 | +* addition metric ssim | ||
34 | +* implement psnr to skimage | ||
35 | +* inference 3 dataset (set5,set14,BSD100) | ||
36 | +* drawing info result | ||
37 | +* drawing background | ||
38 | +* Lossless compression bmp dataset | ||
39 | +* valid dataset | ||
40 | + | ||
41 | +> Image Results | ||
42 | + | ||
43 | +The left is high resolution image, the middle is Bicubic-interpolation image, and | ||
44 | +the right is super resolution image(output of the ESPCN). | ||
45 | + | ||
46 | + ** Upscale Factor is 2** | ||
47 | + | ||
48 | +- Set5 | ||
49 | +<table> | ||
50 | + <tr> | ||
51 | + <td> | ||
52 | + <img src="images/Set5_img_001_SRF_2_HR.png"/> | ||
53 | + </td> | ||
54 | + <td> | ||
55 | + <img src="images/Set5_img_001_SRF_2_bicubic.png"/> | ||
56 | + </td> | ||
57 | + <td> | ||
58 | + <img src="images/Set5_img_001_SRF_2_superResolution.png"/> | ||
59 | + </td> | ||
60 | + </tr> | ||
61 | +</table> | ||
62 | +<table> | ||
63 | + <tr> | ||
64 | + <td> | ||
65 | + <img src="images/Set5_img_002_SRF_2_HR.png"/> | ||
66 | + </td> | ||
67 | + <td> | ||
68 | + <img src="images/Set5_img_002_SRF_2_bicubic.png"/> | ||
69 | + </td> | ||
70 | + <td> | ||
71 | + <img src="images/Set5_img_002_SRF_2_superResolution.png"/> | ||
72 | + </td> | ||
73 | + </tr> | ||
74 | +</table> | ||
75 | + | ||
76 | +<table> | ||
77 | + <tr> | ||
78 | + <td> | ||
79 | + <img src="images/Set5_img_003_SRF_2_HR.png"/> | ||
80 | + </td> | ||
81 | + <td> | ||
82 | + <img src="images/Set5_img_003_SRF_2_bicubic.png"/> | ||
83 | + </td> | ||
84 | + <td> | ||
85 | + <img src="images/Set5_img_003_SRF_2_superResolution.png"/> | ||
86 | + </td> | ||
87 | + </tr> | ||
88 | +</table> | ||
89 | + | ||
90 | +<table> | ||
91 | + <tr> | ||
92 | + <td> | ||
93 | + <img src="images/Set5_img_004_SRF_2_HR.png"/> | ||
94 | + </td> | ||
95 | + <td> | ||
96 | + <img src="images/Set5_img_004_SRF_2_bicubic.png"/> | ||
97 | + </td> | ||
98 | + <td> | ||
99 | + <img src="images/Set5_img_004_SRF_2_superResolution.png"/> | ||
100 | + </td> | ||
101 | + </tr> | ||
102 | +</table> | ||
103 | +<table> | ||
104 | + <tr> | ||
105 | + <td> | ||
106 | + <img src="images/Set5_img_005_SRF_2_HR.png"/> | ||
107 | + </td> | ||
108 | + <td> | ||
109 | + <img src="images/Set5_img_005_SRF_2_bicubic.png"/> | ||
110 | + </td> | ||
111 | + <td> | ||
112 | + <img src="images/Set5_img_005_SRF_2_superResolution.png"/> | ||
113 | + </td> | ||
114 | + </tr> | ||
115 | +</table> | ||
116 | + | ||
117 | +## Reference | ||
118 | +- Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias | ||
119 | +Weyand, Marco Andreetto, Hartwig Adam. MobileNets: Efficient Convolutional Neural | ||
120 | +Networks for Mobile Vision Applications. arXiv:1704.04861, 2017 | ||
121 | +- Chao Dong, Chen Change Loy, Member, IEEE, Kaiming He, Member, IEEE, and Xiaoou | ||
122 | +Tang, Fellow, IEEE. Image Super-Resolution Using Deep Convolutional Networks. IEEE | ||
123 | +TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 38, NO. | ||
124 | +2, FEBRUARY 2016 | ||
125 | +- Jiwon Kim, Jung Kwon Lee, Kyoung Mu Lee. Accurate Image Super-Resolution Using | ||
126 | +Very Deep Convolutional Networks. Computer Vision and Pattern Recognition (CVPR), 2016 | ||
127 | +IEEE Conference on | ||
128 | +- Jiwon Kim, Jung Kwon Lee, Kyoung Mu Lee. deeply recursive convolutional network for | ||
129 | +image super resolution. 2016 IEEE Conference on Computer Vision and Pattern Recognition | ||
130 | +(CVPR) | ||
131 | +- Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P. Aitken, Rob | ||
132 | +Bishop, Daniel Rueckert, Zehan Wang. Real-Time Single Image and Video Super-Resolution | ||
133 | +Using an Efficient Sub-Pixel Convolutional Neural Network. 2016 IEEE Conference on | ||
134 | +Computer Vision and Pattern Recognition (CVPR) | ||
135 | +- Sajid Anwar, Kyuyeon Hwang, Wonyong Sung. Structured Pruning of Deep Convolutional | ||
136 | +Neural Networks. ACM Journal on Emerging Technologies in ComputingSystems.Article No. 32 | ||
137 | +- Darryl D. Lin, Sachin S. Talathi, V. Sreekanth Annapureddy.Fixed point | ||
138 | +quantization of deep convolutional networks. ICML'16 Proceedings of the 33rd | ||
139 | +International Conference on International Conference on Machine Learning - | ||
140 | +Volume 48 Pages 2849-2858 |
images/Set5_img_001_SRF_2_HR.png
0 → 100644
362 KB
images/Set5_img_001_SRF_2_LR.png
0 → 100644
107 KB
images/Set5_img_001_SRF_2_bicubic.png
0 → 100644
296 KB
363 KB
images/Set5_img_002_SRF_2_HR.png
0 → 100644
117 KB
images/Set5_img_002_SRF_2_LR.png
0 → 100644
37.4 KB
images/Set5_img_002_SRF_2_bicubic.png
0 → 100644
111 KB
129 KB
images/Set5_img_003_SRF_2_HR.png
0 → 100644
127 KB
images/Set5_img_003_SRF_2_LR.png
0 → 100644
35.8 KB
images/Set5_img_003_SRF_2_bicubic.png
0 → 100644
109 KB
134 KB
images/Set5_img_004_SRF_2_HR.png
0 → 100644
92 KB
images/Set5_img_004_SRF_2_LR.png
0 → 100644
32.7 KB
images/Set5_img_004_SRF_2_bicubic.png
0 → 100644
93.6 KB
119 KB
images/Set5_img_005_SRF_2_HR.png
0 → 100644
116 KB
images/Set5_img_005_SRF_2_LR.png
0 → 100644
36.2 KB
images/Set5_img_005_SRF_2_bicubic.png
0 → 100644
99.3 KB
122 KB
proposal/딥러닝을 이용한 영상 화질 개선 알고리즘의 최적화.pdf
0 → 100644
No preview for this file type
-
Please register or login to post a comment