류재훈 / 2018-1-capstone_design_2_SuperResolution

Go to a project
Toggle navigation Toggle navigation pinning
류재훈
c93dd9ac 1 parent 9028d193

update result

Inline Side-by-side
Showing 22 changed files with 140 additions and 0 deletions
# Superresolution
___
## Introduction
We propose a super resolution method based on Convolution Neural Network (CNN), which is similar to or superior to existing methods, while dramatically lowering the number of network parameters and computation cost compared to the conventional CNN method. Our proposed network has a much more efficient network structure than deconvolution operations or other methods of enhancing the instantaneous resolution of inputs by using sub-pixel cnn techniques to increase the resolution inside the network. In addition, by applying Modified Depth-Wise Separable CNN, which has been introduced in MobileNet, the number of parameters can be kept smaller while having a more complicated network structure in comparison with SRCNN. We have applied Depthwise Separable as described above to reduce the number of parameters. Finally, our network converts Floating point to Fixed Point operation, which makes it more efficient and faster. Ultimately, our network has a lightweight, simple structure that can be extended to hardware implementations such as FPGAs or ASICs, which makes it possible to extend to these areas.
## Requirements
- PyTorch
- skimage
- python3
## Datasets
### Train、Val Dataset
- download dataset from this [link](https://drive.google.com/open?id=1-5eKvxDnIqrXE3ABSk6RcPwMrgsKeCsw) and put it in this project
### Test Image Dataset
The test image dataset are sampled from
| **Set 5** | [Bevilacqua et al. BMVC 2012](http://people.rennes.inria.fr/Aline.Roumy/results/SR_BMVC12.html)
| **Set 14** | [Zeyde et al. LNCS 2010](https://sites.google.com/site/romanzeyde/research-interests)
| **BSD 100** | [Martin et al. ICCV 2001](https://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/)
## [Proposal](images/딥러닝을 이용한 영상 화질 개선 알고리즘의 최적화.pdf)
## TODO
* Fix PSNR BUG
## DONE
* compare other model
* apply Video
* [Tensorboard](https://github.com/yunjey/pytorch-tutorial/tree/master/tutorials/04-utils/tensorboard)
* make Inference script
* addition metric ssim
* implement psnr to skimage
* inference 3 dataset (set5,set14,BSD100)
* drawing info result
* drawing background
* Lossless compression bmp dataset
* valid dataset
> Image Results
The left is high resolution image, the middle is Bicubic-interpolation image, and
the right is super resolution image(output of the ESPCN).
** Upscale Factor is 2**
- Set5
<table>
<tr>
<td>
<img src="images/Set5_img_001_SRF_2_HR.png"/>
</td>
<td>
<img src="images/Set5_img_001_SRF_2_bicubic.png"/>
</td>
<td>
<img src="images/Set5_img_001_SRF_2_superResolution.png"/>
</td>
</tr>
</table>
<table>
<tr>
<td>
<img src="images/Set5_img_002_SRF_2_HR.png"/>
</td>
<td>
<img src="images/Set5_img_002_SRF_2_bicubic.png"/>
</td>
<td>
<img src="images/Set5_img_002_SRF_2_superResolution.png"/>
</td>
</tr>
</table>
<table>
<tr>
<td>
<img src="images/Set5_img_003_SRF_2_HR.png"/>
</td>
<td>
<img src="images/Set5_img_003_SRF_2_bicubic.png"/>
</td>
<td>
<img src="images/Set5_img_003_SRF_2_superResolution.png"/>
</td>
</tr>
</table>
<table>
<tr>
<td>
<img src="images/Set5_img_004_SRF_2_HR.png"/>
</td>
<td>
<img src="images/Set5_img_004_SRF_2_bicubic.png"/>
</td>
<td>
<img src="images/Set5_img_004_SRF_2_superResolution.png"/>
</td>
</tr>
</table>
<table>
<tr>
<td>
<img src="images/Set5_img_005_SRF_2_HR.png"/>
</td>
<td>
<img src="images/Set5_img_005_SRF_2_bicubic.png"/>
</td>
<td>
<img src="images/Set5_img_005_SRF_2_superResolution.png"/>
</td>
</tr>
</table>
## Reference
- Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias
Weyand, Marco Andreetto, Hartwig Adam. MobileNets: Efficient Convolutional Neural
Networks for Mobile Vision Applications. arXiv:1704.04861, 2017
- Chao Dong, Chen Change Loy, Member, IEEE, Kaiming He, Member, IEEE, and Xiaoou
Tang, Fellow, IEEE. Image Super-Resolution Using Deep Convolutional Networks. IEEE
TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 38, NO.
2, FEBRUARY 2016
- Jiwon Kim, Jung Kwon Lee, Kyoung Mu Lee. Accurate Image Super-Resolution Using
Very Deep Convolutional Networks. Computer Vision and Pattern Recognition (CVPR), 2016
IEEE Conference on
- Jiwon Kim, Jung Kwon Lee, Kyoung Mu Lee. deeply recursive convolutional network for
image super resolution. 2016 IEEE Conference on Computer Vision and Pattern Recognition
(CVPR)
- Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P. Aitken, Rob
Bishop, Daniel Rueckert, Zehan Wang. Real-Time Single Image and Video Super-Resolution
Using an Efficient Sub-Pixel Convolutional Neural Network. 2016 IEEE Conference on
Computer Vision and Pattern Recognition (CVPR)
- Sajid Anwar, Kyuyeon Hwang, Wonyong Sung. Structured Pruning of Deep Convolutional
Neural Networks. ​ ACM Journal on Emerging Technologies in ComputingSystems.Article No. 32
- Darryl D. Lin, Sachin S. Talathi, V. Sreekanth Annapureddy.Fixed point
quantization of deep convolutional networks. ICML'16 Proceedings of the 33rd
International Conference on International Conference on Machine Learning -
Volume 48 Pages 2849-2858