Light field imaging and Dual Attention Networks
Machine Learning
Mentors :
Abeer Mishra
Shreyas Patil (200260052)
Mentees :
5
"Elaborate upon the work and learning involved in the project. Suggest some reading material and resources for mentees to gain context and spark their interest. You may also link GitHub repos/Demonstrations, if the project is an already existing one. Light field technology, also called integral imaging, is an emerging concept in the field of imaging technologies. In this, we use multiple cameras through a camera array, in order to construct an image with both angular and spatial information of the light rays to form the image. Light field (LF) images can be used to interactively refocuss images and increase resolution. Image super-resolution is one of the most extensively researched problems in the field of image processing and deep learning. Various classical image processing techniques have been developed to denoise and increase image resolution. Further after the publication of the SRCNN paper by Dong et al (https://arxiv.org/pdf/1501.00092.pdf), there has been alot of research focussed on developing deep learning based methods for image super-resolution with each method outperforming the previouos one. In this project, we'll aim to make an implementation of a Dual attention network for super-resolution of light field images inspired from a paper by researchers at NUDT (National University of Defence Technologies), China . Check it out on https://arxiv.org/pdf/2110.12114.pdf
Prerequisites : Basic level coding in python is an essential pre-requisite. Can be relaxed in case mentee demonstrates eagerness to learn Python from scratch. Prior experience using image processing libraries like PIL and OpenCV or Python ML frameworks like TensorFlow, PyTorch, Keras or Sklearn is aencouraged but not required.
Tentative Timeline :
| Week | Work |
|---|---|
| Week 1-2 | Basics of Machine learning |
| Week 3 | Study convolutional neural networks and basics of image processing |
| Week 4 | First checkpoint: simple machine learning task based on convolution neural networks |
| Week 5-6 | First major checkpoint task begins - implement the SRCNN network, towards the end of week 6 |
| Week 7 | Study Dual Attention Networks. Begin implementation of the paper: https://arxiv.org/pdf/2110.12114.pdf |
| Week 8+ | Work on the implementation of the paper. It is likely that the full implementation wouldn't be done by the time the project is finished, in that case we'll call off the project possibly in it's training stages itself rather than waiting for full optimization of the model. |
