Stable Diffusion
Machine Learning
Mentors :
Shubham Hazra
Om Godage (21d100006)
Kartik Gokhale (200100083)
Mentees :
5-6
Elaborate upon the work and learning involved in the project. Suggest some reading material and resources for mentees to gain context and spark their interest. You may also link GitHub repos/Demonstrations, if the project is an already existing one.
Stable Diffusion is a powerful text-to-image AI system, can create photos in the style of cartoonists, 19th century daguerreotypists, stop-motion animators and more. Text to image diffusion models are an exciting area of research in machine learning that aims to generate high-quality images from textual descriptions. This technology has a wide range of applications, such as generating images for artistic or commercial purposes, enhancing accessibility for visually impaired individuals, and aiding in virtual reality and game development.
They use a generative approach that involves learning the statistical relationships between text descriptions and corresponding images. This involves training a machine learning model on a large dataset of paired text and image data, which it uses to generate new images based on textual descriptions that it has not seen before.
Students can checkout DALL.E-2 by openAI: https://openai.com/product/dall-e-2
https://www.unrealengine.com/en-US/unreal-engine-5
https://docs.unrealengine.com/5.0/en-US/
It's not that hard!! This guy made an open world game in 24 hours.
https://www.youtube.com/watch?v=3DjY1T42b_M
PreReqs:
Basic Python and willingness to learn
Tentative Timeline :
| Week Number | Tasks to be Completed |
|---|---|
| Week 1 | Basics of Regression & Classical ML |
| Week 2 | Intro to Deep learning & frameworks (Tensorflow, PyTorch) |
| Week 3 | Image Processing using OpenCV & classical methods |
| Week 4 | Dive into CNNs & transfer learning |
| Week 5 | Intro to NLP , text encoders and decoders |
| Week 6 | Learning about GANs, Autoencoders and Attention models |
| Week 7 | Starting with Stable Diffusion |
| Week 8 | Finishing up with the training and implementing the pipeline |
| Week 9 | Finishing up with documentation and submission |
