Comic GPT
Machine Learning, Fintech, Stock Market
Mentors :
Lakshay Gupta
Mentees :
4
Use Large Image and Language models to create a comic generator app. Fine tune the models on your favorite styles and make a fun app you can share with your friends.
We can also explore creating videos out of the comics we generate (basically text to anime😋) if time permits.
We'll be making use of cutting edge diffusion models for image and video generation (think stable diffusion, modelscope, etc) and large language models like GPT3.5 and LLAMA to generate the storyline.
If you think you'll just be stringing together APIs, you're mistaken. We will go into the depths of transformers and diffusers from whitepapers to their implementations so that you don't feel like you're juggling black boxes.
Here's a basic version of the app you're going to make - https://sparkler.club/
Needless to say, we'll be adding many more features.
Diffusers - https://github.com/huggingface/diffusers, https://huggingface.co/docs/diffusers/index
https://ai.facebook.com/blog/large-language-model-llama-meta-ai/
https://platform.openai.com/docs/guides/completion
Prerequisites:
No huge prereqs
- knowledge of python or any other programming language
- willingness to dirty your hands by squashing new bugs
- enthu
Tentative Timeline :
| Week | Work |
|---|---|
| Week 1-2 | Playing around with Image Generation models and building mini projects like finetuning stable diffusion on your face and generating your own custom ai avatars |
| Week 3-4 | Playing around with Language Models, exploring the best prompts for our use case and also toying around with tools like Langchain |
| Week 5-6 | Build a basic version of the app. Armed with the knowledge so far, you can also work on your own ideas instead of a comic generator |
| Week 7-8 | Add fun features to your app like an image editor, chatbot, social sharing etc or explore the use of generative audio and video in your app. Share your apps on social media and get real users. Launch your app on product hunt. |
| Week 9 | Talk to users and make improvements in the app. |
