18. 🖼️ Image-to-Poem
Overview
Transform any image into a poetic verse that captures its mood and key elements—perfect for social media captions, personalized gifts, or creative inspiration.
Primary Use Cases
- Artists & photographers adding poetic captions
- Social media managers auto-generating engaging posts
- Anyone looking for creative flair in messages or greeting cards
Key Features
- Image upload or drag-and-drop
- Auto-tagging: CLIP-based extraction of main themes/objects
- Poem generation: user-selectable style (haiku, free verse, sonnet)
- Shareable cards: boilerplate poster with image + poem overlay
- Audio read-aloud via TTS
Tech Stack
- Frontend: React + TypeScript + Tailwind (uploader, style picker)
- Backend: FastAPI (Python)
- AI Models:
- Vision:
clip-vit-base-patch32
for theme extraction
- Language:
google/flan-t5-large
fine-tuned on poetry corpora
- TTS:
espnet/kan-bayashi_ljspeech_vits
Architecture
- Image Service: preprocess → CLIP encode → top-K theme tokens.
- Poem Service: builds prompt with theme tokens + style → T5 generation.