18. 🖼️ Image-to-Poem

Overview

Transform any image into a poetic verse that captures its mood and key elements—perfect for social media captions, personalized gifts, or creative inspiration.

Primary Use Cases

• Artists & photographers adding poetic captions
• Social media managers auto-generating engaging posts
• Anyone looking for creative flair in messages or greeting cards

Key Features

• Image upload or drag-and-drop
• Auto-tagging: CLIP-based extraction of main themes/objects
• Poem generation: user-selectable style (haiku, free verse, sonnet)
• Shareable cards: boilerplate poster with image + poem overlay
• Audio read-aloud via TTS

Tech Stack

• Frontend: React + TypeScript + Tailwind (uploader, style picker)
• Backend: FastAPI (Python)
• AI Models:
  • Vision: clip-vit-base-patch32 for theme extraction
  • Language: google/flan-t5-large fine-tuned on poetry corpora
  • TTS: espnet/kan-bayashi_ljspeech_vits

Architecture

1. Image Service: preprocess → CLIP encode → top-K theme tokens.
2. Poem Service: builds prompt with theme tokens + style → T5 generation.