Target Funnel Stage: Early Retention (Day 3 Drop-off)
Problem Statement
Users easily complete Day 1 and Day 2 lessons with high initial motivation. However, data indicates a massive 40% drop-off on Day 3. When a user studying a new language track (for example, encountering complex Hiragana characters for the first time in the JLPT N5 track) experiences a sudden spike in cognitive load, they often abandon the app. Users choose to stop playing rather than risk failing the lesson and losing their newly formed streak.
Experiment Hypotheses & A/B Test Design
Test 1: The 'Cognitive Load' Lever (UI/UX)
- Hypothesis: We believe that splitting the Day 3 lesson into two smaller "Micro-Lessons" will increase Day 3 completion rates by 15% because it keeps the time-to-completion consistent with previous days, reducing the psychological friction of tackling a difficult module.
- Control: Standard single Day 3 lesson node.
- Variant: Day 3 node visually and functionally split into Part A and Part B.
- Primary Metric: Day 3 completion rate.
- Secondary Metric: Day 4 start rate.
- Guardrail Metric: Average session duration (to ensure overall engagement does not drop).
Test 2: The 'Loss Aversion' Lever (Gamification)
- Hypothesis: We believe that gifting users a free "Streak Freeze" explicitly right before they start the Day 3 lesson will improve D7 retention because it removes the anxiety of losing their progress if they struggle with the new, harder concepts.
- Control: Standard flow, users must buy a streak freeze from the shop.
- Variant: An interstitial screen gifting a one-time freeze right before the lesson starts.
- Primary Metric: D7 Retention.
- Secondary Metric: Day 3 lesson start rate.
- Guardrail Metric: Organic Streak Freeze purchase rate (to monitor revenue cannibalization).
Test 3: The 'Social Proof' Lever (Copy/Messaging)
- Hypothesis: We believe that adding a pre-lesson screen stating "80% of users who pass Day 3 stick around to become fluent!" will increase the start rate of the Day 3 lesson because it provides a clear milestone and social motivation to push through friction.