PlayAndLearnDE — Learning German from Game Screenshots

Playing in German, you often understand only half the dialogue — and looking things up interrupts the game. A screenshot is enough: the rest happens on its own.

Features

• Screenshot to a Telegram bot, processed through a queue
• OpenAI Vision reads the text from the image and translates it with grammar notes
• The web interface shows the result as it arrives (server-sent events)
• Phrases are extracted and collected in a database of their own
• Review follows the SM-2 algorithm: whatever is hard comes back more often

Result

• 51 commits
• Self-hosted — no third-party service between the user and their data

Stack: Node.js 22, TypeScript, OpenAI Vision, Telegram Bot API, SQLite, SSE