Context
Built a speech-based live boxscore system that converts spoken commentary into structured basketball statistics in real time.
My Contribution
- Designed the end-to-end server architecture
- Integrated speech-to-text using Whisper
- Used GPT-based parsing to convert commentary → structured stats
- Built a FastAPI backend with PostgreSQL
- Implemented real-time game state updates
Technical Details
Pipeline:
- Audio → Whisper (STT) → LLM parsing → Structured event → DB update → Live boxscore
Key challenges:
- Handling ambiguous spoken phrases
- Maintaining consistent game state
- Designing robust stat update logic
Result
- Real-time voice-driven stat tracking
- Full backend + DB system
TODO: Add captions (latency, accuracy, and how you resolved ambiguity/state consistency).