Prototyping a Voice Assistant with Face Detection and GPT-3
Combining OpenCV face detection, Azure Speech Recognition, and GPT-3 text generation into a talking desktop companion — written in Python over a weekend in 2022.
Prototyping a Voice Assistant with Face Detection and GPT-3
This was a weekend project from late 2022, right when GPT-3 started feeling accessible to hobby developers. The idea was: can I build something that sees you, listens to you, and talks back — all in one Python script?
The stack ended up being three APIs glued together:
- OpenCV for webcam access and face detection via Haar cascades
- Azure Cognitive Services Speech for speech-to-text
- OpenAI GPT-3 (text-davinci-003 at the time) for generating a reply
- gTTS + VLC for converting the reply back to audio
The main loop reads webcam frames, draws a rectangle when a face is detected, and waits for speech. Once something is recognized, it goes to GPT and the reply gets read out loud. A small tkinter window shows the transcript and has English/German buttons to switch the recognition language.
The face detection part is purely cosmetic — it doesn't influence what GPT gets. I mainly added it because watching the bounding box react while you talk makes the whole thing feel more alive.
Looking back, it's a product of its time: the text-davinci-003 calls would need updating today, and the latency was noticeable. But as a proof of concept it worked surprisingly well, and it was genuinely the first time I built something that felt like it was listening. Code on GitHub.