Interactive 3D avatar for malls and trade shows
The context
A brand activation company working in malls and corporate trade shows needed an attraction that could genuinely talk to the public: no operator behind it, no fixed script. And with one central business requirement: the same system must be resold to different clients and events, swapping the entire character (visuals, voice and personality) with no development rework.
What we are delivering
A 3D avatar running on a kiosk, end to end:
- Detects presence through the kiosk sensor, greets with gesture and voice, and applies a cooldown so the same person is not greeted repeatedly.
- Listens, understands and answers by voice in real time: captured speech becomes text and the language model generates the answer in the active character's personality.
- Speaks with lip sync: the synthesized audio arrives with the viseme timeline that drives the avatar's mouth and gestures.
Rendering and orchestration run locally on the kiosk. If the internet drops, the avatar stays in idle without freezing the operation.
The streaming voice pipeline
Latency is what separates a natural conversation from a slow toy. The whole pipeline runs in streaming, stage by stage:
- Speech is transcribed in real time, with automatic end-of-utterance detection.
- The model's answer arrives token by token and is sent to voice synthesis sentence by sentence.
- The avatar starts speaking before the full answer exists, targeting 1.5 to 3 seconds between end of speech and start of response.
Language model, voice and speech rate are per-character configuration, not code.
The architecture
Architecture decisions
Character as a configuration package
Each character is a self-contained folder: 3D model, voice, personality prompt and animations declared in a validated manifest. Swapping characters means pointing to another folder. Zero code.
Lip sync through native TTS visemes
Voice synthesis returns the exact phoneme timeline along with the audio. This removes the need to build lip alignment from scratch and works the same for any character that follows the blendshape standard.
Web 3D packaged as a kiosk app
Three.js rendering packaged as a full-screen auto-start app. Runs on any modern PC, with no engine license, and operates offline-first: only the voice and AI API trio depends on the network.
Where we are
- Voice conversation pipeline working end to end, answering within natural conversation time.
- Character swap validated: visuals, voice and personality change through configuration, without touching code.
- Delivery planned with a sample character and a guide so the client's team can create the next ones on their own.
Stack
How we execute
Technical documentation approved before any code: architecture, justified stack and mapped operating cost. Development in milestones with the client following progress, and final delivery with commented code and an operations guide for their team.