How we built AI narration that actually sounds human

One of the hardest problems in building ProxiShop wasn’t the live video, the payments, or even the voice recognition. It was making the AI sound right. When the system reads a product back to you — its name, price, key details, what makes it worth considering — the gap between “robotic” and “natural” turns out to be the gap between a tool people tolerate and a tool people trust. Here’s some of what it took to close it.

The problem with reading a database out loud

Product data is structured for screens, not ears. A typical catalogue entry is a wall of fields: title, SKU, weight, dimensions, fifteen bullet points, a marketing blurb stuffed with keywords. Feed that straight into a text-to-speech engine and you get something technically accurate and completely unlistenable. Nobody wants to hear “Organic Free Range Large Eggs Box of Six, dash, six count, comma, suitable for vegetarians” recited like a legal notice.

The fix isn’t a better voice. It’s a better editor. Before anything is spoken, the content has to be reshaped for the ear: shorter sentences, the important thing first, numbers phrased the way a person would say them, and ruthless cutting of detail that doesn’t help a buying decision in the moment. That editorial layer is where most of the work lives.

Why we use more than one model

ProxiShop narrates using a multi-model approach — drawing on GPT-4o-mini, Anthropic’s Claude, and Google’s Gemini rather than betting everything on a single model. That’s not for show. Different models have different strengths: some are crisper at summarising, some warmer in tone, some handle messy product data more gracefully. Routing the right task to the right model produces a noticeably more natural result.

It also matters for resilience. If a provider has an outage or a latency spike mid-session, a shopper who relies entirely on voice can’t just “look at the screen instead.” Redundancy across models isn’t a nice-to-have for an accessibility-first product — it’s part of the accessibility itself.

For a sighted shopper, a slow AI is an annoyance. For a voice-first shopper, it can be the whole session breaking. Reliability is an accessibility feature.

The things that made it sound human

A few changes moved the needle more than anything else:

• Lead with the answer. When you ask “is this gluten free?”, the narration starts with “Yes” — not a thirty-word preamble. Front-loading the point is the single biggest difference between conversational and robotic.
• Speak quantities, don’t spell them. “Two pounds fifty” instead of “GBP 2.50”. “About half a kilo” instead of “0.47 kg”.
• Know when to stop. Give the useful version, then offer more if wanted — “there’s also nutritional info if you’d like it” — rather than dumping everything at once.
• Keep the human in the loop. The AI narrates, but a vetted human handler is physically in the store, checking labels and confirming what’s actually on the shelf. The AI never has to bluff, because a person can ground it in reality.

What we’d tell anyone building voice

The lesson that surprised us most: the quality of a voice experience is decided long before the audio is generated. It’s decided in how you reshape information for listening, what you leave out, and how you keep a real human accountable for accuracy. The voice is the last 5%. The editorial judgement is the other 95%.

That’s also why we don’t think of ProxiShop as “an AI shopping app.” It’s a collaboration — you, a human handler, and AI — where each does what it’s best at. The AI’s job isn’t to sound impressive. It’s to get out of the way and let a shopping conversation feel like a shopping conversation.

Natural narration isn’t a voice problem. It’s an editing problem with a voice at the end.