Voice-First Wholesale Ordering

A Swiss wine distributor’s wholesale buyers and sommeliers think about wine by name, vintage, bottle size, and quantity — and they place orders standing in a cellar or at a tasting, not sitting at a desk. Typing all of that into a form is slow and easy to get wrong.

I built this product, through the agency, so they can just say the order out loud. A buyer records something like “a Pinot Noir, 2022 vintage, 75cl, twelve bottles” and the app turns it into matched catalog items ready to review and confirm.

I worked as a design engineer — I owned the interface and the code that makes it feel right, not one or the other. I designed the recording experience and the order flow, then built them in React Native.

The whole product runs on one brand type system — a display face for headings, a clean body face, and a single golden accent. I carried that identically across iOS, Android, and web, so the brand reads the same everywhere instead of drifting per platform.

I also tuned the recording visualization to run at 60fps. Live audio metering drives a ripple and pulse on the UI thread through Reanimated shared values, so the animation stays smooth even while the app is busy uploading in the background.

The app is a single Expo / React Native codebase — React 19, Expo Router, TypeScript in strict mode. The same code compiles to native iOS and Android and exports to the web through react-native-web. One product, one design system, three platforms.

Recording is handled with expo-av. When a recording stops, the audio is added to a sequential upload queue that runs in the background with automatic retry and exponential backoff. No blocking modal, no spinner taking over the screen — the order just appears when it is ready.

The backend is an ASP.NET Core REST API. It transcribes the audio with Azure OpenAI Whisper, extracts product, vintage, size, and quantity with a GPT model, then fuzzy-matches each item against the wine catalog — Levenshtein distance plus umlaut and French / Italian accent normalization — and returns a confidence score. Anything it is not sure about comes back flagged for a quick manual check instead of being guessed.

Auth is JWT; orders and per-user audio are persisted with EF Core. Builds and over-the-air updates ship through EAS, and the interface is fully bilingual — German and English — to match how Swiss customers actually work.

The product is used in wine cellars and tasting rooms, not quiet offices — sommeliers explaining wines, glasses clinking, music, sometimes fifty people in the room. So I dropped automatic silence detection and kept the controls manual: the buyer decides when to stop.

The best feedback here turned out to be no feedback. Uploads happen silently, products show up when they are ready, and the interface stays out of the way.

A working cross-platform app — record, transcribe, match, review, order — that showed the distributor what the experience could be. Buyers speak the way they already talk about wine, and the system handles the messy parts — accents, umlauts, vintages, bottle sizes — only asking for help when it genuinely needs it.

It was an experimental build: I delivered a working app across iOS, Android and web from a single codebase, with the same design language and interactions everywhere. Whether it went into production was theirs to decide.