BREAKING: Voice AI is Dominating
120M Conversations a/wk, 4x YouTube
1M+ Devs, 100x TAM, 4x YouTube
Dylan Fox is the founder and CEO of AssemblyAI, the voice AI infrastructure platform behind AI notetakers, medical scribes, drive through ordering, contact centers and humanoid robots. AssemblyAI went through Y Combinator’s first AI batch in 2017 & now serves 1+ million developers with ~100 million API calls a day.
→ Listen on X, Spotify, YouTube, Apple
We get into the volume numbers, why weekly conversations are up 800%+ in 3 years, the three macro trends pulling voice into everything, why coding agents turned small businesses into API customers, how data mixture drives 75% of model quality, the humanoid robot speaker problem, on device models, and the disclosure question the entire voice agent industry is avoiding.
“It’s very easy to optimize for a public open source benchmark. It’s very hard to optimize across all these real world applications.”
We cover
› 120 million weekly conversations and 4x YouTube’s daily voice volume
› Coding agents expanding the TAM by 100x
› Context aware voice models and the McDonald’s ordering problem
› Why humanoid robots can’t disambiguate speakers
› Whether voice agents should disclose they are AI
Special thank you to Brex, MongoDB, & AssemblyAI for helping make this RAISE AI Summit mini-series in Paris, France happen.
𝐓𝐈𝐌𝐄𝐒𝐓𝐀𝐌𝐏𝐒
(00:00) Dylan Fox, Founder & CEO at Assembly AI
(01:06) AssemblyAI's voice traffic just beat YouTube by 4x
(03:33) $100K in GPU credits that started it all
(06:39) Why Voice AI is inflecting right now
(10:11) AssemblyAI's infrastructure-first strategy
(12:41) Handling 120 million weekly calls
(14:49) How AI agents are rewriting Internal Ops at AssemblyAI
(17:05) The Sovereign AI problem nobody's solved for voice
(18:26) Is the Keyboard and Mouse finally dying?
(22:33) The problem Humanoid Robotics hasn't solved yet
(25:14) The real problem behind translating languages
(29:27) Can AI actually talk to animals?
(31:24) The real reason Open-Source benchmarks lie to you
(34:20) Building websites in chat rooms as a kid
(36:48) On-Device AI models are about to change everything
(43:03) The people who shaped Dylan's career
Brought to you by:
Brex—The intelligent finance platform: cards, expenses, travel, bill pay, banking—wrapped into a high-performance stack. Built for scale. Trusted by OpenAI, Anthropic, Vercel, Granola, Deepgram, & Sourcery.. teams that move fast AF. visit → brex.com/sourcery
MongoDB–Millions of developers and more than 65,200+ customers across industries – including ~75% of the Fortune 100 – rely on MongoDB for their most important applications. With integrated capabilities for operational data, search, real-time analytics, & AI-powered data retrieval, MongoDB helps organizations everywhere move faster, innovate more efficiently, & simplify complex architectures. visit → mongodb.com/ai
AssemblyAI–Millions of developers use AssemblyAI to power their voice ai apps & features. One API gives you access to best-in-class speech-to-text, voice agent, & speech understanding models for both pre-recorded and real-time audio. Granola, ClickUp & HeyGen are scaling with AssemblyAI - get $50 of free credits today → AssemblyAI.com/sourcery
Dylan Fox on the Voice Infrastructure Running Underneath AI
AssemblyAI builds the voice AI infrastructure behind AI notetakers, medical scribes, drive through ordering, contact centers and humanoid robots. Founder and CEO Dylan Fox walked through the volume moving across the platform, the problems still open in voice, and what changes over the next 18 months.
→ Listen on X, Spotify, YouTube, Apple
Where the Volume Is
AssemblyAI went through Y Combinator in 2017 in a batch organized by Daniel Gross that contained six AI companies in total. The perk was compute. Production AI deployment was rare at the time and the first TensorFlow meetups were still running.
“We got $100,000 in GPU credits. That was our perk, which now seems cute.”
Weekly conversation volume is now up over 800% in three years. On a peak week the platform carries more than 120 million voice conversations. Daily throughput runs above 2 million hours of voice, which as of December is 4x YouTube’s daily volume.
“Over 2 million hours of voice, which as of December of this past year, was a little over 4X the amount of daily volume that’s going to YouTube.”
There are a little over 1 million developers on the platform. 40% of them signed up in the past year, and close to 100 million API calls a day run against the API.
3 Trends Making Voice Inflect
1) Model capability & context. The newest voice models accept context about the environment they are deployed into. They shipped a few weeks before the interview and are the first voice models that can do it. The same models are faster and lower cost, which widens where they can be deployed.
“Today, if you drive through a McDonald’s and you order with your voice, and there’s a voice AI system there, that voice AI system has no clue that it’s taking a McDonald’s order.”
2) The rest of the AI stack. Reasoning models, vector databases and models across other modalities now exist as commodity components. A company building drive through ordering, AI notetaking or a healthcare scribe assembles the voice layer plus the rest of the infrastructure it needs.
3) Coding agents! (the newest of the 3). AssemblyAI is an API platform, which historically meant engineering and product teams inside software companies. But now, anyone can code. A small business lawn care chain signed up and is automating back office work through AI-driven apps like Lovable, Cognition’s Devin, Claude Code, Cursor and Replit. Global enterprises are now building their own internal software on the API with two people and a set of agents.
“Our TAM has just increased by 100X because we’re not just selling to engineering teams within product companies.”
Infrastructure & Nothing Above It
The company builds models, the inference layer around them, an orchestration layer, and the developer and agent experience. It stays out of the application layer.
“We’re 100% focused on voice AI infrastructure. So we don’t do anything at the application layer.”
Models are tailored to healthcare, drive through ordering, contact center, AI note-taking, and much, much more. Most AI notetakers run on the platform for scalability, cost, accuracy and burst capacity, with no servers for the customer to stand up. About half the company’s engineering work goes into scaling the infrastructure across regions and clouds.
The company is around 80 people, almost entirely product, research and engineering, with forward deployed engineers working directly with customers. Research hires arriving from larger companies come in with a workflow built around building a model and benchmarking it.
“I think that’s the biggest difference between us and a lab at a bigger company. We have a lot of researchers, research engineers that will come to Assembly from a bigger company, and they’re so far removed from the customer.”
What Makes a Voice Model Good
Data mixture accounts for most of model quality. Model updates ship every couple of weeks, and the platform runs multiple model versions and APIs, including one launched the day of the interview for dictation and push to talk.
“Probably like 75% of it is like the data that you’re training on.”
Correct behavior is defined by the application, and two customers can want opposite output from the same audio. Legal discovery teams running police body cam footage want every speaker captured with nothing left out. A drive through order needs everything filtered away (like screaming kids) except the person in the front seat.
“If the kid in the back screams, an extra large Frosty, you wanna be smart enough to ignore that.”
Public benchmarks compress the visible gap between models. The company runs a dedicated internal evals team measuring across a large set of metrics, data sets and languages.
“It’s very easy to optimize for a public open source benchmark. It’s very hard to optimize across all these real world applications.”
The latest model handles 20 languages in one model with automatic language detection and real time switching. Native speaker judgment determines what correct output looks like, and fixing one behavior frequently breaks another. Local voice AI vendors are typically the strongest in their own languages because they understand the nuanced cultural dictations (something Google translate is still trying to understand) and can identify which training data is good.
Voice as an Additional Dimension of Computing
Voice became reliable as a form of data capture in roughly the past year. That covers computer dictation, meeting notes, medical charting during patient encounters and field service coaching.
“Voice is a reliable form of data capture now.”
Ciro AI runs on the platform building software for plumbers and HVAC technicians that listens to a service visit and delivers sales coaching afterward. Those technicians are taking home around 20% more. Large consumer electronics companies are implementing the technology now, and on-device models that run on a phone or a low powered device such as a TV remote are in development.
And voice is now becoming such a reliable form factor that we may actually not need keyboards anymore (shoutout to Coatue’s Max Cook).
“That enables more passive hardware and more passive computing experiences where you can talk to things and you can be walking around and you can be free from being like a prisoner to the device.”
But when it comes to something a bit more complex like humanoid robots, they’re hitting a specific limit. Three people standing next to a robot produce jumbled and merged output because the system cannot disambiguate who is speaking, and cameras do not solve it when a speaker is turned away.
The Disclosure Problem & the Next 18 Months
Text based agents in support flows and outbound sales identify themselves as AI. Voice agents are built to the opposite standard.
“Today when you’re building a voice agent that you’re talking to over the phone, for the most part, the goal is to try to trick the human into believing that it’s also a human.”
Customers building voice agents see the behavior in their own numbers. Users hang up on disclosure because prior voice experiences were bad, which makes mimicking a human the current way to signal intelligence.
“If our customers, if they’re building a voice agent and you disclose upfront that you’re an AI, people just hang up.”
Callers who work it out two minutes in report the experience as strange, and on a hospital nurse line a caller may want a human specifically. Good pacing and natural conversation are achievable without the deception, and the industry has not settled the UX for it.
Sovereign and self hosted deployment is available and rising, though voice sits earlier than text on custom model ownership. Getting the technology to work is the current phase, and the working assumption is another 18 months of it. Over the next 12 months some of the fun & exciting growth areas Dylan is looking forward to are consumer hardware and software including toys, games and consumer electronics.
“In five years, kids are just gonna talk at things, expecting them to be able to understand them.”
→ Listen on X, Spotify, YouTube, Apple
The material presented on Molly O’Shea’s website are my opinions only and are provided for informational purposes and should not be construed as investment advice. It is not a recommendation of, or an offer to sell or solicitation of an offer to buy, any particular security, strategy, or investment product. Any analysis or discussion of investments, sectors or the market generally are based on current information, including from public sources, that I consider reliable, but I do not represent that any research or the information provided is accurate or complete, and it should not be relied on as such. My views and opinions expressed in any website content are current at the time of publication and are subject to change. Past performance is not indicative of future results.
Paid Endorsement. Brokerage services by Open to the Public Investing Inc, member FINRA & SIPC. Advisory services by Public Advisors LLC, SEC-registered adviser. Crypto trading provided by Zero Hash LLC, licensed by the NYSDFS. Generated Assets is an interactive analysis tool by Public Advisors. Output is for informational purposes only and is not an investment recommendation or advice. See disclosures at public.com/disclosures/ga. Matched funds must remain in your account for at least 5 years. Match rate and other terms are subject to change at any time.