BREAKING: SambaNova Hits $11B Valuation
Are $50-100B Data Centers Necessary? Inference 101
Premium Inference 101
Economics & Infrastructure Behind Running Trillion-Parameter Models
Rodrigo Liang is the CEO and Co-Founder of SambaNova. The company just announced a first close on a $1B round at an $11B valuation, led by General Atlantic with T. Rowe Price and Capital Group participating.
→ Listen on X, Spotify, YouTube, Apple
Rodrigo has spent 32 years building chips. We covered the 101 of semiconductors and data centers: what inference is, why it is a different problem than training, and why inference will require orders of magnitude more chips than training.
He walks through SambaNova's chip lineup from SN10 to the new SN50, why a 10 kilowatt air-cooled rack changes where AI can be deployed, and why running a trillion parameter model in a single rack matters for agents, latency, and edge.
We also covered the era of premium inference, how providers measure revenue per rack, coopetition and traffic routing across Nvidia and AMD, sovereign models, the move back to on-prem, and why tokenmaxxing is the wrong goal.
Special thank you to Brex, MongoDB, & AssemblyAI for helping make this RAISE AI Summit mini-series in Paris, France happen.
𝐓𝐈𝐌𝐄𝐒𝐓𝐀𝐌𝐏𝐒
(00:00) Rodrigo Liang , Co-Founder & CEO at SambaNova Systems
(00:59) SambaNova’s Series F: $1B raise at an $11 billion valuation
(03:00) The Inference problem nobody saw coming
(04:52) SambaNova's chip evolution
(07:19) Running a trillion-parameter model on a single rack
(11:00) Do $100 billion data centers actually make sense?
(14:14) What "premium inference" really means
(18:28) Speed is about to become AI's biggest price tag
(20:43) Starlink, edge computing, and AI reaching every corner of the planet
(24:27) Working alongside NVIDIA and rival chipmakers
(27:49) How customers actually measure inference performance
(32:07) The biggest bottlenecks in AI's global land grab
(35:12) Justifying the billion-dollar AI valuations
(37:53) Why SambaNova refuses to build its own cloud
(41:03) The "AI sovereignty" debate
(43:48) Data privacy fears are driving the return to on-prem AI
(47:55) How to actually get ROI out of AI spend
(51:16) The one question every business should be asking about AI
(56:09) The mentors and lessons behind a 32-year career in chips
(58:02) Unveiling SambaNova's newest chip, the SN50
Brought to you by:
Brex—The intelligent finance platform: cards, expenses, travel, bill pay, banking—wrapped into a high-performance stack. Built for scale. Trusted by OpenAI, Anthropic, Vercel, Granola, Deepgram, & Sourcery.. teams that move fast AF. visit → brex.com/sourcery
MongoDB–Millions of developers and more than 65,200+ customers across industries – including ~75% of the Fortune 100 – rely on MongoDB for their most important applications. With integrated capabilities for operational data, search, real-time analytics, & AI-powered data retrieval, MongoDB helps organizations everywhere move faster, innovate more efficiently, & simplify complex architectures. visit → mongodb.com/ai
AssemblyAI–Millions of developers use AssemblyAI to power their voice ai apps & features. One API gives you access to best-in-class speech-to-text, voice agent, & speech understanding models for both pre-recorded and real-time audio. Granola, ClickUp & HeyGen are scaling with AssemblyAI - get $50 of free credits today → AssemblyAI.com/sourcery
Premium Inference 101: The Economics & Infrastructure Behind Running Trillion Parameter Models
I sat down with Rodrigo Liang, co-founder and CEO of SambaNova, in Paris at the RAISE Summit, on the announcement of the first close of their $1B Series F at an $11B valuation.. just 5 months after their last round.
→ Listen on X, Spotify, YouTube, Apple
The $11B Valuation Round
SambaNova completed the first close of $1 billion in strategic financing as part of a Series F round, valuing the company at $11 billion post money, led by General Atlantic with significant investment from Seligman Ventures, T. Rowe Price Associates, & Capital Group. New & existing investors include A&E Investment, Assam Ventures, Battery Ventures, Cambium Capital, BlackRock, Intel Capital, Kabila Capital, QFO Capital, Qatar Investment Authority, Vista Equity Partners, & Volantis.
“I’ve been in this industry for 32 years, building high performance chips for a long time. I’ve never seen the interest in semiconductors higher.”
SambaNova raised a $676 million Series D led by SoftBank Vision Fund 2 in April 2021, past a $5 billion valuation. The company reportedly reached roughly $100 million in annual revenue in June 2025 against more than $1.13 billion raised. Intel was rumored to be trying to buy it for about $1.6 billion. The $350 million Series E closed in February 2026, bringing Intel in as strategic partner and investor. Five months later the company is at $11B. Rodrigo puts total funding at $2.5 billion.
Liang said the capital is going to secure the supply chain, which he called the key constraint on fulfilling orders over the next 12 months, with the SN50 shipping in the second half of 2026.
Why Inference Is a Different Problem Than Training
Training is the one-time cost of creating a model.
Inference is the cost of running it every time somebody sends it a prompt.
“There’s no point of training a model if you aren’t gonna inference it, i.e. if you’re not gonna use it, right? You don’t invent a search algorithm if you’re never gonna do search.”
SambaNova started in 2017 with the goal of lowering training cost, back when the workloads were image recognition & voice. Inference wasn’t a problem then because the only people running the models were the people who had just trained them.
“Now what you’re seeing at scale with Anthropic, OpenAI, & with Gemini, you’ve got millions & millions of people using it every day. And so now you have the problem that SambaNova was originally focused on, which is around efficiency.”
“How do you actually deploy at scale so the whole planet can use it without burning up the planet, without running out of data center space, without blowing up your infrastructure cost? Because at scale, the number of chips deployed for inferencing will be orders of magnitude greater than whatever you’re doing for training.”
Inference has moved from a lab expense to a product gross-margin line, with per-token unit price down roughly an order of magnitude over two years while consumption rose more than 100x. SemiAnalysis reports inference providers including Fireworks, Baseten, & Fal are seeing widening margins on hyper-growth revenue.
The Rack Is the Unit of Measure
“Instead of 130, 140 kilowatt rack of Nvidia GPU, we were outperforming them with a 10 kilowatt SN40 rack.. air-cooled. You didn’t need liquid cooling upgrades.”
By comparison, the Nvidia GB200 NVL72 draws roughly 120 kW nominal with 130-132 kW observed at full load, against a 7.6 kW industry-average rack per the Uptime Institute in 2025. Air cooling isn’t viable at that density, and rear-door heat exchangers designed for 30 to 40 kW racks can’t handle the load.
Most data centers can handle 30 kW racks, more modern facilities 60 kW, and 120-140 kW almost requires new facilities. A rack that runs on air at 10 kW can go into a building that already exists. A 130 kW rack needs a building designed around it.
The SN40L rack uses 16 chips in a standard 19-inch air-cooled form factor delivering 10.2 PFLOPs. Each socket is a 5nm TSMC dual-die design with 102 billion transistors, 640 BF16 TFLOPs, and 520MB of on-chip SRAM, using a three-tier memory system of SRAM, HBM, and off-package DRAM.
“We’re using standard Kubernetes, standard Red Hat Linux, standard Ethernet at the top for networking.. And so that allows people to go in to existing data centers, roll this thing in, pull out the old gear, & you’re up & running with new services, which otherwise might take you nine months to a year, maybe as long as 18 months, to build a gigawatt data center.”
The Minimum Quantum
“With SambaNova, that minimum quantum is down to one rack. Where if you have other service providers, you just run, say, a DeepSeek model, which is now one and a half trillion parameters, just to run that, the minimum for some of the other providers might be 10 to 20 racks.”
The minimum quantum is the smallest amount of hardware you have to buy before you can serve a single customer. If it takes 20 racks to load the model at all, that’s the cost of entry regardless of how many users you have.
16 SN40L RDUs make up a single rack running models such as DeepSeek R1 671B and Llama 4 Maverick. Fifth-generation SN50 RDUs scale to 256 chips across racks, running models up to 10 trillion parameters with context length up to 10 million tokens.
“We take the biggest models in the world & we run them in the original precision. We don’t quantize. Quantizing is, you know, you chop half the weights off. So we don’t chop the model down. We just run original precision, full precision, run it faster than anybody else.”
Quantization compresses a model so it fits on less hardware, at some cost to accuracy. For Llama 3.3 70B, the same weights span roughly $0.12 to $1.05 per million tokens, about 9x, depending only on who serves them, with quantized tiers at the cheap end. Two providers can list wildly different prices for what looks like the same model because they aren’t running the same thing.
What Premium Inference Means: 2 Axes
1. Model size, which today is a proxy for accuracy
“It’s not that people wanna spend the hundreds of millions of dollars to train. ... They’re fighting for that bit of accuracy.”
His example was code generation. If the output is trustworthy, you don’t spend engineering hours checking it, and that saved time is what the buyer is paying for.
2. Speed, & the argument for it is arithmetic
“If each of those 20 agents took two seconds, that’s 40 seconds, you’ve already given up on that prompt, right? So the response time by the end user’s expectation is, say, 1-2 seconds. Divide that by 20, it’s 0.1 seconds per.”
Two seconds is fine for one person talking to one model. In an agent workflow where 20 models call each other before anything comes back, 2 seconds each is 40 seconds of silence.
“When 5G showed up, nobody’s signing up for 2G..
It’s never been the case, ‘Let me pay more for the slow.’ Right? Or even, ‘Let me pay a little bit less for the slow.’ Most people over time are gonna say, ‘No, I want the fastest.’”
“It’s whether today we can offer at low enough cost that everybody can afford it.”
Forbes reported in June 2026 that OpenAI and Anthropic are on the brink of a token price war ahead of anticipated IPOs. Custom chips including TPUs, Trainium, Maia, and MTIA cut inference costs between 30% and 50% versus Nvidia GPUs, with TrendForce projecting custom silicon runs 40% of AI servers by 2030.
Open No Longer Means Cheap
Two weeks after this conversation, Moonshot announced Kimi K3, a 2.8 trillion parameter mixture-of-experts model with 16 of 896 experts active per token, which would make it the largest open model ever released, ahead of DeepSeek V4-Pro at 1.6 trillion. Weights are promised by July 27.
Jamin Ball of Altimeter ran the pricing in Clouded Judgement. K3’s blended price is $5.40 per million tokens, against $2 for GLM 5.2, roughly $9 for Opus 4.8, and $10 for GPT 5.5. His framing is that “open weights were a cost lever and the ‘license’ was the discount.” At 2.8T parameters that lever mostly disappears. The weights alone come to roughly 1.4TB, which means 10-plus H200s just to load the model before you touch the KV cache, and Moonshot’s own launch materials recommend supernode configurations of 64 or more accelerators.
Open models were cheap because they were small enough that anyone could serve them and compete each other’s margins down. K3 is the first one big enough that only a handful of operators can serve it at all.
Gavin Baker of Atreides added the second variable. Per Artificial Analysis, Kimi K3 is 50-70% more expensive to run than GPT 5.6 per task, because what you pay is cost per token multiplied by how many tokens the model burns to finish the job, & K3 burns a lot. Baker calls it a “token wastrel.”
His larger point is that a market with 2 to 3 dominant frontier labs at 90% inference margins turns those labs into monopsonies for power, data centers, & semiconductors, so anything that compresses margin at the model layer pushes margin dollars into every infrastructure layer beneath it. That’s his explanation for why Jensen Huang champions open source. The same logic applies to SambaNova.
The K3 numbers line up with both of Rodrigo’s axes. The open frontier went from 1.6T to 2.8T parameters in one release, so model size is still climbing. Moonshot recommending 64-plus accelerators is the minimum quantum problem, stated by the model maker rather than by a competitor. And if open models are no longer cheap to serve, then whoever runs them at full precision on the smallest footprint captures the difference. What we find out on July 27 is whether third-party servers can undercut Moonshot’s $5.40 at all, given that the spread on serving identical weights already runs about 9x across providers for smaller models.
An open model needs the same compute as a closed model of the same size and architecture. The license discounts the lab’s margin and nothing else. That’s good for whoever sells the infrastructure and bad for the assumption that open weights deflate inference spend.
The Demo, Nvidia for Prefill & SambaNova for Decode
At RAISE, SambaNova ran MiniMax M2.7 on a setup that split the work between two kinds of chips. One Nvidia H200 rack with four GPUs handled prefill, and one SambaRack SN50 with 16 RDU chips handled decode. As benchmarked by Artificial Analysis, decode speeds reached up to 850 tokens per second on short-context workloads and over 450 tokens per second on long-context workloads.
Every request has two phases. Prefill is the model reading everything you gave it, which is compute-heavy and highly parallel, so GPUs are good at it. Decode is the model writing the answer one token at a time, which is limited by memory bandwidth and gets worse as context grows. Sending each phase to the chip built for it is what disaggregated inference means. The demo ran on vLLM, so providers don’t have to change the serving software they already use.
Agents spend nearly all their time in decode, since planning, tool calls, validation, and revision are all generation. Long-context decode speed is what decides whether an agent feels useful or stuck, which is the same problem as the 40-second math above.
The COMPUTEX version of this, with Nvidia B200 for prefill and SN40 RDUs for decode, delivered 2x the inference speed of B200-only configurations as verified by Artificial Analysis, is running live at Vector Core Compute’s data center, and Together.ai is the first commercial customer. MiniMax is the open-weights coding model Rodrigo names twice in our conversation. Artificial Analysis’ public MiniMax M2.7 provider page shows SambaNova leading measured output speed among tracked API providers, at a higher blended price than lower-cost GPU-backed providers. Fastest, not cheapest.
Calculating ROI: Revenue Per Rack
“[Inference providers] purchase per rack, they operate per rack, so they want to generate revenue per rack.
The revenue’s generated per token.. If I put a rack of hardware, I’m just seeing how many tokens am I generating in a particular model, and that model has a price per token, right? Multiply that by 30 days per month, 24 hours per day, number of tokens per second, & you can figure how much money that rack is generating, & you look at how much it’s costing you to operate.”
“We’re very focused on making sure that when you deploy a rack of SambaNova, you generate great margins relative to the model. And you can change the model, different models have different pricing per token, but you’re still generating significant number of tokens per month so that you’re making profit on that rack.”
“For them to sustain themselves, as you know today, inference services, they’re not making enough margin. They’re generating lots of revenue, but you’re not generating enough margin.”
SambaNova sells providers on routing inference traffic to its racks rather than replacing what they already own. In a facility where 70% to 80% of racks are running inference, “Why would you run inference on those racks when you can run it at a fraction of the cost at a higher performance on SambaNova? And so route that traffic to SambaNova, frees up all these racks.. And so the economics starts getting much better because now without buying more hardware, they generate more revenue.” SambaNova’s SN50 target puts numbers on it, claiming that at 500 tokens per second per user, B300 plus SN50 lowers output cost-to-serve versus GPU-only decode, improving provider margin at the same API price.
“People forget, as much as Nvidia costs, it’s commodity. Because what you offer is the same as what your neighbor offers.. and your differentiation is, ‘I can save you a little bit of money because maybe I got a discount from Nvidia.’”
“It’s not gonna be 100 different chips. Right? It might be two or three. Maybe three or four. That’s as heterogeneous as AI infrastructure is gonna get.”
Ok so, Who Buys This?
SambaNova doesn’t build data centers and doesn’t run its own cloud. “Many of the chip companies have chosen to go build their own cloud and compete with the AWSs of the world. We have chosen not to do that.” The company sells to three customer types, sovereign clouds, neoclouds, and enterprises building private AI deployments.
JPMorgan Chase will deploy SN40 and SN50 systems to run AI workloads on its own premises rather than through a third-party cloud provider. Rodrigo told CNBC that JPMorgan selected SambaNova to be the inference provider for the bank. The quote in the release comes from the CIO of Infrastructure Platforms, and the language is about testing speed and security, which reads as evaluation rather than deployment.
SoftBank Corp. is the first deployment partner for the SN50, shipping in the second half of 2026. SoftBank Corp. already hosts SambaCloud for developers in the region, and by anchoring new clusters on SN50 it positions SambaNova as the inference backbone for its sovereign AI initiatives and future agentic services. Its affiliate SoftBank Vision Fund 2 led the 2021 Series D. Intel entered a planned multi-year strategic collaboration in February 2026 alongside the SN50 launch.
Three sovereign agreements were announced in October 2025 with SCX in Australia, Infercom in Europe, and Argyll in the UK. Others include stc Group in Saudi Arabia, OVHcloud, over 30 enterprise customers including multiple US Department of Energy national laboratories, AWS Marketplace availability from May 2025, and an Accenture integrator partnership. Saudi Aramco & several Japanese firms are also named.
Vista Equity and Cambium announced Vector Core Compute, an agentic neocloud built on SambaNova hardware, in June 2026, and both are investors in this round.
Most companies that want fast inference buy an API, not a rack. SambaNova reaches them only if the model they want runs on its hardware, which in practice means open weights, or if their provider routes to it underneath. Baseten resells SambaNova as an Nvidia alternative in multi-tenant environments, and Together.ai is the first commercial customer of the disaggregated architecture at VC2.
Sovereignty & the Return to On-Prem
“I was talking to a CIO recently of a big bank.. they were never gonna go to a cloud, and it’s like, ‘See, I knew all along.’ I was like, ‘Yeah, it’s cyclical.’ A 20-year cycle, wait long enough, it’ll come back.”
He puts the shift down to margin rather than security. “If you actually transfer all of those services, that differentiation to all using the same exact model that’s in the community, where does the differentiation come? And so what you find yourself is in a place of low margin for all these enterprises that historically had been able to enjoy much better margins.” He thinks companies realize this within two years.
On national models, “I don’t want my data trained into a model and have that model shipped worldwide. Can you imagine if your bank account information starts showing up in ChatGPT in some other place in the world without your permission?” SambaNova sells a full stack of chips, systems, software, and cloud directly to enterprises, sovereign programs, and regulated industries, which its chip-only competitors don’t.
Asked whether $50B and $100B data centers make sense, “I think you’re gonna have some data centers like that, because I think there’s still going to be large scale deployments.. And I think you’re gonna find that the world’s gonna be heterogeneous.” His argument is about where the big builds can’t go. “In terms of ultra-low latency in the big cities, you’re gonna have to find smaller quantums.”
What People Should Be Asking
“Just using AI doesn’t differentiate. How do you differentiate?”
“It’s kinda like early days of internet. If we say, ‘What is the ROI for email? Well, show me.’” On where that goes wrong, “If you aren’t paying attention, it becomes token maximizer, just like, ‘Hey, just spend.’”
On scale, “The hints that you’re seeing today with energy constraints, data center constraint, chip availability constraints, cost constraints, all of those things are only getting exacerbated.” CoreWeave crossed 1 GW of active power in the quarter and expanded contracted capacity past 3.5 GW, with capex raised to as much as $35 billion for 2026 citing infrastructure component inflation.
“Business at scale is gonna come with the highest highs and sometimes the lowest lows, and you’ve gotta fight through all of it. It’s never a straight line.” SambaNova was worth $5B in 2021, was reportedly the subject of a $1.6B acquisition conversation, and is worth $11B now. “You can’t control what the world wants at any given point in time. You can’t control what the economy does. You can’t even control what the politics do. What you can control is your conviction around what you’re building and being resilient and staying with it.”
→ Listen on X, Spotify, YouTube, Apple
The material presented on Molly O’Shea’s website are my opinions only and are provided for informational purposes and should not be construed as investment advice. It is not a recommendation of, or an offer to sell or solicitation of an offer to buy, any particular security, strategy, or investment product. Any analysis or discussion of investments, sectors or the market generally are based on current information, including from public sources, that I consider reliable, but I do not represent that any research or the information provided is accurate or complete, and it should not be relied on as such. My views and opinions expressed in any website content are current at the time of publication and are subject to change. Past performance is not indicative of future results.
Paid Endorsement. Brokerage services by Open to the Public Investing Inc, member FINRA & SIPC. Advisory services by Public Advisors LLC, SEC-registered adviser. Crypto trading provided by Zero Hash LLC, licensed by the NYSDFS. Generated Assets is an interactive analysis tool by Public Advisors. Output is for informational purposes only and is not an investment recommendation or advice. See disclosures at public.com/disclosures/ga. Matched funds must remain in your account for at least 5 years. Match rate and other terms are subject to change at any time.