Etched raises landmark round from Jane Street, Jump, Two Sigma and Peter Thiel to build transformer-specific AI inference chips

every company in the in the co-working space potentially. I don't know. We'll figure it out. But fortunately, we have our next guest from Etched in the waiting room. We're very excited for the launch of this company. I did an interview with Gavin. Uh what what was that two years ago? Something like that. You were laying out the the idea for this company very early on. You made a ton of progress. Uh take us through the story. begin at the beginning, introduce yourself first.

Well, uh I'm Gavin. I'm the CEO and one of the co-founders at Etch. Yeah.

And we are building rack scale inference hardware.

And what that means is that well we're building the full system

that is chips, boards, platforms, racks and clusters as well as of course software and importantly the production lines.

Wow.

To build those things at scale. So the original viral sound bite that got so much attention around your company was the idea that you were going to uh bake the you were all in on the transformer. You said AI will advance but the transformer is here to stay. That architecture will be here to stay and so we are going to fully commit to the transformer and we're going to sort of etch that into the chip. How real is that today? has that that thesis feels like it's been borne out very well, but it feels like you've added a lot of new strategies to the portfolio. How are you thinking about the the the bet that you made two years ago?

We looked at a lot of early research directions and we realized the key things that models need are way more compute and way faster memory.

If you think about inference, there's two key parts. Prefill and decode.

For prefill, it's a computebound problem. You need to have more flops, more operations per second on each of your chips.

Yeah.

And on our GPU, the bottleneck's actually thermals. You can't really run a GPU more than around 50% of what it could theoretically do or it'll melt. So, we're introducing a new technology today called low voltage inference to try to solve this problem. And what that is is we bring the voltage of the chip down dramatically which allows us to have way way better efficiency in terms of how much power is drawn per unit of math and thus fit way way more flops onto the chip.

Why can't pausing there? Why can't you super cool the chips liquid nitrogen or dry ice or something to make it colder so that you can run it at 100% power?

It's a great question. If you look on the internet, there are some videos of folks trying to super overclock CPUs by using these things on them. The problem is you actually can't go a whole lot above where they're meant to operate.

Mhm.

For example, virtual CPUs can go ahead and do four or four and a half GHz.

I think the world records on the order of 8 n GHz with the liquid cryogenic helium.

And the problem is you have to get the heat actually out of the chip that you can't physically go touch the resistors with your uh liquid nitrogen. you have to go ahead and go through the silicon itself and that by itself has a good deal of thermal resistance.

Unfortunately, there's no way around it.

You have to solve the power problem if you want to run inference.

So, low voltage and then the other side of the equation. Take us through that.

Well, for decode, it's all about bandwidth.

Mhm.

And it's not just bandwidth on a chip, but bandwidth across your cluster. And that's why we have this technology that we call cluster scale memory.

Mhm. It reduces the amount of time it takes to communicate from one chip to another dramatically. And as a result, we can go use all of our HPM and HBM bandwidth and SRAM and SRAMM bandwidth and our scaleup domain as a single coherent pool. And that means if you're a user, you can go get much faster tokens per second speeds while still keeping your costs low. Mhm.

How are you and the team dealing with various bottlenecks across the supply chain?

How did you get TSMC to work with you as a startup?

TSMC has been a fantastic partner to us that we are so so grateful and one of the great things about them is they're very technical. You can just sit down and say, "Hey, we believe this will be a big market for these reasons, and we need your help to figure out how to go co-design transistors that can run at a much lower voltage." And that was a particularly challenging piece. As you bring the voltage down, manufacturing gets way, way tougher. We would not have been able to do it without a great fab.

Where else are there bottlenecks right now that you're experiencing? I think all across the chip supply chain, there has to be much more buildout.

People don't realize how big of a market interference is going to be. Right now, only a couple million people have access to the frontier models.

And those are going to have to get deployed to 8 billion people across the globe.

Mhm.

And that's not even counting AI agents that exist already and will exist in the future.

There need to be way, way more tokens produced. And that requires both much bigger assembly lines, many more factories, and some new technology.

Two years ago, you predicted that AI would not kill everyone within two years. You were correct about your prediction. How are you feeling about the the general prospect of doom?

I mean, I don't understand it. I think right now is an incredibly exciting time to see all the breakthroughs that are getting made.

Things like the unique distance conjecture,

stuff I learned about in college and to see that now get cracked by an AI model. Man, what a time to be alive and we are still so early.

Yeah,

models will keep getting smarter and like

man, I I could not be more excited for the moment right now.

Yeah. So, what is the

why was why was like now the time to come out of stealth? You clearly have had no issues getting getting customers. Sounds like there's a a billion dollar uh pipeline that you're working through. Uh but what what made you decide to to talk about the business today and and what are you hoping uh to get out of this this uh little media tour?

Uh it's all about talent. I really strongly believe that our company's best asset is the people we've been able to hire. And I'll give a couple of examples. Look at our platform team. Around half of them are from Nvidia.

Mhm.

Or if you look at the uh VP responsible for designing that thing behind me that used to go run Nvidia's HDX and DGX programs that made up most of their revenue for a hopper and black ball. And people like that should move the needle so much more a business especially in a hard space like semiconductors.

And not to mention we're now working on our road map.

Finally, I have Gen One. If you want to go do multiple generations in parallel and take really big ambitious tech bets, you need the best of the best.

Are you a car?

A car. Are you a car? There's a debate around whether modern large chip companies are interchangeable with one another. Whether the moes around the software

if you have a logistics business, you can buy trucks from multiple vendors.

I can get a new car tomorrow. I in fact I am demoing a new car right now. I got in it. I connected my phone over the Bluetooth and it works just like any other commodity product. uh that has not been the pitch in semiconductors for years. For years, we've been told about Moes, about how complex the software is, how hard it is to write efficient software that runs on a particular chipset that you had to be all in on one ecosystem. And with the advent of agentic coding and insanely high rewards for figuring out how to run on other A6 or other other chipsets, uh it feels like chip companies are becoming more carl like. How do you think about your moat as you go into the future?

Well, uh I would say a Toyota and a Ferrari are not the same thing.

You look at the speed gap there, you see what a 2x.

Mhm. And I think chip similarly is very very meritocratic. If you're able to go out and be faster, you can go command a huge premium just like uh the fastest cars can.

Mhm.

The difference here, the way cars work under the hood is very similar. There are no more connectors to be had in cars.

Mhm.

I was thinking chips over the next couple of years. There's going to be a lot more technical innovation. And I'm very excited to go ahead and show the world, not just the current gen, what the road map for these things looks like. We believe we can go push low voltage inference and cluster scale memory much further and get even bigger unlocks here.

So next time you get that question, say I'm not a car, I'm a Ferrari.

Doesn't make any sense. Um, so so is the source

makes perfect sense.

Doesn't make sense. uh is the is the source of of of strength is the reason that you will have positive margins in the future. Uh intellectual property, trade secrets, patents, like what is the shape of owning that technology and not just being copied and commoditized by other chip companies?

I think it just comes down to those things matter. We of course do great trade secrets and patents,

but you need to build a lot of products

and you have to go out and have really great economics as well.

Yeah,

being faster isn't enough.

You have to go be able to do that at a price people can afford.

You see fork stretched thin today.

And that's why when we think about doing the thing, we want to go do the whole hard thing.

That means building not just the chip, but the board, the platform, the rack, the cluster, and especially the production. What is the last AI innovation or product development that has updated you on your strategy if anything? I'm thinking about the transformer-based LLM, obviously a huge moment. Uh but then we got diffusion models and image models. We got agentic software that requires CPUs and and other memory constraints. You got reasoning models. Has there been any development where you've uh either said I'm glad we're on the path we're on and uh we need to double down on that or we need to slightly adjust our strategy to deliver the best performance for the latest and greatest use case.

Well, the thing that's been the most exciting for me has been seeing the models get so big

get so big and demand be so large.

Yeah. When you're thinking about economies of scale, it all just comes down to how much is there to go serve. If the market's relatively small, you justify small factory, but not some gigantic mega cluster. And with what we're seeing right now with these many many trillion parameter models with these uh quadrillion token uh demands that we're seeing that are increasing every month there has never been a better time to go ahead and invest in those economies of scale.

It's great news. Last question. um the customer are you focused on serving hyperscalers, the frontier models, neoclouds, open-source models, all of the above. How do you think about that?

All of the above.

We want to go out and do is build tech that you can go ahead and see that'll make your tokens faster and your tokens cheaper.

And that is working with the full stack.

Well, congrats.

Talk about the round. Who participated? We have a gong and it would be an honor to hit it on your behalf.

Well, I am blown away at the quality of investors we've been able to get on the cap table. Folks like Jane Street, uh folks like Tech Alliance.

Yeah, it is it is a very interesting uh group. I mean obviously a lot of uh a lot of traditional venture capital firms that focus on technology.

I was I was ready to do one hit for Jane Street, one for HRT, one ship for Two Sigma, one for Jump, one for Scott, one for Peter Teal, one for Hinton.

Scott Woo Scott,

Patrick Oan,

what a great

Zell.

Absolutely.

I think the people are fantastic.

Yeah.

Miy Kareem.

Wow. This is I I the mo best best party round of the year. You really put together a good list.

It's a Coachella poster. Well, thank you so much for taking the time to come chat with us.

Yeah. So great to meet you and congratulations to the whole team on the launch. Looking forward to the next conversation.

We'll talk to you soon.

My pleasure, guys.

See you later, Gavin. Goodbye.

Great. Cheers.

Let me tell you about the New York Stock Exchange. Want to change the world? Raise capital at the New York Stock Exchange. You know who won't be raising capital at the New York Stock Exchange anytime soon? these fly by night peptide companies because the peptide battle has commenced, says Jessica Adams. FDA recommends against adding all seven peptides under review to the 503A bulks list. Is that for bulking up? Is that a list of things that get you bulky? No, I think it's things that cannot be produced in bulk. But the FDA has has effectively delivered a a blow to BBC57 for ulcerative colitis, KPV for wound healing, TB500 for wound healing, MOTS- C for obesity, and osteoporosis, Midilide and DI DSIP for opioid withdrawal, CAX and Epialeon. You can tell I do not know my peptides by the way I'm pronouncing these. Anyway, um, a lot of people are fans of this stuff. Uh, sorry if this is a dark day for you. Uh, a lot of people are in the camp that, uh, the FDA serves a functional purpose here to review these things and make sure that they are effective and safe for public health. And it's a blow for them.

Ben texted just about to take all of these.

Uh, no. Yeah. Um, a lot of people are going to be upset here. I think a lot of people would also say that this is like maybe good if um uh you know there's could be some very real downsides with with any of these. Uh I was texting with a friend uh in healthcare this morning. I said, "What does this mean for peptides?" He said, "Danszo." Uh and I said, "Can't sell anymore." He says, "Unless RFK overrides the FDA, not legally." Then I said, "But they were already in a gray area. Now they are explicitly banned." Uh, question mark. He said, "They've been banned other than research use and haven't been enforced for human use." So, they've been banned, but there hasn't been enforcement. Uh, justification for taking them off the ban list was to kill the gray market. A lot of people have been taking peptides from random websites online. they either don't even have active ingredients or they're contaminated uh or yeah they're effectively just just nothing and uh he says I think the real companies that are were moving into the space net new in anticipation of this changing are in a real pickle sole

um yeah so we'll see they now face a newly constituted pharmacy compounding advisory committee making the July meeting one to watch that will be July 23rd to 24.

So, we will keep following.

Well, in other good FDA news, uh, nicotine pouches finally got MRTP, which is modified risk tobacco product. Zen is the first one to go. And Zinn can now market and put on the label and advertise that it is better for you than cigarettes, which is such a mild claim, but it's huge for the industry. If you know what this means, it's an extremely high bar. They have to run a bunch of tests and whatnot. Uh, every product in the category currently says this product contains nicotine. Nicotine is an addictive chemical, but the products for ever have not been able to say this is better for you than smoking, which is the obvious benefit that every company wants to make. They want to make that claim. There's a ton of literature out there. A lot of it's from Sweden, a lot of it's international. The FDA has to review it on a per product basis to make sure that your particular product maps to the data because the the Sweden data that everyone points to is about snooze, which is a tobacco product. It's not exactly the same as as white nicotine pouches that actually don't have tobacco in them. But anyway, the FDA has to review each one. They reviewed Zen and they gave them the MRTP approval. It's huge news for the industry. It's very good news. It's good news for everyone, I think. Uh anyway, the other news is that Nvidia and Eli Lily are building an AI lab, $1 billion lab for drug discovery in San Francisco. Eli Liy CEO Dave Ricks talks about their data advantage. some scale bio tech bio players are just training on public data but there's only 4,000 ever approved drugs Lily alone has three million failed drugs we're we're learning from failure uh which is interesting so a lot of big a lot of big talk about the next biotech boom there's acquisitions going on and thropic bought a company there's all sorts of predictions about the impact that AI will have on bio whether