Groq COO Sunny Madra on the White House AI dinner, the American AI stack, and the Saudi inference cluster

for friend. com. Uh good luck to them. Well, we have our first guest of the show. We have Sunny from Grock infrastructure for inference, purpose-built for speed, quality, cost, and scale. Welcome to the show. Thank you for joining the show today. How are you doing? Welcome to the stream. I'm doing good, guys.

I'm uh actually at the All-In Summit, so it's like, you know, coming in. Yeah, exactly. So, I'm just in a green room here. So excited to be on with you guys. Did you already talk or is that coming up? No, no, no. I'm not talking. I'm just kind of like, but still in the green room. Still in the green room. Oh, okay.

We got I I I I had to get a favor so I could come on with you guys here. So, nice. I appreciate it. Uh yeah, take us through the last uh last week. I I believe you were at the dinner. uh what was kind of what was the what was the vibe like? Just just walk us through your experience.

Yeah, I mean really special like look so first things first I would say you know when you think about you know the president what he's really done is he understands that he needs to enable technologists to basically do their thing and I would I would you know summarize it in the following ways.

one he you know he brought everyone in and said look if you have some kind of issue there's a team of people that you can work with including David Sachs or just himself um to to get things unblocked right and there's many issues that companies deal with whether you know comes to export control or or you know tariffs that are being put on or fines that are being put on our companies and so he's really making himself available to to help our companies um you know broadly I think he's also understood that um you know AI and technology is the forefront of where many of you know the innovations for the country will happen including huge infrastructure projects right and so you heard that when the press came in and when they're asking you know the questions how much each company is spending you're looking at you know trillions of dollars of spend which has downstream impact in manufacturing construction and other places and so you know that that's something he's excited about and I think lastly like this we you know we really want America to win and he wants America to win and so he's basically saying look I've got your I've got your back.

I'm supporting you here and I'm basically, you know, putting this on the forefront. So, um, you know, that's the high level. I would say the experience just diving into it was really special. You know, started with a small gathering on the Rose Gardening Club outside. It was raining.

We were supposed to have a dinner there, but it got moved. Um, and then a bunch of us got, you know, all of us, sorry, got moved into the Roosevelt room, which is connected into the u into the the Oval Office.

And then, you know, he took his time basically meeting with each one of us quickly, getting everybody gathered together and just kind of understanding concerns that folks had. Um, and you know, basically us getting an experience in Oval Office. We all got a challenge coin.

We got a pen, which is really cool, like a little like a little, you know, um, a momento, a very special momento. And then, uh, we all went, uh, you know, it's the East Wing for a dinner. And, uh, you know, private conversation followed by a little bit of the press came in, which you guys would have seen the videos.

and then uh conversation afterwards and he really took the time to address you know everybody that I was at the table and and you know make sure that if anybody had a concern they had a chance to surface it there. What was top of mind for you specifically?

I think like look you know the American AI stack that's really important. That's something that was published at the AI action summit a couple you know basically middle of July. And so what we're really focused on is basically um making sure that we help the government understand what that stack looks like.

And so um you know I don't know if your producers can pull it up, but we published something a few weeks ago uh maybe two weeks ago now that basically our version of the stack and we think it's been positively positively received by the the government and the administration in terms of framework.

We're not saying this is everything. We're not saying it's comprehensive. We're not saying it can't change. But really, you know, the the, you know, kind of the Department of Commerce today is out there trying to understand what it should look like. And this is one view of the stack. And really, what does that mean?

Let's kind of double click into that. The stack is important because, um, if the US is going to keep export controls so that we can control, you know, highly sensitive technology getting into the wrong hands, you need to look at it from a stack perspective.

A chip alone is not something enough that you should basically, you know, put a control on. So you got to look at all the pieces.

And on the flip side, if you are going to be a partner of the US and you're going to want to buy US assets, you want to make sure that you have the whole list of things because if you don't have all the pieces, you're going to end up with paper weights, right? Really heavy and expensive ones.

So you need to have, you know, pieces along all and that's what we sort of took a stab at when we put that out there. Speaking of the AI stack, can you help me understand at a more precise level how you're positioning the company or or at least telling the story around the company in terms of where you fit in the stack?

because we saw last week OpenAI is doing a deal for custom silicon with Broadcom and then uh uh semi analysis came out with this uh this article saying that basically Anthropic and AWS and Tranium are like super tightly co-designed now Google obviously with the TPU and then and then I was I was chat chatting with some open AI people they're like you know we actually talked to Nvidia pretty closely like we're like we like I don't know if code design is the right heard, but like we give input and you know the next generation of Nvidia GPUs will be very capable of running chat GPT.

Um, and so so so where h how do you position where you fit into the the American AI stack? Do you guys mind if I pull up a quick screen share? Yeah, you can, but we are live. So anything you share will be shared. So do not share all uh your your private key to the those crypto holdings you got. There you go. Thank you.

That was smooth. Yeah. Yeah.

then you know um so the way to think about your question exactly can you zoom in a little bit is that possible or maybe we can on our side yeah maybe you have to do it on your side I don't know if I can but I I can try um you know really where that's called out at the bottom is um so ju just to answer your question first and I'll I'll try to zoom in uh without I think on your side um uh so really at the bottom there which is you know this is infrastructure and the bottom is compute so in the compute bucket Right at the bottom here you see Nvidia, Grock, AMD, Qualcomm, Broadcom.

Like I said, this is not comprehensive. You know, Broadcom is helping companies like Google and OpenAI and others make their chips, right?

And so um the idea you know to answer your question is that um the models come higher up but you should be able to look at this stack and say if you picked a piece from each one of these different categories that we've highlighted you can have an end toend AI solution to either train a model to either inference a model or to build an application on top of it.

Right? So those are the three things that this stack should enable and you may not need all the pieces depending on what you're doing. Right? If you're trying to train a model clearly you don't need databases. Right? But these are all the pieces that you need to basically do be able to do something in AI. Got it.

Fascinating. Yeah, this this Thank you for putting this together. This this document is massive and uh deserves its own like deep dive. We'll have to dig in deeper uh in another one. It's if you guys ever want to. Yeah, of course.

What's the update on the projects in Europe and then what what do you guys have cooking uh stateside? Yeah, you know, we've launched close to 14 different data centers this year. Um, including, you know, one that we announced publicly. Yeah, there we mad decent there.

Um, yeah, we've we've uh we've launched 14 this year and and we're we're continuing to expand our footprint. Um, you know, Europe Europe is really fascinating. You know, they've you guys saw the announcement with ASML and their investment into Mistral.

So, it looks like they're waking up and and starting to basically put resources behind these things. So I think that's great.

Um you know our f our focus primarily in that region of the world is centered around a data center and arrangement that we have in Saudi Arabia and so that's in in partnership with Humane Ramco digital where it's the largest inference cluster in the Middle East and we believe all the way into Europe and so we serve you know sort of most of Asia, Europe and the Middle East from that data center.

Um and our our goal right now, what we're cooking up is just an expansion of that, right? So we're working to expand that cluster um add more capabilities and basically more capacity to that cluster because it's it's been wellreceived and well consumed. Very cool.

Uh I know you have a heart out so we'll let you go, but uh last question for me on that Mistral uh ASML deal. Uh it it kind of jumped out to me because it feels like they're almost like jumping a layer of the stack.

you know, like I typically think about like ASML sells to TSMC, which sells to, you know, Nvidia and some other folks, and then and then the application layer and the foundation model labs like sit on top of that. So, you're kind of a couple couple steps away. Um, do you have any insight into what they were thinking?

A lot of people are just saying, "Hey, it's a national champion. They want to support and they're willing to kind of go outside the typical business that they do to to make an investment.

" and or maybe they just see it in purely financial terms, but it seems like if it was purely financial, they would have there there's a bunch of other things that they could invest in. You know, they're not a hedge fund. Yeah. Right. Yeah. Yeah. No, totally. Um I think the points you talked about are valid.

I would add one thing to it, which is if you actually go back and listen to the launch of XAI, what Elon talks about is like, you know, companies and his companies primarily using, you know, AI to better themselves, right?

And I think if you're someone like ASML, if you want a close partner, um, doing that with Mistral is quite smart, right? How can they improve the tools that they make so that we can get, you know, better chips, better stuff in that layer of the the stack.

And so I would say that's something that is getting underappreciated today, but more and more companies are doing that, right? Getting more advanced AI so that they can basically do that. And and from their perspective, they're they'll never be able to do that on their own. To your point, they're too far down.

So the partnership there maybe we'll get better machines, maybe we'll get better power consumption from those things. Maybe we'll get you know better lithography from them.

So that would be something that that I I would you know if I was building a business outside of you know Grock today that's something I would focus on is how can I you really not just use this for the high level use cases we've seen but really have it you know find new materials for me uh look at you know different aspects of physics and those are really hard problems that you need to be very closely embedded with a model maker like a foundational modelmaker if you really want to pull those off.

Fascinating. Uh well, we will continue to monitor the situation as always and thank you so much for hopping on. Enjoy the rest of the convers and the updates. We will talk to you soon. Have a great rest of your day. Thanks for having me, guys. Cheers. Yep. Let me tell you about Adio customer relationship magic.

Adio is the AI native CRM that builds, scales, and grows your company to the next level. You can get started for free with Adio. Uh Theo, we have cognition. One thing, one thing I wanted to highlight quickly because we didn't get to it earlier.

T-Mobile, AT&T, and Verizon all dropped pretty meaningfully on the Echoar news. Yeah, that makes sense because basically going around T-Mobile. It won't need to go through them uh because they'll own their own piece of the spectrum. T-Mobile's market cap. Um 20 billion. Close. How much? 273 billion. 273 billion. Wow.

Beast. Okay. I uh how much bigger than the stock own? So Verizon SpaceX stock question. No, Verizon is 1882 billion. I didn't realize T-Mobile was bigger than AT&T is 206 billion and Verizon is is bigger than all of them. Wow. It's almost like uh it's almost like owning a monopoly is valuable. Yeah.

Seriously, like once you get those licenses, it just prints. Yeah. So, it's so crazy that Echoar was like on the verge of bankruptcy, like not making debt payments and they're just like yolo and they weren't even leveraging the spectrum that they had. No, they just I just it fascinating.

Um, you got a post from uh I I'd rather I'd rather talk about this Andrew Macall story. So, Project Bob has launched. You can go to dashboard. probob. xyz. Uh Christian Kyle says this is easily the coolest thing happening in the world today. a hobbyist launching an autonomous drone ship, hoping to circumn the globe.

Andrew, of course, works at Varta Industries on manufacturing in space. But in his free time, he tried to replicate the LK99 superconductor. I was there on the night that he tried to get the rocks to float. It was very, very interesting. Uh he's just a fantastic scientist and and builder.

And I guess in his free time, he went and built a a boat that drives itself, talks to Starlink and a few other networks and wired it all up. And who knows, maybe uh maybe it turns into a to a business. That's the nature of of launching side projects.

If you're really good at them, eventually people will say like, "Well, I'd like to buy some, sir. " And maybe uh maybe the DoD will come to him or the DOW, as they're called. I'm so I'm so excited for the the drone era of exploration, right?

Just being able to send something off and and follow it along live virtually is so cool. I always like I I I want Andrew to take this and and uh properly send Antarctica