Extropic's Guillaume Verdon on thermodynamic chips, a US government LOI, and why GPUs aren't the endgame

Speaker 1: Long overdue, but very excited to have him on the show. Guillaume, are you doing? Welcome to the show.

Speaker 8: Doing great. Doing great. Long time fan, first time caller. So So excited to have you on here.

Speaker 2: Congratulations. During a massive moment. Massive moment. Thank you. You. Good time coming. Give

Speaker 1: us the state of the union on the company and then the news and the deal that's going on, the letter of intent.

Speaker 8: Yeah. I mean, so, you know, at XTropic, we've been pioneering this new form of computing from the ground up called thermodynamic computing. You know, our whole thesis is that right now, everyone is focused on scaling things up, buying more GPUs, bigger data centers. We wanna scale things down. We think there's gonna be a race to densification of intelligence. So we started that about four years ago, which was very early to worry about energy. It sounded crazy to say we're going to run out of energy four years ago, but here we are. And so we've reinvented how to leverage the transistor for the era of generative AI. So generative AI, for those not familiar, is a probabilistic algorithm. Right? You're you're sampling from these distributions of like, okay, if I give you if I type in the word cat, what sort of image do I get out? It's it's kind of random at the output. And so that is a probabilistic workload. It turns out you could run electronics probabilistically and at at much lower power, and we pay a huge price to maintain our electronics in a deterministic state. So it doesn't make sense to run probabilistic workloads on deterministic electronics if we're going to pay a huge tax to know the state of our computer at all times. So our insight is if you use very little power, essentially your computer can operate probabilistically and you can train it directly just like a neural network, And you can map your algorithm that is a generative AI algorithm like a diffusion model directly onto the physics of the hardware. And so that's kind of the vision, and it's been it's been a journey essentially. Yeah. You know, a year, year and a half ago, we had our our first prototype in in silicon. Yeah. There you go. X zero. And this one we taped out with with TSMC. We've also now done global foundries, that that demonstrated that actually because when you operate electronics probabilistically, you can use far less transistors for these these algorithms. You don't actually need the smallest transistors. Mhmm. And right now, the reason people have to go offshore is because the best cutting edge fabs are all offshore and they have the smallest transistors. And so what we demonstrate is that in principle, we can manufacture these chips in The US. And that was very interesting to all sorts of folks higher up. And, you know, we we had some conversations and, you know, there's a there's a very strong history of, you know, the government supporting the silicon industry. I mean, Silicon Valley was was DARPA. They created the Internet. Right? Exactly. Right? And and and now, you know, we're we're we're trying to go with or I guess, they're going we're going with a modern twist where, you know, if the if the the taxpayers are gonna support r and d, you might as well get some upside as well and and get some equity. And so this is what this this announcement is about. It's with the chips r and d office and Cool. And, you know, we'll we'll get into it. But Yeah. Yeah. In terms of where

Speaker 1: you want the first applications to be, it feels like different models are maybe sometimes designed around certain architectures. You see certain like, GPT OSS runs really well on Cerebras, and then you'll have another model that runs on an NVL 72. It's like RackScale. And then there's other models that can sort of run locally or run on the Mac Mini or Apple Silicon. And there's all these different pieces of the puzzle. And increasingly, we're starting to see where, like, voice models might be really good for those to run locally, but some crazy long running AI agent might be may be fine to send off the server. Do you have a an idea yet of where you think if things go perfectly, the first application might be? Like, where's the sweet spot?

Speaker 8: Yeah. We've been exploring a lot of applications. Obviously, running the model itself, the transformer is a big one. What we found is that and we have a blog post coming soon. I guess it's a scoop here, but we have a blog post coming on on a sparsity scaling law. So if you sparsify your model because our our chips have a sort of sparse structure, so that means sparse is just, you know, print GPUs, they have big matrices, which are these grids, and and basically every entry in the grid is is busy. Right? Like, there's there's something going on. Sparse means there's a lot of zero. There's a lot of stuff not going that's not activated. Yeah. So our chips are really good at sparse operations. That's not originally what we're designed for, but if you really wanna shove a transformer on our chips, you can do it. Mhmm. And if it's sparse, as long as it has the same number of parameters, you could reach the same sort of performance, which is interesting because for us, the flops are much cheaper per watt. Sure. So even though you use more flops, you get more intelligence per watt. So we have something coming out soon on that and teasing it here a little bit. But overall, we're really interested in diffusion models, you know, models for predictive control, for defense, physical intelligence, signal processing, you know, stuff you would use an FPGA for, you wanna use one of these chips. So, you know, we have RTX as investors for that for that reason and similar defense primes. Cool. Very interested. Yeah.

Speaker 1: How are you feeling about the retrospective on effective accelerationism? Because it feels like my interpretation of it is that you were 100% right at the time and that class of models didn't really pose any real risk. There was clear need for a build out and diffusion of this technology. But now we're in this, like, new the last few months, there's been a new discussion over literally slowing down, literally decelerating, and it's coming from inside the leading labs, not just, like, doomers who are outside. How have you reflected on what Yak was, its role now, just AI optimism versus pessimism,

Speaker 8: risk. How are you thinking about all that now? Yeah. I mean, originally, I wanted to just bring balance to the force. It was kind of modern culture in Silicon Valley. It was just just the doomer camp. And I, you know, I was seeing the writing on the wall. If we just had the doomers, then that would eventually affect policy. And and to me, that was a very biased view of the world. And so I think we kinda stretched the Overton window. It was okay to Yeah. Say no to do doomers and say that's a ridiculous model of the future, you know, great goo, foo, and all that stuff. Yeah. But now, yeah, we've reached a point where, you know, the models are very capable and, you know, the EAG view to me is just like viewing the whole world as a complex self adaptive system. And ever since the dawn of time, it's basically been PVP. Every form of life is becoming more complex and harder to predict, and then its adversary has to step up, get smarter in order to predict it, in order to compete. And if you do that slowly, then basically you can have a whole ecosystem and everything's good. But if one one thing becomes much smarter and much more complex than others, then it can kind of control others and that's bad. And so I stopping makes absolutely no sense. Pacing can make sense, but really has to come with a really strong investment in, for example, cybersecurity and hardening things because you can't pause things forever. Essentially, you just want to be adversarially robust and and I'm all for that, personally. It is very interesting reflecting on the Doom arguments,

Speaker 1: how they were like, to your point about Grey Goo and Foam, it was not as precise as as cybersecurity risk, which is some people were calling out, but the conversation was definitely distracted from the more practical, like, difficulties about around slop and, you know, overinvesting and bubbles and all all these different things that that are much more short term that, you know, America will need to grapple with to actually, you know, deliver a positive outcome here. How are you is your development process being accelerated by AI?

Speaker 8: Oh, absolutely. Yeah. Yeah. No. Absolutely. I mean, we we use all the big big model providers Sure. In house and we try them all. We have agents constantly running. You know, for us, it's like we have to speedrun deep learning. It's like you you teleport back to 2011 Yeah. You know, and there's no AlexNet paper, and you have to speedrun ten years, fifteen years of deep learning progress as fast as possible. Sure. Right? But now you it's like new game plus in a video game. You have all the power ups. You have you have the AI. And so we're speed running progress Yeah. Very quickly. And and, you know, we have online learning agents. We're training our own models as well or post training rather, sorry, the cutting edge companies do, and so that's very exciting to me that, you know, current AI can help bootstrap and kick start the next paradigm, the next substrate. And going back to your comment about the cycle, you know, my thesis was that, you know, GPUs are not the end game. And so, you know, we wanna invest in the current cycle, but we should we should hedge our bets. You know, as we saw today with Leopold, RIP, you know, you gotta hedge your bets. And I think for all in on GPUs and we don't invest in the next generation, the next paradigm of computing and have a couple bets there, then when that pops up, you're gonna get wiped out, and we don't want that. And so to me, we're starting the next s curve. We're starting the next cycle. I know people are not over the current cycle, but everything is cyclical. But, you know, it's my responsibility to make sure, you know, we can we can have build outs in the future that are that are really power efficient and, you know, it gets you better return on investment on your capital, whether it's on Earth or in space. Yeah. Or, you know, at the edge.

Speaker 2: Yeah. It feels like the one of the you know, you've been one of the loudest, biggest voices on on X for the entire time that we've been running the show. We've, you know, had a bunch of your posts on the show over the years, and it feels like you picked a you picked a product category that just by the nature of it was gonna take time to evolve. And I think like, when I think back, it's like the challenge of showing progress. I mean, this moment is massive. Right? It's a a it's a vote of confidence and and it's very exciting. But everything that it took to get here, meanwhile, you're getting like hundreds of millions of views from some of the Billion. Yeah. Yeah. Billions of of views. And I'm curious, like, yeah, reflecting on the last two years, do you feel like you've found the right balance between you wanna be getting attention and you wanna maintain your voice, but you also wanna be moving the business forward? At any point, know, X and Teapot specifically, if they feel like those two things are not balanced, they're gonna, like, pounce and you know, take shots and all that stuff. And I know you've gone through that, which is why I'm excited for for this moment to to for but, yeah, how have you processed it all?

Speaker 8: Yeah. No. It it it's been a it's been a journey. It's been a lot. It was despite, you know, allegations, it was never planned to get doxxed. And, you know, I knew I was going after a very much a deep tech moonshot that was gonna take some time to cook. But, you know, exponentials are slow at first and then they compound over time. Could I have had, you know, a product that, you know, would have had a shorter time to market and and use the heat to make it grow? Probably. But, you know, I I was just dedicated to this this one mission. I I I think it's the most important thing I could be working on, and I've been all in since the beginning. But, yeah, there's certainly a tension, like, building in deep tech where you're supposed to be in stealth, you know, you have Yeah. Nation states trying to reverse engineer technology, and you're you're trying to just reveal the the minimum because that's your secrets is are your edge. But at the same time, you want to get buy and you want to shape people's beliefs like, hey, actually, this technology is here. It's coming. Here's a hint. Here's a prototype. Here's a nature paper. Here's this. Here's that. Here's some signal. We're gonna have a lot more coming up next week, a big announcement for

Speaker 1: z one and and all those products around it, including our stacks. I love that framing of new game plus. It feels like there's so many founders from the previous era who grew big like SaaS companies and they're now sort of on New Game Plus moving a lot faster. It's a great formulation, great philosophy, and thank you so much for coming on the show and breaking it down. Yeah. Congrats on all things we're having. Yeah. Great to to you soon. Have a good one. Goodbye. Cheers. Let me tell you about Railway. Railway is the all in one intelligent cloud provider. Use your favorite agents to deploy web app service databases and more while Railway automatically takes care of scaling, monitoring, and security.