George Hotz: humanoid robots are 'as dumb as self-driving cars was,' AGI is a meaningless buzzword, and Waymo has more than 1 human per car

George Hots. How you doing, George? Good to hear from you. What's going on? Welcome. Can we hear you? Oh, can you hear me? Yes, we can hear you. Gotcha. Uh uh let's kick it off with something uh simple. I want to take your temperature on uh AGI timelines, pdoom, the the easy and fun stuff.

I don't know what AGI means and I don't know what you mean by doom. No. Is are are these terms just like entirely irrelevant? I mean now we've shifted to like super intelligence.

They're all buzzwords, but at the same time like there is there is an idea of like the like I don't know that that the conversation is maybe shifting to like the AI generating more economic value than humans. Is that a relevant metric to track?

Machines have been generating more economic value than humans since the industrial revolution. Is there some is there some other metric that that we should be tracking or is it just like irrelevant? You're just talking about like hype. Like I don't know. I mean I I like I I don't know what you mean.

Like you can talk about concrete things. The term like AGI means nothing, right? Like computers everything that's a touring machine is a general purpose computer. Is that what you call intelligence? I don't know what you mean. Is a linear regression intelligent? What if it's big enough? The Chinese does know Chinese.

Yeah. Um, what what I mean what about uh uh your your decision to get on a spaceship traveling at 0. 9 C away from the from the Earth? Like how close are we to that? Are we closer than the last time we talked which was like a couple years ago and it seemed like it was maybe going to happen within your lifetime.

Has it moved at all? Yeah. I don't know. I don't know if I'm actually going to get that spaceship but it's kind of like in an ideal world what I would want to do, you know? Yep. Just just just back away and chill and don't look back. Actually, you can't look back. They're all there.

You need the you need the blast shield, right? You need the information shield. Information. What do you mean? Oh, that's how they're going to get you. Okay. Right. I mean, okay. So, like here's a way you can think about AI, right? Yeah.

Um, imagine there were 10 CIA agents assigned to you and they're running at a thousandx real time. So, they're like hyperfast CIA agents that devote their entire lifespan to your day. Mhm. And they're trying to manipulate you. Maybe to get you to buy things, maybe to get you to vote for a certain guy, whatever.

But like that's what you're going to be up against with AI. What we're currently building, what what what if you think about the biggest companies in AI, what they do is advertising. What advertising is is just manipulation of humans.

Um, so you're going to have a team of CIA agents thinking about you and trying to manipulate you at all times. And now you see why you want to head away at the speed of light, right? Mhm. Even CIA agents can't do that.

Is there is there some world where there's like a capital war and I'm paying for a more powerful ad blocker? Yeah, I mean that sounds good.

Like another question is kind of to say like okay if you think that you either think that current like capital uh accumulation dynamics are going to continue and that uh the rich are going to continue to get richer and if you believe that the question is kind of well how many people are going to survive in the future?

How many people are going to have any modum of independence? Mhm. Um, right. Like you have some far AI people who think that there's going to be a singleton, right? Think that there's going to be literally one, right? Um, you know, some people maybe think it's 10, some people a thousand, 10,000.

Uh, some people think that all the humans will get to continue to exist as independent entities. Are they already independent entities? That's a question, right? I don't know. Good question.

Uh I mean if you were try to put it in like the form of a bet human population above or below 8 billion in 2030 above I think I would just give you a normal trend. Oh is that what the trend says? Yeah. Just go the trend says I don't think there's going to be any discontinuities to any trends really.

Well yeah I mean I mean at some point uh but but the question is like how far out do you have to go until you start seeing these effects? What do you mean by human? Right. What about someone who lies in bed all day and watches TikTok? Are they human? Yeah, that is odd. They kind of drop out of society.

I think I think uh question that that popped up for me is is this uh all this debate about AI safety and what should labs be doing? What should labs not be doing? It feels like your angle is it should be each individual's responsibility look to look after their own safety in the context of of AI. Is that at all?

I I just I I just like this whole like should shouldn't like what I don't know. I'm not a sadistic [ __ ] who wants to manipulate other people like the people in power. Like I don't know. Yeah. But I mean people still look to you as like an example of like uh someone who might have uh answers.

No, I don't have any answers. Not not necessarily answers. Just like uh but you can buy my shitcoin here. Did I shield a shitcoin? Here you go. Just click this QR code and you can buy a George Hots coin and that will give you answers.

You will find satisfaction and fulfillment in your life after purchasing a George Hots coin. Is that Is that the end state? We all have our own coins. I guess. No, no, no. I I don't mean it like that.

I mean, like I think that a lot of people are like they don't really know what they're looking for and that uh vacuum is is a very uh you know it's very dangerous and it's going to be filled by dumb [ __ ] and don't have that vacuum, right? You got to you got to stand for something, you know, or something. I don't know.

Yeah. I mean, do do you think that there's a chance that someone is able to take a stand and and actually uh bend the arc of of AI progress in the way that uh I mean it happened with nuclear, right? Like like nuclear development did stall.

There was a stagnation in real world buildout of nuclear capability on the energy side. Yeah. I mean there's a few things about nuclear that make it different. Uh so nuclear uh even as a weapon is incredibly hard to deploy tactically, right?

So so if a country has has nuclear uh weapons, they're aside from like a mutually assured destruction idea, they're not all that useful. It's not like you can use a nuclear weapon to accomplish tactical objectives, you know, if you could I think Russia would have already done it. Yeah. Right. Right.

Russia has some tactical objectives they might want to accomplish, but uh nukes aren't really going to do it, right? I mean, from a pure real politique perspective, not even from a uh like uh oh, like a taboo moral perspective, like what do you want in a radiated pile of rubble? Like that's what you're going to get.

No, what you want is drones that are hyper specific and can take out exactly who you want, can control areas, right? So like as a military technology, nukes are not that good. AI is way better. Yeah, but what about as an energy technology?

It feels like the it feels like the fear like the mimemetic fear of nuclear war and total destruction caused a whole bunch of regulation to pour into a sector and essentially a stalling of nuclear energy buildout.

And if if if the AI doom scenario, whether it's real or not, becomes so mimetically powerful that someone's able to harness that and actually say if you try and build a big data center, we will shoot you. Then maybe it stagnates. No, I don't really think that's the reason for nuclear.

I think it has more to do with why we can't do other big infrastructure projects in this country, right? Like it doesn't have to do with the new we also can't build dams, right? Yeah. And if you look like that's the thing, if people think that there's some weird taboo around nuclear, right?

But then okay, look at hydroelectric, right? There's no taboo around hydroelectric, but China leads in installation of both nuclear and hydroelectric and coal and everything. It's almost like they're correlated, right? So the thing is not there's a specific fear around nuclear.

It's like, you know, the US decided that they're a developed country and we're not going to develop anymore because we're already developed. You see the D on the end, right? Like, interesting stuff.

Is that So, so is that just cultural then when you are like the Malaysia sets in, would you expect that to happen to China when they catch up? I don't know. I I Yeah, I mean, maybe it's just like this normal story arc uh of like uh you know, it's it's I don't know. I don't know.

I I think that like you have a real problem when the kids can't live better than their parents. Yeah. Um so, but no, I don't have anything more to speculate on that. Do do you have more context on on China and specifically in like the AI context?

Um like US electricity looks like this and China electricity looks like this. Is that all that matters? Pretty much. Yeah. I mean, that's a pretty good proxy for everything, right? Yeah. Um, like there's two things. There's two things. You know, people are like, "George, how do you feel about the Trump administration?

" I'm looking at two things. Yeah. With any administration, I'm looking at two things. Did you decrease government spending and did you increase total electricity production of America? Those are the only two numbers I care about. Those will capture everything. Why does uh why does government spending matter?

We were joking that, you know, Trump must be extremely AGI pilled if he's running up a massive budget deficit. What the hell is AGI? I don't know what this is. Uh like in in this never seen it. Never seen it in this formulation. It's that it's numbers. Yes. Yes. Yes.

But but it but is an extra lever on on labor and capital and it creates more GDP that then can be taxed to pay down the increasing amount of debt. Super super Excel. Yeah. Super Excel. I get it. What is What is What does Super Excel do that normal Excel doesn't? Let's give it up for Super Excel. Yeah. Yes.

We need that Excel 2. 0. Right. Yes. The thing is Excel was the the final piece of software and then but in order to add another, you know, hundred trillion dollars to to global GDP, we needed to like kind of rebrand it. And so now we get AGI. GDP is the the complete it's biggest [ __ ] thing ever, right?

Like I always joke with my friend and I that we're going to start companies and be billionaires. And I'll tell you how we're going to do it. So, okay. All right. I start a company, he starts a company. Uh we both write contracts to each other. Yep. Right.

like I'll buy something from him for a million dollars and he'll buy something from me for a million dollars. We'll just do this real fast. We'll keep passing the money back and forth. Whoa, look at our revenue. Wow, that all contributes to GDP. Wow, we made we're billionaires overnight, right? Yep.

Like like and that's my argument is the economy is just that with a lot of extra steps, right? You can't use services is not part of GDP. This is complete nonsense, right? You can't you can't have services. No, like literally literally you take the steel out of the ground, you grow the corn. Okay, that's GDP.

But is it I mean if if that GDP is fake, is not the de is the deficit not fake? Like is is government spending less? We owe that [ __ ] to people. It's not fake. But can't you just tax the fake the fake money? Like if you tax your scenario where you're generating a billion dollars in fake money.

You can't tax the fake money because we're passing the same dollars back and forth. The minute you tax it, that falls off so fast. Yeah. Yeah. Yeah. You can only tax productive work. Uh, is uh is AMD doing productive work right now? AMD is doing all right.

Yeah, either Nvidia's really overvalued or AMD is really undervalued. It has to be one or the other. How does it all play out? Like what what does AMD actually need to do to get back on track or realize their potential? Nvidia needs to stumble. I mean, it worked for AMD and Intel, right?

Like so AMD ended up beating Intel in the entire like no one would buy a data center Intel CPU anymore. Yeah. And it's just because well, you know, they stumbled and now Intel owns that market. Yeah. So, you know, AMD just sits there in second place.

They're pretty they'd be in a better second place than they were a few years ago. Yeah. And then when Nvidia stumbles, AMD is like, "Oh, hey, we're here. " Is is DGX Leptton like their their cloud offering a potential stumbling block or is it uh or is it the right move for them? I don't know what that is.

What's an Nvidia cloud [ __ ] Yeah, exactly. Cloud's dumb.

cloud though you can you can break AI down basically into like there's like five five tiers right like at the base level you have like electricity and data centers and land and like things like that tier two are like TSMC ASML Samsung Intel right fabs uh Nvidia AMD open AI anthropic and then on top you have like completely worthless things like cursor and windurf um you know these character AI all these people who think oh we're the app we're going to we're going to get the ARR no that worked the web it won't work for AI and I can go into why but it's kind of boring I want I want No, keep going.

Keep going. Keep going. Basically, okay, so like here's the difference between AI and and web. Um when you want to run a service like Gmail, one server can serve 10,000 people easily, right? And there's no demand for like better Gmail, right?

It's not like it's not like I can click and get like, yeah, you can buy Gmail Pro and it'll have a few things, but most people don't really care, right? There's no limit to the ceiling of how good you want your AI to be, right? Or how fast you want your AI to be.

Maybe there's a limit to the speed, but like when you're at like a thousand tokens per second, I want the biggest model in the world, right? Like so there there's very little limit on on that. Uh but suddenly you can't serve one 10,000 users from one server anymore. Mhm. Right.

And the whole dynamics of the web, the whole reason some of the value aggregated to these end players and they still didn't aggregate to the cursor and the winds, they aggregated to the open eye and the entropics. Right. Nobody nobody nobody who built like an email client survived.

They all got eaten up by the the tier fours of the web, right? the Googles, the Facebooks, um, all of these like app providers, right? Where's a Zinga today? You know, like this already happened, right? People just don't, where's Zinga? Oh, Zinga is going to be the next thing, man. Like, no, it's not.

Facebook ate all of that value, right? Google ate all of the value from all the people building on top of Google. So, the tier fours ate all that value. Yeah. So, OpenAI, Anthropic will eat all the value from the cursors and the wind surfs of the world. They'll acquire some of them.

They'll compete with some of them, right? Same as you saw on the web. Uh, but I argue that the tier fours aren't even going to have value because the tier fours, this ain't the web. This ain't where you can have one server serve lots and lots and lots of people. You know, I'm running 03.

I'm running, you know how much I cost OpenAI every month. I pay the $200 a month and I cost him a lot more than that. Codeex, you can now click on Codeex. Yeah. Spin up four nodes. Yeah. Why would I not click four? It's not my computer. You gave me the button.

Hey, I'm just using I'm just using George Hots single-handedly bankrupts. So, $300 million. Is there no value in just being the the the front end to AI applications to be like the the the the front door, just the the the default button?

because we see these we see these these these uh these models kind of go back and forth in terms of benchmarks or what's hot and there isn't as much customer churn as you would expect because people are are just kind of like defaulted into the app that they installed whenever and so even if Gemini gets better in terms of the actual performance metrics people don't switch from OpenAI to Google because it's so it's so negligible you got to make something 10x better right you got to make something 10x better so like this whole game is open AI eyes unless they stumble.

Sure. Um I'm not switching to Gemini because it's 20% better and I download some new app and think about all the thing, right? No one's going to switch. Is there is there a chance for a company to kind of come out with something that's 10x better with an algorithmic improvement or is it just a race for scale?

Like what could actually be that next? It felt like GPT 3. 5 when they really broke through with Da Vinci and then 40 or and then four.

like it felt like this kind of like binary moment when a lot of people realize that this was usable for their daily life, even if it's just a Google search replacement or whatever, write a poem or whatever.

Uh like a a 10x what you're describing in like a 10x improvement feels like that kind of like qualitative binary shift. Is that possible with just scale or is this something that we need a different model for? I don't know. Um I don't know.

I would bet majority still on like these big labs are also attracting the talent. Um, but it is also like it's uh pretty commoditized a lot more so than like Google search, right? Like you can look at people track how far open source is behind. It's not that far behind. Y um so no, I don't know.

I think this game is mostly going to be uh chat GPTs. I think Elon's aware of this too. That's why he's trying to go 10x bigger with the data center. Yep. We'll see. Maybe it'll work. You know, there's there's someone to bet on.

anthropic I'm not that bullish on but uh maybe you kind of predicted the uh the pre-training wall uh but that's not a reputation of the bitter lesson and we're going to see similar scale play out in reinforcement learning or is there going to be something else that we're building the big data centers for there's something that we don't understand uh in terms of data efficiency um so like when you think of how long it takes a GPT to learn to talk like how much data it takes it takes like terab terabytes of data.

In order to make a GPT talk like a normal person, it takes terabytes of data. Okay? Whereas a human trains on megabytes. Yeah. Right.

How is it that if you take all the text that you've ever heard in your life and you you put it to Whisper and you you uh you transcribe it, it's going to be a couple megabytes, 10 megabytes, maybe 100 megabytes. Yeah. So, humans have this thousandx data efficiency uh advantage.

And we're going to have to fix that if we want like reinforcement learning to work. especially like reinforcement learning that you want to do in the real world. A humans could do humans can learn from very few samples. Yep.

Um, and yeah, I think that like it might be okay if these foundation models train unsupervised on lots and lots of stuff, but uh, yeah. Is that a is that something that somebody's working on just like a a new more data efficient algorithm to drop into the pipeline? Or do we have any like leads there?

because it feels like right now we're going down the path of like reinforcement learning with verifiable rewards and we're going after like individual business use cases that are increasingly long tail and that could be kind of like valuable but it doesn't feel like the breakthrough that you're talking about like has there ever been a breakthrough right like people think GBTs were a breakthrough no they weren't like you just if you watch the the world it was just it was all just smooth but but but what I will say about AI scaling laws oh man you see People get excited about AI scaling laws, but here's a pitch that'll kill your excitement immediately.

Ready? AI scaling laws. You can put in exponentially more money to get linear returns. Exactly. Uh do do you believe that uh the real value is investing in in humanoid robotics then? Uh have you heard this theory?

So, so it it So, I mean, if you if you put exponential more money into humanoid robotics, assuming that they work and assuming you can uh you can like you make 10 times as many robots, you get 10 times as much output. Anyone Anyone who wants a humanoid robot has never worked in a factory in their life. Okay. Right.

Bring it down. Anyone wants a human Oh, yo. Yo, it's going to walk around. Oh, good thing it has legs, right? No. Here's what I want. Yeah. Yeah. We got a laugh track. Okay. Can you show me a robot arm that's capable of putting a screw in something? Probably. Put the screw in the thing. Yeah. No, no, no, no.

Not like you carefully jigged up the screw and have a screw dispenser like the way a normal human does it where the screw sitting there in a little bucket on the thing and it picks up one screw and it puts it in. It takes a screwdriver. No, we're No, no, we're we're we're not close, but also it feels like we're not far.

It feels like that's what I feel like I feel like humanoids are this interesting sort of like space because a lot of smart people just say like here's the 20 reasons why they won't work and and like why we shouldn't build them but then so much capital and so many different teams are trying to make them work that they they very well might work for some things and like they just humanity might brute force it because we saw it in a sci-fi movie you know 30 years ago.

Why? Why? Why? Why are we cooked on? This is as dumb as self-driving cars was, right? And nobody learns their lesson. And people like Kyle Boat should be ashamed of themselves. Like they really should.

These people who go and raise large amounts of money for another thing that like they should know, they should know better, right? Here's basically like remember in 2012 when Google said that, you know, my my my 12-year-old daughter would never have to get her driver's license. Yeah. Come on. That's nonsense, right?

And like now, okay, they shipped Whimo. It's in a few cities. They're teley opt how how tea operated are they in your opinion? Is it is it effectively one to one? It's more than one to one. There's probably about I would say there's 1. 2 operators per car. Um but it's not they don't have a steering wheel and pedals.

Yeah, it is it is an autonomous system that they're probably doing some higher level inputs on. They're definitely like saying when you can be aggressive, when you should slow down, uh, you know, whether you can turn to the stop sign or not. Yeah.

Um, you know, again, like here's here's the simple reason to know that it's like that, right? There's definitely some teleopaths, right? Yeah. Have you ever seen a picture of that room? No. Yeah. Why not?

I've always thought it was like an ace up their sleeve because like if if there's a lot of pressure on them to say these Whimos aren't safe, they can pull off pull the the sheet off of the ghost and say there's actually a human in the loop. Don't worry, it's safer than you thought. Yeah. Yeah.

Like you the fact that you've never seen that room tells you that it's way worse than you think it is, right? Tells you that there's way more telly op than you think it is. If it was really one person supervising 10 cars, Google would post those pictures all over the place. You don't see any pictures.

There's so cruise that actually came out in the lawsuit. I think it was like 1. 5 or 1. 7 humans per car, right? Or or vice versa. Like, right. 1. 5 cars per person, right? No. No. Wait. More people than cars. Yeah, that's what he's saying. It's still that it's still that one. An Uber only requires one person. Yes.

Yeah. But but so so maybe the real innovation is just allowing somebody to get in a car with and not have to talk about the weather or or you know. Exactly. Exact. I'll pay more for that. I'll pay more for that. Yeah.

But I mean, is there any hope that we drive this down and we get to two cars per person, then four cars per person, it starts doubling exponentially and eventually like we are there. I mean, yeah, like it's obviously going to happen, right? It's obviously eventually going to happen.

If you want to see where the real state-of-the-art of unsupervised self-driving is today, right? There's no person with FSD. When you get your Tesla, that's not tell. You can go press FSD and that's real AI. Uh, and well, you can see how good it is, right? Would I uh take a nap in there even for five minutes?

No way in hell. Yeah. Um you'd be stupid, right? How are things going on the comma side? Uh give us the update there. Pretty good. You know, we're we're we're on track to um we're on track to be two years behind Tesla. So there you go. So So two years behind Tesla. Uh but you know, here's why we win, right?

Like because like it's cheap. Uh okay. So, when you think about self-reming cars, uh it doesn't look anything like the roll out of Uber, right? Or Airbnb. When you roll out something like that, you're trying to roll out a two-sided marketplace. Mhm. Uh you got to spend tons of money on customer acquisition costs.

You got to make sure that you've perfectly matched that marketplace right away because if drivers aren't getting rides, they're going to leave the platform. If riders have to wait too long for drivers, they're gonna leave the platform. So, it's this careful balancing act.

But once you get this marketplace, you got you got a moat, right? Switching costs are real high. Try to get everybody to switch at the same time. It's a chilling point. ever doing. Self-driving cars don't look anything like that. Self-driving cars look like scooters.

The only thing that it's going to take to roll out big fleets of self-driving cars is capital, right? It's just strictly a capital market. You could just I can if I look at a city, I can calculate how many Whimo there are.

If I want to build my own network and deploy that network and run at a lower cost, it's straight up capital. Easiest thing for investors to calculate, very little risk. So, self-driving cars are going to be this awesome race to the bottom, right?

It's going to be like scooters where there's going to be like 10 providers of these things for a while and then they're going to consolidate and like one is going to do it. But um yeah, people are really going to win.

What is what's most valuable in terms of developing the next like the next better version of full self-driving? Is it having a lot of data, building a big data center, having a great team to actually design the system? What's most important? Are they all equal? Yeah, all those things matter, right?

I think the main thing that matters more than anything else is just time. Like we're figuring things out with research. Infrastructure is getting better. I think that a lot of it's just infrastructure. My new company's AI infrastructure, right? Like the infrastructure gets better.

Um my uh my uh coworker has a saying is like uh what we do is that we make the uh hard things easy and the impossible things hard. And that's like the goal of infrastructure, right? You build infrastructure, your infrastructure gets better.

And then what was what you couldn't even dream of doing 10 years ago is now one command today. And today, you know what what you you you uh Yeah. What's the current use case for most people with tiny boxes? Is that that's by design, right? You're not supposed to know. But I mean, so I sell the computer.

It has specs, right? Like so many people want to tell you, and I hate this. I hate this. They're telling you like how the product is going to impact your life or what you can use the product for. Oh my god, who cares? Yeah, here's what it is. I'm going to tell you what it is. That's your job, right?

I'm not an advertiser, but I mean I mean our intern wants to build something with a tiny box. I want to give him some ideas. Go buy one. I don't know. Why do you want to build something with a tiny box? I mean, is it good? Yeah, it's just a It's a bunch of GPUs in a box, you know? It's nice to know boxes.

GPUs in a box weight to it. What um it what robotic form factors are you most bullish on? We've touched humanoids. You gave a a great review there. We've touched autonomous vehicles. Sounds like generally bullish, but Capital Wars, Race to the Bottom, uh all that stuff.

Are are there any other kind of form factors that you're thinking about that you are generally optimistic or excited about? ARM. Arm the two arm, right? Just two arm, right? Because I look I run a factory. I run a factory in San Diego. We make all the commas right here.

And I can't wait to get a whole lot of robots in there. But uh I don't need humanoids. I'm just going to stick two arms to the table. And then it's going to grab a comma. It's going to put the screen on the front. It's going to flip it over. It's going to put the four screws in it. And then it's going to pass it on.

Yep. Right. Show me anything that's anywhere near that level today. Yeah. What what uh what would you do if you were trying to build like a truly multi-purpose robotic arm? The arms are already good enough. It's the AR offtheshelf arms are fine. It's all software. Again, it's always all software.

Autonomous vehicles are all software. Robotics is all software. But everybody loves to bike shed. Yeah. What color are we going to paint the humanoids, you know, like like let's have a great conversation about that. Well, you know, we don't want to paint them red because that might scare people in a terminator.

This is actually the level of stupidity that I see in most discussions about human robots. Yeah. Would you uh would you trade in your legs for wheels if you could? I got that. I trade in my legs for wheels. Yeah, this is a question from Aaron Frank, friend of the show. He's asking this in real time.

Weird wheel guy then. Like people had asked me if the wheels are like making a statement. I just don't want to have to have this conversation. What What What about the sim tore gap in robotics? Like how how is how is simulated data?

You know, you you build a bunch of data in Unreal Engine, then you try and transfer learn it back. Uh obviously there's been a bunch of experiments of that with self-driving cars. Is that a path that we should be going down for the for the robotic arm development? Yeah.

So I think with a lot of sim to real stuff, the reason people are excited about it is because of that data efficiency gap we spoke about, right? Like current machine learning algorithms like a thousandx less data efficient than humans. Uh so yeah, you're going to need a thousandx more data, right?

If a human can learn something in one example or 10 examples, the computer is going to need a th000 or 10,000. Now, do you really want to reset the stupid state of the physical world 10,000 times? You might do it 10, but you're not going to do it 10,000, right?

So, that's where you want to simulator where you can just click reset and everything's back to exactly how it was. Um, so I think this stuff's going to play a role, but I think more fundamentally that data efficiency gap has to be understood. Uh, we talked a little bit about coding agents.

We talked about how you're bankrupting OpenAI by spinning up a lot of different uh codec agents. Um what uh what other a sort of agentic software are you excited about? Do you expect to uh you know what's a gentic mean? Yeah, basically bots. It's it's like what we're calling bots now.

Um but but but anyways like I just you know from Star Trek. Yeah. Yeah. Maybe. No but but I think about a world in the future you Do do you expect to be I don't know if you're a a Slack guy, an iMessage guy, Discord, maybe no messaging at all, just you know um you know telepathy telepathy.

Uh but but do you expect a world in the future where you're just you know a perfect interaction between you know human employees and agents or is it you know going to be more like you know you'll do the odd deep research or maybe you send some automated outbound emails or have some codeex bots running?

I don't even like follow this. When do you think you'll be able to book a flight just by saying I'm trying to get to New York tomorrow? Oh, see the worst part about this is like like and that's going to come pretty soon actually. Okay. Right.

We're going to pretty soon have computer use models that are actually capable of going to delta. com and booking a flight. Yeah. But then what's actually going to happen is Delta's going to partner with whatever company does that and they're going to put it behind the stupid uh thing and like Yeah. So that's gonna Yeah.

That's going to be here in a few years, right? Not with agentic [ __ ] but just with normal hooking the APIs together, right? Yeah. Wait, so yeah, what is the bullish on APIs? What is the mistake about like the agentic buzzword? Like what what are people like even describing?

Again, it's another thing that I I really have no idea what it means.

You know, I I was hanging out with some friends last night and like like uh my friend in this VR company and the you know, the the CEO is really interested in things being open source, but he's also really interested in making sure that things are protecting our intellectual property and proprietary.

And the truth is he has no idea what the word open source means. He has no idea what it means that they can copy his [ __ ] right? Like that someone else could use it. He just he just heard the word open source in some like buzzword thing and he's like, "Do we have the open source?

Do we have the open source in the thing? Okay, so check the box. Check the open source box. Let's protect our IP. Last question about the a last question about the agentic buzzword.

I think that there is something that people are picking up on, which is that these models seem to be very smart for short amount of time, but if you run them for a long time, they start hallucinating and kind of going off the rails. And so you you have like 10-minute AGI.

feels incredible, but as you let it run and do more work, you can't just say, "Hey, go do a week's worth of work. Come back to me when you're" But it's superhuman in one minute. And so, is that kind of trade-off curve real?

And then is it just a matter of like better harnessing to actually get to two hours of work, which is kind of what the agentic people are like advocating for. No. So, I don't think it's better hard, but this is definitely a real phenomenon. This is definitely a real phenomenon. Uh, you can experience this.

There's papers exploring it which show that if in 10 seconds there's absolutely no way I'll come even close to a modern LM totally um because the first shot from the LLM is great. Yeah. And then it kind of degrades and it degrades pretty quickly whereas humans look a lot more like this.

Humans can stay coherent internally for much longer. Um so yeah I I think that that's a real thing. I think that that's mostly going to be fixed by like long context, just more energy. Long context RL. Yeah, just like you just got to do it. We'll figure out new ways to make the context better.

We'll combine diffusion and uh and auto reggression in some clever ways. Yeah, I think that this is just going to be uh like there's not going to be a breakthrough here. There's not like one magical thing that we're missing. Yeah, I think it will be a continued plot. The same thing with data efficiency.

I think people will start to care about it. Some new tricks will come out. Some of them will work, some of them won't work. We'll continue to do graduate student descent until we find a anything that's last question for me. Anything that you're particularly optimistic about? Anything?

You check the timeline and you think, "This is awesome. I love this. I love this. I want to see more of this. " A little maybe a little white pill to kind of cap it off. Yeah. So, here's something I'm optimistic about.

That fact that the one server can't run 10,000 users, that is most of the reason that the modern internet that that is one of the reasons that the modern internet sucks. That that that so much of the stuff is in non-recurring expense and then it becomes really really hard to compete with these people, right?

Like you could run Twitter on one computer. Yeah. Right. and 20 people could do it too. But like they don't because again these companies have moes and they invest in making sure that their modes can't be broken.

Um with AI I think there's going to be a much less of a mo especially when you look at the move from auto reggression to diffusion. So auto reggression can run in large batch sizes. When you run chat GBT you're running with a whole bunch of other people on that same computer. Yeah, it's only 100.

It's not 10,000 but still it's 100. Diffusion is run in the cloud at batch size one. And once you're in batch size one land, running it locally starts to make sense. Actually running the models locally or at least having your own computer in the cloud. Yeah.

Not being some shared resource that's really controlled by some else. Um so yeah, this was never a thing because you can't put lots of people on a GPU. That makes sense. They tried some weird stuff with the licensing, but yeah. Fantastic. Well, thank you so much for stopping by. This is a great conversation.

Yeah, I wish we had a full hour. This is great. We'll talk to you soon, George. George. Cool. Bye. See you later. Cheers. Bye. Next up, we have uh Joseph Troian, author of The Party's Interests Come First. He was recommended by Jordan Schneider of China Talk. We're very excited to talk to him about uh the life