Nathan Lambert: America needs its own DeepSeek — the case for publicly funding open-source AI to stay competitive with China

first, let me tell you about Bezel. Go to get bezel. com. Your bezel concierge is available now to source you any watch on the planet. Seriously, any watch. Go check them out. And I'm very excited to bring in Nathan uh and talk about Deep Seek Llama. How you doing, Nathan? Boom. Good to meet you. What's going on? Good.

How much this format? You guys got a loaded lineup today. I was like, wow. I got on the same day as Ben. Ben is like the motivation for why I started writing about AI. Somebody has to do this for AI because there's so much to talk about. But all he does now is AI anyways. So he's a competitor.

He's the Ben Thompson for AI is definitely definitely Ben Thompson.

But I mean it is a little bit different in terms of there's so much different space in terms of uh whether you're going after the the the business models or the actual infrastructure or or what you were writing about earlier with kind of the the open source geopolitical angle.

Um, so take us through the recent piece, the thesis, and then I have a bunch of questions about uh both DeepSeek Llama and kind of how this could come together. We were talking to the CEO of Grock yesterday.

Uh, he's obviously extremely long open source and and uh it's very interesting to dig into a million different threads here. So, just kick us off with an overview. Thankfully, we were talking to the CEO of Grock with a Q just at that very moment there was a different Grock hallucinating and and at scale.

Yeah, there should be more Grock news later if tweets are to be believed. But the American deepseek thing is largely a forcing function to make AI research EOS, make the AI research ecosystem in the United States catch up. I think we were talking about niches and lanes and like Ben Thompson, it's the biggest one.

A lot of mine is on the research side and kind of understanding the emerging trends on research that are getting picked up by companies.

And one of the clearest ones that we do is I I lead this with a couple other people as we keep track of all the open models and data sets, a mixture of research and startups are releasing. And there's been a huge shift in the last three to five months and pretty much everyone builds on Glen.

And there's a long tale of kind of business or geopolitical reasons. Some of them are sensitive and some of them are kind of obvious where American companies don't want to build on Chinese models. And that's one thing.

And then also America should just take pride in what has been a great like research ecosystem and we want to have that and own it whole stack which is otherwise most of the leading AI research is going to start coming out of China and I think so politically it should be an easy win and in terms of cost to maintain the open ecosystem it's so much less than what these companies are pouring into their AI models.

So, it's just kind of getting the a bit of the tractor beam of AI onto this open source and open. It's like just building models that have all the data and code released so more people can start building on them. We can go into the details. Yeah, Deepseek versus Quen.

I feel like Deep Seek had this like crazy viral moment. Um, but now you're saying that Quen's been kind of on a on a, you know, compounding growth for a while. What's the dynamic between those two companies and and what's driving Quen's adoption over DeepS? Yeah.

So, this is a great example of one I've started using and we'll loop it into Llama. Essentially, DeepSeek has these frontier class models that are extremely good and they dropped the weights on Hugging Face and they have been switching to permissive licenses.

These are models that anyone could pick up and use and dump into a product that they want to ship. A lot of startups use these things. We see all the clouds hosting them. That's one thing. Not a lot of people actually fine-tune Deep Seek because it's huge. It's already so good. Like, what are you going to get from this?

And then what Quen is doing is they're releasing honestly tens of models at different formats both base models and post-train and from size scales as small as like 500 million parameters up through these bigger models.

So for a researcher it's pretty much or somebody trying to build a really niche product and something at the cutting edge is like Quinn will have the model at a certain size that you need in order to fine-tune it or figure out if you can build a certain thing at a certain cost profile.

And especially for researchers that have some sort of limited compute like training and working on deepseek is super hard.

And we've seen this with llama 2 and llama 3 were much closer to this quen approach when it was seen um both in the data that we have and on the ground is like llama was the open standard for research. I used to joke around that like hugging face is just going to be rebranded llama because you see llama everywhere.

Uh especially around llama 3.

And with Llama 4 meta started to go like release drama aside, they've started going more towards their bespoke solutions and they're also releasing the models which has made this big opportunity for Quen with Quen 3 which honestly earlier Quen models were already starting to fulfill this but that's kind of been a big uh mass shift and attention shift in the last few months where So what do you expect out of Meta with uh with the new talent acquisitions and it seems like a redoubling of the efforts on AI broadly super intelligence uh but maybe you know the the uh the strategy of llama could be shifting um are they the are they the lot I I've often thought that you know right now with the dominance of deepseek and Quen internationally in kind of these like jump ball half ally countries frennemy countries uh Mark Zuckerber should be like a national champion and and and we should be pushing Llama you know at a national level all over the world Um, but what do you think is going to happen there?

Yeah. So, there's two things. Um, mostly I mean I'm going to gossip as anyone will and what will happen with Llama. It's very 50/50. I think with the leadership they've brought in that there's less um attention and value behind the open thing.

So, Zuckerberg historically has been very pro open and if more of it is shifting to other people, they kind of loosen that vision. if other people are at the at the lead of leadership.

Like that's what people are saying is like they need more leadership to build this AI or so a best case scenario is Llama invests more in AI and the national champion becomes even better and they just crush this.

I don't think that's the outcome that I expect to happen, which is I'm saying there's a a cheap offramp if you take the cop packages for a couple of those researchers.

Like that's the cost to get a whole research ecosystem built around a fully open US model where we have all this like the data is released and a nonprofit and stuff can handle data releases a bit better than a big tech company that has all these eyes on their back and then just have research happen on these things.

So what are you advocating for? Are are you advocating for nonprofit taxpayer funding? Um, you know, we've we've seen a nonprofit before and it turned into a for-profit. Um, how are we going to keep this in a for-profit? And then how are we actually mustering the the will?

Because yeah, it sounds like, yeah, just put $100 million into a nonprofit. Like, that's a lot of money. Like, this is where we get to the nitty-gritty. I work at the Allen Institute for AI and AI 2, which historically is even more academic than OpenAI was.

So I think culturally there's there's not that type of feeling the AGI like supervision that Ilia had on the scaling deep learning. That was kind of the thing that I think drove them from the start. They're like we need so much money.

So AI2 is set up it's it's like you can do digging but it's so different in that culturally that it could never happen.

So if if the leadership here tried to do that, the company would just implode um going for a for a profit because I mean most of the leadership has co-appoints with professorships at UDub and things like this. So it is already half embedded in the ecosystem.

And then practically speaking, moving AI talent around is so hard that it's like the government exfiltrating researchers to fund like a open-source government lab doing this is so hard that it's like you have to find the money or the partners to do this where there are people.

So sitting on the ground where I am, we're trying to train our next model. Like it's pre-training now. It's like we just need more compute. So if we double or triple our compute, America will have X or like these open models will be just X% better. And it's it seems tractable.

It's just hard to get the right it's it's a lot of politics to get these things in place. Yeah. I mean I I I guess to to uh dig in there, I'm wondering if like you know the the initial like economic model for Llama was always in a little bit of debate. Is it a recruiting effort for Meta?

Is it their desire to decouple and not be dependent on uh Gemini or OpenAI or Anthropic and just save cost there?

Um there were a whole bunch of different uh you know economic motivations, but I'm wondering if like in the long term we don't see something like uh you know a Red Hat Linux where it is a for-profit company maintaining a nonprofit uh or or an open-source software package or even you mean you can run Linux on Azure now.

Um, and so, um, is there a world where you just have every different piece of the stack, whether it's a consumer app that people pay monthly for or an API that people pay on a per token basis or an open source model that you're paying for, you know, Red Hat style consulting services on top of all within one company?

Like is there no hope that OpenAI's open source model or you know uh AWS or Microsoft open source something that that actually competes significantly with DeepSeek and Quen but is still within the typical corporate structure. I think it'll come from the the biggest motivators have to be Nvidia and AMD.

So the long tale of ifwen is going to keep doing this and then it's something like Huawei they start working with Huawei this like Huawei libraries they want to support them and then US researchers are starting to dig down into those levels because they want to understand how these models were trained.

So those are the people that have the most direct um exposure it would be Nvidia and AMD it's cheap for them to do they get the benefit of the researchers keep working on their hardware and software ecosystems.

There are like outlandish stories that you could tell about open source AI where that type of thing could emerge, but mostly they involve uh technical breakthroughs that you can't plan on.

Like if you could do weird model splicing where you train a bunch of and then you cut ane out of one model because it's really good at healthcare and there's kind of this open marketplace for model parts and then that's all in the open and the person that kind of um writes the software by which those like pieces of models are combined and standards by which those happen that could exist.

So there's kind of wacky ideas, but I think that that's a lower probability outcome. And it's just like let's get good big transformer models trained that anyone can download and poke around at and just get more people involved in AI research that it's just we have complete control over. Last question for me.

Uh what are you expecting out of uh OpenAI's uh open model? Yeah. So I think um one of the core things about OpenAI culture is that they really like to deliver extremely cutting edge and good artifacts and research.

So I I expect it to be one model that fills a niche that they're either hearing from customers or the community that isn't quite filled.

Whether it's a extremely like super long context or low latency for agents or a certain size of reasoning model, it's going to be this type of thing where it's a a certain niche and it it works really well for it where it could be like a like a deepseek style release where it's just super strong model that people can plug into real world products and applications, but it's not going to be this Quen or Llama suite of models that researchers look at for all sorts of things.

and OpenAI has been saying that they hear the license critiques of Llama and stuff and they're going to commit to the actual like permissive license which are things like Apache or MIT that these Chinese orgs have started using again which I think is a nice thing to kind of make all of that simpler to just you release a model it it doesn't have terms and conditions on it like you met Meta is not trying to say like your legal department has to talk to us or avoid these use cases it's just get people using the model that your company released and take a simpler approach That makes a lot of sense.

Anything else, Jordy? That's it for now. Thank you so much for stopping by. This was fantastic. Thank you for working on all this. Yeah, we will talk to you soon. Looking forward to the release. Cheers, Nathan. Have a good one. Bye. Up next, we have Richard from you. com coming into the studio.

Um, do we have any more ads? Go to eight. com. Uh, get a pod five. 5-year warranty, 30