River AI raises $1.1B to bring personalized, self-owned AI to companies and consumers

Speaker 1: Railway is the all in one intelligent cloud provider. Use your favorite agents to deploy web app servers, databases, and more while Railway automatically takes care takes care of scaling, monitoring, security. And we will bring in our next guest.

Speaker 7: Hi, guys. Thanks having me.

Speaker 1: What's happening? Popping on the show. Welcome. Congratulations on the fundraise. But may maybe let's go back in time. Tell us a little bit about your history and journey to starting River AI. And Jordy already has the gong ready, so just tell us the fundraising announcement, I guess.

Speaker 7: Yeah. We we just managed to raise $1,100,000,000 for River AI. Broke the

Speaker 2: gong.

Speaker 1: Broke You broke the the mallet.

Speaker 2: The chosen one.

Speaker 1: The chosen one.

Speaker 2: You are the chosen one.

Speaker 7: Okay. So We'll see if it if it's if it works out. So anyways, I I started my career as a physicist. I was really interested in understanding the universe. Yeah. But then realized that there was something really big happening, which was AI

Speaker 1: Yeah.

Speaker 7: Started to happen. And I think AlphaGo was really the moment when I started to feel like, wow, gotta switch gotta switch and learn how to do AI. So I managed to join DeepMind.

Speaker 5: Yeah.

Speaker 7: It was almost ten years ago. Wow. I worked there on WaveNet. We trained a Starcraft agent that was very strong, so got into reinforcement learning there and then switched to OpenAI. One point was really interested in reasoning, coding with alarms, which now turns out to be something that works. It was which is pretty crazy to to see. And then end up cofounding XAI together with Elon.

Speaker 1: Yeah.

Speaker 3: So I

Speaker 7: thought it was time for for another Frontier Lab. I'm always in favor of more diversity in AI more. More companies doing different kinds of things, so I was happy to to support him building up building up XAI. Yeah. And now with Brannin, we're kind of taking that to the extreme. So I feel like the way we've been building AI is maybe not the way of the future. A few large corporations building these super powerful models. Everybody has to pay them by the token. We wanna figure out how we can distribute AI to everybody in a way in a way where you own it, you're able to shape your own AI systems. Maybe you have the inference running in your home or in your office, that would be the the the best achievement if you can if you can figure out how to do that efficiently. And we're running a few different bets on how to help people build up their own AI. We're helping companies build AI with the River API. That product's already out. So if you go on river.ai/api, you can log in and you can start training models based on open weights.

Speaker 3: Mhmm.

Speaker 7: So we support all kinds of very powerful open weight models.

Speaker 1: Mhmm.

Speaker 7: And then we're also using that platform to build out personal AI agents that are increasingly personalized to you. As you're using them, they understand you better and better and should really feel like you're you're building the AI. You're you're creating it.

Speaker 1: Can we go back to your time at DeepMind working on video games? I'm so interested as in video games as a benchmark. I saw someone using codex to play Slay the Spire. And I've played a lot of Slay the Spire. It's pretty difficult. Now it's a it's not a fast twitch game but I'm wondering if you like how would you think about the value of video games as a benchmark? There was another story about the FAA hiring traffic flight traffic controllers who had previously played video games. So there's some sort of transfer where if you're good at video games and maybe SimCity, you might be good as a flight traffic controller. And you could imagine a situation where AI gets really good at playing video games and then becomes more useful in a whole bunch of different, you know, related tasks. But it feels like the gaming benchmarks are still toys. They're fun. They're people just doing them off on the side. But how do you think about the role of of solving video games or or testing models on video games in the modern era?

Speaker 7: I think it's a great idea, but I I might be biased. You know, I used to play a lot of video games Yeah. Growing up and still do do some gaming from time to time, I I think it's awesome. And I I think the the the idea here is you wanna test your your AI on system on on problems that it hasn't necessarily been trained on directly.

Speaker 1: Yeah.

Speaker 7: So wanna have some level of generalization. And games are are amazing because they have all kinds of complex things you're gonna have to do, all kinds of problem solving. You're gonna have to develop on the fly, and it's kind of measurable how much progress you're making. So if you're getting to the end of the game, you're doing well. If you're making progress from one level to the next, you know, that's, you know, that's measurable. So and and have many, many properties that make them pretty ideal Mhmm. For measuring AI capabilities. And the the craziest thing today is we have these powerful agents that have been trained mostly on coding tasks.

Speaker 1: Yeah.

Speaker 7: And I give them some code base and you ask them to to fix a bug or develop a new feature and they go out and they do all this tool calling to figure out how to do it and rewrite your files. But then you can also hook them up to a game. And give them an API like, here's how you control the units in the game or here's how you manage your resources or here's how you move around in Pokemon and other games like that. And I think it's crazy that these models are so capable at playing games and just shows you how much we've how how far we've come in terms of generality and

Speaker 1: Yeah.

Speaker 2: Yeah. Where are the shortcomings though? Because, you know, I can think of one which is like with a game, like, can you basically run an agent, have the agent play a game effectively infinite amount of times. It can fail a lot. It can learn, things like that. Yeah. One of the challenges in is in the real world, like let's say someone was making a sales agent, like an agent that wants to help you get customers.

Speaker 3: Mhmm.

Speaker 2: You can't necessarily just let the sales agent run wild in the real world as a business at least because you know, it's gonna mess up a lot. A bunch of customers are gonna have a bad experience and you could make maybe the agent gets slightly better from that experience but you could have lost like a bunch of potential customers or pissed a bunch of people off or things like that. And so how do you think about making the jump from agents that are very effective at at playing these games in a generalized way to agents that can be effective at long running tasks in the real world that involve effectively complex groups that are are third parties?

Speaker 7: That's a good question because most of the training today is done with synthetic environments. So it's you're you're building up these aural environments inside of your AI team and you're training the the models for reinforcement learning. And so you kind of go for a simulation, you could say, and they're not really interacting with the real world when you're training them. I think one of the big frontiers right now, one of the big developments you might see is if you train the training moving into an online setting Mhmm. Where the the models are directly interacting with the users, with the companies that are using them. And as they're solving tasks, as as they're figuring out what to do, we update the weights of the model, they get better and better. And it's a big research problem right now. So nobody knows how to pull this off in in general, and the best agents, they're all all been trained in simulation so far. Yeah. But it's one of the things that we're working on at River AI. So if any of the viewers are interested in doing some research on this, reach out.

Speaker 1: Yeah. It feels like we're not that far, at least in the gaming sense, to, you know, in the in the training step, just create an environment that's just like, here's a Steam account and a credit card. Go buy every game and try and get the platinum trophy or like complete the game and feed that in. But gaming thing is sort of a pure benchmark at this point because it feels like the labs haven't identified it as something that really they want to focus on. Can you talk about the trade off between like bench hacking for good and bench hacking for bad? Because there's the game that the labs are playing but then there's also like if you show up with a product and and it does the task and it classifies all of my taxes, I don't care if you bench hacked on that as long as it gets the job done. Right? So there's Yeah. This push and pull between those. How are you thinking about communicating that with your customers, your the companies you work with who might be fine with a model that's only good at their specific task?

Speaker 7: Yeah. Exactly. I think that's a big opportunity for any company out there today because you own your own data that you've collected from your customers or from the work that you're doing. And if you eval the systems on that data, this is, like, the the perfect eval

Speaker 1: Mhmm.

Speaker 7: For you. If you're able to improve your models based on that, you you might be able and you might end up owning the best model in the world for your particular tasks. I think that's actually huge for companies. They should be building these specialized Evals, they should be trying to build their own models. And the River API makes it easy Yeah. To fine tune your own model, given your eval, given your own training environments. But in in general, when it comes to AGI and improving the the intelligence of the of these models, I think we want to hit them with some surprising benchmarks. Want to measure generality then throwing in a game that it hasn't been trained on, that it's never seen before. I think that's a very, interesting measure to see, like, how how far out of distribution can they go can they do interesting things Yeah. And haven't trained them for.

Speaker 1: Huge, huge fundraising round. AMP is in. We've talked to Ajnae a bunch. And he has a very interesting thesis around actually going much deeper in the stack, acquiring compute. How are you thinking about the uses of those funds? Because I could see coming to you as a company and knowing that you have the capital to go and really optimize all the way down to the stack and become a Neo Cloud, build a build a data center for me or help me with the more expensive CapEx piece of the puzzle. At the same time, like, I don't really have a rule, like a solid frame of if I come to you and I say I want to fine tune a near frontier open source model, is that actually that expensive and wouldn't you just ask me to pay for that upfront? So that doesn't seem like a huge capital cost to you, but what is the shape of the of the cost that

Speaker 7: you Yeah.

Speaker 1: Are planning on incurring over the next couple years?

Speaker 7: Yes. So we're charging the customers by the token. So if you have a fine tuning run, you wanna do you wanna do it. If have an Aural run, you only pay for the Xudo. So which first app. Choose some kind of model

Speaker 1: Okay.

Speaker 7: Training run size that you that's perfect for your task. Some customers, they end up training a really, really small, really fast, efficient model because they've got the best data for the task and end up beating the the largest and most expensive models. Yeah. Other customers want something more general or they wanna utilize these larger open rate models like k m a k free Sure. And others are coming out. So that's a that's a more expensive training run. But, yeah, we obviously need a lot of access to GPUs to make that happen. The funding helps with that. But even if you have the funding today, you still need to get access to to GPUs, and GPU prices are increasing Yeah. Steadily. So I think what we're going to see is more and more investors collaborating with our portfolio companies around compute Mhmm. Bringing up GPU capacity, distributing it among the portfolio companies. You know, some maybe some of them need a little bit more in one month than others and so on and so forth. And it's a new kind of strategy that we're seeing, I think, to deal with the fact that compute prices are are going up this much scarcely.

Speaker 2: Yeah. Would you say that's like one of the one of the biggest sort of challenges for for River at this point is just compute planning? I mean, we've seen it. We've seen the full spectrum now. We've seen Sam last year, you know, getting really really really aggressive. And then we saw, you know, earlier this year, Anthropic just sort of being caught off guard by the growth and it feels like that is as as a CEO, you know, you're,

Speaker 5: you know

Speaker 1: It's a completely new skill set

Speaker 2: for Yeah. It's like a new it's

Speaker 1: a Yeah. There's never a moment where Marc Benioff was like, I don't have enough servers for Salesforce, I imagine. Like, they were way different problem set. But now it's yeah. Demand planning is like a key key key skill set for a CEO.

Speaker 7: Exactly. It's a totally new skill set. That's super important, and it's so difficult because as a start up, by definition, you have this variance for the future. You don't know if you're gonna grow by 10 x or if you're gonna grow by three x Yeah. Over the next twelve months. Right? So you have to play this really, really difficult poker game to figure out, you know, what is the right allocation for me or maybe create some deals that are more flexible

Speaker 1: Mhmm.

Speaker 7: So you can actually scale up Sure. Dynamically as the as the demand is is growing. So I think we're gonna see more and more of that's of that happening. And yeah. So so we're bringing up quite a bit of GPU capacity, and it's been important both for research and also to power the API. So as more people are starting to tune their own models, it's becoming very, very popular now. A lot of companies are reaching out about wanting to build their own custom models that they own and trying to train on their own data using their own evals. So demand is gonna keep going up, we expect.

Speaker 1: How do you think about custom silicon over the next few years? YouTube, I believe, has a custom silicon chip for encoding video very efficiently because you upload one video, they need it in three sixty p, four eighty, seven twenty, four k HD. And then we saw Thales bake the weights of Lama, I believe, into their chip and prove that that was, exciting enough that AMD acquired the company. And I'm wondering if you imagine a future where a company comes to you, like, you can imagine like the Visa network is like we want to run a transformer based model on every transaction and it's going to be like trillions of prompts effectively or more. And so custom silicon might actually make sense but then the models jump forward and you can do batches and there's so many different tradeoffs. How do you think custom silicon will play in the diffusion story of AI?

Speaker 7: Yeah. I think we'll see more and more custom silicon. Obviously, companies like NVIDIA and AMD are also going to do extremely well, especially on the training side. Yeah. It's really unmatched what they're what they're able to do. Yeah. On the inference side, there there is some room for optimization because we believe that with personal AI agents coming up, so this is kind of the next evolution of agents after coding agents, which have been super successful. We expect that the demand for tokens will go up even further. Mhmm. So to the point where, you know, we we don't even know how we're gonna serve all these tokens for everyone given limited data center capacity, given all the bottlenecks in the data center supply chain. So I think we gotta be smart and start to develop some custom silicon specifically for inference of these personal AI agents. We could imagine being much more power efficient with these kinds of chips. Can we maybe bake some of the transformer architecture into the chip to kind of despite the fact that we know what kinds of models we're gonna run. So that will make it more specialized. You might not be able to run any kind of model anymore. So Talos takes us to takes us to the extreme, so you can actually bake bake in the model weights. But today, you're not able to fit weights of very large models on a single chip that way. So that's the big the big bottleneck. Yeah. So you're you're only able to do maybe eight eight billion parameters or something like that, whereas the best models have trillions of grams. Yeah. So hopefully we're gonna see some chips that are able to to run those top of the line models very very efficiently.

Speaker 2: Do we need more NeoLabs or, you know, basically like are are are there is there enough idea space that that, you know, more people should be spinning up entirely new labs? I imagine a lot of the people that that would be candidates to spin up labs themselves, you're probably trying to to recruit. But are we past peak neo lab or or is it are we just getting started?

Speaker 7: Hopefully, we're just getting started because, you know, this idea of building powerful coding agents, having APIs and so on, we all had those a few years ago or not OpenAI and other places that that would be the the future. But now that it's actually arrived, I think all of us are feeling like this can be the end of line. Like, there have to has to be a different way Yeah. To own and build AI systems. And so that means there's there are opportunities for research, there are opportunities for new kinds of business models, you know, totally new talents can move in. You know, somebody who doesn't have a big name in AI can do something really amazing today because they have to kind of have to think out of the box, come up with something that's, you the the AI experts that's been doing it for ten years, they might not come up with it.

Speaker 1: Mhmm.

Speaker 7: It's such a wild idea. Right? So that's I think we're gonna see a phase of innovation, totally new approaches to how AI systems are built, how you make use of them, and some really cool AI products as well for consumers. So that's what I'm looking forward the most.

Speaker 1: Yeah. What exciting time. Well, congratulations and thank you so much for taking the time to come chat with us.

Speaker 2: Yeah. Excited for the next one.

Speaker 1: Yeah. We'll talk to you soon.

Speaker 7: Thank you, guys. Cheers.

Speaker 1: Have good rest of your day. Goodbye. Let me tell you about MongoDB. What's the only thing faster than the AI market? Your business on MongoDB.