AWS CEO Matt Garman on frontier agents, Nova 2, Nova Forge, and Trainium 3 at re:Invent 2025

built from first principles and object storage. Fast 10x cheaper and extremely scalable. Uh our turbo is at reinvent.

Amazing. So

fantastic. Well, we are joined by the CEO of Amazon Web Services, Matt Garmin. Thank you so much for taking the time to come and chat with us. How are we doing?

Hi. Thanks guys for having me.

Uh please take us through uh some of the highlevel announcements. Obviously, it's uh it's reinvent. Very exciting. Congratulations on all the progress. Uh would love to know uh what's at the top of your mind, what's on the top of your uh presentations over the over the over the course of the event.

Yeah, we had a couple of really exciting announcements today. Uh a couple I'd highlight. First, we uh introduced these idea of frontier agents. Yeah.

Uh these are agents both uh in Kirao uh for software development as well as uh in operations and security. And these frontier agents are meant to accomplish much much more than customers were a ever able to do uh in the past where we have these autonomous agents that can help customers really turbocharge their software environment. So super excited about that. Um we had some announcements around Nova which is our frontier um uh AI models that we announced. We announced Nova 2 uh and our new sets of models. Um, and one of the things I'm in particular really excited about, um, is Nova Forge, which allows customers to actually bring their own data to pre-training checkpoints, mix in their data with Amazon data, finish training the model, and at the end of it have a custom model that deeply understands their own enterprise data um, and is uh, and is just for them. Um, so that that's another thing that I'm excited about. Um and then the third thing is uh we announced a new chip around tranium 3 um to really turbocharge uh the next generation of training and inference uh for our customers and so quite excited to to get that and that went G today as well.

That's very exciting. I let's go back and start with the first one. Let's talk about uh coding agents and uh the your own proprietary models. How are you thinking about positioning those to potential buyers? uh are you do you like the benchmarks these days? Do you think that uh we're sort of like post all the benchmarks or do you think those are still useful tools uh for a buyer who's making a decision? Is it about integration? Is it about cost? How are you positioning them?

Yeah, when you think about software development, it's it's not about pure benchmarks. It's really about what is going to allow you to get the most amount of work done. And when you think about our offering which is called Curo, yeah,

um it's really focused on in a enterprise uh or environment where somebody's doing high velocity um development, they actually need more structure. People love vibe coding and it's exciting, but you can actually get down a path where you get stuck. Yeah.

And you'll often find actually that you spend just as much time trying to get back to where you were before as if you had just coded it from the beginning.

We have this idea of specs that gives you structure to what you're trying to build. And so you can have agents go and start to build around those specs together with you and your team. And it gives you the structure that allows you to go really fast, can undo if you need to, um can make sure that you're hitting your design requirements. And it and it really allows you and the agents to operate um in conjunction with each other and move really really fast. Um and we're starting to build these much more capable agents that can go and actually do longunning tasks for you on your behalf. But all of it is kind of ties into this structure. And we view that as a way to deliver kind of real development that's going to be meaningful on a large codebase with large teams in enterprises um where they have existing things not just um kind of single individual people sitting there kind of doing vibe coding which you know you can do vibe coding on on Kira as well by the way we think that that's just not sufficient for what makes development's going to need.

Yeah. and uh talk to me about what it actually looks like to set an agent off and say, "Hey, I got a task for you. Come back to me uh in a few days," which it sounds like that's where we're going. Uh we've been tracking the meter benchmark and it seems like we've been seeing doublings there, but again, a lot of those have been the benchmark has been h how long would it take a human to do this task? The actual agent might have done it faster. Um, and so you it's not necessarily that you're actually letting something cook over the weekend. Uh, what's the experience been like and and what have people been reporting about uh these longunning agents? Yeah, I think the first and actually most important thing is thinking about how you actually kind of have a mind change on how you think about software development where you think about not about do this task, get it back, look at it, do this task, but how are you thinking about directing a lot of agents to go out there and do lots of different things and and let those run for long periods of time where they can kind of have amorphous tasks like instead of go write me this function like try to go solve this problem for me and then it'll come back and uh and and then but you can but if you send out two or three or 10 or 20 or 50 of those things then your job as a software developer and as a product leader is actually much more around coordinating those when they come back troubleshooting make sure that you know directing them course correcting etc. Um and so I'm excited about that. We've already seen these um these processes go off and work for multiple hours at a time um on on particularly like really hard tricky amorphous tasks and um and we think those things are are going to continue and be more the norm of how software developer teams change what they accomplish. Yeah,

we think hero is going to be the engine that's going to drive a lot of that.

Yeah. Yesterday we were talking to uh Vincent from Prime Intellect and they do some of this like fine-tuning on smaller models and he has this thesis I think that you share that uh a lot of businesses will need to take a a pre-train and then and then bring their own data fine-tune it not just because it's important from performance and output but also from cost. But I'm interested in understanding um how you think the market will shape out. Do you see implementation partners and like consulting firms coming in and doing that? I was asking him like uh

yeah,

you know, there's a lot of tech startups that are going to be able to do that. They're going to understand I need to build an RL environment around my app. Uh but for larger legacy companies, they might not understand. So how are they going to wind up uh using that tool in particular? I I think they will and and actually just want to highlight one piece there where some of what we announced today. Yeah.

Um is a little bit different. We announced this idea. It's it's an open training model with Nova. Um and so the difference and what you just said is people take a pre-trained model and they'll do RL after the fact and they'll try to do some some fine-tuning. Um which is great, but there is actually limits to where that does. In fact, if you do too much post- training, often times those those models will forget what they've done at the beginning. They'll start to lose some of their reasoning and their core intelligence. Yeah. I mean this is an unsolved problem um except when you go and insert your data in the pre-training phase and so what we do with Nova is we expose checkpoints you can take a 60% trained or an 80% trained um model pre-trained model insert your data into that pre-training phase mix it in we then expose actually Amazon training data to you via an API that you can then mix it together and so it's like you said here's my all my corpus of corporate data here's everything that I need to know about my industry We then mix that in and then [clears throat] finish pre-training the model. So you get a pre-trained model that totally understands your company and your data. And then you can go do fine-tuning. You can go do reinforcement learning gyms. After that you can shrink them down and distill them. You can do all those things but on a pre-trained model that deeply understands what your company does.

And is that called mid training now? Is that the right buzz word for that?

It's not like we're and mid training is a different thing. the first time that anyone's ever exposed this idea to to to deliver pre-training checkpoints where we can mix in your data. No one's ever done this before. It's first time.

Uh, great. Yeah. Well, then yeah. On on market structure uh do you think it's self-s served enough that you know large corporations will do it or do you need like do you need an AI lab? Do you need an AI scientist? Do you need someone to who can you know write TensorFlow or PyTorch or something to implement this or is it something where you know just a normal software engineer at a large company could go and pull this off the shelf and implement it?

Yeah, we we'll see. I think we're going to keep working on the tools today. Um I do think that for some enterprises they'll want to have some consulting um folks that help them with this. I think we'll have some some people where you have some experts that can come and and teach how to do this. And I think we'll quickly get the tools to a point where you know it's not somewhere where uh you know a non-technical person is going to go do this for sure. But um but it may be a software developer that that tends to be a little bit more on the the AI or ML side um that we hope is going to be able to go do this without having to have a whole bunch of expertise about how to go pre-train a frontier model.

Yeah. Uh on the cost side obviously you're working you announced a new chip. Uh I imagine that there's you know the emergence of some synergies across the models that you're developing the software you're deploying the cloud and then also the chips. Uh how are you positioning the like the tranium ecosystem? Is this something that you're you're planning on really doubling down on across the entire stack? Uh or do you want to be more chip agnostic? Are we going to see you buying TPUs in the future? Uh, no, we we definitely um well, a couple of things that that there to unpack. The first is we're very excited about Tranium and think it has enormous potential and we absolutely think there's a benefit to optimizing every single layer of that stack where we have um the best cost performance um that we can deliver at at Trrenium. We have optimized models for you to use and applications and agents at the top of that that we talked about.

So, we think that whole um optimization of that stack is going to be critically important. And of course, we're gonna support choice for our customers as well. And so we'll continue to offer um GPUs from Nvidia as an example and um and we have a very tight partnership there. But but we do think and we're quite excited about what Trrenium 3 is going to offer for customers. And I do think that we're going to see an explosion of that ecosystem as more and more people um get access to those chips and are able to take advantage um of the pretty significant cost performance benefits that you can get from running on training. How are you thinking about uh opensource the open source ecosystem that you need to build around tranium? That's the big discussion with the TPU right now. The question of you know Google has some amazing folks. They have some amazing uh software folks. It seems like uh they don't necessarily need to open source everything. Uh and so a lot of people are waiting to see how much the the industry, you know, builds open source alternatives independently uh versus how much does Google just give away. What's your thought process on building a an open- source ecosystem or even just giving developers access to closed source software to run efficiently on Tranium?

Yeah. No, we're we're all in favor of having a an open set of software to run on Tranium. In fact, we've we have our uh Neuron uh SDK um which is open source today and allows everyone to to contribute to that. We we think that the the more that we can collaborate on that software ecosystem to make it easier for people to to use chips and we of course support um the broad set of whether it's PyTorch or or other kind of open frameworks as well. Um so we collaborate across the industry on that and and are big advocates of um contributing to and uh and supporting that um open ecosystem. Jordy

uh love to get uh your insight on just like general constraints for for AWS as a business. What you guys are doing on the power side is that is that a real constraint? Uh anything that you can share there?

Yeah. Uh you know, look, it's um as we're scaling incredibly rapidly, we've um you know, we recently announced that we've added um 3.8 gawatts of data center capacity in the last year alone, which is just an insane amount of data center capacity.

Thank you. Um, [laughter] oh, you're welcome. I don't know. And uh and and so it's it's ramping incredibly fast and it's it is a constraint. You know, we have more demand than we have supply today for uh for AI. Sure.

Um and as we ramp up the supply chain, we think about all of the constraints. We think about chip constraints, we think about networking constraints, we think about power constraints, we think about networking constraints, um data centers, etc. And so we're we're working really hard to to try to remove every single one of those. And uh when with an industry that's growing as rapidly as the AI one is, there's always going to be some constraint and uh and we work really hard to keep removing blockers every every time so we can keep growing fast.

Makes sense. Um well, we have a hard stop. So, thank you so much for taking the time on such a busy day to come chat with us. Uh would love to have you back on the show and go way deeper, but thanks so much and uh congratulations on all the massive releases. We're excited to dig in deeper and keep chatting about them. Uh but have a great rest of your day. We'll talk to you soon on coming on. Cheers.

Um, let me tell you about public.com. [music] Investing for those that take it seriously. They got multiass investing and they're trusted by millions. Um, we