Red Hat CEO Matt Hicks: 80% of enterprise AI spend should migrate to smaller open-source models — here's how

in the reream waiting room. So, let's bring in Matt Hicks to the TBPN Ultradome. Uh, wait, we have a second. I'm sure he's been tuned out [music] because we had him.

There he is. I'm sorry for keeping you waiting, but thank you so much uh for joining the show today. How are you doing?

Hey, I'm doing great. Great. Thanks for having me.

Yeah, really happy to have you here. Uh would love would love an introduction on your on yourself a little bit. I know that you became uh the CEO in July of 2022, but I'm I'm more interested in understanding the the landscape of the of the business today. how you see the how you describe the shape of Red Hat and all the business all the different business lines that have developed over your tenure to become what the company is today. Yeah, we've uh you know, I've been at Red Hat for a long time, actually over 20 years now. And so I started at Red Hat actually in it success

when it was just uh it was just ra at the time. And and if you if you fast forward to today, we have

re we have open shift, we have an we have Red Hat AI. Um I like to describe as like when I started uh

in open source, you really only had an operating system that was there. And today you have any software that you could create any company idea to help you do that. And and we try to cover that that gamut. Everything from operating systems to you know clustered capabilities with Kubernetes to AI capabilities and smaller models. So it's been a fun uh fun ride for a couple decades here.

Yeah. I I would love to just jump into the the AI side of the business. Um, how are you thinking about bringing a Red Hat appropriate product to bear in AI? Are you, it sounds like you're not training your own huge model trying to compete with the, you know, the big foundation models, but I'm sure there's a bunch of ways that you can plug into your your client base. How are you thinking about um the the shape of the AI business over the next few years? Yeah, I think for us, and this is how we use things internally, we think you're going to end up using Frontier models and the capabilities there. Um,

in addition to smaller open- source models, um, and I I often describe if you go back to like when Google search was

invented,

you had this feeling like it was going to change everything in business,

but you didn't apply a Google search appliance to everything you did. Like you still had relational databases. you still had specialized applications because you needed that control. Um, frontier models to me are sort of like Google search. They're going to change everything. It's going to be pretty incredible. We don't know what will happen there, but they show possibilities.

Small models for us are the things that you can train. You can specialize. Uh, you can run them in a factory or space station or you can run them as a mere mortal yourself in your own data center. Um, and that combination is really powerful. Like that's what we use internally. So, our business is squarely in that smaller open-source models. Um, when I say small, they're still

Yeah.

pretty large compared to traditional computing, but compared to those frontier models, um, they're just a very different tool.

Yeah. Walk me through what a customer might experience if I come to you and I say, "I have a business problem. I have a whole bunch of you know like like data flowing in images that I need to OCR and transform into JSON or something like I have a defined business problem that uh that transformerbased large language models will be good for but I want something that's you know 59s of reliability that's you know I control the the the ownership of the code it's it's economical um what are you bringing to bear to help the customer achieve chieve their goals.

Yeah. So I think if you have that use case and I often talk about them as like the three Ps we use frontier models to understand what's possible

once you know it's possible with frontier models. If it's not possible with frontier models it's like just stop there.

It's not going to translate that well.

But if it is yeah

then your second P is production and your third P is profit with it. uh the models we look at are about a hundred times smaller um just in size. And so generally you could say if you can run them well, they're going to be a hundred times cheaper

to answer those questions you're going to ask it. And so we work with customers to really find that use case. It doesn't always fit every aspect,

but how many of those pieces can you move to something that is a hundred times cheaper with it? And it requires some you know you're competing with the Googles of the world or Microsofts in terms of operational discipline. So you have to be able to run these things well and this is word you hear inference that pops up. We help enterprises with inference just normal mere mortal data centers GPUs Nvidia AMD and running these models efficiently. So you can start to decompose that problem into you know what are the areas we can optimize and really control ourselves. So that that's one of the use cases. We have a lot but that's a a popular one because

so yeah getting getting more specific company comes to you and they're saying like I'm I'm spending $2 million a month with this with the on these like frontier models and and how where do you go from there? are you like breaking down exactly how they're exactly what those tokens are being used on and then basically saying like hey we we can actually break out the spend and reduce the cost by this amount. Obviously, Red Hat has a margin uh but but how did those conversations go specifically and how much spend today on frontier models really shouldn't even be going to frontier models and should be going to uh some of these smaller uh more more um uh uh specific models.

Yeah, I think if you look at not overly simplified but we sort of live in the world of text on it. So if you said, "Hey, my use case is um video generation and all of these capabilities," that's not really our wheelhouse. That that often isn't in that space where um a lot of your business is going to operate. Understanding policies, um understanding numbers on it. So that's the first thing we look for is does this fall into that classic enterprise use case? Um because there are things you'll keep on frontier models. The second part for us is where do you want to run these? can rent space at a core for example to get GPU capacity or it might be data sense enough you want to host it yourself and then you're in a hardware purchase. Um but for us then it's about going one by one. Sometimes it requires changing the data a little bit but it's making those use cases run efficiently on it. when I look at um what's that mix of what we use these big models for that we just don't need to I think it's going to be really similar to cloud spend um in my role they're really powerful I love the learning experience people are going through and using them but I'm going to bet there's a 7030 8020 split of where we could take things that were paying more than we need to and be able to optimize them and it's just in that balance cloud was really good at learning a new operational mechanism and skill set. We're going through that same phase, but I think we'll be able to optimize a lot the majority to smaller models even if you start in what's easiest and shows you what it can do in the larger ones. H

how have your how how have your thoughts on the relationship between building a for-profit business in and around open source technology changed over your career? It's um so I started at Red Hat as I loved the open-source model. I was actually a consultant at the time and you know I had worked with Linux in college and my first experience like I couldn't be stopped when I was working with Linux because if it wasn't working for a customer I could change it. I could figure anything out. I didn't have to be dependent on someone. And I loved that

that feeling that power that came with it. um just open sourcing things and assuming a business will happen I think is also pretty flawed with it like you you have to do something different that customers value that the open source community doesn't do and in in our case open source communities is a great innovation model they move really fast

we help customers that they can't keep that pace and speed of open source they want the innovation model, but once they've built their app, they need someone that's going to help support it for 10 years.

And that's that value we do that communities just don't care about. They're on to the next feature. And so, it's been a really nice balance of we work with a community, we complement it, but we provide enterprise value. I think that's what's going to shake out in this world with AI of how do you find that that symbiotic relationship where you can leverage open source you can you know use innovation models or other areas but you still provide a concrete value to customers where they know what they're paying you for

how have you been processing the the fact that it feels like meta is pulling back from you know largecale open- source AI efforts uh China seems to be running away with it uh Is there any other nuance to is it just like a business model problem that we haven't seen uh like the red hat of AI kind of crop up or or stick around in the case of OpenAI as they've kind of evolved their business model although they do of course have GPTO OSS uh and and you know Metal Lama is is is still a real product. Um, but it does feel like China has just taken open source AI way more seriously. And I'm wondering if you have any read on like uh like was that always the way it needed to happen? Were there alternatives? How did you interpret the open-source AI LLM war play out?

You know, I I put it in two buckets. There's um a race to AGI.

Yeah.

Personally, I'm not a big believer in that, but you know, it's going to produce some incredible things on that journey. you. We'll see if we get there. But I think a lot of the big players when they talk about these billions of dollars that go into training models,

they're in the race for AGI

at that point. And that that's a new and exciting domain for them to chase. And I think Meta is in a similar boat there

what they're investing. Um I love the fact that they open sourced it, but they're in that AGIish

camp and race. Um the number one innovator in open source models I would say is academia at this point. It China has always been strong in open source. US is very strong. Europe is very strong. But if you go look on hugging face at the sheer number of specialized tools we have out there. It's more than you're going to understand how to use well today. Um, so I often the coaching I give customers is I don't think we necessarily need a lot more. It's just that stability in a few. So I worry less about um, you know, China's created some pretty impressive innovations like in the Quinn model on training,

but as long as those are shared in academia, we'll see the next mistra pop up with options there. But but that's how I separate the two. There's an AGI race. I don't think the um the financial mechanics are going to work out for open sourcing there.

But then there's an academled ecosystem that already exists that's incredibly vibrant um that is not chasing AGI that'll fit really really well with the um the boring enterprise use cases like how do you run a company a bit more efficiently?

How do you um chase the next engineering innovation more efficiently? that doesn't necessarily have to be those uh those big guys.

Last question for my side and then we'll let Jordy ask anything he has. But um I I'm interested in advice for we talked to a lot of founders, a lot of startups that are kind of building the Red Hat of X. They're thinking about open sourcing a great piece of technology, building an open source community. Uh we often talk to Y Cominator founders that come on and tell us, I have 5,000 GitHub stars already. I'm raising money. I'm going to build the next thing and they're not going after Red Hat directly. They're just they're they want to learn as much as possible from you. So, do you have any advice for founders who are earlier in the journey? They might have built something um great, a a useful tool, a piece of software, and they really want to get the the red-hot model correct. They want to have an open- source community, and then they also want to build a great for-profit business on top of it. You know, I think the the mistake I see the most is um I want to use it for a marketing tool and then keep all the control

myself. Um almost always those fade out. You can you can do that. You can get a lot of eyeballs with open source. But if you don't know who your community is, it could be a user community, it could be a contributor community,

but if you don't know the value you provide them and they provide you, that marketing effort of stars only will fizzle.

And it is very tough to give up some control and product roadmap to your competitors to build a contributor community, but it's what you have to do. We work with our biggest competitors side by side in Kubernetes. Yeah.

Um, so I think for founders really knowing um, is it the right time for them to give up that control and being authentic of their model, it'll give them staying power in open source. It's a savvy ecosystem.

No one's going to get tricked into like a star count and then you build a, you know, durable forever business off that. But a lot of options down there, but I think knowing those and putting some thought into that is a really important first step.

Makes a ton of sense. Jordy, anything?

What uh do you think working for one company for 20 years is underrated? Feels like uh

company man.

May maybe maybe some of the people that are coming on the show will end up uh

Satella.

Yeah.

Perfect example of this

Satia. But uh but kind of what what what are some like un unexpected sort of positive elements of being, you know, deeply embedded with a company for as long as you have? You know, I think when I walked into Red Hat as a joining the IT team to port Pearl applications to Java, I didn't really see myself in this seat 15-ish years later. Um, I didn't, you know, I was lucky enough to be involved in Open Shift in the early days. I didn't really see that. But the tenure um especially working in IT, seeing every part of the company, working in a startup in Red Hat with Open Shift and just the pain and struggle that goes into bootstrapping something in a pretty successful company. Um it's what lets me do the job that I'm doing now because I I have the depth and breadth to push on things in a different way than if you were an outsider. And I think it's really powerful. It's um you know people bounce around there's there's there's a lot of learning to be gained with that but for me I certainly use every day my 20 years of um you know all the challenges and you know benefits things that have worked things that haven't to guide my decisions dayto day. So that's um you know I still pinch myself every day. I'm like, I'm surprised I'm sitting in this seat. But, uh, [laughter] but it's been a, you know, it's been a great runup to it, and I don't think I could do a good job without that that depth of experience.

How do Yeah. How do you help, uh, a lot of people, you know, are impatient for for the raise, impatient around the the new title? What like what kind of framework do you give to somebody that comes to you maybe at a different company and says, "Matt, I'm at this company. I've been here for three years. I just got an offer that'll I will immediately make more money. I'll have a more senior title. Maybe the company happens to be the other the company making the offer is growing faster. Maybe has more capital. There can be a lot of reasons to jump ship. Uh but what's your framework uh for helping people make that decision?

One, I always warn them it's a personal framework. I'm like it's worth what you paid for it, which is is nothing on it. But uh but for me my driver has been am I learning something new in this space. If I'm learning things I can be patient for the right opportunity to leverage it. Um and I also tell them like you have to be accepting there's some luck in the game as well. Like just being in the right place at the right time with the right skills. You can't always manufacture that. But for me, I always look at opportunities of they're going to come with some pain and challenges, but if you learn a tremendous amount, you're going to leave that being more valuable than you came in to it. And you know, I take that into this role. It's why I, you know, in limited hours I'm still learning AI myself to not just delegate it down the org, but to really build that hands-on experience, but that learning dimension has always been the most important uh dimension for me.

Well, thank you so much for taking the time to come on the show today. Great to meet you. Great to meet you and

open source champion of the world.

Merry Christmas. Have a great rest of your day. We'll talk to you soon.

Yeah, love the show. Appreciate you all having me on. Thanks back on so We'll talk to you soon, Matt.

Goodbye.

Let me tell you about Figma. Think bigger, build faster. Figma helps design and development teams build great products together.

Um, gyms are coming to airports.

This is big.

This is big. Will you on your next

consumers have been saying the other passengers aren't sweaty enough.

Yeah. So, this is from

We got to We got to have gyms in right next right next to the gate.

We We have another We have another guest joining just a minute, but I want to riff on this for like 10 minutes. Uh this is hilarious because it it seems to be a collab between the the uh uh Secretary Kennedy who's the health and human uh services secretary. So there's like HHS crossing over with DOT somehow and they're like let's let's let's make the airports healthier which is like a funny thing. I feel like I feel like these I feel like these like labs don't happen before but they're happening now. Um, I don't know. The the idea of working on an airport, it's kind of crazy because if you are all sweaty and then you have to and they also said like you should dress up when you go to the airport and if you're dressed up and then you are all sweaty, that seems pretty rough. But, uh, I don't know. I I like the idea of doing something new in airports. I think it's cool. I think it's good that there's some some opportunity for some sort of grant program. I guess the question is the $1 billion grant program with how much it costs to make like is this like 10 gyms? Well, so I don't know because like there's a world where you build like a proper gym in the 10 most premier airports that has like a sauna, showers, like full laundry, like you know, you're good to go. You can spend like a couple hours in there, really get a serious workout in. Or it could just be like a couple pull-up bars and you put one in every in every terminal in America. I don't know. It depends on how how like like can you imagine if it's just like okay yeah like you know turns out JFK put in an awesome awesome application they got all one billion

little calisthenics sound

they got it all. I don't know it's it's a very funny story anyway let me tell you about fall the generative media platform for developers develop and