AI agent defeats Excel world champion in live head-to-head competition

Speaker 1: a while. Let me tell you about Codex. Codex is a powerful workspace for getting work done with AI agents and playing video games now. Whether you're writing code, analyzing data, creating content, or automating business workflows, Codex helps you move projects forward from start to finish. And we have our next guest with us in the team. Big. This is And relates to what we were talking about earlier.

Speaker 2: What's going on, guys?

Speaker 1: How you guys doing?

Speaker 8: Yo. Thank you guys for having us. And congrats. On the acquisition.

Speaker 1: Thank

Speaker 8: you. I'm seeing Codex ads, Cerebras ads. Yeah. Yeah.

Speaker 2: You. Great to see you. And Michael, we are honored to have you on the show. So Yes. Nico Don't

Speaker 5: be here.

Speaker 2: Why don't we we've been dreaming actually dreaming about the like dreaming about this moment. We've talked we've we've John and I have talked on and off air about wanting to get Yes. The greatest Excel jockeys on the show. Yeah. We finally did it courtesy of Nico.

Speaker 1: So maybe quick introductions.

Speaker 2: Yes. Nico Set the scene for us. Why are we talking why are the three of us talking with the Excel world champion?

Speaker 1: Yes.

Speaker 8: For sure, for sure. Yeah, guys. I'm Nico. We chatted about a year ago when we launched the first like superhuman Excel agent. We launched Shortcut, caused like a big storm Yeah. And we've been like heads down in spreadsheet hell for like a year trying to build an agent that was really really freaking good. Yeah. A whole combination of things happened that I felt like we did pass this threshold where maybe it's at the level of a Michael Jarman or Demerle, and I'll let Michael introduce himself. But I flew down to San Diego last week, nervous as hell, calendared a studio, we rented it out, and competed live on four official championship models, me versus Michael. And of course, I had shortcut, but Michael had his, you know, big brain.

Speaker 2: There you go. When did you first name do you remember your first Excel session?

Speaker 6: That's a great question. I think it would have been when I was about seven or eight. I think for some reason, I decided I want to wanted to put the 2,002 World Cup in Excel. I mean, somebody else had already done this because somebody else was running the tournament, but I decided that seven year old me could do it instead. That's my earliest memory with Excel. But, yeah, since then, sort of built a career doing various things in Excel, and it's now a competitive esport for reasons that I still don't really understand. But it's it's fantastic. There's a great community around it. It's surprisingly watchable. And, yeah, I won that competition in 2024. There's a financial modeling sort of World Cup that is sort of a little quieter. I won that last year, which is why Nico decided that I should represent humanity and see if we can all keep our jobs.

Speaker 1: What what is the shape of of the challenges in an esports competition around Excel? How much randomness is in there? I mean, I've seen I've seen the the the screen recordings where you have to change the color of a bunch of cells, reshuffle sort things, write formulas. But how much do you know going in? How defined is like the scope of problem that they're going to give you?

Speaker 6: Yeah. I know the videos you're talking about. It's it's some dude who like speed runs like doing all that. That's not what we do.

Speaker 1: Okay.

Speaker 6: That I do not have the dexterity for that at

Speaker 1: all. Interesting.

Speaker 6: So what we do is it's more sort of problem solving in Excel, a bit like a hack essentially. So, you know, you'd be modeling some sort of board game or, you know, navigating around a map. The the year I won, you sort of had 20 characters in mod of Warcraft because they were sponsoring it that year. Yeah. And you sort of had to try and track them leveling up and doing quests and whatnot. It can be very, very random, like, what comes up, like, or not it's your sort of question or someone else's. Like, there's a vast array of things that people can do using spreadsheets. So, yeah, the question makers do make a good effort of trying to make a variety of topics people to answer questions.

Speaker 1: And then on the actual grading, how much is qualitative versus like once the solution has been found, the buzzer buzzes and it's deterministic every single time, like pure math versus there's oh, you know, you got a nine out of 10 from this judge because there's some style points in there.

Speaker 6: No. There's no marks for the style at all, which I think is for the best. Yeah. I think if people had to spend times making things look pretty, I don't think it would be nearly as entertaining. It's entirely deterministic like somebody else has already calculated what the answers are.

Speaker 1: Got

Speaker 6: it. And then as you're copying them in, they either go, yeah, you've got them right or no, you haven't.

Speaker 1: Okay. Is there anyone who tries to basically eschew Excel entirely and solve the problem with Visual Basic? Because if I believe if I understand correctly, Excel has like a VBA back end and you can basically write, you know, you know, programming language. You can just write code.

Speaker 6: Yeah. Absolutely. So there's nobody who tries to solve it entirely in Visual Basic just because anybody who's watched this or watching this who's coded in VBA knows it's a terrible language. It's so slow to get going.

Speaker 1: Yeah.

Speaker 6: There's bits there's chunks of code people have especially especially for for things things like looking at the colors of cells, like if you've got a map and like the blue cells of water, the green cells of land. There's no native way in Excel to do that with formulas. So you need VBA for that. There is one guy who there's now Python and Excel that's been there for about two years, I think. And like you write some Python code, it gets sent to Microsoft. They run it. They send you back the answer. And there's one guy who competes in that and does quite well. His general thing is he's very slow to get going, but when he gets going, he sort of, you know, really starts to pick up pace. So again, it's a little bit of the randomness of what sort of case comes up. Some suit him. Some suit him less.

Speaker 1: And then what is the what what are the rules around artificial intelligence, copilots? There's there's you know, AI is coming to Excel. Do you think competitors will freeze on a particular version of Excel that doesn't have AI? Or will there be an AI subcategory that allows you to use some copilots, everything, unlimited, unrestricted, you know, and no rules apply categories. How will all that play out?

Speaker 6: Yeah. So AI is banned in the main competition. It's getting increasingly difficult to police because it's it's making stuff that looks more and more like a human would compare sometime maybe eighteen months ago. Sure. Maybe they'll start running it like an AI assisted competition. I know a few people have talked about that. I don't think it'll be very popular. Like, you look at the world chess championship, people watch that. There is a world chess engine championship. People don't watch that as much. Mhmm. So but, yeah, it's it's currently banned. I think we're still gonna wanna keep on the latest version of Excel because they're adding other stuff to it that's quite helpful. Like, added Regex about a year ago. So we didn't have that, so people would complain. But, yeah, at the moment, it's kind of an honor code. Like, when you get to Vegas where it is to do the finals live and you obviously can't just be like, hi, chat GPT. Do this for me because someone will notice. Yeah. At home, it's currently an honor code, which is, yeah, something we're finding harder and harder to police.

Speaker 1: Yeah. That makes sense. So Nico, like what are you learning from Excel championships? Yeah. Yeah. Yeah. Yeah. This is your competition. Is there training data to be learned? Are there sort of like the the rough edges of the spiky intelligence you notice things at at the at the top tier level that then can be applied for everyday users that use your product? How are you actually working together?

Speaker 8: Yeah. Yeah. For sure. I wanted to know that like, one, could you really compete at this high level and what has to be true for you to get there and how much of this is like mastering the art of using AI versus how much could you like institute into the product as like a first class experience for everybody. And it turns out that like, you have to be really fucking good at using, sorry, really good at using AI, to even have a chance. So like when I competed, I actually did some practice cases at home all weekend before leading up, and like AI will take the same way in coding, like the path of least resistance. It'll be like, okay, well I didn't check every single question. I looked at the hints and then decided what I should answer. I'm like, no. And you'll see this in my prompts. I'm like, do a blank space audit, spin off adversarial review sub agents of multi different model types at the frontier to see that, like, you know, they think differently. I want you to catch yourself. And with enough advert like, with enough verification loops, you can clearly get a 100% on, like, any realistically hard challenge that is verifiable. But that's not how most people use AI, and I wanted to know what what it required to get there and then work backwards from like, do you bake this into the product? Because, there's a trade off to these things. And you know, I actually didn't even use the best model. Like, didn't use Fable. Interesting. I had to use Opus four eight fast because it's the right spot of the front Because I would probably have lost.

Speaker 1: On speed.

Speaker 8: To my goal. If I if I used Fable because we were getting a 100% accuracy on almost every question, but I was, you know, faster because of the model I had chosen.

Speaker 1: Is there strong transfer learnings from these verifiable tasks versus the less verifiable stuff that people do in in Excel all day long. Like like the average investment banker who's doing a DCF, like, it's not like at the end of the day, they're like, yep. The share price should be $200, you know. You're you're building a case.

Speaker 8: Right. Right. So in I would say AI is better suited for the contest than it is for real Excel work. In that, this is like you asked first, which is, is this programmatically verifiable? Like, there true or not true? And the answer is yes here because we have a contest. But if this was judged by a committee of, like, experts based on taste, it would be way harder. Now you can say that as AI becomes more tasteful, and it is, you see this on front ends, like it's getting a taste for what looks right, we benefit from that. But there's another more difficult form of verification, which is like, okay, well, there's 20 ways to make a DCF. They all look great, but one is specific to your company. Like how do you learn that?

Speaker 1: Mhmm.

Speaker 8: And that is like the real long tail product learning curve that we bake in with our customers. Like that's the real challenge right now. It's AI will do things but there's a million there's a million right ways.

Speaker 1: Among among your customers, what what does demand for speed look like? I mean, we were talking about Cerebras, like Oh, man. There's this trade off. And I feel like for like, the nature of an investment banker, is at least on the junior level is often the MD is like, turn this around ASAP because the client is meeting in an hour and I need you to do it as fast as possible. And there's no excuse for, well, I'm using a really solid model, but it's gonna take five hours to get back to me. What does demand look like? What what are customers telling you?

Speaker 8: Oh, man. I I don't think enough people are talking about Cerberus or whatever other alternative ways there are to get to like whatever tokens per second that was. Yeah.

Speaker 5: It was

Speaker 1: like 50 you

Speaker 2: know, every company has a different style.

Speaker 8: Like it's going to open up computer use. It's going to open up everything.

Speaker 7: Now you

Speaker 8: have to hit you have to hit a certain level of competency and accuracy first before it even matters at

Speaker 5: all.

Speaker 1: Yeah.

Speaker 8: But we are sort of there with Soul and Fable. Yeah. Now, like for example, we even with Opus four eight Fast, we turn like an operating model build out with like 15 quarters of historic fifteen years of historicals for quarters, like Mhmm. That can take thirty hours for a hedge fund to build, and then two hours to update every quarter. We do that like in one hour and then ten minutes. Wow. Now we could get that to like 10 times faster. Yeah. And that's actually what's going to be a huge unlock because hedge funds, for example, want to track not 200 companies a pod, but 2,000. But you cannot do that because they're Excel firepower bottlenecked. There's another bottleneck, which I think the future will care about, but it's like, even if it's faster, it has to run headlessly because you're not going to open up 50 models or a thousand models like on your computer, and Excel does not run headlessly. I bet you it will soon though.

Speaker 1: At all? There's no there's You

Speaker 8: can do some clever clever arrangement of like vms vms, like expensive licenses and like some companies might do this to get around the bottlenecks, but I'm I would I actually don't know for sure, but I would bet my life that Microsoft is working on this.

Speaker 2: Interesting. Niko, are you recruiting Michael yet?

Speaker 8: Michael. No. Don't think I could afford Michael. No. I would say Michael, the best form of recruiting was after the contest Michael asked me if we can get his team on shortcut. And we did.

Speaker 2: There you go. Wait. So Michael, what do you what what's your what's your what do you do day to day when you're not

Speaker 1: Winning championships. Yeah.

Speaker 6: So I've about six months ago, moved into private equity investing in freight rail across North America.

Speaker 1: Thank you. Thank you

Speaker 5: for your service. Alright. Thank bro. Appreciate it.

Speaker 1: Faith in humanity restored. I needed that today.

Speaker 2: I needed that today.

Speaker 1: Thank you, guys.

Speaker 2: To know that our best and brightest are still choosing private equity

Speaker 1: Yes.

Speaker 2: And not giving in to, you know, the Nikos of the world building our our future.