Claire Vo of ChatPRD on GPT-5's developer-friendly design and the need for role-specific models

go back and forth between those is is is super interesting. So, wave 12 on the way soon. Hope hopefully we'll we'll have a lot more more to share.

Last question. Uh, yeah, hit the soundboard, Jordy, for that. We're wave 12. Wave 12. Fantastic. Um, uh, last question, we'll let you go. Uh, what is your probability that AI will get a perfect score on the IMO next year? Interesting. Um, so we, by the way, we just had the IOI, which is the the programming version, like the programming Olympiad. Um, and I think there's a good chance that we'll have a golden medal at the IOI for for this year announced as well. I think perfect score for next year.

Wait, uh, we as in humanity or we as in cognition

as in humanity. Yes. Yes. Yes. And AI perfect score. Yeah. Uh, sorry. And AI gold medal, right? Uh, perfect score in the IMO next year.

I think it's got to be north of 50. Honestly, I I would put it around like 75% or so. We'll see.

Well, thank you so much. I we'll be following you closely. Uh and uh good luck to you and uh congrats on all the progress. Very fantastic. We'll talk to you soon.

Awesome, guys. Talk.

Bye.

Let me tell you about bezel. Getbbezzle.com. Your bezel concier is a is available now to source you any watch on the planet. Seriously, any watch. And we are joined by our next guest, Claire Vo from Chat PRD. Welcome to the stream, Cla. How you doing? What's going on?

I'm It's a fun day today, isn't it? It

is a fun day. Uh, what was your reaction to the stream? Um, what was your reaction to GPT5?

You know, GPT5, the first thing I said and I got a little early access is I said it's a developer for developers by developer. This thing is built to be a software engineer.

You've seen a long string of your guests come on and really speak about the coding abilities of it. And what I think is interesting about this particular model, especially because we're seeing them deprecate the old models in the chat GPT experience,

and we're seeing a lot of positive feedback, but I do think there are drawbacks to a model that's so clearly tuned to a developer use case. And as somebody who's building an application um that isn't focused on agentic coding, I have noticed some personality quirks that are going to be really interesting to see how they shake out um

as we roll out this model to our users.

Walk me through those. What are the what's the timeline? How how much like how much time do you have to kind of move users over to five before

Yeah. Yeah. So I I mean I think we have tons of time from the API side to move move users. And in fact, you know, our strategy at ChatPD is not to just upgrade to the latest model. I know Zach at Warp said like why wouldn't you want the latest intelligence? And the reality is because we're doing a lot of business strategy and business writing. I actually want to validate with our users that they're they're getting the quality of strategic thinking output writing that they really want. So we actually AB test every single model roll out and really evaluate for user quality, token generation, all those things. And you know, looking early on, it yaps. Man, this thing just wants to go through tokens. Right now, I'm seeing four to 10x the number of tokens generated between the, you know, four generation models and five. And when you're in a business context, you do not always want longer

words, you know, and so it'll be really interesting. there. It is certainly focused on execution. So I, you know, I've heard a lot from the OpenAI team, it's steerable. Yes. And its natural inclination is to drive you towards like how, what, very tactical, very specific. And and so if you're trying to zoom back out at a um strategic level or focus on a business initiative, it's actually a little harder to tune in that direction. So, you know, I think there's a lot of positive things for me as somebody who uses Agentic coding platforms, who writes a lot of code. It's my daily driver now. I love it. Um, but for other use cases, I think it's going to take some time to figure out if it really is optimal in use cases where intelligence actually isn't the differentiating capability.

It's very interesting to think uh the the best the best product manager is not the one that writes the most the longest doc.

No. and you don't send your engineer into your executive meeting like I and I I really am looking forward to the time where we're not getting these numberbased models where actually I can get like GPT developer

or GT GPT strategist where they're pre tuned and trained for the role they're going to play as opposed to general purpose but clearly oriented towards a set of set of tasks. And I just think if you look at this model, it was oriented towards um engineer software engineering at least in my experience.

So have you been tempted to launch any type of agent like agentic coding products? You are you guys are obviously responsible for creating documentation and if you look at the other guests that have joined today, many of them are competing with each other in different ways and trying to own different parts of the stack. you guys have seemingly stayed really really laser focused and no one else uh is doing anything like you're doing at least on the show today. But talk about like picking your your lane and kind of like

optimizing.

Yeah, we're integrated with a lot of those platforms. So a lot of the kind of like prototyping platforms, vzero.dev, lovable, all those we integrate. We just released our MCP. So I use chat purity pretty consistently inside cursor through our MCP. So I think of we we think of ourselves as the product pair to the AI engineer. Now what's really interesting about my experience with GPT5 is the one place it actually does really well is technical specs and that's a place where u chat PRD has sort of bridged into engineering execution. often our product managers are generating a PRD or some sort of business document and they're actually going the next layer and developing a technical spec. The GPT5 technical specs fed into these agent coding frameworks or prototyping frameworks output output much higher quality assets on that end. So I do almost think there's going to be this kind of like right model for right use case especially in our kind of business and so we think of ourselves as integrating. The one thing I have thought about with GPT5, it's the first one where it feels really simple to just go ahead and roll your own agent coding framework or um prototyping framework inside of our application. So never say never. It's something that we get asked for a lot, but we were we're friends we're good friends with almost all your guests on your show today. And so we like we like the role we play in terms of being the product manager pair to all these AI engineers.

Yeah, that makes sense.

What are you looking for next? What am I looking for next? I mean, in terms of um model capabilities, what I think is really interesting about OpenAI and why I'm really committed to the OpenAI ecosystem, even though I test and use a variety of models, is I think developer support is a real differentiator here. So we spend a lot of time talking about model capabilities and for application developers certainly ones that are doing more complex applications like agentic coding model capabilities really matter like core IQ of the model matters but the other thing that matters you know as somebody who has built developer tooling products it's developer experience matters the primitives in these APIs matters and so what I'm really pushing the open AI team to think about which is in addition to the core intelligence the model. What are the developer tools you need around these models to really make them a platform on which a variety of applications can build? And I do think that OpenAI has disproportionately invested in developer experience, but I'm always looking for like give me better out of the box tooling, give me more control over these models, um, give me more hosted services, all those things that as an application developer are just going to make it easier to deploy these models in production beyond the core kind of intelligence of the models themselves. What was your read on 4.5? Is there a world where, you know, I'm I'm I'm thinking about the the the product manager versus the engineer. You have your 03 go crunch some really hard reasoning and then you have four five turn it into uh you know stronger pros or or like more you know a human language.

Yeah. So I did a lot of experimentation around 405 and 41. 45 was my favorite pros writer

by far. It was was loved from a business writing perspective. I thought the pros was the most natural. It was really slow. Like

untenably slow. And so the um the compromise we made in our testing is we ultimately ended up with 41 as the fan favorite for business writing when we were balancing off both quality of pros and intelligence as well as performance which for application developers is a real consideration. So I landed on 41. 41 is the model that's being tested right now against GPD5 in chat purity. And one of the things that I have to go do now is figure out how to get chat GPT or GPT5 to stop writing it. It writes a lot and it only wants to write in bullet points. So I've got to go back into our prompts and figure out how to direct it to be a little bit more businessoriented.

Bullet point maximalist.

It's the new mdash. I'm telling you, you will not be able to stop seeing it. It it just all it wants to do is write a bullet point and call a tool. like it I was using it in cursor and it just kept maxing out

my tool calls. I'm like you do not need to read 50 files to to do this. So I do think you know application developers are really going to have to think about how they slot this into their current workflows. There's definitely tuning that needs to happen. But I am telling you you are going to see a lot of bullet points when this thing rolls out.

Yeah.

In 60 seconds where is product management going? A lot of people talk about the, you know, examples of product managers that are starting to ship code themselves, ship whole features,