Substack partners with Pangram to fight AI slop with content detection — Chris Best explains the Claude-fishing problem

Speaker 1: Amazing. Thank you for coming on the show.

Speaker 2: Great

Speaker 1: to Talk to you soon. Have a good day. Cheers. Goodbye. Let me tell you about CrowdStrike. Your business is AI. Their business is securing it. CrowdStrike secures AI and stops breaches. Our next guest is the best. We got Chris Best, the founder and CEO of Substack. Welcome to the show, Chris. How are you doing?

Speaker 8: I'm doing good, man. Thanks for having me.

Speaker 2: The great slop wars Yes. Have a new era.

Speaker 1: It's not just nice to talk to you. It is amazing.

Speaker 2: It is the best.

Speaker 1: It is the best to talk to you. Tell us about the slop problem on Substack. What is the shape of the problem? What motivated this? Take us through the new product. What's the state of the union in the slop fest? The The state of the trough.

Speaker 2: The state of the trough.

Speaker 8: I mean, place I started with this, I've had the experience that I think a lot of us have had, not on Substack, but just on the Internet in general.

Speaker 1: Yeah.

Speaker 8: I'm scrolling the timeline, I'm reading stuff, and sometimes I just have this experience where I'm reading something, sometimes it's by someone that I think I know, sometimes it's something, and I have this kind of invasion of the body snatchers experience where you realize you're sort of reading something that was actually just pure Claude. Yeah. Purely generated.

Speaker 1: Yep.

Speaker 8: And it's kind of, you know, it it makes the experience reading on the internet worse because you're constantly wondering. Yeah. You constantly have to wonder, is this real? Yeah. Am I actually reading something this person believes in?

Speaker 1: Yeah.

Speaker 8: Or, you know, did Brian Chesky get hacked? Or whatever is

Speaker 1: going on. Yeah.

Speaker 8: And so I like, I've been feeling this. A lot of the people I know who spend time online have been feeling this. And then the guys at Pangram did this study because they've been running this browser extension, right, where they have their own browser extension you can install and you can just have stuff labeled. Yeah. And they studied different platforms and they found that on some platforms like, I think LinkedIn, 40% of the long form text is generated by AI. Mhmm. And, you know, in my view, it's not the case that all

Speaker 2: I would expect it

Speaker 8: to least

Speaker 2: But Yeah.

Speaker 1: I'm Yeah. Going

Speaker 2: 40 is

Speaker 1: a lot.

Speaker 8: Well, and the thing, you know, a good point here is not not all slop is AI, you know. LinkedIn, even pre AI had a lot of had a lot of stuff on there that's, you know, maybe the best.

Speaker 1: Pretty sloppy sometimes.

Speaker 8: Depending on your perspective. I mean, some people like it. That's fine. Yeah. And I think, you know, they had sub stack coming in about 10%. I started looking at this. I started thinking about this. And to to my mind, the problem is not people using AI. The problem is when you have kind of the deception that comes with, expecting that I've got a real connection to a person

Speaker 1: Yep.

Speaker 8: And I'm actually reading something that's written by no one.

Speaker 1: This photo

Speaker 2: is my experience when I realize I'm reading AI tech.

Speaker 8: Yeah. This yeah. This is the moment you realize. And the moment you realize, that's when I think of this as clod fishing.

Speaker 1: Yeah. Yeah. Right? It's about the violation of expectations. Right.

Speaker 8: It's the fact that you realize and then that you're like, oh, this changes the experience that I'm I think I'm having. And it's so I'm not just reading AI text. I read AI text all the time.

Speaker 1: Yeah. We when when there were those rumors that, like, OpenAI was thinking about a social network, and it was still really, really early, I was like, I would love to be able to subscribe to a feed of Tyler Cowen's deep research reports. Because I know he's using I these know he's coming up with really interesting and creative prompts, asking about, like, what's the food like in this random town internationally? And, like, I would like to learn what he's learning. But the but the whole premise of that would be that I'm just getting a link to chatgpt.com. And people do share these around, but we haven't really because we're not in, the link sharing world, people haven't actually coalesced around, yeah, I'm just gonna share you a link to my deep research report. But sometimes that would be better than you took you you copy and paste it and you forgot to put the is there anything else you'd like me to help you with or whatever?

Speaker 8: Or you left it in, then you get these sales emails that leak the prompt to you.

Speaker 1: Yeah. And so I can imagine that there's probably a really banger sub stack out there that where the the journalist or the writer is gonna say, look, I'm gonna go I'm gonna go find an interesting fact or I'm gonna give you a roundup and I'm going to do the work of prompting and fact checking and making sure that this AI instantiated newsletter is high quality and high quality Yeah.

Speaker 2: Example that I always give is like, if Alex Heath Yeah. Was using AI.

Speaker 1: He uses AI often.

Speaker 2: Yeah. All the time. He's talked about it. But like, Alex Heath using AI, I'm like, great. I I'm I'm not subscribing for the like writing. I'm I'm subscribing for the information, the facts that he's finding Textualization. Plus the takes Yep. And like his point of view on that. Yep. If he's just if he's just ram if he just like finds the facts and then does dictation and just kind of rambles for ten minutes, turns it into something that is, you know Yeah. Short and

Speaker 1: But the disclosure is important.

Speaker 8: So Right. Yeah. The way you can tell that it's good is because the disclosure wouldn't ruin it for you. Totally. And we have all takes on the platform. We have everybody from, you know, Sam Chris wrote a piece recently that said, if you let AI do your writing, I will come to your house and kill you.

Speaker 1: That's a death threat.

Speaker 8: I think I wouldn't be able to tell. I can tell. And then, you know, we have people that are doing the thing you're suggesting that are that are using these tools Yeah. To like Slop is not something you aid with AI. Slop is something that nobody believes in Mhmm. That you didn't actually care about. Right? If you just go and say, hey, write me a thousand viral LinkedIn posts, make no mistakes

Speaker 1: Mhmm.

Speaker 8: That's Slop. Yeah. If you spend know, if you're doing this deep thing where you're going back and forth and you're doing it I talked to a bunch of people who do this on the platform and they're worried about slop too. Mhmm. Because in a world where you get a bunch of things that are like low effort or like zero authorship stuff masquerading as somebody's careful work, it drowns out the actual careful stuff. Yeah. Like, you can have a platform where like the good high effort thoughtful things get out competed by people just like spamming the zone with stuff. Yeah. And I just think that's gonna be I think that's already a real problem on some platforms. It's not a huge problem on Substack yet, but we kinda wanna make sure that we're just, you know Yeah. Keeping the bar high and having giving the people who are making stuff they believe in the chance to win.

Speaker 2: Have you seen any like generational divide between how publishing AI writing is perceived? Because there's been a number of VCs at least over on X that are that are maybe maybe in like the back half of their career. Well, they'll just publish an essay that's just like incredibly obvious that it's just like, you know, they they maybe didn't they maybe had like a couple key ideas that they gave the model and then they just took the output and just published it. And they'll usually get kind of dunked on. But it it seems as though the author author was not even or the author was not necessarily even sort of aware that that there's like a negative perception or that it's maybe obvious to other people. Have have you seen anything like, have you seen it divide at all or is it just sort of random?

Speaker 8: I think we're in a period where these there's, like, new norms that are getting figured out. It's not clear what the rules and expeditions are, and there are there are people that have wildly different ideas of what good is. I don't think it's purely generational. I think there's people on all camps. To me, the thing you know, I think that anti slop is a pro AI stance. Yeah. Right? You don't see Sam Altman or Dario Amade writing doing these like Claude posting things and it's not because they don't believe in AI. Yeah. So I don't don't think that it's a it's an anti

Speaker 2: AI I I think you can be incredibly in favor of maximally using AI to to do research and be a be a, you know, even a thought partner and edit your work and help you think of even new ideas to include. But it's like, but it's about it's it's about ultimately Yeah. It's about ultimately creating something that's worth people's time. Right? That they couldn't have just gotten by asking the same question to the same model, at least in

Speaker 4: my Right.

Speaker 8: Claude is great. But if I wanna talk to Claude, I can just talk to Claude. Yeah. But if I'm reading you, it's because I want there's like something else. I mean, another way to look at this, you know, you gave the example of like, what if Alex Heath were making this really thoughtful thing? I would love that. I think everybody has an example of something where they're definitely cool with it. Everybody's talking to AI. It's it's valuable. On the other end of the spectrum, do you remember the guy when sort of OpenClaw was at its fervor that famous for having the bot text his wife? He was like answering his wife's texts. And like, I think almost everyone can look at that and be like, okay, that's probably not legit. Yeah. And then there's a whole bunch of stuff on the Internet that kind of falls in the spectrum between, you know, thorough research report with AI pros all the way to, like, you know, having not having to answer my wife's texts. Yeah. Like, if I'm going back and forth in the comments with you, what's the expectation? If you're writing a piece, what's the expectation? Does it is it different if you're writing a personal story versus a I think these things are all, like, new norms that will have to get created. I don't think we at Substack have all the answers. And so this is why we're kind of leading as a first step. We're like, hey, let's just have some transparency. Let's make it so that readers can scan this text if they want and have that information and do with that, you know, make their form their own judgment.

Speaker 1: Yeah. How do you think the how do you think writing is changing for writers who don't use AI? I've noticed personally I've been writing video essays in the pre ChatGPT era, then all the models were not really great for a while. Then I started doing daily essays on Substack, got accused of using AI even though I wasn't. Sometimes we pass them through AI to just fact check and grammar correct, then that gave it the AI flavor. And so I made the decision for with the team to just, like, not proofread them, basically. And so I will often be doing research in a bunch of different AI tabs and collect information. Yep. But then I'll go and just complete stream of consciousness, nonsense grammar, all these weird twists of phrases like I'm just speaking out loud. And I think it's remained like I'm getting less and less AI accusals, guess, even though I'm like I I I just writing daily, I think you're gonna get accused of using AI because it's so obvious that you would save a lot of time by not actually going and doing the work. But I'm wondering if you think that that is just a unique thing to me or if that

Speaker 2: was Well, it's not even just a problem with the writing. I I've noticed on YouTube Yeah. There's creators that

Speaker 3: Yeah.

Speaker 2: That that I used to love Yeah. That would make like history history geopolitical content or just like weird niche content, even car content. And now, I have that same experience of like ripping the headphones off because

Speaker 1: Oh my god.

Speaker 2: I realized that somebody that used to put like all this energy into like writing these great scripts and telling these stories, you just start hearing the the claudisms in Yeah. The video. And even though they're just like making an output, making a script and then reading it, you it's still like coming through and you can still tell that that they didn't put craft into it. Yeah. And so I've seen on YouTube, there's creators that are just like in the description or even in the introduction of the video saying, and to be clear, I don't use any AI in my videos. And Is it a different even friendship? As somebody who's very pro AI, I find myself like I I I feel like a little bit comforted hearing that because of all the bad experiences where I've gotten three minutes into a video and and realize like, okay, this person is just reading a script that was at at one prompt. Mhmm.

Speaker 8: Yeah. And we added this to the feature we launched too. So if you if you're writing someone on stuff, like, you can set like a how I make this statement.

Speaker 1: Oh.

Speaker 8: Where if somebody's scanning your thing and it can say, hey, I never use it or hey, I use it in this way or even Yeah. You know, hey, I use it extensively, but here's the here's the perspective I'm bringing. Here's what you can get from me. I think it's I think and I think this is true no matter how it gets more true as an AI gets more powerful. Yeah. Right? One critique is is oh, I I don't like it because it's bad, because it's like a claudism or because it's just like detectable. But I actually think the underlying thing is like, am I getting a human perspective here?

Speaker 7: Mhmm.

Speaker 8: And the more powerful the a I AI becomes, the more important it is to know that. Because if I'm not, it might still be great, but again, it's like if I wanna talk to Claude, I can talk to Claude. The question is like what human believes in, stands behind, has the perspective that I'm spending my time and energy ingesting.

Speaker 2: Yeah. The other the other thing with Substack is like you're paying for a media product when you subscribe to someone, but you're also paying to, in many cases, at least personally, allow somebody to pursue a life's work. And if and and that to me is like part of the sort of like social contract that you have when you subscribe to someone, which is like, you know, I am in a small way, I may I might be one of 10,000 people, but I'm allowing you to ditch your job and just pursue like an understanding of this of this industry or, you know, this market or whatever. And so that's also like, you know, if somebody just decides one day, okay, like I'm kinda done with this whole, you know, independent writing thing. I'm just gonna slop it up for the as long as I can. People will churn but, you know, like that that that would be, you know, a violation at least in my view.

Speaker 1: Do you think your recommender system will naturally sort audiences into what they like? So if there's someone who's subscribing to more AI generated substacks, they would get recommended into that and they would be satisfied with that. Is there gonna be like a barbell effect here where you imagine there's like power law creators who use no AI, power law creators who use all AI?

Speaker 8: I think there will be some of that. And I think these these tastes will change over time. But something I've noticed in myself and in people I talk to about this is a lot of these things are kind of an acquired distaste. Right? It's something where, you know, a few years ago maybe I wouldn't even recognize this as being different.

Speaker 1: Yep.

Speaker 8: And now I sort of have that skin crawling feeling. I think more people not everybody has developed that already Yeah. But I think more people will develop it than have today. But then, look, I think ultimately, you know, our perspective on this, we're pro freedom. We're not sitting here telling you what tools you can and can't use.

Speaker 1: Yeah.

Speaker 8: We're not sitting here telling you what you can and can't read. Yep. As long as you're making the decision, you know, as your best self, what do I value? What do I want to spend my time and money on? We're happy with that. We wanted to kind of help people do that. And so if, you know, I I if there are places where people are, you know, paying for AI stuff and it's the stuff that they want and the person may like, we we think that's great as long as it's, you know

Speaker 2: Yeah.

Speaker 8: People are doing it with their eyes open. They know what they're getting.

Speaker 2: Yeah. It's funny like how I I would certainly pay $5 a month to just get a feed of Tyler Cowen's like ChatGPT prompts. Right. Right? Because I'd just be interested to fall like, I would just be interested to like sort of passively And learn about I'm surprised no one has like figured out a way to create that kind of product yet.

Speaker 1: Fair enough.

Speaker 2: What do you think is, like, the is there an AI writing product that is overtly AI that's actually been, you know, a massive breakout hit and that they've they've fully disclosed that this is, you know, AI generated, but but it still works? Any any that you're aware of?

Speaker 8: You mean, are there examples of texts

Speaker 2: Like, I mean, somebody that, like, made a made a sub stack and it's like, hey, this $5 a month and I'm just gonna give you breaking news and analysis on these types of topics and it's all AI generated. And, like, actually, have has anyone built a a real place?

Speaker 1: The equivalent would be like demon flying fox on YouTube. He's the one that did the Harry Potter Balenciaga. He is like an AI first video creator. It's very clearly disclosed. There's a number of Instagram accounts where like the whole thing is like imagined AI futures and they are creators, but they openly use AI. I'm wondering if there's a version of that for text or sub stack.

Speaker 8: There that definitely exists and that's the thing that I I believe in. Right? The idea of of a like an AI like an AI powered creator

Speaker 1: Yeah.

Speaker 8: The way I think of this is like AI gives you superpowers Yeah. And it just also gives you new ways to be annoying on the internet. If you're Both. If you're, you know Yeah. The a in AGI is not supposed to stand for annoying. But the like, okay. If if you're saying, like, I have this perspective and I'm using these tools, like like, I think that Harry Potter Balenciaga thing is genius.

Speaker 1: Yeah. It's great.

Speaker 8: It's genius. And I don't care that they used an AI tool to make in fact, it's great. It's like it's a new way to express the thing. Yeah. And we have we actually there is

Speaker 7: a bunch of

Speaker 8: that on Subsidydeck. There are people who are doing analysis, people who are sometimes writing things in other languages. There's people that are like writing English is not their first language, but they have some important analysis, or they're like, you know, they're doing some deep research or they have some perspective, and, you know, the final prose is generated, but they're transparent about that. I think that is great. The the question the deeper question to me is like, do you actually have to have a a human artist behind the thing? Like, if instead of that, if if instead of I I've used AI tools, if you're kind of doing the thing of like, hey, you know, generate 1,000 viral posts, make no mistakes. Or generate 1,000 viral bots that each do 1,000 viral posts, make no mistakes. Mhmm. And my view is like, that may be compelling, but that's not going to be valuable over time because it's gonna be there's gonna be no differentiation. It's the thing of like, yeah, talking to Claude is great but I can talk to Claude

Speaker 6: if I wanna talk to Claude.

Speaker 1: That makes sense.

Speaker 2: Do you know of any historical comps to this conundrum that we're in as citizens of the Internet? Like, was there a period where where authors weren't real authors if they used a typewriter? You know, you had to you had to write it with a with a pen or or or it was frowned upon? Or is this just like a novel experience?

Speaker 8: I found when I was looking for the art for the announcement post I did, I found a a a reverse example from history. Reverse clod fishing which was the mechanical Turk. Do you guys know the story? It was it was supposed to be a robot that plays chess.

Speaker 1: Yeah. And so it was it

Speaker 8: was supposed to be like a magic strong chess playing AI, but the the the trick was there was actually like a chess master kind of like hiding in the bottom, controlling it. There's sort of it's the reverse of having the, you know, realization of you're absolutely right while you're reading something.

Speaker 1: Yeah. Yeah. It's great. Well, congratulations on the launch. It seems like it's going really well and we're huge fans of Pangram and the team over there.

Speaker 8: Those guys are great. They're excited to see. We're proud

Speaker 1: of our team. They really are. Yeah. Yeah. What a great In

Speaker 2: two more minutes before you leave Mhmm. What what can people expect out of Substack for the rest of the year? Like, are you are you

Speaker 1: Yeah. How is the business doing in the age of AI broadly? Like, there's a narrative where it's like, oh, people would just talk to the AIs to get information. They're gonna spend no time on Substack. And then there's another one which is like, you'll be 10 times more efficient because all your software engineering can be automated. Like, what how is the business actually changing?

Speaker 8: Business is doing well. We're growing. Cool. You know, the way I think about this is I think as you get sort of plentiful you can get anything from the AI, you can get any content, the value you you have infinite kind of, like, things that you can spend your time on. The value of the trusted human connection and perspective goes up. Mhmm. And as long as we can have the platform reward that, as long as you feel like, hey, you know, like, even if you're reading Tyler Cowen's prompts, the fact that you know have some idea of Tyler Cowen as a person and have invest some faith and trust in him Yeah. Is a fundamentally valuable piece in this new world. And we actually think this is again, it's like if you take sort of a pro AI stance, you can also have like a pro human culture stance that thinks that we're gonna be able to make that even more valuable.