Andrej Karpathy's 'Software 3.0' vision: the neural computer is becoming real

The Neural Computer Is Becoming Real

Andrej Karpathy's concept of "Software 3.0" — where frontier AI models render interfaces and execute entire workflows without intermediate code — is no longer theoretical. It's reshaping what gets built and how.

Karpathy frames the neural computer this way: a device that takes raw video or audio into a neural net, uses diffusion to render a UI tailored to that specific moment. No predefined interface. No app layer. Just input, model, output.

The shift shows up in real workflows. A single prompt requesting a financial comparison of GameStop and eBay produces a pixel-perfect side-by-side table showing revenue ($50B vs. $15B), growth rates (8% vs. -5%), operating income ($2.28B vs. $232M), margins (20% vs. 6.4%), and valuation multiples — all formatted as an image, shareable and digestible. In the pre-ChatGPT era, this required tabs, copy-paste, spreadsheets. With frontier models, it's one request and done.

Karpathy's own MenuGen example is starker. He built a coded app that uploads a menu photo, OCRs the items, generates images of each dish via image generation, and re-renders the menu on Vercel. Then he saw the Software 3.0 version: take a photo, send it to Gemini, ask it to use an image renderer to overlay dish images directly onto the menu pixels. Input: photo. Output: enhanced menu. No app in between.

The vibe coding boom was a temporary aberration. Millions of people have used code generation tools like Codex and Claude Code, but adoption sits far below chat app penetration — which hovers around 80%. What's revealing is that non-technical people pitching vibe-coded ideas are often building solutions that the chat apps themselves already solve in one thread. They're creating apps that don't need to exist because the capability is already there, free, and widely available.

This creates a decision tree: when do you actually need to code and deploy versus just prompting the model? For calendar management, food tracking, financial comparisons, menu enhancement — the frontier models in ChatGPT, Gemini, or Claude handle it end-to-end. No deployment. No infrastructure. No friction.

This echoes the "Fat Protocols" thesis from Union Square Ventures' 2016 analysis of blockchain value capture. In web 1.0 and 2.0, protocol layers like HTTP accrued little value; Facebook captured the upside from the protocol's existence. Crypto aimed to reverse this by baking value capture into the protocol itself. Bitcoin sits at a $1 trillion market cap; the largest applications built on it are worth hundreds of millions to tens of billions. Ethereum follows the same pattern.

In AI, models are getting fatter every month, eating away at what required application layers to do. The protocol — the frontier model — is capturing an increasing share of the value chain. Applications still exist, but many serve as thin marketing arms or awareness layers, not technical necessities. A company can deliver value and acquire customers by presenting raw model capability in a digestible form, even if the underlying capability is already available elsewhere.

The walled garden problem is real but not technical. There's no technical reason a single LLM couldn't query every web resource. The constraint is business: Apple, Meta, Google, X, and others don't want their data and services accessible to each other's AI agents. But models are finding ways around. OpenAI's agents have already penetrated some walls. When companies like SAP try to lock agents out, the workaround becomes human: a person takes a screenshot, an AI tells them where to click, or electrostimulation devices puppeteer their fingers to execute model instructions. Until biometric verification locks down every interface, compute will find a path through.

The practical implication: a huge class of software that required coding, deployment, and maintenance is collapsing into single-prompt model calls. Vibe coding was a waypoint, not a destination. What remains valuable is either irreducible complexity that models can't yet handle in one shot, unique data sources, or differentiation so distinct that the open model capability can't match it. Everything else — the thin applications, the obvious tools, the things that could be one-shot prompts — will compress.