Light Matter's Nick Harris: photonic interconnects deliver 100x bandwidth for AI data centers

huge gigas scale AI training runs. Uh it's a fascinating company, very deep tech basically. Um he's going to have to do a lot of explaining for us because I'm not up to speed on his technology, but I'm excited to talk to him. So Nick, are you here? Yeah, I'm here. Good to meet you guys. Good to meet you, too.

Um, how you doing today? Doing great. Yeah. We just had a big launch event yesterday sharing our products and excited to chat and tell you guys what we do. Yeah. Yeah. Yeah. Why don't you just start with a breakdown of yourself, your company, and what you announced yesterday. Yeah. I'm Nick.

I'm co-founder CEO of Light Matter. Been building a company for about eight years now. Um, we're based in California in Mountain View and, you know, we're about eight years old, spin out of MIT and we're looking at the future of computing.

Um so our goal is to continue to make progress on uh the cost of computation, the scale of computation and the energy efficiency in a time when uh computer chips are no longer making progress.

You know really principally the only way that chips get better these days is by if I want to double the performance I double the number of computer chips inside a package. And so you're seeing over the past 10 years these chips getting absolutely massive. Yeah.

And then the next challenge is around how do you link them together especially in the context of training these large language models for AI and that sort of thing. And that's what we do is we build these optical links between GPUs to power foundation model training and and that sort of thing.

So how does that interface the rest of the products in the market? I mean Nvidia is known for NVL link. I used to have two uh 3090 graphics cards that were kind of linked together over the PCI slots.

like I imagine it's much more complicated on your side, but uh can you give us a little a couple more concrete examples of like how this rolls out, what the timelines are for this type of thing, and what kind of performance gains we can expect to see?

Yeah, so right now uh in terms of timelines, chips are coming out this year. We've got big semiconductor partners that uh will be releasing uh products this year based on our technology passage. And the way passage works is it's a silicon phatonix engine.

So it leverages all the semiconductor manufacturing supply chain that's used to build all modern chips today. So it's built in these big semifabs. You hear about companies like TSMC, Global Foundaries. That's where we build the stuff.

And what we do is we 3D stack GPUs and switches, these really like heavy data center AI products on top of our phetonic wafer. And then you cut out the die size that you want, attach optical fibers, and now you can wire up these massive AI data centers. You know, you're hearing about XAI with a 100,000 GPUs.

You know, Elon and team deployed that thing in what was it 30 days or something like that. Yeah, that's insane. Which is break neck speed. We are the wires, the fetonic links that connect up these massive data centers and the speedups are are enormous. So, we just announced yesterday 114 trillion bits per second.

State-of-the-art is about 1. 6 trillion bits per second. So, about 100x. Wow. So, uh obviously there's a lot of trade-offs here. I imagine that you're Yes, thank you. The gong goes off when we hear a big number.

Um uh I I imagine that there are trade-offs probably at the early stage cost, but uh are we just talking about speed or is there also a heat component that's important to to think about? But I know heat dissipation is super important on these big uh training runs. Yeah.

I mean on the heat point, so something to notice like at a global scale, we're building data centers. Humanity is building data centers that use as much power as the biggest cities on the planet. So 5 gawatt data centers. New York City is 7. 5 gawatt. LA is 2. 5 gawatt. Yeah.

So if you were to look at planet Earth through a thermal imager, the brightest objects you would see would be the mega cities, the deserts y under uh sunlight, volcanoes, and these new data centers. Fascinating. That's the scale of of what's being built. And the real task at this point is how do you wire them up?

How do you continue to drive performance and build bigger monolithic computers to train these AI models? Got it. Uh how did you process the sort of deepseek moment?

It feels like a decade ago at this point, but it it it you know, the narrative was that they had GPU constraints and so they found more ways to be efficient.

Do you think the American v, you know, sort of like tech world just hasn't cared enough about efficiency to date because it's all been about speed and we sort of haven't had that much capital constraints just because of how much investment is sort of pouring into the category.

But I'm I'm you know sort of broadly curious how how you and the and the team processed it. Yeah. On the DeepSeek side, you know, what people are thinking is that you don't need as big of AI clusters to build a frontier quality model.

Um and they had some innovations in how they train the model and and some people suspect that a lot of that came from using existing LLMs to build the new model. And that's not novel. Think about how we design computer chips. Like the CPU today was designed using the previous CPU and you're stacking up this ladder.

That's how exponentials work. Yeah. So to me it's like yes obviously that would that would happen. I don't think it's really slowed things down.

Uh and I think that reasoning models are going to require an enormous amount of computation and they require lots of links like the kinds of fatonic links that we build so that you can take these models. Have you ever tried deep research? It's like 12 minutes to get a a result back.

What if you could get it back in one minute? And what if the energy cost on communications was one? Yeah. Can you talk about a few of the other paradigms for fighting with Moore's law and improving the speed of these models and inference?

I've you know there's etched which is baking the transformer architecture down into silicon. There's you know if you look at the history of Bitcoin they went from CPU to GPU to FPGA to ASIC.

Um are you uh excited about in uh data interchange in a world where you know the the latest and greatest Dolly 3 or four model is baked down into silicon into an ASIC. Uh is that still valuable? Well, the past 30 years have been a millionx improvement in operations per second for GPUs and for AS6.

A millionx in 30 years. Interconnect is only improved by about a thousandx today. Interconnect is the bottleneck and the next thousandx in performance for AI training and AI inference will come from interconnect and that's fundamentally what we innovate on. Uh so so that's the thing that we're excited about.

I think if you look at AS6 it's a spectrum. You start out with the GPU which is relatively general purpose and then you go to an ASIC that's like okay I'm going to target AI training or AI inference. If you talk about etch that's saying like let's go even further all the way to the bottom.

I'm going to bake one model into this thing. Yeah, and of course you should be able to get efficiency gains that way, but these are architectural tricks. You could have built that computer 30 years ago if you wanted to. So it's a one-off sort of hit. What you really need are new scaling laws.

And at Light Matter, we're focused on the fundamentals like how do you make compute faster and more efficient? Are there new ways to do compute? How do you make the interconnect faster and more efficient? We need new scaling laws. Got it. I I I don't know if you've been following George Hatz's journey with AMD.

Uh but some of the foundation model labs and the folks who are working on these big AI models have been kind of banging the table saying like the cost per flop should be better with these other chip manufacturers and yet I'm locked into Nvidia's ecosystem effectively because of bugs and just go fix the bugs.

Uh do you have any uh push back that you have to overcome when you're pitching a somebody who's building a big cluster to say hey slotting us in is not going to create any extra overhead on your software engineering team. Yeah, that's you know that's one of the exciting things. We're not building processors.

We're building the interconnects that link them together. It requires no change to any software and thank God for that. That's amazing. There's there's no lift. And the TAM for this thing is absolutely massive. 300 billion a year spend on AI hardware. About 30% 25 to 30 is networking. And that's where we play.

So it's an enormous market that requires no software change. You can run CUDA, whatever you want. It's all fine. So what are the major hurdles or or like next milestones for you in terms of scaling up? Do we need new machines? Do we need new machines that make the machines? Is it just are you are you demand constraint?

supply constrained. How are you thinking about the next roadblocks in the business? So, I would say that the the ramps are happening now in terms of volumes. So, this technology silicon ponics attached to GPUs, attached to switches, it's happening now.

You're going to see roll out in 26, 27, some of these big data centers are going to be using it. And the trick will be can we make sure that the supply chain can handle it if everybody comes online at the same time. right now.

I don't think that that's possible, but I think you could handle um at least half of the world demand. There's about 14 million GPUs or XPUs that are shipping per year. I think you could get at least half of that. And it's not too hard to imagine building out the capacity to get the rest of the way.

But the tech's been ready for prime prime time, you know, for a couple years now, uh with us. And it's been an eight-year journey. So, it's not exactly overnight. We've been working on, you know, all the problems on every front for quite a while.

Well, you we're still going to call you an overnight success when you go public and finally, you know, deliver some massive quarter. We're going to say, "Oh, yeah, you know, it just happened all of a sudden. " Because we love we love overnight successes.

How skeptical are you when you see new seed stage anything on the the hardware side of AI? People love to come out and sort of make these big promises and then some companies actually can manage to get public just purely on big promises. Um, how Yeah.

How like do you think we need more founders coming in and starting to build at at the hardware level or is it just so hard that you know focus on the rappers and you know do we need like to start a an activist short hedge fund because he seems like he knows the science here and can probably make some good calls but I'm sure he's too nice to do any of that.

Yeah.

So I would say that um the pitch early on so we started in 2017 that was the AI accelerator boom you think about graph core senova like all of these companies Sarah all coming out largely that's been a really tough slog like it hasn't really worked out I wish those guys the best yeah you know you know to be frank here I wish them the best but it's hard the software mode around Nvidia the the adoption the models are designed to fit on Nvidia GPUs like that that is the moat like everything is architected for Nvidia and so good luck bringing in something else.

So the new pitch, the new like uh pitch for these guys is everyone did this wrong. I know what they did wrong. If you just do this, it's going to be fine. But my fundamental issue with it is you're all using TSMC in the exact same node with the same packaging technology.

You can't tell me you're going to have some breakthrough. It's the exact same tech. That's fascinating. Well, good time to be in the compliments to Nvidia, not in the not in the substitutes business. Love this. I like your strategy. Uh, this was a fantastic conversation.

I honestly I didn't know about the business last week, but I'm super excited we got to talk to you and uh we'd love to have you back. This was fantastic. Thank you so much. Awesome. Yeah, happy to be on and and thank you guys. Yeah, congrats on and congratulations and we'll talk to you soon. Cheers. See you.

Switching gears, we're going over to overnight success. an 8-year overnight success. An absolute dog. Uh we got to we got to hit the size gong again for all the big numbers we heard from that. Didn't understand most of them, but uh liked his attitude and size gong. I love the size gong.

Uh well, we got Rome mortgage coming in the temple of technology in just a few minutes. Uh fascinating business. Heard about this and the pitch made so much sense. You're locked into a 2% a 3% mortgage fixed. You want to move. you feel chained to that specific house forever.

That shouldn't be the way it is so you can move and and and and uh kind of keep your rate or sell your sell your mortgage. Uh but I want to hear from the