SPEAKER_01 0:00

I'm like, Jensen, why are you signing on a memory wafer? We need that. There's been a lot of noise about how Micron's out, not going to be on the Vera Ruben. And I was just, I was like, oh look, uh Jensen must not have gotten the memo. He must not have gotten the memo that Micron is not designed in. Hello listeners and welcome to another semi-dope podcast. I'm Austin Lines with Chipstrat, and with me is Vic Shaker from Vic's Newsletters. So, Vic, what's going on? What's up this week?

SPEAKER_02 0:35

A lot of things, right? This was the GTC conference week and OFC, which is Optical Fiber Conference. So after Nvidia's announcements and all the stuff that happened in the optics world, I don't know. I I'm kind of like like seriously, I'm like filled up. My mind is like filled up with all this news. And I'm like, I've come to the point where this week I actually made a bunch of tweets that made no sense at all. Like I'm just tweeting up stuff that makes like no sense. Like I don't read through stuff fully, and I just like say this, and then I'm like, what am I doing, man? Like, calm down. I think my brain is like fizzling out.

SPEAKER_01 1:13

Totally. That's why so much to stay. Yeah. So much to stay on top of. And yeah, I was at GTC, and then I was trying to like keep up with what was going on in OFC, and then I was just like, never mind, I can't do both. You know, I'll just have to read it next week. But today I'm not reading any of it. It's it's Friday, and I'm just gonna go outside because it's nice here.

SPEAKER_02 1:36

Yeah, that's nice. Yeah. Then I mean you just got back from California, right?

SPEAKER_01 1:40

Yeah, yeah. Which it was hot there. So when I left Iowa, it was like seven degrees Fahrenheit, negative 17 wind chill. I had to park on like the outside of a parking ramp, like at the top. It was like freezing. Um, and then when I came when I got to California, it was like 88 degrees when I landed. They're having like a heat wave. Um, and then uh even when I got home, it was like 78 here, and my kids were sunburned from playing outside.

SPEAKER_02 2:05

So it's like, oh, it's spring. Actually, my day-to-day temperature uh in Bangalore is uh around 80, 85 degrees. And if you're like at, you know, I think you're easily at minus 15. I don't know. Yeah, uh yeah, we have a hundred degrees just between this podcast between us.

SPEAKER_01 2:23

That's hilarious. That's so good. Yes, I'm not used to it. Uh uh, yeah. But uh okay, so yeah, where should we start? Um let's let's start not NVIDIA. What do you think?

SPEAKER_02 2:33

Yeah, so I have to tell you this first. Like, I heard I've heard so many people like right now saying, Oh, I spent so many tokens uh in like agents and this and that. And then NVIDIA comes and says, like the guy Jensen uh in the all-in podcast, I think, he says, if you're a$500,000 uh dollar engineer working for a company, I expect you spend at least$250,000 in tokens. I'm like, wait, that that's not gonna go down well. That's like kind of like imagine going to your boss and says, like, yeah, I want to spend$250k in tokens. I'm like, I thought AI was supposed to make things cheaper. What's going on?

SPEAKER_01 3:11

That's hilarious. Yeah, you know, so I I've got some interesting thoughts on this. So I actually I was in an industry analyst QA earlier before Jensen said that uh the podcast thing. Um and I asked him the question because he had said in the keynote, uh, hey, you know, I think that we'll soon get to a point where a token budget is a differentiator for employees. Um and so I and and I said to him, like, hey, look, I agree because to me, I think that agentic AI is the killer app that we've all been waiting for for generative AI, right? When Gen AI first came out, it was just a chat bot. Everyone's like, okay, this is cool, but what's a killer app? I think agentic AI is, and I think like for me specifically, it's coding, but I think it'll be more broad than that. Um, but maybe it will just be coding. Maybe it's just like everyone can code, you know. Um uh so anyway, so I said to Jensen, I was like, hey, I I actually believe you that once you go agentic AI, you'll never go back. So if you're at a company where you're they're just like, ah, sorry, we've got strict token budgets, you can't spend much. And then some other company says, like, dude, you can go to town, claude code all you want, like you'll never go back. You'll be like, that's amazing. But to your point, I started thinking about it and I was like, well, wait a minute. If you give people an actual token budget, like let's say$10,000 a month, we'll just use round numbers. Well, what if I'm super productive and I'm going to town and I blow through it in a week? Now, am I gonna just not get to use claude code for the next three weeks until it resets? Right. And so the the the incentives are maybe a little misaligned. Um, so what I said to Jensen was, are companies in this situation going to buy tokens for every employee or are they gonna buy token generators? Are they gonna buy the hardware itself? Because then you essentially have like unlimited tokens, if you will. Now, I was also hoping that that would open the door to talking about like on-prem inference, like the um Dell Pro Max with the GB300, where you can literally like air-cooled, plug it in at your office um and have the data center GPU, because and there's I think a lot to explore here and a lot more I want to hear from Dell, which is like, hey, what if for a team, instead of saying, yeah, you um we're gonna spend a million dollars for this dev team in tokens per year, what if you're just like we're gonna get, these are like$120,000 machines. What if you're like, we're gonna get like four of them and you can all remote in and share them and go to town and I think it's got like 768 gigs of memory or something now, not all that's HBM, but like, hey, run, run big open source models, do what you can. Uh you know, maybe that's actually gonna be a lot more cost effective than people just getting these crazy token budgets where once you've spent it, it's gone. And it's a whole the whole OpEx versus CapEx type conversation and stuff. Like, I just feel like CFOs are gonna really want something other than, oh great, now I get to spend an extra 200 grand per person with open AI, for example.

SPEAKER_02 6:16

Yeah, that's that's the way I think it's gonna go. And uh the OpEx CapEx thing is obviously very important. But you know, even when you deploy on-prem, I don't think you're gonna have virtually unlimited tokens or anything because you know there's a token per second generation thing for a given uh quality of hardware, and then there are so many engineers, so it's gonna be divbied up at some point. Just think of it as like EDA licenses, right? You you sign up for 20 licenses, and when somebody is using it all up, I have done this personally. I go to somebody and say, Hey, can you please get off the license if you're not using it? Because I want to run something, and then they do, and then that's how it goes. So this is will be on a similar note, I think. Not exactly the same, but I think it'll be like, yeah, you're you've used up your token budget this week. Can you chill? Can you like do something else and maybe next week get back on it, right? Sure, sure, sure.

SPEAKER_01 7:08

Yeah. I I I see your point. And and that would be the the trade-off is like, well, this machine can generate a certain number of tokens per second for a certain number of concurrent users, 24-7, but now the onus is on the employees to have to kind of manage that and share it and whatever.

SPEAKER_02 7:26

Yeah, it's a resource, it's a hardware resource like any other that has to be shared. Right. Like, you know, we used to have these like big computers where I used to do a lot of like electromagnetics work, which is like heavily computational. And yeah, so that was how it was. Like you submit a job to it, um, and then you you hang out or whatever, and then it gives you the results back. Or think of it like LSF clusters. Like you submit a job, you don't see the machine, right? And it when the simulation is done, it gives you the results. It'll be like an LSF cluster of uh tokens, right? So you're gonna be using tokens from this cluster instead of compute, which is kind of what it does anyway, LSF. Uh setting facility, I think it's called. So it's gonna be like that. There's gonna be a bunch of servers that every big company at least will deploy. Um smaller companies will deploy smaller versions of the same thing.

SPEAKER_01 8:19

Yeah, exactly, exactly. But the nice thing is you you have a fixed cost, you know exactly what you're gonna spend, as opposed to just like APIs where everyone can they don't have to you kind of don't have to share in some respects when you use APIs, but you could go to town, you could blow through things or or blow over and spend too much, you know.

SPEAKER_02 8:40

Yeah, and nobody wants to send their tokens anywhere else outside premises. There's some of these industries are very, very sensitive to this, and you just can't do it. It's very because even if how do you know that like your tokens don't like go somewhere else in the cluster and some somebody else does something? Because you're using a bunch of CPUs and GPUs in a like a uh facility data center somewhere. What is a guarantee that your bits did not go to your competitors' bits, which are using GPUs from the neighboring rack? Like that kind of security guarantees are coming into newer NVIDIA GPUs. They have been talking a lot more about security guarantees. That's coming up, it's gonna be a thing, but a lot of companies will not like that, right?

SPEAKER_01 9:26

Totally. And and you know, Jensen, when I asked him this question, he spoke to it and he sort of said, like, hey, I think that there'll be a hybrid, like there'll be times when you want the biggest, best frontier models, and it's gonna have to run in the cloud, whether it's your own cloud or you know, someone else's cloud. Um, but there's to your point, there's gonna be sensitive data. Also, he he said he believes in like a lot of local, fine-tuned models where a company will say, like, hey, we have our bespoke data, we've trained our model for our particular use cases. Let's let that run on-prem and handle and then any of that stuff, you know. And so I definitely do think we'll get to a world, of course. Then there's lots of questions around like, how's all this orchestration gonna work and who's gonna build it? But hey, now that we have AI, maybe it can get built easily and quickly.

SPEAKER_02 10:16

And yeah, I you now that you mentioned that like these fine-tuned models. I have to mention uh Matthew Berman. I don't know if you've seen his YouTube channel, it's pretty amazing. Yeah. And uh yeah, he he has all these amazing videos and he mentioned this one thing that he's working on. He said, Yeah, I had the my open claw agent has all of my information, it has all of my memories, everything I prefer, don't prefer, all of that stuff. And all of that has like the outputs from a frontier model. So he's like, I'm gonna take like a lighter model, like an open source model, smaller one, and I'm gonna train it just on my inputs and outputs that I have already stored in my open claw uh environment. And that's gonna save my token cost enormously. And he said he's gonna like look at it and like like post a future video or something. It's very exciting. So I decided after watching Matthew Berman, I'm gonna like take this open claw thing seriously because Jensen was also like so emphatic about it at GTC. I'm like, okay, fine, it's time. Okay, let's I played with it a little bit uh earlier on my laptop and then you know all the security things. So I didn't, I just quit it there. But now I deployed it on, I have a little home server I run where you know I keep my own files. I run my own little cloud uh with all my family photographs and stuff like that. So I deployed it in a Docker container within my home server, and I put in like a reverse proxy uh to some kind of uh like an open claw dot something.something. You know, so if I go there, uh I can go to my dashboard now and do whatever I want. And before anybody listening to like saying, like, hey, why are you reverse proxying it to an URL? That's like the worst thing you can do to agents. To be fair, this is entirely within my local network. It doesn't go outside, it's not exposed to the internet. If it is, it's through Tail Scale. So I have a VPN Tail Scale that's going. So I do have security practice in this place. But yeah, I set it up and I bought a bunch of credits for like Claude. And I'm like, okay, fine, let's let's do this, let's set this up. And uh I've been playing with it.

SPEAKER_01 12:18

That's amazing. You I you'll have to share what you do with me sometime. I'd like to set up something the same. One of the things that I really want to do, because I like my mantra right now is ABC, always be clauding. And so like I I always just want to check on my jobs, my agents, and then just just because usually it's like, okay, they're good on their own, but then after a few minutes, they'll just ask for something. And I'm always like, yes, yes, yes, you know, like even if I try to give it like disable all things, never ask me anything, just go to town. Like it seems to always still have to ask me. And so I want to just be like, from my phone, be able to like check, you know, uh go to that URL, check on the dashboard or uh have a terminal or something and just be like, oh, yep, keep going, keep going, you know.

SPEAKER_02 13:00

Wait, doesn't it didn't you set it up with like Telegram or whatever? WhatsApp?

SPEAKER_01 13:05

I haven't yet.

SPEAKER_02 13:05

No, well, there you go. That's a great idea. I think that was like the one of the initial initial use cases that like got everybody hooked on this because you could just like hook it up to WhatsApp and then you could chat with it on your phone, like messaging. Oh, beautiful. And it's like keeps doing whatever you want on the phone. I think it's already quite easy. Okay.

SPEAKER_01 13:23

Well, it's solved, it's a solved problem, and yes.

unknown 13:26

Yeah.

SPEAKER_02 13:27

Anyway, I don't know if I'll continue this Docker container approach or I maybe I'll get I have an old Mac M1, uh, so maybe I'll like set it up on that. I don't know. But yeah, I think it's kind of cool. I'm gonna agentify this thing because like I told you earlier, like I'm overwhelmed with the news. I can't take it. Like I am, I have uh so many subscriptions on Substack that I have to read. With the news, I'm I'm kind of done, I'm overwhelmed. So let's go agentic on this one and we'll we'll talk about it at least briefly on the podcast. I'll tell you how it's going.

SPEAKER_01 13:57

Sounds good, man. Sounds good. All right, so okay, on top of all the conferences, there was also micron earnings. Did you did you pay attention to that at all?

SPEAKER_02 14:07

Uh a little bit, I think. I I didn't do too much of it because as I said, I mean like memory now. I mean, okay, Micron's doing great. That's the that's the takeaway I have. They showed a lot of earnings, and uh their stock is up, I don't know, some 60% this year. Amazing. And they are supplying HPM4 to Vera Rubin, uh, which is uh something you posted on X. And I was like, wait, what? That's cool. And Jensen's like signing a wafer, uh, and Sanjay Marotra is holding the wafer. I'm like, Jensen, why are you signing on a memory wafer? We need that stuff. Stop signing.

SPEAKER_01 14:46

Yeah, that's way too expensive to be signing, dude. What do you think? You don't need that, Jensen. That's hilarious. That's good. Yeah, yeah. I posted it because you know, there's been a lot of noise about how Micron's out. Uh their HBM4, it's not going to be on the Vera Ruben. And I was just, I was like, oh, look, uh Jensen must not have gotten the memo. He must not have gotten the memo that Micron is not designed in. Somebody bought, somebody paid up for the wrong thing.

SPEAKER_02 15:15

No, I'm kidding. Yeah, I think maybe there was so much news about this uh a couple of months ago that Nvidia is not going to be choosing Micron and they have zero percent share. And a couple of people like uh on Substack wrote about it and pushed back and said, like, no, we don't believe that's the case. Um, maybe SK Heinex is gonna have the lion's share of it, but it's not zero. I mean, I think it's a smaller percentage. I still don't know what the percentage is, to be fair. And then uh there was this uh you know argument that even if Micron was out of HPM, it's not a bad thing. Because have you seen DRAM prices recently? You know, one it's like easier to make DRAM, just make DRAM and then sell it. You can like ship so many bits, you don't need to stack them, package them, test them. HPM is a nightmare. So I know you have like a lot of uh up margin for selling HPM. It's a high, high margin, high, high uh dollar count product, but just ship DRAM and you're gonna be fine. Is what I read from some other Substacks. And I was like, yeah, I I kind of buy that.

SPEAKER_01 16:20

Yeah, totally. Turns out they were in. They're in. And well, and they Micron had a press release. I think it was maybe it was part of their earnings, whatever, but they had specifically said that they have been their HBM 4 has been designed in. Now everyone's asking, okay, but does that mean you're qualified yet? But no vendor has yet said they're qualified. So I think Micron has publicly said probably everything that they're allowed to say, which is look, Jensen signed our our thing, you saw it, it's public, and we are designed in, and we, you know, have the right pin speeds. Uh, so you know, we can't say any more, I'm guessing, but all signs point to they're they're gonna play ball here.

SPEAKER_02 17:02

Yeah, the pin speeds is an interesting thing because I'm glad you mentioned it. The Jedix spec for HPM4 is I think about eight Gbps uh per pin. I hope I got that right. So I remember eight Gbps. Yes, okay. So like anyway, let's at least look at relaptoid numbers. I probably got the units all wrong. But then um the news is that they have been pushing uh the ZX speed per pin higher and higher so that Nvidia can like stay ahead of AMD. Uh whose uh MI450, I don't know, is arguably really competitive or better, I don't know, than Vera Rubin, we have to see. But they wanted to get ahead of AMD and they've been pushing all these HBM vendors to make faster and faster memory. Now, one of the things is like it's gone up to 11 Gbps, right? I mean, that's the kind of speeds that like SK Heinnix is reporting. Oh, yeah, we did 11 Gbps. So in one of the earnings calls, I think not this one, but the last one, or the one before that, uh the detail was that the base die, the logic die that sits below all these DRAM chips in a HPM was actually designed with a memory node. So a memory node is like a one base one beta node or a one alpha or a one gamma. These are how this memory chips are actually uh designated. It's not so much as a two nanometer or three nanometer, those are like logic nodes. So memory is like a different process technology because you've got to make those very tall capacitors that sit on the transistor for the one T1C cell that goes into the uh DRAM, right? So the one T1C cell uh requires like a unique amount of engineering. So the entire process of making memory uh is entirely very different on a wafer level than logic chips. This may not be um like news to some people listening to this, but uh if you ask me the same thing two years ago, it would have been news to me because I thought maybe everything is like a chip, right? Whatever. So memory chips are very different. The thing is, these memory chips are typically not designed for speed. I mean, they are designed for memory density. They are not designed to be the fastest transistor ever made. That is what logic chips do. That's why you go from a 3-nanometer transistor to a 2 nanometer transistor and you get all these speed gains, and you know, this is how you know chips have been scaling forever, right? They get smaller and faster and all that. Memory nodes are not built for that. So the whole argument was Micron HBM used a memory node for its base die, and therefore they are struggling to meet the speeds that is being demanded by NVIDIA. And so that was like another case against Micron. But I think they repeatedly came out and said, no, no, no, we are fine, the speeds are fine. And then there were like questions about like, yeah, really, how many speeds are fine? One out of ten are fine, or five out of ten chips are that's you know, that's right. All valid questions. I mean, these are like legitimate, good questions that have to be asked, but this is what is behind all of this stuff. Right now we see that yes, uh, it is being designed into Vera Rubin. I still don't know what the speeds are. I don't know whether they're different from SK Heinz. So yeah, you know, take take all of this with a grain of salt, whether you're like in the engineering world or um in the chip world in general, or whether you're an investor. Like I think if people had sold off on the 0% um content in Vera Rubin, they would have, I don't know, lost out 60% uh share growth. No, yeah.

SPEAKER_01 20:42

Totally, yeah. You make a good point, which is it's very nuanced. Just because you can hit those speeds, how many chips can hit it? And then of course, you know, what is how are you yielding? It which is asking the same thing. So essentially, like how expensive are your chips going to be that can hit that 11 gigabits per second per pin or whatever. So, yes, lots of nuance, um, lots of noise. It'll be nice once these things just ship, and then the companies can just tell us.

SPEAKER_02 21:12

Yes, we you know Yes, eventually we'll know. But the whole point of the investor game is to kind of figure this out early.

SPEAKER_01 21:19

Correct. Yes, that is 100% correct. And I think um, so another thing that stuck out, so obviously Micron, you know, crushed their earnings, they beat on all fronts, and I think they even surprised investors with just how how much the revenue was. Um, they one of the things that I did notice was there are lots of investments in new capacity that'll be coming online in like 2028. So there's a DRAM fab, I believe, in Taiwan that they have acquired and are kind of pulling that forward. There's a new NAND fab in Singapore, and I think they're ramping up fabs uh in Idaho and New York. And so, of course, naturally, I think in memory, there's obviously gonna be this tension of like great you're winning and you're reinvesting in in some in more capacity which is good because AI demand will continue to increase capex will continue to increase but there's always the like caution that people have of like oh this feels cyclical in nature and so it feels like you're doing what always happens which is you're winning and so you're investing in more capacity and eventually there's gonna we're gonna be flooded with more capacity and what's that gonna do to prices. But do you have any any reactions on that front on the the in the CapEx investments?

SPEAKER_02 22:33

So yes I understand the fear and we've been through this cycle many times in memory it's a brutal of all semiconductor verticals I think memory is the most brutal one it has had some really bloody stories in the past so it's very valid that like people who are either working in memory or in have invested in memory in the past have been through these cycles. So the fear is legitimate. But looking at uh you know Vera I think I was looking at the Vera CPU tray you know that has eight CPUs and I'm like what's all the black stuff around the CPUs like look at all that like tiles. Like the entire the tray is like huge. It's all of that is that all DRAM like it's incredible if that's all DRAM. Where the memory is not going away anytime soon. I don't see this going away next year or the year after. If we are talking in five years from now I don't really know things change way too fast. Look five years ago we didn't even have AI so I couldn't have told you anything. Right? But where the the the need for memory is getting incredibly uh you know stringent and very important. And the other reason why is that all computers are now going to do agentic AI work. You like it or not. Like you know um everything is going to do some amount of AI work. Like and memory is what is going to drive all of it like you know HP and the Dells and everybody needs memory because everything is going to run AI at some point or the other the phones. So I don't see the demand going away. So I I'm quite bullish about this stuff.

SPEAKER_01 24:13

But yes so yes my argument would be very similar which is there is a fundamental sort of step change that happened with generative AI where huge value is being unlocked but that requires these massive LLMs, right? So we've always, you know, we've had AI, we've had ML for a long time and and every sort of step change in the past, um like to personal computers or to the cloud, you know, demanded more memory in that we started to get more machines but the applications running didn't necessarily demand like orders of magnitude more memory. So more higher volume of machines but not necessarily higher volume of memory per machine. But now we're in this era where we do need a massive you know increase in volume in AI accelerators and but those also need a you know massive it's like an order of magnitude increase or more orders of magnitude in memory per accelerator. And then to top it off to your point, the whole surrounding ecosystem whether it's the CPU trays to support the GPUs or even just like all of our you know thin clients will still also want more memory because they we are going to still be running even more generative AI at the edge whether it's on on-prem, racks or potentially you know even our devices which although today thin clients work now so you know I can I can get to the agents even if I don't have a ton of memory but surely there will be a pull for doing more and more locally. So so I do think that the Generative AI has definitely there's a step change in the amount of memory we're going to need on every device yeah and I'm not saying for edge AI that necessarily means we'll be running full models on the edge devices.

SPEAKER_02 26:07

I think the models can remain on the cloud but just inference and agentic use cases will need memory.

SPEAKER_01 26:13

Yeah yes yes 100% right like the like all of my tool usage stuff today it'll be like oh yeah go hit these websites and scrape them or make these API calls and then get a bunch of JSON or HTML or whatever and process it and then feed it into the LLM right so like these are still you know memory intensive memory dependent things.

SPEAKER_02 26:37

Cool I think we should hit up one more topic and then we'll keep this episode rather light. Okay so recently there was this optical compute interconnect multi-source agreement okay like that's so many like letters I had to like look it up uh OCI MSA. Okay so what that really means is like this bunch of important companies Meta, Broadcom, Nvidia, OpenAI, you name it, all the names are here, they kind of got together and formed this multi-source agreement for optical components that is going to be used primarily for scale up I would imagine. And so this is very important because this is like an official declaration that optics for scale up is now important and big players are getting together to hold hands and make this an open ecosystem where many people can make components and supply into the industry as opposed to like one player getting in and doing things there with.

SPEAKER_01 27:35

Yeah. Yes this is an industry consortium and just like say UALink or eSun or something they're saying like let's get together define sort of I'm assuming I didn't read too deeply and uh but like let's define some standards and then we can all agree to those and then we can all compete based on performance and anyone can buy us and plug us in.

SPEAKER_02 27:56

Yeah so there are a bunch of technical specifications defined on the website OCI hyphen MSA.org. So yeah so this is going to be like an evolving thing like the version 1.0 is out but I'm sure it's gonna evolve there are other MSAs like I think there's like dense wavelength division multiplexing which is another optical technique where you send multiple wavelengths through the fiber that has an MSA because multiple people are going to supply into that and it's been around for a while because this is used in telecom and things like that. So this is a big movement. But typically when you talk about optics for scale up you know there are going to be a lot of uh connections because you know how many GPUs they say like connected in an all-to-all fashion so scale up represents a very large interconnect problem because there are just literally so many cables that have to go in scale up which is why everybody in the optics industry is extremely excited that optics for scale up will be a thing whenever that is next year two years from now three years from now can't really say but it is coming when we are heading towards that direction copper at some point will reach its limits we're going to need optics. So now here's the big question what kind of optics do you really need for scale up? The easy lazy answer to it is to say like all other optics why is it any different like why would I not use indium phosphide electroabsorption modulated lasers actually I don't think anybody says that except us nerds on the pod but you know what I'm talking about. But essentially why not you know do up do optical connections like they've always done like uh the telecom companies have done it for decades optical scale out has been doing it between racks scale across between data centers. That's because you know the scale up problem is a small one. It's within a rack let's say or very close by rack. So now I have to ask whether the technology we already have which involves indium phosphide lasers is actually the best suited technology for optical scale up. Right now people assume it is like you just put that in you put CPO whatever you hook it up and then you do it like you've always done optics. But I think that in the industry there is a whole different bunch of technologies that are being looked at or are evolving that could completely change the equation here.

SPEAKER_01 30:21

And I think this is important to talk about yeah yeah that makes a lot of sense right so like um I guess for listeners thinking about the distance you if you need to send something 10 meters versus one meter versus a tenth of a meter versus a one hundredth of a meter maybe you would have different requirements for the light source to use if you've got like if you're only connecting chips that are really close maybe you would use a particular technology.

SPEAKER_02 30:50

Is that kind of where you're going yeah so that's what that's what I'm going towards. So one option that uh I think people have been looking at uh is micro LEDs and it is interesting because these LEDs are basically gallium nitride on silicon devices and the way they are done is they are manufactured like separately and then they are like like placed onto um you know a silicon wafer that has you know um transistors and circuits and all of that stuff. So these are GAN LEDs and GAN LEDs uh they require what is called a microlens and the reason for that is the light from a GAN isn't like very pointed like you know lasers you know you you point you just go from one place to another this is like turning on a light bulb okay yeah it like diffuses yeah it goes out like in a in a cone outside so you need to put these micro lenses on top of each GAN LED that actually focuses the light. So that's an important part of this thing. Okay so that's that's what people are looking at and it's difficult because this is not a very you know high quality light source and its characteristics uh for modulation like you know you want to send on off bits right so you send a light or you don't send a light that is basically uh called intense intensity modulation right uh that that kind of modulation can be done but it cannot be done like the way you do it with an indium phosphide laser because the characteristics of that laser are exquisite and they are basically built to do this. But GAN LEDs are not like that. You know they're like micro LEDs are just they kind of they have all kinds of crappy characteristics. So you can then run 200 you know gigabits per second like data on an Indian phosphide laser uh you can probably run two gigabit per second on a micro LED laser. So almost per wafer per per channel basis it's pretty crappy. Like you can't drive this thing to 100 GP or like even 50, I don't know. I don't think so. But you definitely won't compete with an indium phosphide EML. So what they do is they take these things called imaging fibers. Imaging fibers are what is used in uh let's say like endoscopy applications. Like you have a a cable with multiple strands of fiber in it. Okay you can have like thousand strands of fiber. Like you think about these long like you know this Golden Gate bridge has this I don't know if you've ever been to San Francisco you should go see the museum at the Golden Gate Bridge. There is the cables in the Golden Gate Bridge that hold it up are not one single cable. They have strands of them inside it's actually made up many cables that you know comprise of the Golden Gate Bridge. So this imaging fiber is exactly what that is so you can send thousand different lights uh light uh signals through one imaging uh core fiber and so what you can do is like this is the slow but wide approach to optics so what you do is you send many slow signals but you sent a thousand of them at once right it's like the HDM of optics essentially totally totally okay so we've got slow and wide versus fast and narrow.

SPEAKER_01 34:18

So instead of like a ton of cars like a highway with two lanes but everyone can go it's an Autobahn so you can go 150 miles an hour this is saying okay it's gonna be a 100 wide lane but you're only going 30 miles an hour but the throughput is the same you're getting the same amount of people because you're wide even if you're slow.

SPEAKER_02 34:38

Yes so that's the whole idea there are like a few companies working on this. So for example like Avicena's uh presentation is something that I saw last ISSCC, the Solid State Circuits Conference in San Francisco they had like a whole demonstration of how this works and the slides had like this oh why are you taking an airplane to the grocery store? It still stands out in my mind. So they had this picture of this person taking airplane to the grocery store. That's what it is like when you're using an indium phosphide laser to do scale up that's their argument at least so why do you need this technology to do why do you need an airplane to go to the grocery store? Say okay this micro LED is supposed to hold a spot between copper whose reach at high speeds is very limited now you know at 400 gigs per lane like Hawk is Hawk Tan or Broadcom is talking about your reach is probably going to be a meter or a meter and a half at most okay that's that's the kind of reach. Maybe that's enough for some applications but maybe it's not like what if you're going to go to 800 gig it's copper is dead in the 800 per lane era. So optics is like power hungry like whatever you do with optics is quite power hungry. It is not like copper. I know people say CPO this and that yes optics is expensive to make all the CPO stuff and it is power hungry. So unless you really need to go there people are still looking for other solutions. Micro LED fills the gap between the two it has like copper like cost structure is at least what companies claim but it has the reach that is could be 10 meters or even up to 30 meters which is plenty to hook up like GPUs in a rack or multiple racks.

SPEAKER_01 36:23

Yes yes in and I if I recall maybe I saw this so I know Avaseina's doing it in then Credo bought a company I think maybe Hyperloom or something that's also doing micro LEDs and and and I believe everyone also talks about like the energy energy dense efficiency per bit. So it's like picojoules per bit times the gigabits per second per millimeter or something like that where like micro LEDs are kind of like in the sweet spot there.

SPEAKER_02 36:54

Yeah that's that's what it is. And Hyperloom acquisition by Credo was interesting. And Credo holds an interesting place right uh they have a lot of copper expertise uh they have active electrical cables those are the purple cables you see everywhere and everybody had so much love for them like a year and a half ago everybody is all about uh uh purple cables and in one Amazon picture they uh in you know rack picture they didn't see purple cables they saw orange cables and their stock dropped because of the cable color because everybody's like where's the purple cable and then later it came out that like Amazon said we don't want purple for whatever reason branding I don't know they said please make it orange so credit was like fine we'll make it orange no problem they stock dropped crazy crazy it's it's nuts so yeah so they have that and uh they recently in OFC26 they announced like a fair bit of like optical content they have like a 1.60 optical DSP they have new products at the 400 and 800 gig uh ranges they have some long reach optics all of this is uh covered with like their you know their zero flap technology where they have telemetry and uh I think telemetry is getting increasingly important actually today I saw a piece of news that Marvel is also include including telemetry in their uh interconnect products uh because you can't have we've spoken this in this about this in this podcast before you can't have link flaps you can't have links going down it's a very expensive thing so by monitoring the health of each link and taking them down before they become an issue this is a very big feature so both that that is becoming important and I think this micro LED thing fits somewhere in the middle. So credo has like a spectrum of products that's very interesting um and people I don't know why people tend to discount them in the light of optics news so I'm like actually this is a good company they they have a lot of stuff like really so I'm not very anti-Credo but stock will not speak to my convictions here. So let's leave me out of it.

SPEAKER_01 38:59

Totally totally yeah I think uh obviously none of this is surprising to Credo they work in this industry and so they're building out beyond active electrical cables to micro LEDs and then eventually to you know optical scale out transceiver DSPs and that kind of thing. It'll be fascinating to watch this micro LED space with the the CPO and the vixels and everything else going on and see what ends up winning.

SPEAKER_02 39:26

Yeah Vixel I'm glad you mentioned Vixel almost slipped my mind because vertical cavity surface emitting lasers are really mature technology. People have shipped billions of these things um they are used in like LiDAR sensors and also in like augmented reality has a lot of actually no I take that back augmented reality has my micro LEDs interestingly because I was looking at who makes all these like lenses and you know focusing things and I came across like a whole bunch of augmented reality like uh content. I need to go look at that. So this is why you have to subscribe to our Substacks. You know this is the kind of thing we do go look at like strange stuff and write about it on Substack right but yeah other than that like Vixel is interesting because Lumentum in OFC26 mentioned uh that they are working on a 1060 nanometer Vixel for optical scale up which is kind of cool. This is also a slow but wide approach and uh they said something to the effect of I have it in my notes here that that this is actually better for like better speed um and higher temperature performance and they said exceptional long-term reliability I'm like what oh that's that's good uh because I think lasers people always worry that lasers are going to fail and there's a good reason for that you know there they they when you then they heat up especially their lifetime drops like a rock so lasers are inherently like that. So if micro LEDs and vixels and all these things have better lifetimes remember we were talking about these link flaps in the scale up domain believe me you don't want link flaps because there are so many cables you don't want to be swapping them out because of failures.

SPEAKER_01 41:12

So that's an another interesting aspect towards scale up interconnects that are coming up and whatever has better reliability might be a selling point just saying oh definitely definitely agree uh so I guess that's probably enough for today but you'll definitely need to write for us more about micro LEDs and get deeper into this optical scale up because I think a lot is happening right now but it's still very fuzzy as to what's going to come out on the other side.

SPEAKER_02 41:41

Yes it's very interesting and I for one I'm not discounting micro LEDs just yet but before I say anything further I need to go dive deep on the Substack. So until then uh you I will have to hold off on making big calls on this thing.

SPEAKER_01 41:58

I love it. And I appreciate that. It's easy to make calls but I appreciate that you as an engineer want to first study it and come to your own conclusions about the benefits, the pros and cons. How about that?

SPEAKER_02 42:08

Yeah I'm a very low conviction guy until I do a whole bunch of research and then I go from low to like medium low. And then don't invest in any of these companies and can I look at the stocks and I'm like what what's going on? Like why didn't I make any money out of this? I need to work on the inner investing aspect. Yeah. Totally good stuff.

SPEAKER_01 42:27

Okay guys well thanks for listening. Oh and I will say last thing so when I was at uh GTC several people came up to me and said like hey I love the podcast or they say like oh are you the dude that does the podcast with that Indian dude with long hair and the guitars and I'm like oh yes yes me and Vic. That's right. So thank you there you go. Um thank you everyone for listening to Semi Doped. Uh subscribe to our newsletters give us a five star rating uh thank you YouTube people who love to leave us comments uh and anyway stay tuned we'll be back next week have a good weekend talk to you later