With AMD 13 years, Mark? Is that right?
It's coming on 13 years, 24— by the end of 2024. That's right.
Yeah. With all the technology change that we're seeing in the market today, particularly as the AI compute market inflects, I think it's great to sit down and get Mark's perspective on how the future of compute's likely to play out. So, Mark, appreciate you coming on to talk with us today. Looking forward to the conversation.
Brett, thanks very much for having me with you and with our listeners this afternoon.
Before we jump in, just to say we're gonna work through specific questions we have from Mark over the next 45 minutes, and then we'll open up to Q&A for the final 15 minutes. For those that are registered, please type in your questions, and we will try to get to your questions a little later on the call. So with that, Mark, maybe we can just start by summarizing the market outlook for AI. AMD late last year laid out a $400 billion market size. It's a big upgrade on where you were last thinking about the market. So can you talk a bit about what's driving that change of thinking, and what portion of the market AMD's gonna focus on? Yeah.
You bet. Well, Brett, I think we surprised some folks when we came out with that type of view of a TAM projection as you look to now through 2027. But when you think about what's happened since we made that announcement, I think there's a lot of corroborating evidence that, in fact, the market is exploding at that type of rate and speed. Since that time, you saw major hyperscalers announce, you know, even their imminent CapEx spends going significantly year-over-year. You've had Sam Altman come out and call for trillions of dollars of investment to build up the necessary AI infrastructure to drive the capabilities that the researchers see are imminently achievable if the computing infrastructure can scale. And that's exactly what we were looking at as we looked at the TAM projection.
When we think about $400 billion, it's an accelerator TAM. It is GPUs. It is the memory around it. It is other bespoke accelerators specifically targeted for this AI infrastructure buildout. And it is just that. Think about it as a buildout like when the internet launched and you had to, you know, you know, an entire buildout, not just of compute but networking, infrastructure, etc. It's a new platform. And so it is indeed a major investment. And with that, that investment comes the monetization, which will occur, as you'll see, you know, thousands upon thousands of applications, which are now at a torrid pace of being developed. So there's the economics that have to be behind it and are behind it.
And when you look at that $400 billion TAM, a lot of it, of course, is the hyperscalers and these massive AI cluster buildout. And so that's gonna be first-party workloads of the hyperscalers but also third-party. And that's where the largest models are needed, the LLMs that are taking on, you know, broad questions that they're helping answer, broad productivity savings that they're driving. And so that is a big piece of that $400 billion TAM. But it is much more than that because when you look at what's gonna happen over the next several years, AI is not just in the domain of these largest hyperscalers, these massive clusters. What happens is businesses as they focus and they have AI needs that address their business needs, not the world's AI problem set that needs solved but actually driving productivity of their business.
And those models are typically smaller in size. They can be handled, they can either be run in smaller clusters on the cloud or on-prem. And frankly, where you need more quick response and even lower latency, think about automotive applications with self-driving. Think about the factory floor. You know, that buildout will be embedded devices. And then right to the endpoint, where you're seeing now PCs. We launched our AI-accelerated PC last year with the Ryzen 7040. And now we look at 2024 as a big transition year where AI comes to the PC, and we're out ahead of that. So there's a lot behind that TAM. We're expecting the AI market to grow at a—you know—a 70% kind of CAGR.
I know that seems like a huge number, but the work our team has done, we think, you know, you can debate the numbers. But the fact is it's very large, and we are investing to capture that growth.
Yeah. And, Mark, just in terms of the pricing for compute, you know, the platforms today, they're clearly built around aggressive pricing, you know, high pricing from NVIDIA. And there's been a lot of inflation as you look generation to generation, you know, whether it's the A100 to the H100 and now the GH200. Obviously, you know, AMD's starting to ramp into this market now. But, do you think the pricing of AI compute in general needs to reset for this market to really grow, you know, grow meaningfully towards that TAM that you're suggesting?
Well, Brett, first of all, I think there's an underlying premise that I think everyone inherently knows. But you have to think about it in regard to pricing. And that is that the compute demand for these AI applications, these large language model applications, and driving to more and more accuracy and more and more human-like, you know, artificial general intelligence type of capabilities, the compute demand is insatiable. And so, when you think about what's driving the compute devices underneath, it is trying to drive more and more compute into a smaller area and into a, you know, a more efficient power per flop, floating point operation, the key element mathematical element underneath the AI calculations, which are done for the training and inferencing. And so that's the backdrop you have. So the base devices are growing in content.
So I think what's important, when you just step back, is to look at total cost of ownership, not just one GPU, one accelerator, but total cost of ownership. But now when you, you when you also look at the macro, if there's not competition in the market, you're gonna see not only a growth of, of the price of these devices due to the added content that they have, but you're, you're, without a check and balance, you're gonna see, you know, very, very high margins, more than that could be sustained without a competitive environment. And what I think is very key with as AMD has brought competition in this market, market for these most powerful AI training and inference devices is you will see that check and balance. And you know, we have a very innovative approach. We've been a leader in chiplet design.
And so we have, you know, the right technology for the right purpose of the AI buildout that we do with. We have, of course, a GPU accelerator. But there's many other circuitry associated with being able to scale and build out these large clusters. And we're very, very efficient in our design. And so we think, one, we will bring competition. And that will be an ameliorating factor. But more importantly, you have to look at AI from a portfolio standpoint. It's not all the LLM of the, you know, these largest cluster buildouts in the cloud. AI, as we're getting more and more astute at how to tailor our applications to the problem which is being addressed, as I mentioned earlier, you're gonna see, you know, a buildout of really leveraging more of the CPU buildout that you have.
A lot of inferencing can be done in the CPU. We certainly are supporting that, with our EPYC servers. You're also seeing it move to the edge. And we've added an AI acceleration into our Versal line of products, our embedded devices, and then right into the endpoint, as I mentioned, with the Ryzen PC. And so that also has an effect of helping manage the overall cost structure, the CapEx spend, as AI is really spread across, from the largest cloud installations, these massive clusters, all the way to endpoint devices.
No, no. I, I know I know the MI300X is off to a great start. Can you maybe give us a sense as to how you're shaping the roadmap beyond MI300X? And I think you've always said in the past, "Customers buy roadmaps." So, you know, what are customers asking for or what's the discussions you're having that may be landing in two or three years' time? Maybe share with us, you know, how you're thinking about the evolution of the roadmap.
Oh, absolutely. Well, I, I think the, the first thing that I'll highlight is what we did to, to arrive at this point where we are a competitive force. You know, we have been investing for years in building up our GPU roadmap to compete in both HPC and AI. We had a very, very strong hardware train that we've been on. But we had to build our muscle in the software enablement. And so we started years ago development of the ROCm software stack. It competes head-on with CUDA. We're able to go head-on. We're a GPU company, just like NVIDIA. We've competed with NVIDIA for years. So it's not surprising that a lot of the even the programming semantics that we use are similar, because we've been, frankly, traversing the same journey for decades.
And so that brought us up to December 6th when we announced the MI300. We brought that competition. And so I know, some people ask, "Well, why did we not present the whole multi-year roadmap at this time?" Well, the first thing that you have to do in a race is you have to click the starting gun. And you have to start the race. And that's exactly what we did in December. We put out there a highly competitive design in AI inferencing, a leadership design. And in fact, you know, we executed that plan to a T. We brought it out to market as we projected in 2023. We're now shipping. We're now ramping. And that's exactly what we wanted. And it allowed us then to create yet a different environment of how we're working with our largest customers.
We worked closely with them, and got input from them on the MI300. But now as they're adopting and you saw the large companies on stage with us, hyperscalers, OEMs, end users, application developers on stage with us because we demonstrated that we could compete. We demonstrated that competitive and leadership design. And so that got us a seat at the table. It's incredibly hard to earn that seat at the table, to really understand the details of what's needed next. And what you saw play out is, in fact, NVIDIA reacted to our announcement. They've actually accelerated their roadmap. We're not standing still. We made adjustments to accelerate our roadmap with both memory configurations around the MI300 family, derivatives on MI300, the generation next. And so we've been very closely working with our customers.
And what we can tell you is I'll tell you right now, that race has begun. And it's gonna be a competitive race. You're gonna see that back and forth like you've always seen when you have competition. It's gonna be great for the market. It's certainly spurring us at AMD to be at our very best of innovation. And I think it's gonna spur everyone to be at the top of their game. So, very, very exciting. We've launched that foundation with MI300. And stay tuned with us as we'll share more details forthcoming on that roadmap because it is indeed a multi-year roadmap that we've laid out.
Yeah. And one thing we keep hearing from the work we do speaking to folks in the industry that looking at the roadmaps, they say, "Look, there's a memory wall in this whole roadmap." You know, memory's pretty challenging. You need a lot more density, faster memory. How do you solve that memory wall? You know, look at the MI300X. And you've got a lot more memory than other AI platforms in the market. So how do you go even to an even greater extent with memory, 'cause it has such a big effect on performance and cost of ownership?
Yeah. You know, Brett, that's a great question. I mean, when you have these massive compute engines and again, it is math that's driving AI. It's the it breaks right down to the fundamental floating point and operations, the multiply-accumulate math functions. And you have to feed that beast. You have to bring memory in at high bandwidth and high capacity, or you don't get the end performance that you need. We get that at AMD. We were actually the first to bring high bandwidth memory, HBM, to market in a 2.5D configuration. What do I mean by that? Our GPU chip sitting on silicon connected on a silicon substrate. So a silicon-to-silicon connection to the HBM memory. We launched that with our Fiji product in 2015. So that was nine years ago.
So we have been extremely experienced at bringing memory into the GPU compute cluster. We led the way in what is now CoWoS at TSMC, which is the most widely used silicon substrate connectivity to have the most efficient connection of high bandwidth memory to compute. And we worked extremely closely with all three memory vendors. So, that is why we led with MI300. And we decided to invest more in the HBM complex so we have a higher bandwidth. And that is fundamental along with the, you know, the CDNA, which is our name of our IP. That's our GPU computation IP for AI. Along with that, it was that HBM know-how that allowed us to establish, excuse me, a leadership position in AI inferencing. And with that, we architected for the future.
So, we have eight high stacks. We architected for 12 high stacks. We are shipping with MI300, HBM3. We've architected for HBM3E. So we understand memory. We have the relationship. And we have the architectural know-how to really stay on top of the capabilities needed. And because of that deep history that we have, not only with the memory vendors but also with TSMC and the rest of the substrate, supplier and OSAT community, we've been focused as well on delivery and supply chain.
Maybe we can talk a bit about inference, Mark, because, you know, I, I often say this. We're at the R&D stage of AI, you know, that very training-centric today. But as we deploy services and we're just starting to see, you know, Copilot and ChatGPT get rolled out, but there's a wave of things gonna come. Obviously, the deployment phase is gonna be pretty significant for inference demand. Can you maybe just share us how you're thinking about inference over the next couple of years? And, you know, y-you know, do, do you think we're gonna see a bifurcation of training platforms need lots of, you know, large clusters? Inference maybe takes a different direction. And maybe there's you don't need clusters and maybe less HBM. And maybe it's a different product line. Can you maybe just talk a, a little bit from a computer architecture perspective?
What's different about inference requirements? And yeah.
Well, first of all, you know, there, of course, is a big difference between the training and the inference. I mean, the training is much more dependent on, you know, just the raw, you know, computation, the floating point operation per second that you can build out in a vast clutch cluster for the toughest AI training needs. So, you know, as you have ever larger models, I look at GPT-4s over a trillion parameters in that model. It's massive.
Yeah.
And so there, you have to have the raw horsepower, Brett. So it's about, you know, building out these massive clusters. Again, that's what we've, you know, we're attacking with MI300, because it does have a scale-up building the, the base compute node. If you look at our, our capabilities, we, we partner with the industry. We don't have just one solution to scale out and build clusters. We partner with the ecosystem to give vendors options as how they can build out that training infrastructure, you know, tailoring the networking providers that they have, tailoring, you know, how they, they may use different vendors to provide that solution. So, you know, we're very, very open as we build out our, our training solutions. That trend will continue. So larger and larger clusters for training.
For those large language model inference applications, it will still largely be GPU-based like those large massive LLM needs for the training. But they're different.
Rather than being that same massive cluster buildout there, it's about latency. How quick can you get that answer? Think, Brett, if you asked GPT or Bard that question. It's a very broad question. You're waiting for the response. You need that calculation to be done very, very quickly. And again, you know, that's where the memory configuration is extremely important, and how you architect to improve latency. And so those transformer LLMs will continue to build out in terms of large GPU-based clusters. As you get more and more applications and those applications aren't just broad LLMs and, you know, artificial general intelligence targeted cluster buildouts, and you start seeing more bespoke inferencing applications. I've trained my model, but now I actually wanna tailor it. And I want it to do more bespoke tasks. There you will see more bifurcation, Brett.
You're gonna see, y ou know, leverage of, you know, with a well-trained model, and with not being a massive model size, you're gonna be running on the CPU buildout that you already have today and running your business. You're gonna see, as I said, more edge applications. Llama 2 is a great example of a model. There's Llama 2 70B, 70 billion parameters. And then moving down from that, smaller models that actually are incredibly effective. So when you take the problem space down that you're, you know, that you're addressing with that AI application, the inferencing can become more and more concise, and, you know, less power demand to get the job done and l ess compute demand. It's more energy and cost efficient. That's the bifurcation you're gonna see. Continue buildout, for the broad LLM, in the search for AGI.
And then a drive to much more cost and power-efficient solutions for the broad spectrum of AI applications, which you now see in development. You know, and we can talk more about that in our chat here. But there's a lot of other elements underneath that, including we think open source will be a huge factor in the buildout of those applications. But we can get to that later.
Sure. I'm sure we can. Maybe just on that same lines, Mark, I wanted to get your perspective on ASICs. You know, we've seen some news from NVIDIA the other day that they're gonna be developing ASICs. And, you know, there's been a few hyperscalers that stood up and said, "We don't think GPUs are efficient enough for inference. So we're gonna do our own ASIC." How do you look at the role of ASICs? I mean, I guess, on one hand, the technology's moving so fast. That doesn't lend itself so well to an ASIC model. But how do you see the ASIC opportunity? And does it make any sense for AMD to offer an ASIC solution to customers?
Well, there should be no surprise that you're seeing ASICs that are being targeted to, you know, an area that a large customer that has, you know, a massive compute demand, when they have a piece of an algorithm, an application they're running that's well-defined, and they are the absolute expert of that application. They control the parameters around it, then it makes sense to create a more tailored solution because you can really optimize and suboptimize on exactly the specific elements of a more stable application environment. And then you'll get the return of a tailored, bespoke, ASIC silicon device. We've seen that time and time again in the industry. I mean, you go back and I talked earlier about a comparison to the whole networking infrastructure buildout. You can look at Cisco, one of the leaders.
They would, you know, often, you know, create tailored ASICs for a number of those applications. Yet when it became, you know, and, you know, where you really need high programmability, it's the programmable devices that are used. It's FPGAs. And it's CPUs and GPUs that are being used. And you're seeing the same thing here. So we work closely with all of the players which are building, in fact, these tailored ASICs. And the size of the demand is, as I said earlier, growing so insatiably. You actually need all of this. You're gonna need continued growth. They need us and our competitors to continue to drive more performant, more efficient CPUs and GPUs, these heterogeneous, highly programmable elements forward. And that allows very rapid innovation on algorithms because the hardware is not tailored to any one algorithm.
It's supporting the basic math functions. It's accelerating those math functions. It's providing incredible bandwidth and low latency to memories and providing an incredible scale-out efficiency. That'll continue. And you'll see a continued development of these ASICs which are focused on, you know, more specific algorithmic needs. So when you have such an expansion of TAM as we're seeing now, that, you know, we need innovation on all fronts. And that's what we're seeing.
Interesting. Another topic that we've been hearing a lot about recently is power, data center power. I guess as we see more AI compute, you know, you talked about insatiable demands. The power requirements here are very different. You know, the power densities, the power per rack, the requirements for water cooling. We're moving up to higher-speed SerDes. How do you think about the, this challenge that is, is this gonna slow down the industry? Do you think there's gonna be a difficulty getting the power infrastructure that's needed to really, you know, deliver the, the demands, you know, deliver supply?
It is an absolutely huge challenge. When you look at data centers today, they are power-gated. So it's not floor space. It's not another limiter. It is power-gated in terms of being able to drive up their compute capacity. And so what are the trends? The trends is that the basic building blocks that are predominant today for AI training and inference is that GPU, that CPU/GPU cluster, as it gets built out. And we are very, very focused on the innovation around that cluster. I mean, the optimization of that CPU and how it works together with the GPU is critical. We've been doing that since 2007 with the acquisition of ATI, is when AMD started working on optimizing CPUs with the GPUs. But it's more than that, Brett. It's really what I call holistic design.
So what we're seeing to drive the inherent energy efficiency now is we can't just be a hardware vendor anymore. We have to be thinking about all the way through the application. So yes, we're driving energy efficiency in our design itself. We have many, many power-gating features. And we innovate at every generation on the power elements. We have entire microcontrollers whose job, their only job that they're doing, is optimizing the power of every program that's running on our compute clusters. We also work with our foundry partner. Our, you know, our primary partner is TSMC. We operate in a deep what we call DTCO, a design technology co-optimization. And we're driving, you know, a very high optimization of how the transistors themselves and how they're manufactured drive less energy consumption. But then we work ourselves up to the entire stack.
We're putting elements in to really drive optimization with math formats with approximations. So, you know, where you can run in AI a math approximation, versus, say, let's say a traditional high-performance application would be run at, you know, 32-bit and 64-bit floating point. Well, HPC applications needed that incredible accuracy. AI applications do not. So that's another source of energy efficiency. So it is this holistic design that we're driving. We've actually committed, from 2020 with our heterogeneous HPC and AI design base, to drive a 30x improvement by 2025 with all those elements I just described. And, we're on track to that. We're on track to that. Yeah.
So, it's gonna be a huge piece to manage the energy. And the other piece will be the trend I talked about earlier, that all models aren't created equal. You need models which are smaller and therefore much more energy-efficient computing, based on the application at hand. But I have to say, what you'll see then, what we will do is we will be increasing the power and generation on that base compute, CPU and GPU cluster for the largest LLM applications. And yet, we'll be offering much more energy-efficient solutions, as you look at edge applications, and end-device applications.
And so that, y ou know, it's very important when people think about energy consumption of AI that they step back and look at the macro. And the macro is, as you're gonna see, an incredible growth of AI applications. And those AI applications all won't be on those highest, most incredible power consumption demands in the cloud. But they'll be moving to the edge as well, to the endpoints.
We see, you mentioned some of the low, smaller models. Do you think AMD needs a separate roadmap for the low-end AI? You know, like, model sizes that are maybe under 100 billion parameters. Maybe MI300X is too powerful for, maybe not, that's not what it was designed for. So, you know, do you think it makes sense to see a segmentation at some point where AMD maybe promotes some of their higher-performance client GPUs into a distinct category for lower, smaller model AI, if you like?
Absolutely, Brett. What we have done at AMD is we were a leader in modular design. It's been a key to the turnaround at AMD over the past decade. And that modularity gives us a chance, the opportunity that we've been deploying now for years. And that is to make sure that we right-size the product offering to the task at hand. You will see that in our GPU roadmap. So, again, the first thing we had to do was create competition at the most demanding training and inference applications because that's what proved that there can, indeed, be competition in the market. That's what earned us that seat at the table. And so what you'll see now is, as we proceed, you'll see other variants. We already have today the MI250 in a PCIe form factor.
So it's a much lower power, PCIe data-attach, for those smaller model sizes. It's incredibly cost- and energy-efficient when you have those smaller model sizes. And you'll, y ou'll see a continuation of that, of that type of PCIe product line, going forward in our roadmap. As well, we have added AI to every product in our portfolio. You are seeing it, that we again, we already started last year with the Ryzen 7040 with our AI-enabled PC.
We boosted the AI inferencing capabilities with our announcement at CES. And we brought it into the desktop form factor. We have also a broad acceptance in our embedded devices. You can look at Alveo and other aspects of our FPGA and adaptive compute roadmap that brings AI into embedded. You're even gonna see it in gaming, because AI in gaming is giving you more energy-efficient image visualization and rasterization. So, it is across our entire portfolio. And as I talk about gaming, we have enabled now AI in our ROCm AI enablement stack to Radeon. And so that was a big announcement we made last year. And we look to expanding that AI support across Radeon because that gives a broad range in our portfolio, our GPU-based portfolio. From the, you know, the absolute optimized CDNA devices for the highest capabilities. Also, it now extended with our RDNA devices.
Yeah. Yeah. Let's talk a bit about AI PC, Mark, because you were touching on some of the changes you've made to Radeon. But can you maybe share with us what is an AI PC? Is there a specific spec? Do you think that, in general, we should be expecting GPU attach rates in PC to climb because folks are gonna wanna buy a more performant machine that's accelerated PC to do AI? Or is it specifically more about sort of like NPUs, conforming to, you know, maybe the next operating system that Microsoft's gonna roll out? But any sort of help from a high level, how should we think about AI PC? And why are you so excited that this is gonna be something that might drive an upgrade cycle for the industry?
Yes. Brett, it's a great question. I do believe that this indeed is a new cycle for PCs. You know, you think back and PCs, you know, were thought of a few years ago as potentially having lived their life. And what we saw is, by no means, the PC remains the dominant content creation device. It's a dominant workforce in both our business needs, as well as any of our content creation and often our intercommunication needs that we all have. In our work and business lives every day.
So what drives a new cycle in PCs? It's indeed new capabilities. So it is, the new capabilities that can leverage an NPU, a neural processing unit, like, you know, we have embedded in our Ryzen 7040. We've actually shipped millions of units already. So we're out ahead of the curve. We're actually future-proofing those that purchase Ryzen AI-based units today because the applications are now just started. When you think about here we are in a video conference. And I don't have my background blurred. But often, you may want to choose to blur your background. That's an application today that is, you can offload onto that neural processing unit, that inference accelerator.
And AI, as well as, you know, some of the AI we clicked an option here that said, "Hey, let's create captions. Let's translate the words that you and I have into, you know, written text and have that caption." And that's another example is today. Well, that's nothing compared to what's coming, because when you look at the kind of applications and Microsoft has spoken publicly that 2024 is the year that they're gonna enable, you know, very broader set of AI inference and applications for the PC and a developer's ecosystem around that, it's going to create, indeed, a new cycle.
Because now, you're gonna have capabilities we've never had before. You know, this is an international broadcast we're doing today. So imagine if, you know, you're, you know, in France, listening to this and say, "I would like to hear this in my native language," click French. And you get, you know, a very accurate simultaneous translation of the device. And it's not tying up your capability to process a conference and be doing other tasks on the side. It's in that inference accelerator. Think about content creation, where you look at, you know, Adobe and others are creating an incredible capability to interface with the natural language and create, you know, new visual content capability. It's gonna transform content creation. The PC is a content creation device.
With that, AI offload capability and then working in conjunction with that integrated CPU and GPU, it's just incredible capabilities that are coming. And Copilot, you know, I'm, w e're using a number of Copilot applications at AMD. We're using that. And I don't think there's an email I create that I don't have Copilot help me draft that email, certainly check the email, make sure it's context-aware, it's appropriate. You know, all of this is still at the very, very beginning. And we're really pleased with our work not only with Microsoft, but with, you know, many ISV developers out there. So we're very excited with the roadmap, just launched a Ryzen 8000, and again, the first desktop CPUs that are out there to be AI-powered.
And we have a very strong roadmap. So we're driving up significantly that very energy-efficient, very low-latency AI processing c apability in our PC roadmap.
Great. We're gonna open up the questions in a second. Maybe just one question before we do that. I wanted to ask about Intel 18A. You know, we get a lot of inbound from investors who are quite keen for your perspective on the impact that 18A might have on Intel's ability to rebuild. And as they ramp their platforms for PC and server, does it make AMD wanna pull in their process roadmap? Do you see sort of two-nanometer-based chips as something that needs to happen quicker for AMD? Anything you can share in terms of that process upgrade that's coming from Intel and how AMD thinks about it from your own perspective?
Well, certainly. I mean, you know, Intel's been very public on their efforts and driving their fab technology. And you know, what we do at AMD is we always assume the competition will execute. And so we drive ourselves to make sure that we will continue to be competitive in actually leadership capabilities. And that's exactly what we're doing in our PC roadmap. We, for years, Brett, when you go back, you know, over really a decade ago, and you look at, you know, where we're at before we launched our Zen product line and before we shook up the market with the 7-nm and lower topographies, we always competed with a disadvantage on process nodes. And so we had to really hone our design capabilities.
And so, you know, you had to win by design and m ake up where you might have shortfalls in process. What we did from 7-nm and beyond in our client roadmap across, as well across more broadly our roadmap, but I'll focus here on this topic with our Ryzen line for PCs, is again we leveraged that DTCO, that deep technology co-optimization, with TSMC. And we've made certain that we're hitting the sweet spot on the curve of every node so that we get the right performance, the right power and performance trade-off, and the right cost, power, and performance trade-off. And that's what's enabled us to position our roadmap very, very well. We've grown PC share tremendously since the launch of our Zen and our Ryzen product line.
We're very excited going forward. We have, you know, we've worked with, you know, the right timing of when N3 and N2 comes on our roadmap. But I will say, again, it's not just process. You have to look at the core design. So we've leveraged in our Zen CPU product line, we continue to have our high-performance optimized cores. But we also have cores which are very dense and power-optimized. And we have the ability to support hybrid cores across our design. So the applications that don't need the highest performance can run more energy-efficient. And the ones that do need t hat performance, get it. And oh, by the way, it's the same implementation of the instruction set architecture.
So programmers don't know the difference of which core it's running on. It behaves exactly the same regardless o f the hybrid core which it's being run on. So we put a lot of design innovation in, and continue our deep partnership with TSMC on node technology, as well as packaging technology.
Yeah. Great. Well, look, I think this is a good juncture to open up for Q&A. So I have my colleague, Yanku, also standing by to help on the questions. So, Yanku, do you wanna take it away?
Yes. Thank you. The first question comes from Akhilesh Kumawat from Bernstein. What is your strategy to ROCm to compete with CUDA? What are the key technical milestones for us to keep an eye on to validate ROCm's competitiveness?
Mm-hmm. No, it's a great question. Thank you. You know, ROCm, as I said earlier, is our software enablement stack. It's critical because when, you know, customers run, they program often at a high level and framework. More and more people are programming today, at a framework level which is actually independent of the ultimate device you're running on. Are you running on NVIDIA? You're running on AMD. You have to have a stack that translates from that high-level framework or a programming language like Triton, which is again vendor-agnostic. And you have to optimize it to really deliver that value, that total cost of ownership. We've been developing for years, ensuring that we have a competitive stack. ROCm first and went to a fully competitive position in high-performance computing, HPC, with ROCm 5.0.
And all that while, we were preparing that stack as well for the broad range of AI applications. And when we announced ROCm 6.0 at our December 6th AI event, and now it's out there. It's open source, which is a huge differentiator for us because it's not just us developing. We bring the community with us. And so it's now out there today. It's running highly performant. It's supported by all the widely used LLMs today. And as I said earlier, it's now expanded as well to Radeon. But I talked about how we're optimized for that general use case of running from a framework or, you know, a vendor-agnostic library.
But the other thing that we've been investing on is the ability if you did indeed write code at a low level in CUDA, how to very, very efficiently port that over into ROCm, into our stack. And we've done that. And we have a number of customer testimonials out there. In fact, we shared that at our December event of those who took existing CUDA applications, ported over. Again, we're a GPU just like NVIDIA. We have a shared history, a shared journey over decades. And so, it's not surprising that that porting is a very straightforward process. There's a lot of work underneath, because you have to have very detailed and performant libraries that get called from those GPU-type semantics. But we've done exactly that. We've gotten it out there. We've tested it. We have a very, very strong at-scale suite.
And now, we've earned that credibility. We have the seat at the table. So we're working with customers, building out more and more of those use cases. You know, I would point right now to, for instance, Hugging Face has thousands of open-source AI LLMs out there. They are regressing not just on NVIDIA but also on AMD before any of those models go through their nightly releases with updates. PyTorch, we're a founding member of the PyTorch 2.0 and the PyTorch Foundation. And again, we are a full-fledged supported offering of PyTorch. And with more and more customers every day, we're building out those examples of where very fast away customers are able to adopt and really get TCO advantage with ROCm. So it was an absolute turning point for us with ROCm 6.0.
We're not slowing down. We've created, you know, we're really growing the division we have. The AI group is an entire division in the company focused on our software capability, and ISV enablement.
Great. Thanks, Mark.
Thank you.
Yanku, can we take the next question?
The next question comes from Fred Holt from Polar Capital. There was some news that blew up on Twitter overnight, and that has been around for a while about a company called Groq, which is claiming that it can blow everyone out of the water on AI inferencing. They have designed and built an LPU that used SRAM memory and now HBM or CoWoS. Given MI300 is looking to take share in the inferencing market, do you have any view on Groq LPUs versus AMD GPUs?
Well, what I'll say broadly is there is an incredibly broad range of inferencing applications. And it turns out that the correlation of what you're trying to do in your inferencing to how the model was actually trained, and how it can be deployed in a very, very broad inferencing application really matters. So if you're looking at the largest LLM applications that were trained on GPUs, it turns out the inferencing needs to be run at scale on GPUs. That doesn't mean that there's not room for innovation across inferencing. We're gonna see, per my comment earlier, there's so many inferencing applications that there's gonna be, you know, just a huge continued demand for our CPU/GPU-based clusters on both training and inferencing. You've seen how we've expanded our portfolio to have the kind of tailored inferencing needs where it makes sense.
We talked earlier about ASICs that large hyperscalers are creating. And of course, you're gonna have startups that can create efficient inferencing applications. But the trick is to look at those applications and say, "What is the software stack and application demands? Can it be widely deployed? Or is it, in fact, a more bespoke applications?" Which is fine. That still means that there's markets for it. And that's exactly what you're gonna see, that again, the compute capacity demands are growing so astronomically that when you hear these announcements, you don't have to think, "Oh, well, this must disrupt someone else in the industry." It's not necessary at all. You know, of course, there's, and I would never say there's not, an opportunity for disruption. But I haven't seen it yet in the AI space.
What we're seeing instead is just a broadening of the tool chest of engines that can be applied to the broad set of training and inferencing applications.
Great. Thank you. The next question comes from Greg Hart from Viking Global. Can you talk about AMD's AI networking roadmap and strategy to close the GPU networking gap versus NVIDIA in building multi-GPU training clusters? To what extent does AMD plan to develop its own networking silicon or new networking fabrics to improve training performance?
Thanks. It's a great question and absolutely pertinent in terms of our scale-out capabilities to take on those largest training clusters. We're incredibly focused on that space. First of all, I'll say we differentiate from our competitor in this regard in that just like our software stack is a fully open software stack, and is explicitly put out there to drive collaboration and community engagement, all the way down to our, you know, to our library optimizations up through the end application stack. Likewise, when you look at our hardware build-out strategy and our networking and scale-out capability, it is the same. It's about ecosystem.
We announced at December 6th that we're opening up the key aspect of our Infinity Fabric, that fabric that allows our CPUs and GPUs and our GPUs to GPUs to communicate very, very effectively and efficiently. And we're opening up the key aspects of that specification to networking vendors, so that in their plethora of solutions that they have today, leveraging Ethernet. Ethernet is, you know, the most vastly used scale-out networking capability out there. And so adding that protocol to be able to very efficiently scale out our GPU complex and to build the largest of training clusters is fundamental and not only our training capabilities but the ecosystem around it and our customers having choice with what vendor configurations that they may wish to use in building out those huge AI clusters. The other aspect is in the pod build-out itself.
How do you build out the GPU cluster? That does involve the know-how that we have because we're using, we're finely tuning those, you know, the scale-up capability. And we have that know-how in AMD. So we have, through the acquisitions of Xilinx and Pensando, we have excellent outstanding networking skills, and they're focused on ensuring that we have the most efficient scale-up capabilities. The other thing you're gonna see, as well, is innovation around switch devices that, you know, can really, you know, bring, you know, that type of capability to expand GPU clusters to a higher radix, using industry-standard switches. And we're excited about that as well in future roadmaps.
Excellent.
Thank you.
Thanks.
The next question comes from Richard Clode from Janus . Sam Altman and Masayoshi Son all jumping on the AI silicon bandwagon. But if you started today, even with limitless money, how long before you had a viable chip in production with the necessary software stack?
Well, that's a, it's a great question. It's a tough question. I mean, I made the analogy earlier to our journey. You know, frankly, our journey's decades long. People might look and say, "Look at that. AMD came out of nowhere. And here's this, you know, this, AMD Instinct MI300 that is taking on not only taking on NVIDIA, you know, beating them in, in these, key, you know, large language model inferencing applications." It didn't come out of nowhere. It came out of, a long journey that we've been on, a hardware and software journey that we've been on, the GPU heritage that we have, you know, which is, you know, so many years in, in the making.
And again, it took all of that to earn the seat at the table to where we now really understand where the algorithms are going. The hardest thing for a startup today is if you haven't earned that trust of the leaders in the industry, you're not understanding where the new algorithms are going. You're not understanding where the new demands are, where they need flexibility and programmability because the change is occurring so quickly. So I think it is a high barrier to entry. You know, there will, of course, be more competition. And I think where you'll see the most traction is where it's focused on more bespoke applications that you can tailor, and you can develop a unique advantage, and you can gain that toehold in the market.
for these broad, you know, highly performant GPU applications, that's where the highest barrier to entry would be. And again, one that we were working on many, many years to arrive at where we are today with our Leadership Instinct roadmap.
Thanks. Yanku, next question if we can.
The next question comes from Peter [Richter]. Will you bifurcate your CDNA devices at some point offering a device optimized for scientific compute, double precision optimized, and a device optimized for AI, small data type optimized?
No, it's a good question. What I'll tell you is, if you look at what we've done across our roadmap, we did bifurcate, first of all, our GPU roadmap with CDNA and RDNA. So one, the RDNA to focus on gaming applications. It still supports AI, and it's gonna share the base constructs and the learning that we do of AI optimization with our CDNA, but it's gaming-focused. The CDNA, on the other hand, is all about HPC and AI. You know, we'll mature that roadmap over time. At this point, all of our devices do support both HPC and AI, meaning for HPC, it does support those high-precision floating point operations, FP64, FP32, you know, single and double precision floating point, which you need for HPC. But we'll continue to watch the space.
What I will tell you is, we're really seeing HPC and AI on an eventual convergence path. The, the developers of HPC are finding that they can, in fact, in many cases, take advantage of the math approximations. And so, we'll continue to watch the space. If it's necessary to have a version of CDNA, that, that one that supports single and double precision, another version that, doesn't, if that provides a, a, you know, a strong TCO advantage, we'd absolutely consider adding, adding that. But we're gonna listen to our customers. It's what we do best at AMD. We partner. We listen. We will adapt our roadmap to what our customers need.
Great. I think we've got time for one, one more question, Yanku. So we'll, we'll make that the last one. Thank you.
The last question comes from McLane Cover from First Republic. Can you talk about instances and applications where AMD's GPU can be used instead of NVIDIA's H100?
Any application. I mean, if you don't have something that you wrote at a very, very low level to some unique NVIDIA constructs and which you can. I mean, you can actually code such that you are tied to an H100. So, you know, those you have to segregate and put aside. But, you know, we are a, you know, a high-performance data center GPU that supports all the common frameworks. We support even low-level semantics that may have been written in CUDA that you can port over unless you used, as I said, some very unique elements that are highly proprietary. So, we are after, indeed, the broad TAM of data center GPU compute.
Does that, Mark, just to clarify, are you seeing more demand this year for internal requirements? So, you know, some of your customers keeping the GPU rather than put them in public cloud, they wanna keep the GPUs for their own use? Or, are you expecting the majority to be rolled out into public cloud? And when do we see instances with MI300 in the market, roughly?
Yeah, great question. I mean, this is by far the fastest product launch that we've ever had. You know, we, w e first, you know, Lisa put out there that, you know, we felt we had a $2 billion market opportunity in 2024 as she revised our last earnings to greater than $3.5 billion. And we're tracking, you know, to that growth rate. It is being led by first-party applications because you have hyperscalers who are really starved for the kind of compute capabilities that they need. But you're gonna see a quick follow. You're gonna see in the first half of this year—here we are in February. But in the first half, you will see instances stood up that are there for third-party applications. And you're seeing the OEMs that will be servicing the enterprise market and bringing out their products to market.
And so we're really, it is not just the fastest ramp that we've had from a revenue opportunity. It's the fastest ramp that we've had across a broad data center application suite from f irst-party applications, third-party applications, and enterprise.
Exciting times. Yeah. It's gonna be an interesting year.
Very exciting.
Mark, just wanted to say huge thanks for coming on today and sharing your perspective on, you know, what is a really exciting time for AI and for AMD. Fascinating discussion. Thanks very much. And I also wanted to extend m y thanks to Suresh and Mitch, who's on the line as well for making this possible. Any final remarks, Mark, that you wanted to leave with the audience?
Yeah. Just well, first of all, thanks for having me. I mean, we're really excited. We're passionate about what we're doing at AMD. We're thrilled to be bringing competition to the highest performing levels of AI. And we're thrilled to be bringing AI across our entire portfolio, all with one software stack that we have essentially developed and supported in the company. As you said, Brett, exciting times ahead. Thanks so much.
Yeah. Great. Appreciate it. Great discussion. Thanks, Mark. And thanks, everyone, for dialing in. Thank you.