All right. Well, thank you everyone for joining us at our Needham Tech, Media, and Consumer Conference. I'm Matt Calitri. I work on the infrastructure software and cybersecurity research team here. It's a pleasure to be joined by Penguin Solutions today. We have CEO Kash Shaikh and CFO Nate Olmstead. They're gonna run through some slides, do a little background, then I've got a list of questions on my end, but please feel free to. I wanna make this interactive, so if anyone has anything, we'll have plenty of time for that too. Yeah, you guys take it away.
Thank you. Thank you for having us, Matt. We have a few slides, but we will make it conversational after these slides. So starting with some of the trends we are seeing with AI is at a fundamental inflection point. We are always talking to our customers, understanding what's going on and how they are using AI, what's happening in their environment, and why are they investing in AI? Are they seeing ROI, not seeing ROI, and all that stuff. We will start with some of the industry trends that we are seeing and how we are helping our customers as AI moves from experimentation to production. AI is also moving to the next phase, which is inference.
The first phase primarily focused on model training, hyperscalers using AI for model training. This next phase where it is much more of the implementation of AI with agentic AI using inference and really automating the workflows, which is one of the reason the adoption for AI is going up because the enterprises are realizing the value of how AI, especially agentic AI, can help them. For example, the use cases we see our customers using agentic AI range from internal usage. They are using agentic AI to automate the workflows within their customer success, customer support, departments, where they can have the agents help answer the questions, resolve the cases, deflect the cases.
We are also seeing customers use agentic AI for software development within their IT, even if they are not a tech company. Obviously, IT has developers that is building the solution for whether it is their e-commerce or running their infrastructure is another use case where we see agents automating the workflows, creating the support for the developers. All that is driving the demand for the infrastructure, especially when you have the agents that are working 24 by 7 on behalf of the users. These agents consuming more infrastructure, more compute, more memory because they are working on behalf of us 24 by 7. AI is also moving beyond the hyperscaler to enterprises. Enterprises are deploying it across their operations as we discussed.
They are also using AI to create new business cases, which is again, driving the demand for the infrastructure. As AI move from model training primarily to inference, there is a lot more need for general purpose compute. You may have seen in the agentic AI adoption, a lot more demand for processing CPUs beyond the GPUs. At the same time, these general purpose processors or the CPUs are using a lot more memory beyond just the High Bandwidth Memory that is used with the accelerators or the GPUs.
A lot more workloads with agentic AI, a lot more usage beyond beyond the accelerators and High Bandwidth Memory to CPUs, as well as the memory that is becoming a key requirement, both in terms of scaling these workloads, because these agentic workloads, some of us have 2 or 3 agents. In fact, some of the software developers that are using agentic code development, they have up to 10 agents. That as a result, a lot more infrastructure requirement beyond beyond the accelerators to compute and the memory. This is a high-level view of some of the things I mentioned. If you look at on the left-hand side, this is the pre-inference or pre-agentic AI era, where primarily, obviously, we were using the chatbots. We are still using the chatbots, but this is before agentic AI.
Primarily at that point, you would have the inference, which is primarily the chatbot. You're asking a question, ChatGPT or whatever you're using, it's gonna give you a response. These workflows were not as intensive because it's like a signal way communication. As AI moved from just a chatbot to agentic AI, these agents are doing the workflows. They are integrated in our system, which you see on the right-hand side. It drives a lot more demand on the CPUs as well as the rest of the infrastructure, beyond just the accelerated compute or the GPUs. What you see in this right-hand side, the memory requirements become much higher. This is just the general purpose memory as we discussed.
At the same time, within the inference environment, a lot more need for the memory for the inference to be making the decisions. To give you an idea, for example, if the agent is writing a book, there are two ways that, you know, the agent can work. Obviously, let's say the agent has written about 75% of the book. Before writing the next sentence, if the agent is not using the memory which is direct attached, it will have to do the compute again before writing the next sentence. Having a piece of memory which is KV cache, CXL-based KV cache is one of the instantiation that we provide, allows the LLM to use the book that is written 75% within the memory, access it directly and write the next sentence.
What is the next net effect? The net effect is LLM responses becomes much faster and whatever task they are performing as a part of the agent, it becomes much more faster and much more easier. The net of it is agentic AI with inference driving a need for more memory, which is related to our integrated memory business, as well as we are seeing increased demand with our AI infrastructure business as these inference workloads requires a lot more memory than model training. This is our AI factory platform. We have two main businesses which are AI-driven. We have integrated memory business. We have AI infrastructure business. This is our strategy for these two businesses. We call it AI factory platform.
This platform is a combination of products that we provide for our customers to build and manage the AI factories and the services we provide. We provide design, build, deploy, and manage complete end-to-end services. Starting with the first element of this platform, ClusterWare. ClusterWare is a software. You can think of ClusterWare as an operating system. Just like with Windows, for those who are using the laptop with Windows. Windows operating system uses the memory processing capabilities and allows you to run the applications on the laptop. Our ClusterWare is an operating system for AI factories.
As customers are building these advanced, data center, AI native data centers, AI factories, one of the approach they can take is to configure each of the GPUs, each of the compute and networking to be able to stand up their AI factories or manage their AI factories. Alternate approaches, they can use our ClusterWare, which allows them to create cluster for all of the infrastructure, including GPUs, including the CPUs, networking equipment, the memory, and able to manage it, build it and manage it in a much more automated way, which is the advantage for us, when we are providing solutions to our customer that can simplify their deployment and management. The second piece of AI factory platform is a new line of products.
We introduced MemoryAI, as we discussed, in inference workload, memory becomes a lot more, they are much more memory intensive, and memory can increase their efficiency as well as the response, which is very critical in inference workload. We introduced a new product in this line of products called KV cache as Appliance, and this is an appliance that can accelerate the responses as we discussed. The third element of our AI factory platform is compute. We create custom compute based on the requirements of our customers. The fourth element is OriginAI. OriginAI is a blueprint as an end-to-end solution for AI factories that we provide to our customers that allows them to have the requirements that whatever specific workload they wanna deploy, we provide them end-to-end architecture with OriginAI.
The fifth element as we discussed is our end-to-end services. We have products. We provide end-to-end services so that customers can have a single point of contact that is responsible for everything. Even if some of the product we don't have, such as storage and networking, we will source it for the customers, design it and manage it on their behalf, working with our ecosystem. That's at the high level our AI factory platform, which is very helpful obviously for enterprises. We are winning customers across enterprise, sovereign AI, and we announced five new logos as a part of our Q2 earnings announcement, and this is how we are providing them the solution.
At the high level to net it all, AI factory business providing product and services as a full-stack AI factories that customers can deploy. Large financials, very large enterprises are building their own AI factories so they can have much more performance latency that is needed in inference workloads. Obviously, economics is much better for them. They are using cloud for model training. They are using inference and agentic AI for on-premise factories as they build their factories. Governments around the world, they are building sovereign AI factories, and the idea is they want sovereignty. They don't wanna rely on U.S.-based infrastructure.
We built one of the largest AI factory in South Korea with our NHN Cluster, which is giving us the visibility to help other countries around the world, where we are working with them on building their own sovereign AI factories. MemoryAI line of products, we discussed our memory business. Having the memory business and AI infrastructure business gives us the insights, deep insights for our memory architectures, deep insights for what is needed in this next phase of AI with inference. That has allowed us to build these kind of appliance that are one very unique. Nobody else in the market has a product like MemoryAI, which is helping our customers and helping us win the deals.
Last but not least, our integrated memory business where we create specialized memory card for large vendors, OEMs such as Cisco, Google, and some of the names you see on the slide. Memory business driven by AI infrastructure business, using some of the differentiation with our understanding of memory to provide full stack solution for our customers as a part of these two main businesses that we have within our company. I will leave you all with this. You know, we have three businesses at the high level within the company as we discuss integrated memory business, AI factory platform business. We also have an LED business.
The high level view of our AI driven business between integrated memory and non-hyperscaler AI HPC business, that makes up for 60% of the business in the first half, and that business in the first half grew about 50%. This is where, you know, this business is AI driven between memory and AI. Just wanted to show you guys the view of the scale of the business on a run rate basis. You know, this is well over $1 billion growing in excess of about 40%-50%. We are very excited about all the build-outs that are happening in the market and how customers are using it and it's an exciting time to be in this business providing memory solutions and AI factory platform.
You wanna add anything, Nate Olmstead, that I may have missed?
No. I think you covered it. Thank you.
Great. Yeah. Well, thank you so much for the breakdown and it is a truly exciting time to be at that intersection of memory and AI infrastructure like you guys talk about. I wanna get into some of the more product specific stuff but maybe to start, Kash, you took over as CEO in February. What drew you to the role and how have the first couple months in the seat been?
Good question. I have been in this space, infrastructure across, you know, large companies for 30+ years, between large companies and small companies actually. I've spent time at Cisco when Cisco was building the data center products, entering into the compute space. I was at HPE in the networking business unit, more interestingly, the relevant piece, more relevant piece is my time at Dell. I ran Dell's AI HPC business. This was before AI HPC was as hard as it is right now between 2016 through 2020. Interesting coincidence was it was a $1 billion business. Obviously, Dell is a scalable company.
It was a $1 billion business when I took over and we competed against Penguin Computing, which was the company that the parent company acquired. Anyway, having been in this space, seeing, you know, the requirements and understanding of the market, when I found out about the opportunity at Penguin, I was excited about that. Knowing the space, as well as the opportunity here, as we discussed, the memory business as well as the AI infrastructure business, that got me excited about the opportunity and, obviously talking to the team and talking to a former CEO, I realized that there is a lot of opportunity here to grow the business. That drew me to the business in terms of why Penguin.
To answer the other part of your question about what have I learned, some of the things I shared, I would say at the high level. I've spent a lot of time, obviously, with the customers, around the world. I've visited pretty much all of our regions, talking to our team members, talking to customers, talking to partners. There is an inflection point happening within AI, from model training, experimentation, to production, to agentic AI, and that is creating a unique advantage for us. That's why we have prioritized our AI factory business. We came up with the AI factory platform strategy. We are investing more in innovation, investing more in our MemoryAI appliances. We are investing more in our Scyld ClusterWare AI.
We are planning to make it agentic so that, you know, it can perform all the tasks on its own, only with the human in the loop. Those are some of the things that I have learned in terms of what's happening in the industry, and it's, as we discussed, it's exciting time to be in this space.
Yeah. No, absolutely. That's great. What have been your first couple key initiatives as you stepped in?
Yep. Number one is, you know, more investment both in product and R&D for our AI factory platform. This is the growth strategy which we believe can help us capture more demand that we are seeing in this segment. Second initiative is just like we discussed, all of these enterprises are using AI within their own operations, company operations. We have a priority where we are investing in AI across the company. You know, we have a software development team that is now using Cursor to develop the code with the agents, and we have introduced agentic AI in our customer success team. All of our teams, including myself, we are investing time and resources to what I call drink our own champagne and take advantage of the opportunity and efficiency that AI provides.
Great to hear. You guys reported very solid second quarter results. When you think about what went well in the quarter from an execution standpoint, what stands out and what's your focus as you look to the rest of the year?
Yeah. A couple of things, as we discussed in the earnings announcement. Memory business is performing really well, and as we discussed, this is the driver for this demand is AI, agentic AI that we are seeing, which is, you know, driving the demand for the business. As well as even if the pricing is one of the advantage, there is a supply constraint in this market given all the requirements AI is putting on the memory. The demand, we believe, is much more durable based on the fact that it is AI-driven. Team executed really well in the opportunity for integrated memory business. Our AI HPC business, we acquired five new logos, and some of these are very large customers.
Tier 1 financial, top 10 energy companies. Deepgram, which was a logo we shared publicly with which we acquired as a part of our partnership with Dell and our partnership with NVIDIA is also getting stronger as NVIDIA is entering into enterprise space. Given some of these hyperscalers, which was the driver for their growth, are developing their own chips. That is creating a synergy for us. The team executed well on the demand, and we are acquiring new logos and growing the business. Those are some of the highlights from the earnings for Q2.
Sorry. To stick with memory for a second, you made a point on the call to emphasize the opportunity that the MemoryAI line unlocks, and particularly as memory becomes more central to AI infrastructure and production. I know you hit on this a bit in the slides, but what exactly is the goal with that product line, and how has the memory business evolved at Penguin Solutions?
Yeah. That product line, MemoryAI KV cache server, is one of the many products that we are going to provide to our customers. MemoryAI product line, the reason we were able to develop is we have deep understanding of the memory. We have deep understanding of AI infrastructure architecture. Even if it is built within our integrated memory team, the buyer for that product is essentially the AI infrastructure buyer. For example, this customer, Tier 1 financial customer we acquired in Q2, they acquired MemoryAI KV cache server. KV cache is a technology that allows you to, you know, as we discussed, keep the content, right, accessible much more faster for the accelerator to use that content and respond to the queries much faster.
There are other appliances in the line of MemoryAI that we are working on. We were an early investor in Celestial AI. Celestial AI is photonics high bandwidth photonics memory company that got acquired by Marvell for $5.5 billion pre-revenue. In addition to being the beneficiary of receiving some of the proceeds from that acquisition as an early investor, we continue to work with Celestial AI to develop photonics memory appliance. That photonics memory appliance will provide even more access, faster access to the memory required in the inference workloads. This is, you know, obviously, a growing area and as enterprises continue to deploy inference, that's gonna help them with the MemoryAI appliances.
The MemoryAI appliances is really the representation of the uniqueness of the company at the intersection of memory and AI infrastructure. Having the ability to develop the products and drive the synergy and provide the products to our AI infrastructure customers are truly a unique advantage for Penguin.
We've seen no shortage of headlines about memory shortages and stuff, and I was even reading recently about, like, worries about a strike over at Samsung, and what are you guys seeing in that DRAM memory market as far as supply chain, price increases, stuff like that?
Yeah. Supply is constrained, obviously, as we discussed, because of all of the demand of memory, whether it is High Bandwidth Memory or the regular memory. One of the advantage we have is we've been in this business for 40 years. We have relationship with all large many memory manufacturers, which is allowing us to have the access to the memory to serve the demand of our customers. One of the main supplier, SK hynix, we have a deep relationship with them in addition to working with them for decades. SK is an investor within the company. We work closely with SK Telecom and other SK companies, having that as a part of our relationship allows us to have the access.
The supply is constrained, so we have a backlog, but, you know, based on our relationship, we've been able to access it and serve as much as demand as we can. Nate, I don't know if you wanna add anything to it.
Yeah, I mean, you've been reading the same thing, right? It's tight. We're fighting every day for supply. Building a very strong backlog, going out a few quarters. Kash is right, our relationship probably helps us a little bit at the margin, but it's very tight.
Yeah. Very important, though. Turning to advanced compute. Good quarter there, but actually end up reducing guidance looking forward. What changed during the quarter and how are you guys looking at that business?
Yeah. That business, very strong pipeline, very strong bookings, and bookings growth in Q2, I'd say above market conversion. The challenge in that business is by the time we book to the time we recognize the revenue, it's about 3 months. However, recently it is between 3 to 6 months, and part of the reason is we are landing new logos, acquiring new logos, as we discussed 5 new logos in Q2. When you have a new customer, you have a new process, it takes longer for them to get to the production, get to a state where we can recognize the revenue. The other challenge is also the material availability. In some cases, it's taking longer. The good news is business is doing really well.
The challenge is time to book to revenue is increasing. For example, when we announced our Q2 results about 6 weeks ago, even at that point, most of the bookings from that point onwards for the second half will be recognized in the first half of next year. It's really a timing issue. We are seeing pretty strong demand, and it's a matter of execution for us.
As you work through some of those timing issues and stuff, what's the best way for us to sort of judge the underlying momentum of the business? What are you guys looking at to sort of to determine you're on track there?
Part of that execution is we've been focused on diversifying the business. Acquiring new logos, continue to have new customer is one of the metric we keep in mind. Diversification is working well as we discuss new logo acquisition, and we have several new logos in the pipe for Q3 that we are working on executing. Bookings obviously is a leading indicator, and we are very closely monitoring the bookings. As we discussed, the bookings are growing pretty strong. Revenue is a lagging indicator, and it's a matter of timing. Making sure we are focused on the right segments, as we discussed, making sure that we are leveraging our differentiation with AI factory platform. We feel confident that, based on the execution, that we will continue to deliver on our commitments.
Great. I wanna make sure I'm not chewing up all the time here too. Do we have any questions from the audience?
Can I ask one?
Yeah.
When you were going through your initiatives since you took the role as CEO back at Bedrock, one of them you said was deploying AI internally, right? I'm curious, like, how are you rolling out AI internally to drive? Like, what efficiencies are you trying to mandate? How do you push that out to your employees to ensure that they're using AI appropriately without just racking up a token bill?
Right. Right. First of all, you know, it's, it depends on the department. Let's say if we are talking about software engineering, which is, as we discussed, we are investing more in software engineering, but making them, first of all, making them the tool available that they can use. In addition to the giving them the tools and access to the tools, we are working on providing them the training because they need help in terms of how to effectively use the tool. In, at the high level, in this use case, the goal will be how much of the code can be developed by AI. For example, in my previous company, which was an agentic AI software-focused company, we were developing 30% by the time I left end of January.
We were developing 30% of the code by AI. What does that mean? Let's say if I had 100 engineers, I was developing the code equivalent to 130 engineers. That's the value that we are, we saw, and that's kind of the goal over here. For the amount of headcount we have, how do we make them more efficient so we get more for what we are investing? In terms of the usage and the tokens, as long as we provide them the right guidelines, as in it's all about providing the guideline how to effectively use the tool, because tool is a tool.
If we help them use the tool effectively, then they will be able to create the efficiency we are expecting, and we are not gonna spend as much as, you know, they can spend if they are not being trained. I think training and enablement is equally important as making them the tool available and helping them realize why we are doing it. You know, obviously, a lot of people are worried that, you know, if they use the tool, they can lose their job. You know, there's always this discussion. We've been very upfront and one of my advantage is having done it in the previous company, and we were on the leading edge, and having gone through this process, we've been upfront that we're not focused on reducing the headcount.
We wanna create that competitive advantage that if we don't use it, we're gonna have a disadvantage against those companies that are not using it. We are not using it to reduce the headcount. We wanna move fast, we wanna deliver more software kind of capabilities as an example in this case. That's how we are, you know, working with our team members and helping them understand while providing them the training and coaching to use the tools effectively. That's an example. I mean, different departments have different tools and different outcomes we are expecting, and we are making sure that they understand this is how they should use the tool and this is what we will be measuring for us to see the ROI on our investments across the company.
Thank you.
You're up? All right, jump back in here. Sorry.
Yeah. One of the big questions and sort of big picture questions that investors have been asking across the board is return on investment in AI. You gave a good example of how you get 130 engineers production from 100. Is that across the industry? How prevalent is that? What do you see going forward? Is there more headroom to continue to have ROI that's gonna justify these mega billions in investments?
Yeah. It is not to answer your specific question that all enterprises are looking at the efficiencies like us. To be honest, the answer is no. I see a range of, you know, clarity on why enterprises are using it and if they have specific outcomes in mind. We talked to some of the enterprises where they are still experimenting. They're making it available to all of the users without having a specific goal in mind, which in my view is still effective because the first thing is getting over the hump of having team members understand this is not gonna replace your job, right?
At the same time, while that is the first goal, which is to make sure everybody's using it, having the clarity is also important and may or may not be the case because some of the enterprises are still learning. You know, they quickly realize they may not have the goal upfront. As soon as they start burning the tokens faster, it becomes a discussion of, you know, Are we seeing the ROI? Not seeing the ROI, let's get back to the focus, which is getting more for the investments we are making. I see a spectrum of companies that may be not exactly using the starting point, but they are coming to the same ending point, which is focusing on the ROI.
Is this a tip of the iceberg? Are we just beginning? How many years can you see this computer return on investment spectrum developing or across enterprises? Are we talking about a couple years, three years, more than that?
I think it's this year, especially so in the last 3-6 months where we have seen a lot more production deployment, experimentation to production, and which is why you see a lot of demand across companies that are providing the infrastructure for that. I'd say that the adoption will increase significantly in the next 1 year because especially for those who have been using it, the tools are becoming more effective as well. That's another dynamic because these agents have made it much more impactful when they are automating the workflows, when they are providing the outcomes in a way that, you know, you're getting more for each of the employee from their outcome perspective. As with any other technology, as we know some companies are embracing it sooner than others.
It's gonna vary based on where they are in their cycle of using the technology to differentiate and invest in the technology. For those adopters who have either started it or will start it, I think the technology will show more returns, which is why the adoption will continue to increase in the next 1 year.
One thing we've spoken about is before you can get to end user adoption, you have to have everything set up on the back end, and just how difficult that is for companies to stand up and so they'll turn to a solution like Penguin Solutions to get that done. What are you guys seeing in terms of the split of organizations that, like, have the internal sophistication that they sort of just need racks as quick as they can get them versus companies that maybe try to do it on their own and then realize they really need this full stack solution from Penguin Solutions?
Let me provide a little more context for the audience in terms of how enterprises are approaching AI infrastructure. The first thing, whether they are enterprises or mid-market, the easiest thing for them is to access a cloud-based infrastructure, right? Go to AWS or one of the new cloud providers, get, you know, access to the infrastructure. That is a facet. They don't have to worry about setting it up. What happens is, you know, acquisition is easier, your team gets started, they start using it. What we are finding is the reflection point for them to move from just using cloud-based to building their own factories is the scale. When you are at scale, especially very large enterprises, for example, this large tier 1 financial bank we acquired, they have 40,000 developers within IT.
When those 40,000 developers started using agentic AI, the budget they had for one year, they exhausted that budget in three months. They started realizing, "We need to have our own AI factory so that I'm not just using the subscription as I scale." They started with, you know, they are gonna build their own factory. They started looking at options, "Are we going to build it on our own or we are gonna go find another one?" They put out an RFP with a very, you know, formal process, and we were among one of the many who responded to that RFP because they had decided that they need help, to your point, right? At the end, based on our value between having the products and the services, helped us win that account.
To your point, why they are selecting us is because we provide the full stack end to end. Rather than them going to 1 company for compute, another company for storage and so on and so forth, we have the ability with our AI factory platform to provide them the solution. We provided them the solution, and now they have a setup. Interestingly, I asked the CIO, are they going to move on premise based on those solution that they have from us with some expansion? The answer was no. They are going to continue with the AI cloud for model training, and they are going to use expand their on-premise factory for inference and agentic AI workloads.
That's kind of the dynamic of why enterprises are, you know, building their own factories and when we go in, why it is more relevant for them to work with a provider like us.
When you say that they're building out their on-premise for infrastructure, are those conversations becoming more prevalent as we see companies sort of move from training into inference?
Yes, we are seeing enterprises, but these are large enterprises that I mentioned. Large enterprises in regulated industries, whether it is oil and gas and others, where there are sovereignty and privacy requirements in addition to the economics, which is at scale, is not feasible for them in the cloud.
Awesome. We've probably got time. Yep.
Is most of what you're doing is system integration and, like, more of consulting, or do you guys actually own assets that you lease out, or is it just, you're just helping people build this stuff on premise?
Yeah. Our AI factory platform, we have products that we provide as a part of the system integration. The Scyld ClusterWare is a product, a software product, that is a part of deploying and managing infrastructure. It's a heterogeneous product, so it is vendor agnostic that provides them much more simplicity in its operating system of AI factories. MemoryAI is a product that we provide that's our own IP that we provide as a part of the solution, and we have compute that we provide. Then we resell the other products that are part of the infrastructure because there's storage and there is networking and so on and so forth.
We have our own IP and in fact as a part of the AI factory platform strategy, as we discuss, I'm gonna invest more in our IP so that we provide them, unique solutions beyond the compute and the storage.
Do you have risk to, like, GPU prices or memory prices, or is that mostly pass-through?
We are reselling it. We have prices. Then we have the resell business as a part of selling the compute.
Pass through with some margin on it for the integration work and the supply chain.
Right.
All that. Yeah. One more quick one.
What kind of long-term growth rates do you see for yourselves on those three large end markets that you categorized up here?
We have not shared the long-term growth rates for those markets publicly. You can see the growth in those markets, right? Even if, you know, we have not shared the specifics, those are very high growth markets, where we are focused on between the large enterprises, sovereign AI, and new cloud infrastructure.
All right. I think we have to wrap it there, unfortunately. Kash, Nate, thank you so much for being with us. If anyone is interested in learning more, we'd be more than happy to connect you with the team.