Good morning. Thank you for tuning in to our State of AI livestream with Microsoft. My name is Brent Bracelin. I'm the Senior Research Analyst with Piper Sandler, covering the cloud software and analytics space, tuning in from Bend, Oregon. I'm joined by Harsh Kumar, our Senior Research Analyst covering semiconductors, tuning in from Memphis, Tennessee. We're very pleased to have three distinguished speakers from Microsoft joining this morning. Doug Burger, the Technical Fellow, and one of the leading active researchers in computer architecture. Welcome, Doug. David Carmona, General Manager of Artificial Intelligence at Microsoft, and Jonathan Neilson is a finance director in Microsoft Investor Relations. He is online, but not live streaming at this point in time. Welcome, Team AI at Microsoft.
Thank you very much.
Thank you. Thank you, Brent.
Before we dive into Q&A, maybe let's start with a brief description of your background, kind of role at Microsoft, and maybe what location you're joining us from this morning. Doug, why don't you kick things off here?
Good morning, everyone. Brent, thank you for giving us the opportunity to participate. I'm Doug Burger. I am a long time, formerly a researcher in computer architecture. I was a professor at The University of Texas for 10 years, running a research group there. We built CPU processors, new types of hardware architectures in Silicon. I moved to Microsoft in 2008. They started up a research group in computer architecture in Microsoft Research. My team and I built a number of new things, one of which was the Catapult FPGA platform, which forms the basis of Azure's smart networking and accelerates applications in Bing. That was actually a large-scale distributed system that we built, not just the use of the chips.
About a year and three quarters ago, the company asked me to move most of my team into Azure to really start a new direction as our cloud grows and hardware innovations and full stack innovations become more and more important. More recently, I've taken on a more active role in AI architectures and systems. That's really my focus now is AI infrastructure at the hardware and low software level. That's what we've been doing.
Great. Very good. David?
Yeah, connected from Redmond. Actually, I'm connecting from probably a short walking distance from the office, but it's useless these days. I could be anywhere. Yes, I'm in Redmond. I joined Microsoft almost 20 years ago. My background is software development. I was attracted at that time, Microsoft was very famous for Windows, right? I was actually attracted to Microsoft because of the developer tools, which is like the DNA for Microsoft. I held a variety of roles, all connected, some of them in engineering, some of them in business, always connected around the developer business. Used to lead the business for Visual Studio, Visual Studio Online, Azure Tools, and so on. Five, six years ago, I noticed that something was changing in the industry, and it's that there's a new software, that is called AI.
I made a transition to AI, and now I lead the AI and innovation team in the business side in Microsoft.
Very good.
Brent, I'm sorry, I forgot to mention, I'm in Bellevue, probably just a couple of miles away from David.
Right. Yeah.
At the main campus.
Well, that's the power of Teams, right? We can all do this meeting virtually. Very, very cool. Let's dive into the kind of current state of AI to kind of frame the discussion here. If you reflect back over the last year, what would you say were some of the kind of landmark AI innovations that occurred either across Silicon or systems, or these new class of AI models? What were the landmark things that investors should kind of pay attention to in the last year? David, go ahead.
Let's start with the technology. Just recapping on Microsoft Build. Let's go to last week. Last week we were, and I'm sure that we'll go in more detail today, but one of the key things, and it flew a little bit under the radar, you have probably heard the term of the AI supercomputer that was announced last week. The change behind that, the implications of that are far beyond hardware. The key thing that we saw with this AI supercomputer is that you can train massive models. By massive, I mean in the billions of parameters where we were in the order of the millions of parameters. That which can sound very cool, that, yeah, you can get better models, more accuracy.
What it really meant, and that is that the key change that is certainly something to be on top of, is how it's changing the way that you develop AI. AI as a platform was something that was not clear before. People, companies will have to develop AI models on a case-by-case basis. What we realize with these massive models is that they can be multitasked. They can be more generic than the traditional, very customized vertical models for specific scenarios. That will bring a very interesting motion in the market, which is companies embracing these massive models and then customizing those models for their particular scenarios and tasks. Therefore, bringing state-of-the-art AI to more companies. That's certainly one thing that I would highlight to be on top of.
Doug, I'm sure that you can provide more in there, but before that, let me just mention another thing also from last week that I think is very important, and it's the concept of autonomous systems. We have this chip, everybody, that is associating autonomous systems with autonomous driving, a lot of cases, but we're forgetting that autonomous systems are much broader than that. Last week, what we announced at Build was our first preview. Every developer now have access to that, to our autonomous systems platform. It goes beyond motion control. It not only targets robotics, but things like process optimization, things like machine calibration, and a lot of scenarios that are much more real in the current situation for companies to address. That's another one that I would highlight that I'm sure that you can add on top of both a lot.
I don't know. That was great. Very complete. Let's see. A couple things I might underscore, and particularly with the AI supercomputer announcement, is really what David said, that the size of the models has been growing much, much faster than I think any of us predicted. It was growing, but now, a round number is about 10X per year, which is a growth rate that far surpasses Moore's Law while it was active, and we all saw how disruptive that was over 50 years. Those giant models, what we've seen so far is that the bigger we train them, the more they can do.
That breakthrough towards being able to train semi-supervised, especially with Transformers and natural language, about a year and a half ago, really turned the field in that domain from a data-bound problem into a compute-bound problem, which is why we're now racing to build these supercomputers so that we can train these models in a reasonable amount of time, days, weeks, months. It is very, very disruptive when we look at the capabilities these things are able to drive, not just in terms of language capabilities, but like David said, multimodal. What we're seeing is, I think, a very rapid infusion of AI across our product families. Now, obviously, these supercomputers are expensive. The infrastructure to do them is to train those large models is very different than what people have built in the past. Once you train them, you have to serve them.
If the models are giant, serving them in real-time is also challenging, and can be expensive. I think that is a transition that I really don't think the industry has grokked yet or appreciated how fast this is going to drive the capabilities of AI and how different the infrastructure needs to be with these 10X requirements.
Before we go into the architecture, because it does sound like there's some unique things you're doing to create that AI supercomputer, I want to go back to the comment you made around just the size of the models kind of changing. Is this at the bleeding edge where it's 1% of use cases? Or when you think about the size of these models and moving from data-bound to compute-bound scenarios, is this more prevalent than maybe we appreciate as outside looking in?
David, go.
Yeah, I can take it. There are two use cases in here that I think we have to separate very clearly, right? One, which is that 1%, is companies that will be creating those massive models, and we don't believe that that will be every company in the planet. The interesting thing here is not actually building the model. It's that these models, as we were mentioning before, they are more generic. They are multimodal, they are multitask. You can centrally train those models, but you can have companies with no such a deep expertise or access to these compute capabilities, customizing those models for their own needs. To give you an example, that's the way that we're actually doing in Microsoft. We transition from having a product-by-product AI approach. You have Office creating their own models.
You have Dynamics creating their own models. You have Bing creating their own models. To having a culture where we centralize the creation of these massive models, these teams can customize them. We have features now in Office, like auto-reply or document summarization, that are coming from customizing this model without requiring that massive infrastructure and expertise. That's the two angles that I would separate that.
David, as you think about going from silo to centralized larger models, is that accelerating the feature? What's the big benefit? Is it the time to market with new AI functionality, or is it more advanced AI functions you can do?
I would say it's several, right? The first one, let's go one by one. At least I could mention three. The first one, super obvious, is skills, right? These are state-of-the-art models that now everybody have access to. For example, you don't have the same level of almost, I would say, researchers, PhD, creating those models because you can reuse it. That's one super important that will say a lot to the democratization of AI in the industry. The second one is data. Because these models are trained initially with huge amount of data, that is again central. In our case, we train it with all the index web in Bing, right? Huge amount of data that we're adding in there. For customizing those models, you don't need that big amount of data.
For example, in the case of Dynamics, they custom train that model with then additional data that are coming in their case. They do things like customer churn identification. They only custom train the model with that delta, with their domain, right? Very interesting. The third thing that I would say is compute power. Because the customization is using a technique called transfer learning, you don't need to train the model from scratch, you don't need that huge amount of infrastructure required to train the model initially. Doug, am I forgetting any other thing that you would highlight?
No. I think that captured it very well. I would add one point. You asked about the model size, you can think about the largest model that you can train and in the billions, we talked a little bit about that in our announcement. Kind of view that as the front of the bow wave. I wouldn't call it research, it's really pushing the envelope in the systems and the algorithmic techniques and the data management. You find out what those models, what capabilities they have. If you take one click down in size, that was radically large six months ago, those models are now being used just kind of throughout our infrastructure. I don't really think of it as a bimodal distribution where there's like a 1% thing, there's everything else.
The whole space is just moving very, very fast, and there's the new thing, the biggest thing at the edge, and then things that were huge one year ago, kind of now empowering all the products. If you think about how fast this is happening, I think in the notes. It takes you years to build custom hardware, silicon, and then build the systems and deploy them. If you need to make a major change in your hardware platform, that's years before you can do it. Yet, this huge shift is less than two years old, and we're still pushing the envelope on the rate of growth.
Yeah.
We're kind of doing whatever we can with whatever we can get to drive the models up. Fortunately, we had made a lot of investments in the past that we're benefiting from now. A second comment I would make, if you think about Microsoft from a strategy perspective, this goes on to something that David mentioned. We have a very successful public cloud with infrastructure that benefits our customers. We also have this massive PaaS and SaaS business that really empowers our enterprise customers, and consumers, frankly, with Office and Teams. Changes there, like David was saying, can really reach so many people and transform their business. Those infrastructure changes that we make to get those models out to such a wide group of users that see value, we can then take to our infrastructure for our infrastructure customers to use.
Those two halves, I think that's why Satya talks about the intelligent cloud. It's not enterprise software and cloud. It's one thing where benefits can slosh back and forth and empower everyone. I think we're really the top company that has both of those halves in a very strong position.
It is fascinating just seeing the pace of change in the space. It's hard outside looking in to really fully appreciate it. All we get is from consumers of the technology. We see the advances, right. We see small changes, the software is getting smarter, right. It's hard for us to quantify how dramatic things are changing on the back end. Let's drill down in, Harsh, you kind of weigh in here, too, on the architecture of the AI supercomputer. I'm just trying to understand. You have Azure, and you can kind of build your own kind of infrastructure to run your models on Azure. Now you have this AI supercomputer that's different.
Maybe walk through architecturally what's different about an AI supercomputer versus Azure and why you think that this needs to be optimized versus building it myself through my own APIs on Azure itself. Walk us through just the architecture of it.
David, do you mind if I take the initial crack at that one?
All yours. You are the architect.
First of all, I think it's a mistake to view this as Azure and not Azure.
Okay.
Okay. There's a lot of common infrastructure across both sides. If you think about what you want for an IaaS customer, let's say a big enterprise IaaS customer. What you need is incredibly high reliability and availability. You want great stability. You want predictable performance. You want access in many regions. It's really a planetary computer, and people build their infrastructure on our computing utility, and it has to stay up. If you go to the AI side and you're training one of these large models, you're running a single program or job over a massive collection of machines. It's okay for some of those nodes to go down, right? Because you can build resilience into that large-scale distributed system.
We have a lot of experience doing that, for example, with platforms like Bing, which has to stay up and respond, but it doesn't matter if one node drops out. You're building a distributed system that needs to complete the job as cheaply and as fast as possible. The reliability characteristics of the components can be different. You need, for example, a lot more networking to train these giant models because they're data-intensive and the communication is global. You'll provision much more aggressive cross data center networking than you need to for an IaaS customer. There's a different mix of hardware of silicon components because you're doing massive amounts of linear algebra. I do think that if you look at the high-speed networking, the ability to communicate quickly, the power requirements that you need to drive this, just raw performance.
Those are all things that accrue to the core Azure business as well. They're just an extreme point, that business will follow along, will benefit from the technological advancements that we're making.
Doug,
Go ahead.
Sorry, I had two questions. Earlier, I want to go back about five minutes ago, I think you mentioned, or maybe David mentioned, transfer learning.
Is that effectively, you would train this large data set, and then you would give it to an enterprise data center to run? Or are we talking about something even smaller than that, like running it on a small chip at the edge? Or are we still talking about a data center type process here?
David, why don't you take that one?
Both. At the end, these massive models, of course, they are thought to be run in the cloud. At the same time, we see many scenarios where through optimizations, the inferencing of those models can be done at the edge. They go side by side. Yeah, absolutely. Doug, anything that you add on top of that technically?
Yeah. I think the way I view the transfer learning is there's building your base, your model, which is super expensive, and then there's flavoring it with specific data. Just like an analogy might be, everyone on the call here is speaking English, but we all have slightly different vocabularies and nuances, and choice of words. If I train an English model, I could flavor it with some history from an individual to customize that model to the individual's speech patterns. As David said earlier, it's a lot cheaper to do that flavoring than build a whole English model from scratch. That's the training thing, and then there's the question of, do you deploy it in the cloud, on the edge? I think that's a separate question.
That's just absolutely incredible. My second question was on the preferred computational mode. Is Microsoft a fan of FPGAs versus GPUs, or you simply don't care? The common knowledge is that ASICs are cheaper. Once you get the parameters down a little bit on what you want to accomplish, you always want to reduce the cost of computing down with ASICs. I was just curious what flavor you guys prefer, Doug or David.
This one is for Doug.
I get asked this a lot. I'm not going to say Microsoft believes in technology X. Let me give you some framing that I think will help people to understand the space. Okay? If you think about what you're doing in AI for these training supercomputers, you're doing a massive amount of effectively linear algebra. You need an outstanding system architecture for low latency communication. You need a robust software stack. You need resilience at the distributed system level, like we said. Once you get down to the silicon, you've got a number of trade-offs. You've got the efficiency of the silicon, like how fast can it do X, just raw flops. You've got the future-proofing. If the models shift, how well does the silicon map onto that shift?
You've got other innovations that you might have in the hardware that help the model training, that might help it to converge, might save you cost. You've got this flex set of trade-offs, and we actually invest in multiple technologies. We disclosed what the AI supercomputer is built on because that part has been really great for training these large models. A second thing I would say is that I really don't like the term ASIC because in this space, it's kind of gotten overused meaning. It just means custom chip. Right? In the past, it meant something very different, right? You're building a chip or a circuit for one purpose. What these things really are is they're programmable engines that do linear algebra flexibly. GPUs, FPGAs, we've talked about our engagement with Graphcore.
There's a ton of startups, and across cloud and edge and Intel. These are all programmable engines for AI-based linear algebra, which have different sets of trade-offs in the efficiency of the math units and the front end that feeds them. Just like we had CPUs for many years, this is kind of a new class of programmable architecture that's emerging in a very rapidly evolving space. At the end of the day, GPUs, FPGAs, what you called ASICs, they're all chips that have a lot of math units on them, different architectures to feed them.
Thank you.
What that ultimately will look like, I don't think we know, or at least we don't talk about outside of the company.
We'll go back to that. We'll kind of try to get a different angle on that because that was a big question. My question, just as we think about the industry pace of change. 10x, we're thinking technology changes around Moore's Law every 18 months, doubling transistors. If you think about 10x every year, is there just a limited number of companies that can keep up with that pace of change? How nervous should investors be around things moving so fast that it's really going to be a limited number of companies that can manage that pace of change? Any thoughts, David, as you just think about guardrails that we need to have in the industry, and do we have efficient guardrails there?
Yeah. I think this is counterintuitive, but actually, technology like this, instead of limiting the access to AI to fewer companies, it does the opposite. It opens it to more companies. Let me elaborate on that. We don't expect, as I think we were mentioning before, companies to building these massive models from scratch all the time. What we expect is the concept of AI as a platform, just like what we had in the past with the internet, with then, of course, mobile, and now with the cloud. What is the equivalent of that for AI? If you look at how companies are developing AI today, their concept of a platform is, I'm sorry, Doug, this is going to hurt, but it's limited in the sense that it is very close to having something very specific for my solution.
I have to build on top of the platform to have something customized for my solution, for my scenario. Where we want to go is a place where AI can be accessible like a platform. You can use these AI models, and you customize those models for your particular scenarios. That's the big change that I think we will see coming very soon, right? This applies also, I know we're talking a lot about NLP, natural language processing, but this will apply to every other scenario. Again, coming back to the previous thing with autonomous systems, very similar pattern that we see with autonomous systems. Would you say that creating an autonomous system is very limited to a few companies?
Well, if you look at it from the angle of building all the components from scratch, like deep reinforcement learning, super complex technique that is not available for every enterprise to address. Our approach is very different. It is to provide a platform abstracting on top of those techniques, like reinforcement learning, so companies can address, can embrace autonomous systems by using their expertise on what they have expertise, which is their business. Right? The same comparison, it applies to many other different facets on AI that we are pursuing across the company.
Got it. It sounds like similar to how you are moving from siloed to more centralized models available to all of the different internal departments at Microsoft, you are going to start to expose some of those same models externally on a kind of as-a-service basis. You almost create a new PaaS layer for AI, essentially. Is that the right way to think about?
Exactly.
What is the timing around that? Is that something that is going to be like a 2021, 2022 kind of opportunity, or are you going to start to expose that?
Yeah, no, it is a good question, and I will go back to Doug's positioning on it. This is not like getting to the finish line. We will see value being delivered in this journey, right? The beginning was last week. There is already a ton of value that companies can use today because of that dual approach that we have in the company, which is, I would say, there are three things. When you look at our technologies, I would position them in three different stages. The first thing that we do is be very open about our innovation. For example, we open source these frameworks, right? That is, for example, last week, we open sourced the key framework, the key distributed framework engine is called DeepSpeed to enable this training, right? That is something very core, right?
We go into a phase where we infuse these models into our products. We already do that too. You have a lot of features that are part of our products that customers can get access today that are used in that manner. The third stage that I would highlight is how we also incorporate this into Azure for developers to also build on top of that, right? We also have for that things like Azure Cognitive Services, which at the end are these AI models that are served as a service, right? They are very easy to use by developers. With those three things, we cover the entire spectrum, and it's not that, hey, today we are releasing everything, right?
It's that as we move and we make progress in this journey, things are getting infused into those three main channels.
Fascinating. My last question before we shift gears.
Hey, Brent, I'm sorry, could I add one point onto that?
Yeah.
I think is important for your investors. If you imagine today that I start a new company, and I want to become a public cloud vendor. I want to build another cloud business on my own infrastructure and compete with the big mega clouds. The capital I need to raise, to deploy, to build these giant data centers and have them be available around the world and in many regions, and the number of software features that I need to provision and the number of SaaS and PaaS services that I need to onboard. It's just not achievable for a new entrant into the space. Especially when you look at some of the advantages that some of the big players have, and I refer to our enterprise software business as well.
In the AI space, this, I think, goes directly to your question, you need to be able to build the biggest AI supercomputers that the world has built to train these models. You need to have really deep distributed systems expertise. You need a really strong AI team that understands this. The capital to do that as well and the level of expertise and the depth is also, I think, not quite at the point that you would have to struggle to start it in the public cloud, but it's very large and growing. Just by definition, that's going to limit the number of players who are able to do this. Microsoft's position is that we want to empower all our customers, like David said, and give access to this technology.
The number of organizations that can build that infrastructure from the ground up, I think, is going to be small.
That totally makes sense and kind of leads into my next question around OpenAI. Maybe could you just walk through that OpenAI relationship? You put obviously a bunch of money there, a lot of really great talented engineers there kind of aggregating at OpenAI. Maybe just walk through that Microsoft-OpenAI relationship. What is it, and how does it relate to the AI supercomputer announcement?
I'll say two things, then I think David should weigh in. I want to be judicious about commenting about a close partner. Number one, OpenAI is among the most ambitious organizations in terms of what they're trying to do with AI and their mission. They, like our internal teams, push the infrastructure very hard. Okay. I think given the scope of their ambition, the fact that they chose to enter into this Microsoft partnership, I think is a very strong vote of confidence for our infrastructure and our roadmap that I think you can look at and say, "Wow, one of the most ambitious AI organizations on the planet was willing to sign this deal and commit to our infrastructure." That should tell you something about our roadmap, which we don't publicize.
Very helpful. Anything else, David, you want to weigh in there?
Yeah, I think it's coming back to Doug's points before, right? In the sense that in order to serve these massive models, you need to build just to throw a number in there. The AI supercomputer that went off last week, it became immediately the number five supercomputer in the world, right? That's a sign of the scale that you need there. To build a system like that, and the other comparison that I do all the time is that you should look at this as the Formula One, right? All the innovation that you do in that Formula One, you make it available to the typical cars that we use every day. The same thing we are doing with this investment.
It's not only about creating this AI supercomputer to build these massive models that we believe will be the foundation for that AI platform that we're mentioning, which only will be available for a few vendors who can really have the capital on that. The other thing is how we are taking all of that innovation and infusing it into Azure for everyday Azure, right? That's the other big aspect to consider in this.
That's right. Just to underscore that point, Brent, I'm sorry, I'll shut up in a minute.
No.
Think about what David just said. Fifth largest supercomputer in the world with the models growing 10x per year. Just think about the implications of that for a minute and where this is likely to lead.
Version one, fifth largest supercomputer in the world. That's the trying to keep up with the modeling kind of appetite that's growing 10x a year. Very fascinating.
Yep.
Harsh, you had a question?
No. I'll wait, actually. I had a question on quantum computing, but I'll wait for you to finish.
Let's shift gears to kind of more of an enterprise reality. The enterprise reality is we're going through a pandemic. We have work from home. It feels like this is an environment where you could see things and AI projects start to pause and slow. What are you seeing in this post-pandemic environment relative to AI, the appetite for AI? Is it slowing? Is it accelerating? Just walk us through what you're seeing there.
You want me to take this one, Doug?
Go ahead.
Yeah.
I'll weigh in if I need to.
Please build on top of that. Yeah. I have to say that even before the pandemic, we were seeing a change in the conversation with enterprises. I remember maybe four or five years ago, every conversation that you had with a customer was about, "Hey, what is this AI thing? Should I be on top of that?" Right? The next thing is then finally companies realize, "Okay, this is big. I need to embrace AI if I want to transform my business." The next thing was, how do I get it started? We saw a lot of adoption in the enterprise, but in the first phases of that adoption, right? Many pilots, many proof of concept, et cetera. From that time to now, we've been seeing that shifting in the market to really, how do I have an impact in the business?
How do I move beyond the proof of concept? How do I connect AI with business outcomes? That conversation was already happening. This pandemic, this global crisis, what it has done on this conversation is accelerating it even more. We see now customers just cutting to the chase, telling us, "Hey, what I need now is putting AI into action, and I need it now." We usually have that conversation in two fronts, right? One is for the current crisis or for the response phase. How can I use AI to help me deal with the disruption that I'm experiencing today? Even in the longer term, we know that we're never going back to the previous normal.
It's going to be a new situation with a lot of economic uncertainty, with a lot of care on how do I optimize the revenue, how do I optimize my operation, et cetera, in my company. How AI can help me there. In the first one, let me maybe, because I think your question was more on the first one, right? It's more today, what are companies doing with AI? How is AI helping in the disruption that they're experiencing? We see three primary scenarios. These are the three use cases that we actually saw a raise on the demand from our customers, and we see that across industries. Not for a particular vertical. The first one, which is very obvious, is of course, customer service. That is the most visible one, right?
Companies that were in many cases piloting using AI to streamline their customer service, they are accelerating those projects. We see examples of companies that have been working for 18 months on a project, and then with the current crisis, they put it in production in two days. That's the acceleration that we see in some cases. That goes, in many cases, it's definitely customer support. You have a perfect storm with more requests from your customers, but at the same time, your operations are being impacted by the crisis. In other cases, we're seeing broader scenarios like really being able to identify your best customers, being proactive, personalizing your service to those customers. We see that across a number of customers, and I'm happy later if we have time to go into some examples from real companies. That's one scenario.
The other one that we see hugely happening in this crisis, of course, business process optimization. I like to separate that in two, because we see that the first thing that customers are noticing is that processes that were very established in the past, now they have been disrupted dramatically. Think of supply chain, right? Think of forecasting demand. Fraud detection, customer churn, those processes are dramatically different right now. They need to put in place processes that are more agile, that they are more flexible to those changes, AI is a solution that is helping in that area. That's one thing. The other thing, of course, is specifically because of the economic situation, we see customers looking for cost savings, right? That in business processes, that means a lot.
We see many customers that are using AI to shorten their business process, to streamline business processes, and to make them more productive. That's the second scenario. The third one that we see a lot, because of another perfect storm right now, is employee productivity. That includes, of course, more working, but it goes beyond that. We see another perfect storm with very complex situation that employees have to deal now from the business point of view, but then an impacted productivity because of the current situation. We see companies looking at AI to how they can, in the short term, increase that employee productivity, we see many examples of those. Those are the three.
Doug, I don't know if you want to add something on top of that, we can go deeper into examples if you want to later.
Yeah. I thought that was super interesting. It was even interesting for me, and I'm in the company, so thank you, David. I just would underscore what David said. What we're seeing is that this crisis is accelerating companies' desire to do digital transformation, partially because they need to, and partially because if you're going to optimize a process that you deferred for a long time, you might as well just do it, do it right. Satya has invested enormously in having Microsoft try to be the lead company that will help people solve problems through digital transformation. I think it's an opportunity with our infrastructure and our services to really help our customers.
As you think about the types of AI use cases that seem to be resonating most right now, you gave three examples. Is it NLP? Is it text, speech, computer vision, all of the above? Is there one type of AI that's really like, "Oh my god, it's taking off much faster than I would have thought?
David, go ahead.
I feel all of them, if I have to pick a favorite right now, that's NLP. NLP is right now in a very hot moment because of this acceleration being especially relevant to an area that was very tricky, such as NLP. NLP is very complex. It requires generalization. It is very, very complex to solve. With this new state-of-the-art models, we see huge potential in that. In the examples that I said before, NLP is there, we see all the techniques of AI across the board.
Right.
David, I can vouch that it's been about nine months since I have spoken to a live person without speaking to a machine first.
I think we all experience that. Absolutely. We have about 15 minutes here left. What I think would be helpful is to maybe get an update on Project Brainwave. Obviously, Doug, I think the last time we met, it was about three years ago, and did a kind of a deep dive. Love to understand, and maybe Harsh is probably better here to go down on this, but to Alfred or to Harsh to get maybe an update on Project Brainwave.
Sure. David or Doug, why don't you give us a 30-second or one-minute overview on what you're trying to accomplish with Project Brainwave, and then I can jump into some questions.
Doug, it's all yours.
The project, we're continuing. It's continuing to go super well. We're using it, continue to use it at scale worldwide, to serve the models that we train. Like I said, we definitely have a mix of technologies in our infrastructure.
Yeah.
With that program in particular, we're able to incorporate innovations fairly rapidly, to really push what the models on the serving or inference side are capable of doing. I think we have really focused on our internal businesses, because that's where the need is pressing. If you think about the need to serve these giant models and the latency and the cost, the expense, that's where a lot of innovation is required. I don't want to talk too much more about it. We are going to be saying more publicly this year.
Great.
I believe that'll be a chance to get updated then.
One term that we hear a lot, we obviously see this in Hollywood movies and stuff like that as well, is quantum computing. I was curious, since you guys are some of the top leading experts in the country on AI, I was wondering, is quantum computing even applied to AI in some manner, or is it used for completely different things? If so, how do you guys feel about the potential for that maybe commercializing in the next five, 10, how many ever years, if it's out that long?
David, do you mind if I start with this one?
Yeah. Sure thing.
There is certainly an overlap between the two in the research community looking at doing machine learning with quantum algorithms and quantum-capable algorithms. What I would say right now, my personal view, I'm not speaking for the company here, is that initially the two technologies, AI supercomputers and quantum systems, will be targeting different applications. Like the things you can solve with a lot of the quantum algorithms are just very different problems than what we're doing with these large-scale AI models. Eventually there may be some convergence, but where quantum, I think, will do best initially is in problems that have huge compute requirements, but not a lot of state, simply because of the difficulty of keeping large amounts of state in the superimposed domain. If you look at the amount of state we're doing to be driving and train these models, it's massive.
I think of them as disruptive technologies that initially are attacking different classes of problems, and then theoretically, there is possible for overlap in the longer term.
Doug, when you say state, what exactly do you mean? I'm just wondering.
Information. Bits, bytes.
Okay. Got it.
How many bits of state do you hold in a computer? Is it 1,000? Is it 10,000? It's not the terabytes of data we're operating on in the AI supercomputer space.
It sounds like we're far away is the best way to think about quantum computing is at least a couple of years away, if not more.
I would say, I don't want to take a position on the quantum roadmap. I would say that for large-scale AI, I think we're pretty far away in the quantum space.
Ben, do you have anything?
Yeah. Let's shift gears a little bit maybe and talk about some real-world use cases. We've covered a lot of ground here. You gave us some scenarios how people are using it. You just recently announced [NDA], you announced all sorts of new use cases, I think at Build last week and 2 weeks coming up to Build around some new large enterprises, where AI was Microsoft AI, Azure AI. I think there's like 20,000 enterprises that have deployed Azure AI at some point in the last year. Walk through how are people actually using it. Are there interesting scenarios that you've come across that you'd like to highlight for us to take the technology into real-world scenarios?
This is a question for David.
Yeah, I can take that one, Doug. I would use the frame that I was using before. I would add one component. The frame that I was using before was those three key scenarios. I can work through some canonical examples in there and even bring some of the customers, real customers that are in those three scenarios. I would add an additional one, which is the post-COVID-19 moment. That's the moment where we'll see companies reimagining again in this completely new normal that we'll experience there. I can bring some examples of things in that area as well. Let's start with today. Let's start with customers that are using AI today for those three scenarios. We started with customer service.
Customer service is a very easy one because it's weird to find a customer that is not now using AI for their customer service and in general, their customer engagement. We have many examples. I think the most exciting one that went out last week was actually in the healthcare area. In Microsoft, as you know, not only we provide the platform for customers to create, for example, bots to be used with customer service, but we also provide SaaS solutions that they can use directly for those solutions, for those scenarios. One in particular is called the Azure Health Bot Service. It's a vertical bot that health organizations can use today for their customer service, in this case, their patient service. What we have seen in the last month is an exponential usage of that service.
Just to give you a sense, we have seen 1,500 health organizations between public health organizations or health providers, et cetera, that are implementing new projects. Think of that, 1,500 new projects that were put in production specifically for COVID-19 management. Those bots that are being deployed in just these few weeks, they have a reach of 30 million people. Think of the scale of that. They provided a huge tool to really unload, to reduce some of the load that we were seeing in health professionals, like doctors and nurses, to have a first-level interaction with patients, because it was a self-assessment tool that patients can take in order to be redirected then to the right resource. That would be one example that I would bring.
In business process optimization, we have seen, I don't know where to start there, because, again, every company is doing a lot in that area. One, I think you mentioned that one, but one of the ones that went out last week was FedEx. FedEx is the perfect example of business process optimization. They had this massive amount of data. They have a very granular data on their shipments in FedEx. With this agreement, they are going to add AI on top of that so they can have better intelligence on what is going on, so they can not only identify trends that are happening or problems that are happening, but also to optimize that supply chain in their organization. Really a perfect example of business process optimization. It was really everywhere. I can think of [IndiaLends] is another customer.
[IndiaLends] is underwriting a credit platform in India for 50 banks in India. What they did in their case is using AI for their credit approval system, they were able to decrease the internal processing time for credit approval 50%, which in these times, imagine the impact of that, is having the ability to process twice the number of credits in a moment like this, which is absolutely critical. Manufacturing, we have seen also many examples. I think I mentioned this one before, where we see companies starting to use autonomous systems, not for the moonshot of autonomous driving, but things like motion control or process optimization in manufacturing that are getting big results from day zero. We see many customers in that front. What else can I bring in there?
Well, you tell me, business process optimization, actually, we saw it also in healthcare, where we have such a limited resourcing as we have right now, it is critical to things like medical supplies and even hospital beds are optimized with things like AI. Very typical. The other one that I mentioned was employee productivity. Let me bring just a couple of examples in there. One generic, we see that a lot, is AI working with humans to augment their productivity. Just to give you an example on that, Reuters. In the case of Reuters, you have, again, you have their journalists that are doing what they do, which is writing articles. In this case, they were using AI supporting them so they can attach relevant videos to those articles.
Not only they didn't have to go through that very manual process of finding relevant videos, but by using AI, they also increased the average completion rate 80% with this technique. Not only better employee productivity, but then better results. The other one that I would mention that is very connected to that is not only don't think of employee productivity only removing tedious tasks or repetitive tasks. It's also about making better decisions. The other example that I will bring in here is Team Rubicon. They are managing the more than 100,000 volunteers in the U.S. for COVID-19. Think about the massive scale of the solution required for that. In their case, they are using AI to really identify and optimize the deployment of those volunteers across the U.S. Very important work that is really about making better decisions.
Those were just a few examples on the first part, how companies are using AI in the response phase. Let me just use a couple of examples on the reimagine phase. I think that is the big conversation that we should have, what is going on after this health crisis is over and it turns into an economic crisis. We see three key things happening in there. Let me use some examples in there. The first one that we talk a lot today was AI at scale. I know that we talk a lot about AI at scale in the concept of the model, but there's a bigger motion that will happen in the enterprise that is really applying AI at scale in their business. It's moving from that pilot phase to really infuse it into everything that they do.
For that, the key thing in there, and I will bring an example right now, is how you need to scale the usage of AI across your business units and move it beyond your technical units. We see the business being more connected to the AI transformation than it was before. That is something that will happen certainly as we move into this phase. An example that I would bring in here, because I think the key lesson here is that we have gone through that before with software development. Remember this conversation 10 years ago, we were having it for software. We knew how to do that. It was called DevOps, and it was all about bringing developers, technical units, and the business together in a combined life cycle. We don't have that for AI.
You have technical units that are working in silos, in these pilots, but they are not being infused into the business. The equivalent for DevOps in AI is called MLOps, and we see that as a huge trend moving forward. An example I will bring in here that we just published is TransLink. In their case, think about the scale. We talk a lot about the size of the models, but what about the number of models that you need to transform an industry, to transform your company? In the case of this company, they have 18,000 models. Think of the number of models. Think of managing those models. There's no way that you can do that with just a silo technical unit doing that.
They brought together the business and the technical units with a common MLOps process that is across all of that. That would be one. I know that I'm talking a lot here, let me just go to the other two because I think it's important. The second one is really going.
You got one-minute drill, then we have to close
I will do it in half a minute.
Yeah.
The second one that I want to mention is how the next step in this transformation is to empower the business. We have talked a lot about technical units developing this AI, but the next step, and we made very big steps in Microsoft, is to empower the business so they can also apply AI. With things like the Power Platform, with Dynamics 365, we see that as a reality moving forward. The one example that I will bring here is Novartis. Novartis, a pharmaceutical, as you know. A year ago, we signed an agreement with them where they are basically empowering their 50,000 employees, so almost half of their employees, with AI. Don't think of it as using AI, but more as applying AI with flexibility, with freedom to their processes.
You can have researchers that are researching new drugs or new treatments or new vaccines, and how they can be augmented with AI by using in their processes to drug manufacturers and any subject matter expert. That's a huge motion that we will see moving forward. The third one, and this could be my last one is let's not forget about responsible AI. We see responsible AI, we had this conversation for many years already, and we are seeing also a huge shift in the conversation from, "Hey, what are the challenges of AI? What are your principles, Microsoft?" To now, "Hey, I'm under a huge risk as I accelerate my adoption of AI.
Help me to implement that AI responsibly so I can mitigate those risks." We have many examples in here, but just to bring one, TD Bank, the bank in the U.S. from Canada. You can see how they're using responsible AI to mitigate things like bias or things like adding more transparency to the models that they are deploying.
Thank you, David.
I will stop. Sorry for all the time.
You guys have so much to talk about AI. Jonathan's extended this conversation for another hour. I'm just kidding. Last question from me. Doug, as you think about the pace of change here, 10 times in the last year on the modeling side, what are you most excited about, thinking about the AI industry and Microsoft's opportunity in the next three years? One thing. What's the one thing you're most excited about?
Are you thinking, Doug?
Go ahead, David. I need to give that some thought.
I keep going back to the concept of really putting AI into action. The concept of democratizing AI, I know that it's not so technical, but there's a lot going on to empower that motion that we're doing, because it goes beyond just the infrastructure of the software.
Yeah.
It's really about our comprehensive solution, from research to the three clouds to our tools. Really powerful if we get there in the next year.
Driving those business outcomes, for sure, for you.
Yep. I'm going to give a little bit of a squishy answer, but it's a true answer. You very rarely get the opportunity, and this is a personal view, to work on things that will really change the world in a positive way. I think the capabilities we're going to be able to generate, just from my perspective with the hardware and the systems work, can help really solve global problems. That is a once-in-a-lifetime opportunity to build something that allows us to really make meaningful shifts on personalized medicine, climate change, efficiency, security. It's coming back to this concept of responsibility. For me, I just really feel driven by a mission and a purpose, to really move the needle and make the world a better place. It's a blessing.
Boy, Doug, that was so nice. You should have said at the beginning so I change my answer.
Listen, we're out of time. Really appreciate, David and Doug, sharing your views on the current state of AI here. As always, it's a pleasure, thank you so much for spending the time.
Thank you so much, guys.
Thank you, Jonathan.
Thank you.
Nice to chat with you all.
Yeah.
David, great job. I should tag team with you more often.
It was a pleasure, as always. Bye.
Carol, bye-bye.