Done, and it's amazing.
It's funny to think if big elephants can dance.
Yeah. Maybe you get everyone on the same page. Talk a little bit about yourself and your role in Microsoft.
Yeah. I lead the Experiences and Devices in Microsoft, so that, that would have Microsoft 365, Office and Teams, Windows and Surface, and then also our search and our browser efforts.
So if you think about the last 12 months, it's been like a crazy journey in terms of new innovation, et cetera, like that. Can you talk a little bit, like, how did you experience it? You know, like, how was the last 12 months for you guys, for you and running it as well?
Yeah, I mean, first of all, let me just say, the last 12 months have been as exciting a time as any.
Yeah.
I've been at Microsoft a long time. I came when it was user interfaces and, you know, we thought that was a big deal, and then mobile. This last 12 months, I actually feel generative AI is as big a technology shift, if not bigger than any or all of them put together. And so for us, in the last 12 months, we got a lot done. You know, like, GPT-4 has now been out for production for about a year. So we, you know, again, we add APIs in Azure for that, with all the safety and trust that our enterprise customers expect. In February, we took the large model, we put that in our search and browser.
Mm.
We did think, you know, this represented a new way for people to find information, to reason over stuff. In March, we took a step back and we thought about what about productivity? How would these generative AI, the power of these, affect or impact the way work is done? So in June, we brought a private preview of that for a lot of our enterprise customers, and it's been great to be with them on this journey. November, we launched the Microsoft 365 Copilot. But I—you know, when I reflect back, of course, GitHub Copilot was the first one that we went after because, it was very clear, that we could help the developers be incredibly productive. So we had that.
In the last few months, we've also taken a look at how business, business processes and business workflows can be rethought in the world of generative AI, where you take systems of engagement, and you take natural language and natural language interface and how you bring, whether it be a Salesforce's workflows, a service provider's workflows, whether it be a doctor's workflows, we build a Copilot for those things.
Mm.
We thought about how to extend the Copilot, you know, to allow customers and developers to extend the Copilots with specific information. So a lot happened, and the thing with this is, a lot of this, technology shift also intersected work that we've been doing anyway. On Microsoft 365, post-COVID, we saw a lot of customer engagement intensity, and that populated what we call the Microsoft Graph, which represents the customer's most important database, I think, which is the projects, who knows what in terms of knowledge, be calendar, email, document, the right permissioning. So this is where the generative AI reasoning engine meets the Microsoft Graph to provide generative AI in the flow of work.
Mm.
That's still very early. Very, very exciting to see the entire company rally to this.
And you guys must be thinking a little bit more ahead. Like, I just wanted to ask, like, slightly more bigger picture. Like, you think now, generative AI, what it can do, and you have more insight than us. Like, how you think the way we work will change over time, like, going forward? Is there, like, a glimpse that you can give us here?
Yeah, I mean, look, I'm not a soothsayer on this stuff, but let me just share what we are already seeing with them.
Yeah.
When we did the early adopter program in June with the Microsoft 365 Copilot, we had many hundreds of customers jump on the opportunity.
Yeah
Because they see the same thing. And hundreds to hundreds of those, we went really deep with. They invited Microsoft to go deep with them and analyze the impact on productivity. So you know, we looked at three dimensions of productivity. One is efficiency, just how quickly can you get the job done or how much more can you do, go do? But we balanced that by taking a look at both quality of work. It's not just enough if AI assists you in doing something quickly. I mean, what is the quality of the output in terms of accuracy, in terms of up-leveling the skill level? And the third thing we looked at was effort. How much effort, how much grind did we take out?
And the results have been, you know, really impressive, and we are going to publish all of this in the Work Trend Index in the coming days.
Mm.
But about 68 or 72 people felt it made them more efficient. It improved the quality of work, that it felt they had to put less effort in, so they had more time to do what is innately human, which is more creative, more reflective work, more collaboration. 70% of them would say, "Of course, the Copilot makes me more... The generative AI makes me more productive." What's also telling is that, you know, more than 55% of people feel that AI makes them more creative.
Mm.
This is today, here and now. Now, you ask the question: How do I see this move forward?... I think workflows are going to have to be reimagined.
Mm-hmm.
So today, what it does is the Copilot or AI helps you in your existing workflows. But let me just take a step back. In the Copilot, our vision of the Copilot is not an autopilot. It works on your agency, it works with your permissions, it works with your context. So today, it's helping you in your existing workflows, but because it's helping everyone, how can entire groups' workflows change and be reimagined? And we are starting to see that both in my team and with customers.
Yeah.
Just one. I want to leave you with one thought. You know, so far, AI, prior to generative AI, it was as if the human beings did all the work, and the AI was the editor. You know, it would show up in autocomplete, grammar check, spell check. But now, AI is doing the work. They're doing the first draft, they're doing the summarization, and the human beings are now editors. So we've gone from AI being editors to now where it should be, which is the human beings are the editors, and the AI is the assistant.
Rajesh, do you have any, like, on that reimagining, workflows, et cetera, do you have, like, examples to make it slightly more tangible for us? Like, I hear you, but I don't hear you. You know what I mean?
Yeah, yeah, yeah. No, it's... You know, so let me give you just a couple of examples. Maybe one, couple of customer examples, maybe one from my team. So one of the customers, what they did was... So when people start using Copilot, the first thing they try to figure out: Hey, how should we go do this? They ask us, "Should I distribute a few seats to this across different departments, or should we go all in in one department?" Then, based on our work with them, we tell them, "Go into one function," because we find taking a peer group and giving them all Copilot helps reimagine the workflow.
Yeah.
So this customer, what they did was they gave their service department. They're an operational company, and so they had very complicated processes for a long time because hundreds of people, incidents would be day-long. This was the core of what the company does, and then they had very complicated overhead of how, you know, when people work four-hour shift in the incident, and the next group came in, who was going to transfer the knowledge? How were they going to come up to speed, and so on. So they reimagined this, reimagined the entire workflow in the context of Copilot in the meeting. So when a new person would come in, the Copilot would just summarize. They could just read and, "Hey, can you summarize what happened in the last episode? Which ones apply to this specific function?
Mm.
So they were able to get rid of a lot of the manual overhead, the human overhead of transferring knowledge. And that was an example of just reimagining the workflow. There'll be one other drug research company where there's a corpus of a lot of data around this clinical trials and drug research, and but that's in a different system. And so, but what they did was they extended the Copilot. So in the Copilot, people could summarize quickly when they came in on what were the latest findings from the last week from their colleagues-
Yeah
In the context of all the emails that have come in around that, all the chats and documents, plus what's in the system. In my own team, this super interesting thing, you know, if a product manager wants to go incubate an idea, the process used to be he or she would come up with an idea in the product management team. She'd go lobby for some resources in the engineering team and design team so they could go, go incubate. But now she just uses the Copilot. So this, product, early in career, she had this incubation idea. She just used the Copilot to generate. She doesn't know coding.
Mm.
She's not a designer. She had an idea. She used a Copilot to create a perfectly workable mobile app to incubate a bunch of stuff.
Wow!
Got to try a lot of different things, and then once the idea was just finished, she brought it up to the core teams to actually implement it. And so this has saved, in that specific scenario, months of just time coordinating people. So I think what you will see is workflows are going to have to reimagine, in addition to augmenting the human ingenuity and creativity and taking the ground.
Yeah. And then, maybe I should have started actually with that. Speaking about the Copilot and the productivity gains. And, you know, you started on GitHub, on the Copilot side.
Yeah.
Now we have, like, you know, personal side using, using it on the O ffice side. What do you, in terms of productivity gains and interrupting GitHub on that one?
So GitHub, you know, it's the first thing, the GitHub Copilot in many ways have blazed the trail for us-
Yeah
In the design language. So we spend a lot of time on how does, how does the AI get instantiated with teams and user organizations? I mean, it's no secret that the AI, AI can get something wrong. Now, of course, when you ground it in your information through a RAG pattern or what have you, then you're using it as a reasoning and artifacting. But what the GitHub Copilot taught us was the design language of it being a copilot, not an autopilot.
Mm-hmm.
It's working on your behalf, making sure that humans were in the loop to actually test the changes or to get the action.
Yeah.
And then when we saw the GitHub Copilot, you know, end up to 50-55% efficiency for developers, the other thing that we did was we created the Copilot stack. What I mean by the Copilot stack is, if you take a look at Microsoft 365 Copilot, it's not just the largest model, the most capable model, it's the fact that that model is grounded with the Microsoft Graph, with all your permissions, and it's then brought into a Copilot design language in the experiences that we use.
Yeah.
And then they need to be extensible. This entire thing is what we call the Copilot stack. All the Copilots at Microsoft have been built with the same orchestration, same extensibility, same design language. And we've made this thing available as an Azure service, whether it be Azure AI Studio or for low-code, no-code, the Copilot Studio. I think in many ways, the GitHub Copilot has blazed the trail for us into this modern graph language.
And then what are you seeing in terms of the customers that's been working with the Copilot and adoption curves here? And telling you where I'm coming from, it's like, look, if I—the productivity gains we're hearing is somewhere like 20, 30, like 50%. And the people that are working with it are kind of more developers, or like workers, they're expensive. So here you think, like, hang on, like, everyone should be rushing out, getting this. Like, what are things that on their adoption that we can consider there and, like, did you have experience so far?
Yeah. So great question. So it's, it's clear that for the end user, whether it's in the context of creation, summarization, first draft, all of this, the payoff. I mean, this is what we... I was just talking about, and what you're gonna see from the Work Trend Index. Benefits of 50%, 60%, 70%. You know, in terms of meeting summarization and follow-up tasks, about 80%-85%. So it's clear value from the end user. For the IT admin and enterprise. They're also responsible for security, regulations, compliance, and so on. One of the things that we've done with the Microsoft Copilot is actually built it to be enterprise grade. What I mean by that, it understands things like conditional access.
If IT had a vault in place, which is if Raimo's outside this zip code tries to access corporate data, but you're not allowed to ask under challenge and authentication, the Copilot policies.
Mm.
Things like all interactions with the Copilot are discoverable for regulatory reviews, or it generates an audit log. So we've done that work, but I go to the validation for these things.
Yeah.
So one of the pieces of feedback we got from IT early was: Hey, you know, we love the notion of the Copilot helping the meetings, but we don't wanna create a data retention policy where transcripts have to be retained to any Copilot. We did the work, so you could now get the Copilot to perform in a meeting without creating a transcript that lasts beyond the meeting. So there is a bunch of IT evaluation and adoption, but we have an adoption guide with Microsoft, adoption. microsoft.com for IT for the, for those reasons. The other one that we hear from customers is: How do I know the ROI?
Yeah.
So we built a Copilot dashboard, so they can literally go take a look at the impact of the Copilot. They can join that with their custom data. So you can build your account, you can bring your sales performance custom data, join that with the Copilot data, and see what the ROI is. So we wanna know that. And then for the end users, we are having to really teach them a new way to be computing, grown-up keyboard and mouse, swipe, gestures, and so on. But now we're using prompts. What is a good prompt in the sales process? What's a good prompt in this organization in the manufacturing context? So we have that now in the Copilot Labs.
In the end of the day, the Copilot is gonna go through a bit of the adoption cycle of any enterprise, because you got to work with risks, you got to work with compliance and security, and, you know-
Yeah
Make sure there is ROI. But the time to value for the Copilot, I feel, unlike anything that I've done in the M365 suite in the last 10 years, is much, much, much better.
Yeah.
Because the Microsoft Graph really exists. It's built to be enterprise-grade, and it shows up in the flow.
With that, like, and you mentioned the Microsoft Graph alone, and actually, I think on our side, we pay not enough attention to that. Was that before and you just got lucky? Or like, how did I say...
It's super, both, I would say.
Yeah.
So here's what happened. You know, if you go back to the Office business, the Office business, when we were on premises, was a bunch of pilots.
Yeah.
We didn't understand the user well. We understood the jobs. With Exchange and Outlook, we understood the mail and calendaring job. With SharePoint and Office, we understood what document management was. With Lync, at that time, we understood, you know, meetings and telephone, but they were all stovepipes .
Mm.
When we moved to the cloud, we really got to be user-centered. That's what the Microsoft Graph is.
Yeah.
We then and the reason we were stovepiped on-premises was no customer could buy all our products at the same time. So every silo had to be standalone and be able to exist, even if the other product silos were different versions. So in the cloud, now we had one data structure. Customers, we were doing the job of keeping it up to date. That's what allowed the graph to get created. And what COVID did, what it accelerated, the use of Microsoft 365, a lot of intensity.
Mm.
The more you use Microsoft 365, the better your graph is.
Mm.
Now, after COVID, what we have now is tons of customers deeply immersed in Microsoft 365. The graph is healthy, it is permissioned, and now you get generative AI. You take the graph, you ground the generative AI into the graph context-
Yeah
And you say, you know, bring it to the user experiences that people are using today, and that's what the Microsoft Copilot is, and that's the differentiation of the Microsoft Copilot.
Yeah. And then I wanted to shift gear a little bit, like-
Yeah
What has been on the way how you do compute, like first fundamentally, but then also how do you do it in Microsoft at the moment? Because you have so much demand on your side, on the Copilot, et cetera, then you have all the external demand on Azure. So like, who's getting the GPU?
Yeah, I mean, first of all, I mean, look, this GPT-4 was built on Azure, and we've been optimizing all layers of the stack on over a year. That being said, to answer your question, you should think of it a couple of ways. You know, of course, the customer, when you come to Azure, you can use not just the particular models-
Mm.
from OpenAI, but you can use models from Hugging Face, from Meta. You know, we order, we now have Models as a Service that we've announced, where after you can use Charles premium model, here's model, Jais model. You can use lots of models. For the large model that we use in our first party applications in Azure OpenAI API, we've done the work to leverage one implementation and one stack. It's not that, you know-- I can optimize the usage of the GPUs across Windows, Office, Dynamics, and the Azure OpenAI because we all use API. If third party is using our APIs, you can use, can bring their own models.
You can use a platform model, but if you use the large model, we are getting the leverage of the fact that we are all using the same API. And then, of course, underneath the model, we are working clearly in close partnership with NVIDIA, but also AMD, and we have our own AI-first solution picture. So you should think about below the waterline or the systems level work, we've been optimizing for a long time, for a year. We have silicon level work.
Mm.
And then above the waterline, when you are getting the large model, we get to optimize the traffic across all our first parties and anybody using the Azure OpenAI API.
Hmm. Okay. Okay, makes sense. And then, I have, and this relation is getting a little bit nervous now, but I try to ask, like, a number—not a numbers question, but like more a monetary question. If you think about, you're creating with AI a lot of value-
Yeah
For the client. If you look at your pricing, it's still like, you know, if you look at the $30 per user, it's still like a very classic model of like, here is it price. I think it was like the GitHub Copilot, was like $5. I tell you, I talk to customers and they're like, "Just, you know, because 20% productivity gain, and I'm paying $9 for a guy that costs me $150, and I get 20% more of him. Like, give it to me," you know? How do we think about that dynamic going forward? Like, do you have to rethink or reimagine that?
Well, I mean, let me just say, early days, yeah. The thing that we believe is, take a look at Office. Before Office existed, Word was a business application. Excel was a business application.
Mm.
What we did was we said, "Now, you know, we want to democratize this for all the users." So Microsoft Copilot, GitHub Copilot, is about democratization of AI for all the users. Basically, same thing we want to take the core capabilities of what used to be productivity, now AI-enhanced productivity, I want to take that to all the users.
Yeah.
Just like we've done with Office. In addition to the per user, we have lots of other constructs that we have on top, whether it be E5 construct, whether it be, you know, Teams Premium construct. So right now, really, the way we are thinking about AI is let's bring it to every information worker, every first line worker. We get to enhance the productivity, and then there will be lots of opportunities for us to add value and capture value.
Yeah. Yeah, yeah. And since you're kind of working on the experience side from Microsoft-
Yeah.
How will I need to reimagine my UI going forward? There's a classic way, you know, like, a good generated AI system should be interactive. Like, what's the work that you're doing here?
Super good question. You know, for 35 years, computing hasn't changed in terms of the way, you know, you're on a phone, you swipe on a, you know, on a PC, or you use the code and mouse, and the operating system does a very basic job of abstracting away the hardware and allowing you to get an application.
Yeah.
That's gonna change. And so the way we—you know, you'll see us integrate the totality, for example, in Windows, and we think that the ones between the shell, search, browser, file, is going to blur. Because, you know, you enable something that understands your intent at a higher level with natural language.
Mm.
It can take a lot of the grunge out for you. The application relationship to the operating system, to the Copilot, is gonna evolve rapidly, and you'll start to see us do this stuff by bringing the Copilot into it. It's gonna be, you know, it's gonna be very exciting, but I do think it's a new paradigm for user experiences.
Yeah. And that kind of gets me to the other questions around devices. Like, you guys were always very good of showing your partners what's possible. Like, and actually, you know, I remember when the Surface came out, I bought it, I loved it, and it was like: Oh, this is cool. And then you kind of, in a way, almost forced innovation on your partner. Like, this in a way, like you need to kind of almost reimagining devices, et cetera. Where, I mean, not that you're announcing anything with me, you know, but like, where are you in that thinking?
Well, I do think this natural use, you know, this natural user interface, these are multimodal, by the way. This isn't just natural language, but speech-
Yeah
Video and all of that stuff. I do think our existing hardware form factors are going to evolve to account with this.
Yeah.
You know, and there will be new form factors.
Yeah. Even like that?
Just do it like that.
I tried. Yeah, okay. The other thing is like, so switching, shifting gear a little bit, back to the kind of adoption that you see, like, what's your thinking in terms of the speed of adoption you can support with a given shortage of GPUs? Like, how do you see that playing out? Do you give, like, some to every client, like a limited amount to every client? Is it like one client gets a lot, so you learn a lot more? How do you think that will play out in the field?
We are open for business now.
Mm.
With Copilot. You know, it's really like the earlier conversation I was having with, you know, where the customer is going through their evaluation of the security policies, governance, procurement, like, how to start with Copilot. How do you use it? We have to figure out, you know, how to-
Yeah, yeah
... you know, optimize capacity.
Oh, okay. Yeah, yeah. So there wouldn't be like, so if Copilot says, like, we are, I think we're a big Teams shop for you guys, if we say we want Copilots for everyone, it's not going to be like, "No." You could in theory, yeah.
I'm happy to sign you up right after.
Yeah, it's not me.
I'll talk to your IT-
Yeah, yeah
Tell them how we are enterprise ready. We are.
Yeah, yeah, yeah, yeah. I mean, and the last question for me is like: how do I think about you guys working with partners? Because, you know, one is like a very Microsoft-centric group, but on the other hand, like, you know, it's not just you, there's like a whole broader world out there. Like, how do you see that evolving for you guys in terms of working with other people, bringing other people into the equation as well?
Super good question. I mean, and again, back in the days that Bill used to run the company, and yeah, I remember being in meetings with him, where he would take incredible pride that for every product that Microsoft would create, we would quickly create a system. So we are really a platform company, and not just at the Azure layer, but even at the M365 layer. And, you know, already with Teams, we have more than 2,000 ISV and countless other line of business applications that extend Teams. And you will see the same platform sensibility I talked about during Build, and, and in this coming year, Build, we'll talk about how developers or line-of-business developers inside of organizations, as well as ISVs, can augment the Copilot-
Right
with their custom logic, their interactivity, because not all productivity is still human in all its various forms. We're not gonna build all the most interesting applications. And so we want the Copilot, Microsoft 365 Copilot is a platform, and that is we are working. I mean, I don't know if you had a chance to take a look, Raimo, at Ignite-
Yeah
Three weeks ago. We showed lots and lots of customers and partners actually in the Microsoft Copilot.
Yeah. I think, yeah, and then, yeah, that's—I mean, the conference was really exciting. Like, it's really nice to see all the innovation coming out of there.
Great.
So just, hey, it's 58 seconds, and I'm German, so I have to be on time. So, hey, thanks for, thanks for joining.
Thank you very much.
Thank you. I really enjoyed our conversation.
Thank you very much.
Thank you.
Thank you, everyone.