Good morning, everyone, and welcome to Moderna's Second Digital Investor event. Since our first digital event in early 2020, we have made great progress in our digital strategy and the role of AI in accelerating innovation, scale, and value creation at Moderna. Today, you will hear from management and members of Moderna's digital team on the company's AI vision, impact, various use cases, and our efforts in transformation. Before we begin, please note that this conference call and presentation will include forward-looking statements made pursuant to the safe harbor provisions of the Private Securities Litigation Reform Act of 1995. Please see slide two of the accompanying presentation and our SEC filings for important risk factors that could cause our actual performance and results to differ materially from those expressed or implied in these forward-looking statements.
With that, let me now turn the call over to Stéphane for an introduction to the presentations.
Thank you, Lavina. Hello, everybody. Thank you so much for joining us at Moderna's second Digital Investor Day. Before I talk about digital and actually hand over to the team, I need to reframe why this is so important to us and has been so important to us since the very start of the company. As you all know, we built Moderna on the promise that we could make medicines out of mRNA, and the future of mRNA that excited us since the beginning, and even more today, is that mRNA is an information molecule. And because of that, we believe the promise of mRNA was very, very large. We believe we could do medicines that could not be done using other technology, like secreted protein, transmembrane protein, and also intracellular protein.
We believe that we could have a higher probability of technical success with this technology because we give the same chemical to the human body, unlike the analog medicines from the pharmaceutical industry. We also believe that if we invested in IT and robotics, we can accelerate research and development timelines. We believed, because mRNA using the same manufacturing process for all the products, that we could have greater capital efficiency over time, versus recombinant technology, where you need a manufacturing process for every product. With that frame, you understand that since we started the company, our imperative had been to invest in digital and robotics to enable learning fast. We obsessed since the beginning, and as you know, it's one of our mindsets, how do we learn fast so we build the best mRNA company in the world?
The other piece that we just discussed, because mRNA will lead to a platform, we ask ourselves: How do we scale this platform? How do we scale the science of it? How do we scale the clinical trial? How do we scale manufacturing? Because it made no scientific sense this would be a one-drug company. You will either be zero, because we fail in that scientific and business endeavor, or you'll be a lot. And so, of course, we built for a lot, and we built for scale. So this is why it's important to understand that since we started, we have obsessed about how do we be the digital company, how do we use robotics to enable scale and speed. So let's talk a minute about AI now.
We really believe, I personally believe, that AI is a very important technological revolution that's happening to our life at home and to work. And if you think about what's happened in the last 40 years, first, the personal computers transformed our life at home and transformed how we work. Before, it used to be paper and fax machines. And then in 1990s, the internet started like a toy. That's what some people thought, but it became a way of working and living. All of us use the internet many, many times a day, and actually, we don't even think about it or realize it now. And then in the 2000s, we moved from flip phones, where we're able to just exchange discussions, to having basically a computer on...
In our hands, actually combining the two previous technological revolution, having computers as a personal device and having the internet in your pocket. We know what that's done to change our life and to change also our work. Well, today, we believe at Moderna that AI is as a profound technological revolution as the three before, and we have now for several years, we embrace this new technology to transform how we work at Moderna across everything we do, including how I work as a CEO of a company. Moderna, of course, was very well positioned for scaling up using AI, because, as we discussed, we are a platform company. We've been thinking since the beginning about how to run, how to build a platform company.
Because we've been digital since day one, we're a very data-centric company, which is fundamental, as you all know, to be able to do machine learning at scale. We have an integrated AI ecosystem, as the team will present to you. Since the beginning, because we are lucky enough to start a company, recently, you know, around 2010, we have been cloud native since day one. We never had server farms like a lot of bigger companies have had in the past and had to migrate. The company, since the beginning, has been cloud native. So, as I said, I know a lot of people in the world have been excited about the last year or so because of OpenAI and ChatGPT. Well, we started to do AI way before I even heard the concept of ChatGPT.
Actually, in 2014, the company was only a three-year-old startup. As many of you that were investors already at the time remember, this was a company of around 100 people. We actually started to use machine learning for mRNA design algorithm, as we built the drug design studio to do medicines design in silico. In 2016, the team built the ML platform called Compute internally, which is a multi-purpose algorithm platform that we use in research. And then we're all excited about INT, our new approach to individualized neoantigen therapy.
Well, some of you might not know that in 2016, when we designed the clinical trial setup and all the technology how to make the mRNA, how to design the mRNA drug that is unique to every human being, we actually built an AI algorithm to do the INT drug design, and it's being used for every patient who receives our INT product. In 2021, a few years before OpenAI started, it was very clear to me that one of the biggest challenge we had as a company to become a truly AI native, where every employee, and I mean every employee, will use all day long AI, was a change management problem. Because a lot of people in the company had five, 10, 15, 20 years of work experience and never really used AI.
Machine learning was really used mostly by the digital team, serving the scientific and the manufacturing team. So we embarked on building an AI Academy, that we've talked about before, with a goal to train every employee to understand the key features, the key principles of AI, so that they could be a good user because they understood the technology. Of course, ChatGPT came online late last year, and GPT-4 this year. And what we did, and Brad will talk to you about it again, and we mentioned it in our Q3, Q2, sorry, earnings call. We launched mChat, which is Moderna's own version of a large language model systems, because we wanted to be able to use extraordinary confidential information to teach the system, and we didn't want to do that on a public system, as you obviously understand.
So mChat is basically our own version of ChatGPT that many of us use many times per day. And this is the pipeline that we talked about at R&D Day in September. We are very excited that we believe we can launch up to 15 new products in the next five years, with several of them in the next two years. As you know, RSV is under regulatory reviews around the world, and we also have, you know, flu respiratory phase III. We have the 1083 flu COVID combo that is also moving into phase III as we speak. So very exciting time to deliver sales of new products in 2024 and 2025. So with this, I'm going to turn over to my partner in crime, Brad Miller, who is our Chief Information Officer.
As you can see, Brad has spent a lot of time in tech companies and then went more into the financial services sector also to use his insight about tech. He has really grown in his career in tech companies, and I want to say leading tech companies, and he really thinks, tech first. So with this, let me turn over to Brad.
Thank you for the introduction, Stéphane. Good morning, good afternoon, and good evening. I'm really excited to share with you all today the innovative work that's been going on at Moderna, specific to our advancements made in AI, that have been built upon a foundation of machine learning and research and development that we've been up to for the past 10 years. As you've heard, AI is not new to us. We're an information-based company. We use information to build great medicines that deliver information in the form of instructions to the body to do what it knows best, to protect itself.
Given our mission is to deliver the greatest impact on humanity through mRNA medicines, we're maniacal about ensuring that every second spent by our employee base is optimized on value creation, and therefore, how we work and the tools we enable are critically important to realize our mission.
It's no secret that as you add more humans to an organization, you increase complexity and viscosity, and therefore, the value derived flattens over time. Looking at other companies that have embraced the digital transformation, specifically the utilization of AI to transform their workforce, there's a dramatic increase in value per employee derived, and we're on that same mission. The impact, for Moderna has been very profound and very specific. We're increasing our speed to market, from drug development to commercialization. Our effectiveness in terms of being right, increasing the quality of our deliverables, being better, and continuously improving, has increased significantly for us. And our efficiency, our ability to deliver faster and quicker to market medicines that matter most to our customers, has been critical impact of what we do.
The intangible is that our workforce is on a journey to work on what's most important to them and our patients, which is delivering more value, increasing their enjoyment and happiness of their work. So our AI vision is to enable the overall company vision. We're not doing AI because it's cool, because it's trendy, or we want to play with it. We're investing in AI because of the profound impact it has had and continues to have on the lives of our patients. Therefore, we believe that we must become a real-time AI company, embedding AI in every aspect of our company. The success of our AI platform today stands on the shoulders of the giants that have built this platform over the past 10 years, starting with ML for R&D. But we haven't done this all alone.
Thanks to the great partnerships, as you see here below, we have been fortunate to ride the wave of technology innovation. Given that we are built in the cloud on a modern tech stack, without legacy holding us back, we're able to natively adopt new technologies quickly. Everything from cloud storage, Compute, quantum, large language models, Copilots, open source, GenAI projects, we can pivot fearlessly and pursue multiple options in parallel. But as I mentioned in my introduction, we're a scientific, information-based company. Data is at the core of what we do, and since our inception, we've been collecting data across all of our activities, as well as third-party data sources, to create our own proprietary data ecosystem. Given the structured and unstructured nature of iterating on scientific advancements, we've created capabilities to create structure out of all data sources.
It's upon this data ecosystem and data platform that our AI platforms function with consistency and precision. Over the past five months, with the advancements in GenAI, we have seen convergence of platform capabilities that we've already built for our ML platforms traditionally, and applied them to the foundation of our GenAI platform. This has provided us with great leverage to rapidly build out new applications that enable direct value to our internal teams across the company, beyond just R&D, and positively impacting our patients. One great example of very quick innovation was the build of our very own internal version of ChatGPT application, called mChat. By leveraging an open-source project, within two weeks, we built and delivered a beta version of mChat to a subset of our company for testing.
The beta provided us with tons of learnings and feature set requests from our business partners. Within a month, we launched our mChat V1 company-wide, and since then, we've been continuously adding features, with releases every two weeks. By having a product in production, we were able to learn as a company, embrace our learning mindset, and watch our employee base leverage this new technology to deliver results we didn't even expect. We didn't run from GenAI, we ran towards it. Given ongoing innovation that we're all witnessing in the GenAI field, and given our platform mindset to build and in a distributed way, we have set ourselves up to be able to, again, leverage advancements that come from the industry and build upon our platform, continuously improving as we go.
But what I'm most impressed with is how fast our company adopted the technology.
We have invested heavily into the transformation of our people through a broad reach of mechanisms to transform how our workforce thinks about their work and adopts new technology. We started with exposing the technology to our senior leadership group, teaching them how to have conversations with mChat versus searches in Google. We taught our leaders how to write prompts, and then we ran a company-wide engineering contest to develop prompts through prompt engineering. We had accountants, we had financial people, we had HR people, we had researchers, we had computational engineers, all writing prompts. It's amazing what you can do when English becomes the programming language of the future.
And so by having access to the technology, hundreds of hundreds of use cases were brought forward, and we taught our employee base how to fish for themselves to solve their own use cases.
And what we learned was, in some cases, we actually needed to build small versions of mChat application that's embedded into the workflows, workflow of our employees, like specific legal document review cases requiring their own small version of mChat. We're very excited about the impact of AI will have on our workforce, transforming how we think about the work we do to have the greatest impact on humanity. And so today, we're gonna dive a little deeper. Dave Johnson, our Chief Data and AI Officer, is gonna talk about the evolution of our platform from R&D to GenAI. And then Brice Challamel is gonna talk about how we're using GenAI to transform how we work at the company.
Brice is our VP of Product of GenAI and Transformation, and so we're gonna learn about how we transformed our workforce to adopt this technology. Then I'll come back and wrap things up, and then we'll be available for Q&A. So without further wait, please welcome Dave Johnson.
Thanks, Brad. I'm Dave Johnson. I'm Chief Data and AI Officer here at Moderna, and I'm responsible for delivering enterprise data and AI solutions across the whole company. What I'd love to talk to you today about is what we're doing in the data science space to deliver AI solutions across the entire enterprise. Now, there's a challenge around delivering data science and AI solutions in an enterprise setting. If you go poll tech executives, nine out of ten of them believe that AI is the center of the next tech revolution, and this is probably even higher nowadays because of the advance of ChatGPT and large language models. But if you look at the same companies, you find that only one out of ten of those AI projects actually make it into production in use.
So there's this big discrepancy between, on the one hand, there's massive potential for AI to disrupt and transform business, and then on the other hand, the practical realities of turning those AI models into solutions into production. So this is something we've thought about for a long time at Moderna and thought, "How do we ensure that we can get these solutions that we create into production, delivering real value?" And so part of it is a lot of those platforms that Brad talked about, the infrastructure and the ML and the data platforms, but then there's some other key ingredients to this.... The first is having the right impactful use cases. So Moderna, we're not interested in use cases around, you know, doing a really novel thing or playing with some new technology.
What we care about is use cases that transform the business, where by putting this in an AI model, by automating this process, we're delivering a tremendous amount of value to the business and transforming the way that we work. Now, I'm not gonna go too deep into the AI, the impactful use case, and how we come up with those. My colleague, Brice, is gonna come on a little bit later and dig into the transformation we're doing in the organization, to help uncover more of these impactful use cases. But I'll spend some time here instead focusing on the technology side. So it starts with data. Everyone always says that the key to machine learning and AI is a lot of really good, high-quality data.
So this is something we spend an immense amount of time on, in putting in place systems to digitizing processes such that we're collecting data as we go. We've been doing this for over 10 years, and as Brad talked about earlier, we have a large set of proprietary data, particularly in areas like research, where we digitized very early on. Now, once you have that data, it then needs to be structured in a way that is useful for algorithms. These don't work on kind of random text and spreadsheets. They work on clean, structured, machine-readable data, and so we integrate these data sources. Sometimes we have to go change processes upstream to make the data collection better and more structured, and then we put all of this in our data lake, where it's available for algorithm development and data scientists.
Once you have that, that's when you can start building these predictive models, and we use the latest and greatest technologies, a multitude of different computational techniques. But what's really important is these models don't just stay in, like, a Jupyter notebook for a data scientist to get something that looks good. We deploy all of these models on that computational infrastructure, that ML platform that Brad referred to earlier. What that lets them do is run at a scale and integrate into our business processes such that they're running live and making decisions themselves. Now, this leads to a really virtuous cycle, where the more predictive models we have running in production, the more that drives digitization of our processes and the more data we start to collect, and then we can improve those models and make better models. Right?
So it continues to cycle like a flywheel. Now, as I said, we did a lot of this early in research, 'cause for a long time, we were a research company. But as you look, we've expanded from that point. So it went from research use cases to clinical development, to manufacturing, to commercial use cases as well. And these aren't, you know, disconnected, disparate use cases. They're actually part of an integrated system here, with all of the data between these functions kind of connected and sharing with each other, and all the learnings we get from one model to another, and all the processes integrated together. And so this is across the entire enterprise.
Now, if you take those functions, and you can put them in a really simplified value chain here of what, you know, drug development looks like, it starts on the left with, you know, researching new therapeutic ideas. Then we develop them into medicines, then we manufacture at scale, and then we commercialize them, right? Super simplistic picture. But what we're doing is implementing AI use cases across this entire value chain. And as we do that, what that lets us do is create better medicines in the research and development space, and then optimize the speed at which we get them to market to bring mRNA medicines to market faster.
So what I'm gonna do for the rest of the talk here is highlight a few examples of this to show you the kinds of things that we've been doing across this value chain.
So the first is in the drug design space. Now, to talk about drug design, I think it's helpful to just step back a little bit and look at our products themselves. So the centerpiece here is obviously mRNA, and these are large, three-dimensional macromolecules here that fold up in, you know, these 3D conformations. But then we combine those through our formulation processes with small molecules like lipids, to create what we call lipid nanoparticles, or LNPs. Now, these LNPs are important because they protect the mRNA, but they're also what's key to get them into the cell through the cellular membranes, so that the mRNAs can attach to the ribosomes, which then translate those into proteins.
Now, the proteins here are the workhorse in the body. They're responsible for a myriad of different functions, like structure and signaling and enzymatic reactions.
And these proteins get their function from their three-dimensional shapes here, which you see in this picture. That three-dimensional shape is driven by the sequence of the proteins, which is, you know, there are 20 amino acids, and they're in a long strand, and they fold into these three-dimensional conformations. Now, if you change any of those amino acids or you change the order of those, you can have a radically different protein structure, which then has a radically different function, right? So by configuring those, you can have the protein be whatever you need it to be. Now, for an individual protein, it turns out that there's also a large degeneracy of different potential mRNAs for that.
And that's because for most of those amino acids, there's a bunch of different possible nucleotide codon choices that you can use to code for those.
So, you can actually have the same protein coded by a bunch of different types of mRNAs using different ACGs and Us here. So, what you see here is an informational center to our products, right? Informational nature. They are defined by code, by the sequence of nucleotides for the mRNA and the sequence of amino acids for the proteins. And this is very different from a traditional drug discovery approach, where you have a particular target of interest, and you're trying to find a small molecule that can interact with that target. Instead, this is really a design problem. It's how do we design the sequences for our mRNA and the sequences of the protein in the optimal way to have the effect we want in the body?
Now, if you zoom in on the LNPs, these small molecules themselves have their own structure.
These are really informational in nature in the same way that the mRNA and protein sequences are, but they can be represented by things like graphs and strings, such that you can have optimization design algorithms, AI algorithms, to create new, small molecules or to optimize the ones that you already have. Now, what's really exciting about this, though, is if we can come up with better small molecules to create better LNPs, that doesn't just help one product that we have, because these are the carriers of the information of our mRNA. They are, impact the entire, portfolio, whole platform of medicines that we have. So by making better small molecules, we're actually improving all of our medicines at once.
So as you think about drug design here and, and the potential for using AI, the important thing to note is some of these challenges.
The design space is huge. It's unfathomably large. So the protein space, as I said, there's 20 amino acids, and so all the permutations of possible proteins are 20 raised to the power of the length of the protein, which is hundreds or even thousands of amino acids long. Similarly, the mRNA space is huge because of all of those different codon choices for those mRNAs, also raised to the sequence length power. And then on the LNP side, the organic chemistry space is estimated to be about 10 to the 60 or more. So these are huge numbers. This is a truly massive space. We can't even keep in our heads how big this space of search is.
Even if you were able to kind of explore that space, the relationship between those structures, those sequences and the structures, and then the function of these in the body, can be really obscure and hard to understand. It's not obvious how to go from sequence to how this thing is going to behave. And then exploration of this space requires experimentation in the lab, right? It requires running assays, putting things in cells, and that can be really slow and expensive. This kind of manual exploration of such a giant, vast design space is just not practical.
But what we want to do, if you look on the goals here, is we want to be able to reduce the toxicity of our medicines, so such that we can dose at a higher level, increase the stability, so they last longer, and we don't have to dose as frequently. Increase the expression of the protein, such that the same mRNA can produce more protein. And then increase the manufacturability of this, which will reduce our cost of goods, so we get greater yields out of our manufacturing batches. And then last is getting the right biodistribution to ensure that the mRNA and the LNP go to the right, the right cells and the right tissues in the body.
So the idea here is, instead of trying to explore this massive space manually, what we do is we use artificial intelligence to form these relationships, to understand that space based on historical data we have, based on the entire evolutionary history of proteins in nature. It can incorporate all that information and help us generate new candidates to much more effectively explore this design space. Now, as you might imagine, this is a really monumental task. There are lots of different aspects of mRNA, protein, and LNP design, and we've been working on this for many years with a number of different products around this. Early days, it was, you know, mRNA translation and mRNA design algorithms, and now we're in the midst of our next generation mRNA design algorithms.
We also last year published a paper on a generative AI model for creating new protein candidates. We have two very exciting collaborations going on with IBM, one around a generative AI model for small molecules, and another cutting-edge one around using quantum computing in this same drug design space. We have algorithms around DNA template design and antibody design, and we're in the midst of a really exciting project around our formulation design, where we're leveraging really high throughput automation and robotics to explore a massive number of permutations in this that will ultimately be guided by AI-driven experimental design and AI analysis on the end. So huge amount of work in this space, and we continue to push the boundaries of what's possible with AI to develop mRNA medicines.
Now, if we go back to that simplified value chain here, another really exciting example to talk about is our individualized neoantigen, or INT use case. This touches on two parts of the process. First, in research on the drug design, but then as well in manufacturing and how we're optimizing things. Now, for those of you not familiar with our INT therapy, this is essentially a product designed for an individual patient's unique cancer, right? Because everybody, every patient's cancer is unique to just them. So, what we do is we take samples of their tumor and their blood normal, we perform Next-Gen sequencing on those to understand the sequences and compare them to find what are the mutations specific to that person's cancer.
Then we use algorithms to turn that list of mutations to figure out which of those are going to show up as neoantigens on the surface of those cancer cells, and then design an mRNA unique to that individual. Once we've designed it, then we go into GMP manufacturing and create one lot of material for one patient. We ship that to the sites, and all of that happens in about six weeks. Now, AI comes in place in two different areas here. First is in the neoantigen therapy design, and then it shows up again in the manufacturing process. I'm going to walk you through both of those. So first, if we double-click into that box on the neoantigen design....
Over on the left here, you see the, the raw NGS sequencing data that we're performing, and then over on the right, you see the end product, which is an mRNA designed to target 34 neoantigens. In the middle here are five different components that are multiple AI algorithms connected together in a fully autonomous, fully integrated way. There's no humans involved, there's no manual activity. It goes straight from NGS data all the way through a designed mRNA sequence automatically. Now, what I find particularly exciting about this is this block here at the end that says, "Design mRNA sequence." Now, before, right before this process, that's where we have the amino acid sequence designed of those concatenated 34 different neoantigens.
Now, what this does is it, it's actually not algorithms that we created specific for INT. Because we're a platform company, we're designing algorithms for general purpose.
So we already had algorithms here for back translation, for DNA template design, for mRNA optimization design, and those very same algorithms I was just talking about a moment. We leveraged here. And that's what's really exciting about this, is we're able to take these as building blocks and combine them together to do new and exciting products like INT. Now, on the manufacturing scheduling side, there's some interesting AI work here. So what we needed to do in this program is create some digital infrastructure that enables every patient to receive that INT administration within six weeks of enrolling in the study. And this solution has to deal with a manufacturing schedule where, you know, there are a lot of unique aspects of running global clinical trials.
There's a lot of intrinsic variability to what's going on, to when patients enroll, to when they screen, to when they dose.
Those are their schedules, and so we have to meet them where they are, and we have to make real-time adjustments to our manufacturing schedule across a number of different patients, such that we can ensure that we're really matching their schedules in a really patient-centric approach. So what we did is we built an integrated end-to-end solution to enable the best site and patient experience. It includes a site-facing interface to connect to an individual patient's schedules, and it includes AI-optimized manufacturing scheduling that looks across all of those patients and all of the manufacturing activities to ensure timely administration for those, those INT doses. Now, to actually walk you through what that interface looks like, I'm going to pass it to my colleague, Wenhao Liu, who is a director of software engineering and led the development of this really unique solution.
Thank you, Dave. I'm here to introduce Maestro, which is core to our individualized neoantigen therapy, or INT, manufacturing process. Our vision is to build the best individualized medicine platform in the world. This vision is made up of several platforms and modules, but at the center of that vision is Maestro, the first Moderna-built software solution to be validated against the FDA's 21 CFR Part 11 compliance regulations. Maestro brings together the end-to-end digital pieces needed to enable an individualized medicine platform. While AI is used in multiple steps in the production and delivery of INT, today, we will discuss the use of AI to coordinate the scheduling of our manufacturing resources to enable the production of the thousands of individualized medicines needed to meet patient demand for our ongoing phase III global clinical trials.
For every patient in the INT clinical trial program, Maestro creates a cross-functional view of all patient tasks. This cross-functional view brings together all the activities related to a single patient, allowing Moderna to tightly coordinate all these activities across functional areas to achieve efficiency. Shown here on the timeline are tasks spanning across clinical operations, manufacturing, quality assurance, quality control, and shipment logistics. All this data against an individual patient is anonymized, protected, and centralized, and can be fed into future developed machine learning models to help us continually optimize our end-to-end operations. If a delay to any activity happens, all downstream events are automatically rescheduled, allowing related downstream supporting functional teams to plan ahead and react to changes proactively.
This timeline view also critically synthesizes the data from our clinical trial partners, which informs us in real time the patient's target dose date, shown here as C 2 D 1, or cycle two, day one. We are able to track that our targeted clinic shipment can complete prior to this C 2 D 1 dosing date to ensure INT is delivered to the site and administered to the patient in a timely manner. Given manufacturing adjustments must be executed on a real-time basis to accommodate updates to a patient's schedule by clinic sites, in order to ensure a patient-centric approach, we developed an internal AI scheduling algorithm. Shown here is our in-house developed clinical manufacturing scheduling system used for drug production scheduling for all of Moderna's platform clinical programs, and we have integrated this with INT scheduling.
Our platform manufacturing scheduler receives real-time signals from Maestro for when an INT patient is anticipated and displays them here. From here, we must place the seven total INT manufacturing steps required for each patient against the anticipated INT administration date, while accounting for multiple other patient and non-patient activities happening in parallel, competing for the same resources. Utilization of this integrated manufacturing approach allows us to optimize delivery of the patient's INT.... Clicking on this button triggers our AI scheduling algorithm, which is still under active development, to automatically place this patient's batch schedule onto the timeline, taking into account a real-time snapshot of our manufacturing capacity and all constraints. If a block needs to be adjusted, all related downstream steps are automatically rescheduled.
As this happens, the data is flowing in real time back to the scheduler view to update the full end-to-end picture for that patient.
If a clinic site updates a patient's target dosing date, their manufacturing schedule will be automatically reoptimized by our algorithm, not just for that single patient, but globally for all active patients simultaneously to prioritize meeting every patient's target dose date for INT. As patient demand ramps up in the next year and we have hundreds of patient batches executing in parallel, the use of this algorithm will be a crucial enabler to allow our supply chain planners to have a full end-to-end view of every patient batch, and to react quickly to changes to an individual patient's timeline to ensure we can deliver every INT dose on time. This fully connected and integrated ecosystem is the result of years of focused investment on digital and a massive team effort across multiple functional areas within our digital organization.
A digital orchestration tool like Maestro and its related ecosystem enables us to execute a patient-centric approach and deliver on the development of an individualized therapy at the scale of our ambitions. The foundations we lay here can be leveraged as we look forward to the future of INT, including potentially supporting commercial operations. Thank you for your time.
Thanks, Wen. So I hope that's given you a sense of all the kinds of amazing things we're doing across the value chain, but we're nowhere near done. We continue to push the envelope in clinical studies and pharmacovigilance, and regulatory, and elsewhere to really push AI everywhere across the company, and we're designing products to do just that. And this is all enabled by what Brad talked about at the top. We have this infrastructure layer, we have data, machine learning, and generative AI platforms built on top of that, and that allows us to deliver products to market like Maestro, like mChat, and like a variety of other products that I've talked about here. So thank you, and I'll pass it back to Brad to pass it on to our next speaker.
Thanks so much, Dave, for that deep dive. So insightful. It's truly incredible and amazing how we've built the foundation of such a great AI platform over the years. So now I'm excited to introduce Brice Challamel, who's leading our GenAI product strategy, as well as our overall AI transformation. Brice is gonna walk us through our journey over the past few months and where we're heading next to transform how we work at Moderna. Brice?
So hi, everyone. So glad to be here and talk about our AI transformation program. As it was mentioned before, one of the key roles of good technology adoption and creating proficiency beyond adoption, so that people use it to the full potential, is a rule of 30/70, where 30% of our time and energy is dedicated to making sure that the technology works right, and 70% that the users understand it, are aware of its momentum and its development, skate where the puck will be, and prepare themselves for the next generation of the technology, and anticipate how it can help their business and create value for them.
As you see on the left part of the slide here, the basis, the bottom line for this is the cultural mindset, and we are very lucky at Moderna to be a cloud-native company, to have a fantastic AI culture, and to be well-versed in the potential of machine learning and to the generative AI to support business. The cultural mindset was always there and is there more than ever. Then the leadership guidance matters. It's really hard to move against the current that is given by your top leadership, and in our case, the current is so pro AI that we are always moving with the current in those efforts, both from the E system point and from the leaders cohort.
And I will talk a little bit more in a minute of how we have worked with those leaders to make sure that they were feeling empowered by the technology, and they could help their teams make sense of it and create value with it. Then we need early adoption, which are the symbols of success, the very first leaders who create something new and exciting with it and show the way to all the others. Business integration, when we connect the data to the processes, to the new ways of working, and everyone understands and feels reassured and feels elated once they are reassured with the technology.
That leads us to employee training, which is, in a way, the first thing that people think about in transformation, but really truly the last thing that will happen when everything else is in place and when the training can lead to use cases and to better ways of working. So as you see how we distributed on the right part of this slide, the transformation pillars that we addressed, first and foremost, and this is out of decades of experience doing this, are fourfold. Grow the AI culture.
Keep growing this strong AI culture that we have at Moderna from the core of the champions and early adopters, about 100 very, very strong users, to 2,000 enthusiasts, 2,000 of them, active users of our forums, our tools, in connection with us all the time, and beyond them to everyone in the company. We're a company of roughly 5,000 people, behaving and working and achieving, like, 50,000 because of and thanks to AI.... The second pillar is empower the users and leaders with action learning and events. There is only so much we can learn about swimming in a classroom. At some point, you have to go in the swimming pool and then into the ocean.
So making sure that they have the tools in their hands as early as they are available, and that they learn from playfulness and from intuition and from trial and error, and creating events when they can share their key learnings with one another is very important. The third pillar, generate those early successes that I talked about, symbols in every business line. So we went one by one in every business line, we ran company-wide contests, which I talk about, for the best use case of generative AI, and we received more than 180 propositions for the best symbolic value of what AI could accomplish. And it ranged from, researchers who proposed tools, coded in Python, thanks to generative AI, within interfaces coded in HTML, thanks to generative AI, with tutorials recorded, thanks to generative AI.
So, you know, like the Russian dolls, the goodness of Python code within the goodness of an HTML interface accessible to everyone, and on top of it, a tutorial written by generative AI. All the way to storytelling about why hygiene matters in labs, for instance, that will be stories told by our generative AI engine for the rest of the companies and for the people who need to be very careful about how they come to a clean room and how they behave in these clean rooms and controlled environments. The fourth pillar, productize AI with self-service, business-driven platforms. This is so important. If the users need to come to digital to use the technology and ask us for permission, or for advice, or for guidance every time they have an idea, we are never going to benefit from the democratization of AI.
The true potential for our organization, and I believe every other on the planet, is that generative AI delivers the promise of internet and machine learning all at once, in conversational mode, in plain language, whichever your language is. For this, we need to make sure that someone from the business who needs to deliver value with generative AI has all the tools they need to go from end to end to production for the largest numbers of use cases. Some of them will require engineering, but most of them don't, and so making sure that they have something that they can use is of the essence. My best analogy here would be Excel spreadsheet in the 90s.
There was a time when this was new to use spreadsheets, but everyone could do this, and you didn't have to go to IT to ask for a new spreadsheet or to enter numbers in a spreadsheet. And yes, sometimes you had to do a pivot table, and it was hard, but you would have your friend next to you who's a champion and knows how to do pivot tables, would show you how, and you're self-sufficient. And that's how we're thinking about AI in the age of democratization of generative AI. So we have known very early on, and it also makes common sense, that...
This is a research that was done with the support of the Boston Consulting Group and published in the Harvard Business Review, that we can move significantly with generative pre-trained transformers, which is what GPT stands for, so generative AI. Pre-trained, because these models are trained once and for all, and then you can fine-tune them, but the training is done already. And transformers is a technology that's binding together all the generative AI models, so GPT. And when you use GPT, you move by an order of magnitude. We go from a scale of four to six, so we augment by 50% the efficiency and the quality of people at a given task.
And if you give them an overview, a layer of help and enablement and support through the platform or through learning and enablement, it reaches another 10% in the density of what they can accomplish. And on those two scales, the gains in productivity we're talking about are in the order of 50% task-based. This is twice as much as the Industrial Revolution, which is most often credited for 20% gain in productivity. We are about to have twice the gains in productivity of the entire Industrial Revolution, and this in three to five years at most. So I want to explain a little bit more of the details of how we do this without going too much in the weeds.
There is a top-down process, and there is a bottom-down process of enablement that meets in the middle, in the teamwork, where the magic of leveraging technology happens. From the top-down process, we've had executive committee sessions, one-on-ones with all of our leaders, our RC leaders. We went with them and their senior leadership teams for off-site, for seminars, for trainings, and then we had a seven-week learning and enablement session with the senior leadership. We call them the SLC, the senior leaders cohort. And during those seven weeks, we ran 21 sessions, once a week for learning, twice a week for office hours, in which they would bring their own use case to productize and activate generative AI.
Those 21 sessions ended up making the 170 leaders of Moderna fully understanding and proficient in generative AI, so that when they talk about it with their teams, they are not just adapting to it, they are leading this. The thing we wanted to avoid at all costs is to have leaders, as I've seen happen 30 years ago when I started my own career and computing arrived in the enterprise environment, who would say: "What are those things? I don't need them. They're like new typewriters. Put them on the, on my secretary's desktop and, and stop bothering me with it," right? Like, the natural reaction of people who are unaware of the potential of technology, and that can completely destroy the value that the teams would be able to bring.
So we have a very, very strong and committed leadership, and they are fully trained and enabled. From the bottom-up standpoint, empowering users, we ran a number of company-wide events. I talked about the company-wide prompt contest. We recorded the podcast in the early days of generative AI with Stéphane, to explain to everyone the technology and rally everyone around it. We opened teams, forums, and channels. We have a EdCast learning path. It's a learning journey application that everyone has access to at Moderna, and it has four levels of learning with little videos of three to five minutes each that they can use.
So we fired on all cylinders to make sure that every single person who has access to the technology has the means to be aware of it, to learn about it, and to use it to support them in their day-to-day life. Then we ran AI awareness and AI applied training sessions, in which you actually put your hands on the keyboard and deliver a use case, which delivers those capstone projects, which are the productization of generative AI into applications that the whole team can use, and that is done by a few core members in a front-end, back-end way. They work in the back end to create the code, and then they create an application on the platform, and everyone can use it on the front end. This ends up with two very powerful groups that are at the core, the center.
The first is a champions group. We call it GACT, which is the four bases of the DNA, guanine, adenine, cytosine, and thymine. But also, in our case, means generative AI champions team. And this is a group of 100, top users that leads the effort and is present in every business line of the company. And by their side are all the teams that they belong to and that have turned teamwork into AI-assisted and AI-driven teamwork. And I can't emphasize enough how this is the place where the magic happens, because if your colleague next to you is not using it, doesn't understand it, doesn't want to understand it, because they feel threatened or just overwhelmed by it, it is very hard to make concrete day-to-day value and applications of something like artificial intelligence.
But if everyone you're working with pings you about something that they've just discovered, shares a best practice, or an anecdote of something they've, they've achieved, then suddenly you're in a much better space because you are invited in the culture, because you, you are surrounded by the culture, and now you're swimming in the ocean. And this is what we're achieving now with this program. So as I move forward on how we create customized technology tools, the first way is that we have a privacy data policy that was both a technical and a legal achievement, to make sure that everyone can use this with every information that they need for work, including sensitive information from a PII standpoint or from PI standpoint, the intellectual property.
So we've created a system that is called zero data retention, in which we have absolutely no leaks or no privacy leaks or no technology leaks from the data, that it is safe, it is streamed throughout the models, and when we run a query, and that we know for sure, and we tested this also on the back end, that every piece of data that is used is safe and secure, right? This helps us create responsible AI use cases on top of safe guidelines. The second is to expand mChat with capabilities such as document upload, so that people can leverage their own data, their own day-to-day data, into the environment. mChat's instance for Moderna is our client, our universal client for generative AI, that we developed ourselves.
It multiplies by a hundredfold the amount of information that can be shared, because if you only prompt and use the pre-trained model, you have access to information that's a few years old and that's not your own. But if you can upload your documents one by one, or pointing to folders or pointing to connectors to your industrial systems, then suddenly you can use generative AI with everything that's in your day-to-day life and your day-to-day work. The third one was to introduce a simplified interface for easier use. So we worked again and again relentlessly to make AI feel like a something that a child could use. Something that only requires for you to converse and discuss naturally.
I think a lot of companies have done this very efficiently, and we have examples from public vendors, but we wanted to make it in a way that the Moderna users feels completely at home instantly. So, we've helped with very simple systems to store prompts, to call them, to share them, which we call persistence, to store conversations, because sometimes it takes five or six or 10 iterations to get to a good outcome of generative AI, and they can store these conversations and share them with one another. And that empowers a much broader employee uptake because everyone feels that this is something accessible, as easy and simple as Spotify, YouTube, Google Maps, all those applications that they are happy with and use on a daily basis.
And it's not requiring from them to have data science knowledge or a heightened sense of awareness or expertise in the field. Lastly, we run data analytics and proficiency tracking. So we have a crawler that goes through the logs and looks at how often and how frequently you use the application, who uses it for what intent, with what level of proficiency, and sends them automated emails back powered by generative AI to give them hints and best practices on how to improve the way they use it based on the analytics of how they've been using it. And that automates the continuous enablement and helps us inform how we conduct learning and improvement sessions.
This has delivered you probably heard this before because Stéphane mentioned it but the fact that more than half of the people in Moderna are active users of mChat of our application. We have here to remember that only 70% of our people have access to a device with which they can use it because the other 30% are lab technicians or manufacturing agents who work in clean environments and don't have devices to have personal accounts and access to mChat. It might seem like an easy thing to achieve to have about 60% of our employees using actively generative AI for their work but we're almost at full capacity there because it's 56% out of 70% of potential users.
It's nothing short of extraordinary, and a testament to how eager the people at Moderna are to try new technologies such as generative AI. So, now the next part I want to talk about is how do we productize AI? What are the buckets of value that we go out to reach? One thing that's interesting is that it is a very democratic technology. Half of the use cases can be simply developed, remember my Excel spreadsheet analogy, by using mChat and using the tool as a self-service capability with what we call prompt engineering, which is how good you are at writing your questions, your conversations, your idea generations into the client directly of mChat. So, a good example of this, we created a speech generator.
What this means is that if you want to work on talking points on a presentation, if you want to generate a toast for a dinner to which you're invited, or a 10-minute keynote because you're a speaker in school or in a conference, you can use prompts that we have provided in a library, and with those prompts, it will ask you a few questions: What is your audience? What is the duration? What is your topic? What would you want to talk about? And propose to you an elevated speech with introduction jokes, with historical references, with interesting and funny connections between different data points. That's going to make your speech much better than if you were writing it yourself.
It's like a, an expert in speech writing that's by your side, helping you do this, and you don't need anyone for this. And that's actually half of the use cases and half of the value of the use cases. So the second example that I want to give is the example of custom workflows. That's something that requires advanced users. It's something that is maybe a three-step or a four-step process. And there we go from using Excel as a spreadsheet to using pivot tables in Excel, something that does require a little bit of knowledge and of know-how. But if you do this right, you're gonna have the ability to tap into a much more feature-rich environment than just the prompt on the first column. And that's about 30% of use cases. I'll give you an example here.
It's the Stardust email that we do at Moderna. So what is it? We take all of the data that we have on how people use generative AI. We have a super prompt that looks at that data, understands the strengths and, and the potential for improvement, creates a recommendation that is generated from that data. At that point, no human being has been involved in the process, and puts it in an HTML format within an email sent to everyone every other week.
Bi-weekly emails delivered in your inbox, and you see an example on the right side here, from mChat Stardust, talking about your mChat adoption and proficiency metrics, and giving you tools and web pages and links and best practices, exactly where you are to go to the next step of your journey and of your proficiency and your learning on generative AI. So this is an interesting way, of course, of using generative AI to support generative AI. Sometimes we call this internally mChat for mChat, which is how we can deliver faster and better than any other team, I think, or any other time before the benefit of new technology. The third is agents and plugins. So this time, you won't just need to have an advanced user, you're gonna need someone to code that agent or that plugin.
If we still remain with my Excel spreadsheet analogy, that's when you need one of those plugins that is gonna do waterfalls for financial analytics, for instance. So we have a team, an engineering team, both on ML that does the compute jobs and on generative AI, that can provide platform components. And a good example for this is image generation. It's something that is expected of generative AI, to produce images, as you would see with Midjourney or Stable Diffusion or DALL-E. And we have this also in mChat, and we're exploring various ways to make this useful for our users' benefit, both in research to illustrate papers, in manufacturing for standard operating procedures or in marketing for marketing campaigns. So here I'm giving you a simple and cute example, but we can actually do very sophisticated image creation.
But just the hashtag create image will create the possibility for a prompt to open an image, and we're working on that plugin and that module, and we're improving it a lot actually these last weeks because of the access to all the new APIs provided by OpenAI with DALL-E 3, and with other vendors who have opened the possibility to us to access the latest generations of image generation and put it in the hands of all of our users for their communication needs and for their presentation needs. Last, it's when we need to have a net new product.
So this time, generative AI might be a feature, the same way that AI is a feature in Google Maps, and Google Maps is basically a map product, but you can know how long it's going to take you to go home or to get to work, thanks to AI prediction. Well, a product can be a product, for instance, to create marketing campaigns locally, but AI is going to be a feature that helps you generate text or customize images, or just converse with you and ideate with you on the best way to make that marketing campaign work for your audience. So that's a very small number of use cases. It requires a lot of development and cross-functional developments across the various teams of digital. But we partner with the product managers, in this case, the product manager of commercial, to develop an interface.
And within that interface, we have generative AI plugins and capabilities, which you see on the right column, to help our people in Japan develop marketing campaigns straight from the generation engine, with all their input and their knowledge, and their proficiency in the market. Instead of having to translate something that we would produce centrally or to simply entirely reinvent campaigns by themselves every time, they get both the benefit of the marketing tools at their disposal and of generative AI into those marketing tools, both to generate content and to support the conversation on the content on the side. And that creates the next generation of products, which I think we're going to see everywhere, and is going to be very important in corporate life and the way that we deliver value. So here we go.
Four types of examples, from the most self-service, but also the most frequent, to the most engineering heavy, and also the ones that require to do prioritization, understand the value we're generating, and decide the order in which we're going to develop them. And as you can see right now, we have a very important priority assigned to commercial. So we're working on helping our commercial team promote the current and the future products in the pipeline with generative AI. So, a couple more points, and I will be done with that part of our presentation. The first is that we've done a lot of training sessions for the employees. I talked about it.
What you see here on the left side is a visual of one of our key speakers in front of an audience of 500 employees, which is 10% of our population, talking about the future of AI, because we don't want to just skate where the puck is, but where the puck will be. And we are learning with the tools of today to anticipate the generative AI of tomorrow, when the models will get more powerful, when the clients will get more intuitive, and when we can fine-tune on our own data for very high-end use cases.
On the right side, you see an example with my friend and colleague, Ed Miracco, there, of us, in a town hall with the research and early development team, exploring potential use cases with expertise on how to produce research papers, but also how to digest the library of research papers that is presented to us every day on our topics. We go team by team, town hall by town hall, event by event, sometimes in large settings of 500s, sometimes in smaller settings of a few dozen people in town hall, sometime in a short, leadership team session to unlock value with a use case or an idea that they've had. We cover the full spectrum because there is no place where we believe there's not a future for generative AI. It's like electricity.
We don't ask ourselves what are the use cases of electricity anymore. Our purpose here is to make sure that no one asks themselves what are the use cases of AI anymore, because we are swimming in the ocean, and AI is everywhere around us and at our service. So last, this is my last, slide. Of course, we are always measuring everything. We would not be a data and AI team worthy of their salt if we were not looking at how everyone is adopting it. Our R&D team is heavily pushing forward, has a fantastic adoption curve. And we're looking at the teams that have least adoption, most often because their data is so confidential and so specific that we have to find the right data connectors for them.
But the surface here is very indicative of how close we are to 60%, 80%, 100% adoption in all those teams. Technical development, the legal team, finance, commercial, brand, I'm going a little bit faster there. But as you can see, we have a huge ramp-up, and there's a huge number of teams and volume of our employees which are 100% at the team level that I spoke of, where the magic happens, entirely in adoption and entirely in proficiency already. And this is less than six months after the technology appeared to begin with, a testament to Moderna's passion for AI and relentless commitment to make AI work, to create value, and to save lives better and faster than has ever been done before. Thank you very much, and I'll hand it back to Brad.
... Hello again. Thanks, Dave and Brice, for sharing with everyone how we're building out our AI platforms in order to deliver meaningful outcomes for our patients and customers. As I said before, we're not doing AI for the sake of it. We're building out AI capabilities by riding the wave of innovation that's happening in the tech industry, and we're investing here so that we can build the best possible version of Moderna. We believe that in order to democratize AI and transform how we work at Moderna, it's critically important to put the user, our employee, at the center of our design. Our platforms, our applications, work to simplify the work of our employees so that they can quickly make right decisions for our patients. AI is really all about gathering data, asking great questions, so that we come out with great answers to hard problems with speed.
In order to have the greatest impact and build the best version of Moderna, we need to transform our culture, embed AI technology into our businesses, and build the necessary tech stack to bring to life innovation of our people. It's their ideas that we are manifesting in our environment to create greatness for our customers and our patients. Back a year ago, we would host AI workshops, where we'd seek out use cases from employees, typically by department, like HR, Finance, Legal, R&D, Commercial. We had hundreds of use cases. However, the bottleneck at the time was really developer and data scientists' time to build out these models and software to respond to the use cases.
However, with the democratization of AI across our company, by teaching people how to use mChat, build prompts, upload documents, point mChat at large document repositories for summarization, we have unleashed the potential of AI on our workforce. We've now stopped counting use cases and focused more on the measurement of that impact that we're delivering. We've also seen that many ML use cases can now be solved with GenAI applications. We've seen a conversion of our platform capabilities, and because we've built in a very distributed way, we quickly respond to technology needs by reusing components to solve many different problems. We're only at the start of this next AI revolution, but the focus of our efforts is clear: to have great impact on our business, our patients, and our workforce.
For instance, in commercial, we're using technology to replace large sales force of typical pharma companies while still obsessing about our, our customers. In manufacturing, we're leveraging AI for automation and robotics, simplifying how we manufacture drugs and reduce the overall cost of goods. In R&D, we're decreasing the time to drug discovery, accelerating clinical trials through the process of AI-enabled data analysis. Finally, we're increasing the overall efficiency of our workforce, simplifying tasks, focusing our people on what's most important. So we're in the process of building a real-time AI company to ensure that we deliver on our vision of becoming the gr-- of having the greatest impact to people through mRNA medicines.
We're on that path to delivering this impact as we leverage AI to accelerate our product portfolio to deliver 15 product launches over the next five years that will have a massive impact on humanity. I hope you've enjoyed learning about how we're taking into account AI as a core part of our strategy moving forward, and now we'll open it up for Q&A.
We'll now move into our Q&A session. For those of you who are joining us via Zoom, if you would like to ask a question at this time, please raise your hand by clicking the Raise Hand button under the Reactions at the bottom of your Zoom window. Once called upon, please unmute your audio to ask your question. Please note that if you are watching the webcast, you are able to submit questions through the Ask a Question tab on the webcast. Thank you. To start, we would like to ask our first question. Our first question is from Hartaj Singh at Oppenheimer & Co. Please unmute your line and ask your question.
Hi there. Good morning, and thanks for the presentation. Really comprehensive. I guess I've got just two questions. One is a very specific question, you know, asking about where Moderna stands today in terms of the speed and scale of the learning process cycle, for high-throughput screen, for example. I know that maybe the mRNA high-throughput screening is not the actual analog, but something in that context. So the speed and the scale of the learning process cycle, and what are the tools you use to accomplish this? So that's number one. And then number two, you know, for example, with INT, we've actually had calls where we've with physicians, and we've asked them: how important is it, you know, to have a really precise and accurate neoantigen selection when using an INT therapy?
And they felt it was extremely important and that companies that, you know, were good at this, would be, you know, a higher priority for them in terms of clinical trials and in the commercial process. So any thoughts in that regards, in terms of, you know, the, the algorithms you're using for your INT projects? Thank you.
... Dave, if you wouldn't mind kicking off on both of those questions, given the R&D aspect of them both.
Yeah, happy to. Thanks for the questions. So the first one, the question around speed and scale of the platform, this is something we're continuously working on. So we talked about some of the algorithms we're using in the design space here. One thing we didn't really go into was what we're using, kind of the operations and research. So how we are putting in place algorithms to optimize our processes and scheduling, as well as analyze data. Some of these things may seem kind of small incremental improvements, but you have to remember, research is a cycle, and so by improving processes in that cycle, we're able to run that loop faster and faster and learn more and more.
And on the high throughput side, we're in the middle of designing a lot of work around our process development, particularly around formulations themselves, to explore the really massive design space. And, still early days, we're working through a lot of the automation and robotics for that, but the ambition here is to use AI to drive the optimal search space for that, use robotics to execute the assays, collect the data, and then do a feedback loop and a circle to continuously improve that. And on the INT side, we do think it's incredibly important. The whole purpose of the program is to be personalized to an individual patient's unique cancer to them.
So, we're very proud of the results we've seen so far, and we continue to assess the efficacy of those algorithms as we look at the next generation ones as well.
Laura, do you have another question?
Yes, our next question is from Jasmine Fels at UBS. Please unmute your line and ask your question. Jasmine, please unmute your line and ask your question.
Hi, this is Jasmine on for Ellie Merle at UBS. Can you hear me?
Yes.
Yes.
Okay, great. Yeah, we have two. First is, how are you thinking about partnerships and collaboration strategies in the AI space? And then, secondly, on AI accelerating Moderna's drug development timelines, both in early and late-stage development, what are kind of the timelines where we're going to be able to start tangibly seeing this effect? Thanks.
Brad, do you want to take the partnership, and then I talk about the timelines?
Yeah, you bet. So, one of the strategies that we believe in at Moderna is to certainly ride the wave of innovation that's happening in the industry, and so partnerships are really critical for us. Since day one of our AI journey, we've been partnering with key players. Most recently with GenAI, we've been partnering with a multitude of the key players, from OpenAI to Microsoft, Amazon, Google, to investigate the use of their capabilities. These large language models are, you know, different in their use cases, and what we believe in is having the ability to pivot fearlessly and fast to follow the newest trends and enable access to our employee base with the latest technology.
And so, we've built capabilities that allow us to utilize multiple large language models in the back end, partnering with multiple partners, and it's a strategy that we're following at this point, especially given the flexibility that's happening in the market today. So, we're just learning, we're partnering, and progressing forward with our strategy of enabling us to have multiple options at the same time.
Yeah, may I just add one piece on the partnership, and then I'll talk about the speed to development. I think as you saw with mChat, I think the team did a great job to not only get us our own version of mChat quickly on the OpenAI, you know, code, but also to adapt it to allow us to be able to upload documents. And that was a big game changer for the team because, you know, you can do so much with a, a large language model if you don't have the data that you want in the model, obviously.
And so the day the team was able to upload that feature very early, just changed how we use the tool, because you could put a PowerPoint document and start having a discussion with mChat or PDF and so on, and the team is doing a lot of loading of documents across the enterprise that everybody can use. So it's both a partnership and then a quick adaptation to our data set, which of course is key. On speed, I think it's a bit too early to be clear and to give timelines of how much time we'll be able to cut, both in preclinical and in clinical setting. But we're obsessed about how do we do things as in real time as we can.
So, if you think about it today, when we file with regulators, for example, [inaudible] that we filed recently as our last filing, there's a lot of exchanging back and forth with questions from regulators. You know, Dave and his team has put a very cool tool in place where basically you get the email coming from regulators, it's standardized, and then there is basically proposed answers from the database that we have to just reduce the cycle time for each of interaction. So, I think, you know, it's gonna take us a couple products to really reduce drastically the time, but as we obsess about learning and we compound this, and you see this on the entire platform, the time across everything we do is gonna be reduced. Because we don't believe in the magic bullet that's gonna make everything goes away.
I think it's what can you do into the lab to go to a clinic faster? How do you do into opening R&D, doing as many as we can, phase I/II, so we don't have to do the traditional phase I, get the data, then go to phase II to get enough of a safety database to go into pivotal. And so, we're trying to do all those things at the same time to just save every day that we can, because it will help this drug get to markets faster, but then all the learnings will apply to the next drug and all the next drugs moving forward.
... Laura, are there-
Our next question is from Jessica Fye at JP Morgan. Please unmute yourself and ask your question.
Hey, guys. Good morning. Thanks for taking my question. Three from me. First, can you provide some tangible example of how AI can help identify and reduce potential toxicity and increase stability when designing or optimizing your development program? Love to hear a little bit, more about how it helped you. Second, do you feel the algorithm that picks the neoantigens for the INT program is proprietary relative to competitor approaches, or is everyone kind of tackling this in a similar way? And third, what areas do you see as the low-hanging fruit for AI to help reduce drug development timelines out of all those areas that you mentioned? Thank you.
Dave, I think most of those are-
Yep.
Yep.
Yeah, so I'll start on those, and then maybe folks can chime in. So the first one about tangible example around reducing toxicity. So one of the things I mentioned earlier was a paper that we published last year on generative AI for protein engineering, and in that very specific example, we were looking at a protein called human OTC, one of our rare disease programs. And the entire purpose of that was to come up with one that had reduced toxicity versus a, you know, a bacterial version and increased stability relative to a human one.
So we were able to incorporate into that model the entire genetic history of OTC variants through, you know, evolutionary history, and we used it to come up with novel variants that we couldn't have picked with traditional techniques, and then we screened those, and we found, you can see the results of the paper, that the vast majority of those variants not only were more efficacious, the proteins had greater activity, but they were all more stable, which was an interesting finding from that. So that's a very tangible example, and we're applying those same approaches to other protein engineering cases, and we have that algorithm kind of implemented internally for use for our researchers for a variety of targets. We're also applying those same ideas to our mRNA design as well.
On the INT program, I, I can't really speak too much to what the competitors are doing with their algorithms. There's certainly gonna be a lot of similar themes across those, but we do feel that the algorithm that we have is proprietary in nature and, and are based on our unique understanding of how that works, and it's also based on, you know, the years of work we have done in optimizing mRNA design itself. As, as I talked about, those algorithms are baked into it as a piece of that, and so that's something very unique to us. And so the last question on low-hanging fruit, you know, as Stéphane said, it's not one answer really.
The R&D pipeline is an incredibly long, complicated process, and so there's a lot of different things that we're doing that we think will accumulate, to a great degree of value. I would say the thing that we're really excited about right now is the impact of large language models. So in particular, in the development space, there's an incredible amount of text going on, and for a long time, that was really inaccessible to us in leveraging AI. And now we see with the advent of large language models to be able to do things like reinvent how medical writing works, to reinvent how clinical data review works. We do have, you know, really high ambitions and are really optimistic of that having dramatic impact, in the, in the near future. So thanks for the questions.
And maybe if I can add to what Dave said. Of course, today we're talking about AI a lot. That's the purpose of this session, but we're really obsessed about how do we keep shaving time, enabling scalability because now the pipeline is 40+ programs and reducing cost. I mean, we talk, of course, about AI and digital, but also robotics is another key feature and internalization. You know, in the olden days of Moderna, all our blood work in clinical trials was all outsourced, so basically, you know, in a queue of the CROs doing that for us. What we've done, a year ago, in the fall of 2022, we basically launched our new lab in Norwood, which is in a new building in MTC East.
For those of you that have been in Norwood, the first building was south, and in that building, the team basically built an amazing robotics setup where the blood samples are coming from the studies and being analyzed very quickly. There's very high throughput systems that goes directly into, you know, the lab information management system, the LIMS, so to be able to provide that data in terms of clean data and structured, that allows us to go very quickly, and we can reprioritize studies. Because we have so many studies, sometime you might have two studies that the blood samples are coming around the same time, and Steven Hogan and his team are able to decide which one goes first, whereas before we just have been in a queue of the CROs.
That's another example of what we are doing to just trying to compress and compress and compress, as much as we can, the development timelines.
As a reminder, if you would like to ask a question at this time, please raise your hand by clicking the Raise Hand button under the Reactions at the bottom of your Zoom window. As there are no hands raised at this time, I'd like to turn the call over to Lavina Talukdar, Senior Vice President, Head of Investor Relations, for submitted questions.
Great. Thank you, Laura. I do have a number of questions that were submitted through the webcast. The first one, it will probably go to, Stéphane, and Brad, if you can also chime in. It is early days for measurements, but how would you quantify the value you are getting from your AI investment, and over what time horizon do you expect to see the return on your AI investments? Stéphane, maybe you can start?
Sure. Thanks, Lavina. So I think speed, which we had a couple of good questions around speed. Speed is really our obsession, and we understand that to create return, time is a very important element. And as Dave mentioned, you know, we're obsessed about learning on the platform, on early research.... And so if we can just accelerate the learning by having shorter cycle time, that's extremely valuable to us. Of course, time to market in terms of creating value is another very important one that we had questions on. But I think it's true everywhere. It's true in manufacturing, it's true in commercial, it's true in SG&A. And how you're able to also enable the scale-up of a company.
You know, we had a quarterly call last week, and we talked about how we believe next year and in 2025, we can have, you know, flat to down SG&A, where we're gonna be launching RSV, and then flu, and then COVID + flu, in 2025. And there's a lot of those learnings and the scalability that we're building as we speak, that we feel very confident will allow us to launch our products very well, gain good share, like we just showed we're able to gain share, you know, on, on COVID, and not increase the cost structure, and that's an important element. And how we just make decisions faster as well, which is very hard to quantify in terms of precise dollar numbers. But, you know, just another example of how it just change how we work.
You know, the other day I had to prepare a speech for a conference in Asia. In the olden days, I would have spoken to the comms team, and I would have to describe what I wanted to talk about, and then they would have written a speech, and then come back to me, and a couple iteration, and then we have a recording. Where I literally, you know, opened Microsoft Word, I typed in the prompt in Copilot. You know, I had a 10-minute slot, so I—of course, in one of the thing of a prompt was in 10 minutes. I had a proposal based on all the themes I had in the Copilot prompts, and then I read it.
I changed literally three things, and I started recording by myself, and then I sent it on our cloud to the Asia team, who couldn't believe literally it took me, like, 20 minutes to do the whole thing, and I didn't use anybody to help me do that speech. So just think about the productivity gains, because exactly I have a comms team working on, you know, the COVID, you know, winter season. And so this you start to see all over board, and I don't know how to quantify those things, but the piece for me is on the quantify, is when we on, on speed or we to scale with flat-ish headcount, as, you know, Brad was talking about, and then really show on cost.
Yeah, I mean, the only thing I'll add to that, Stéphane, is that early on we were taking in use cases, and we used to measure our success, and, oh, we have 125 use cases, and then the next week we had 350 use cases, and then next week after that, 650 use cases. And so we stopped counting use cases because we've democratized AI with the launch of GenAI within Moderna by the use of mChat. And so there's no longer this use case count. It's really becoming part of how we work every day in using the tools and capabilities, and that work was led by Brice and team in terms of really driving that transformation that we brought to our employee base.
Just last week we presented an award, for instance, on being bold and how they're utilizing AI, and we had two individuals who optimized the process that made them able to deliver three people worth of work in two weeks, to which they weren't able to do before. We're seeing optimization of our resources, and we're seeing avoidable effort being removed out of our system, and that lets our workforce focus on what's most important, rather than mundane task that we're eliminating, which allows us to become more efficient and have greater efficacy of what we're doing, so.
Yeah, and maybe to go back to another example, just to give you a sense of how profound this is becoming at the company. Brad and I, with Tracey, you know, head of HR, and Jamey, our CFO, we're doing a quarterly review of a quality organization a couple of weeks ago. We had a demo from a woman in, actually our Basel office in Europe, who had used our mChat to teach her how to write a macro in Excel. She had never written an Excel macro in her life, and she taught herself how just using mChat, and then she developed the whole macro, and she copied and pasted it into Excel.
Just what is very interesting, you start to see employees that have never coded in their life starting to create tools, to enable them to remove repetitive work from their workload and then share it with colleagues, which is very interesting, the example of that, that woman. That tool is now used across Moderna quality worldwide.
Yeah.
Well, thank you for those responses. And another question coming in, relates to AI and how our AI work is truly differentiated versus competitors', and what allows us to do it successfully where our competitors may not be able to? So I'll, I'll open that up to the panel. I'm sure everyone has an answer to this.
Yeah.
Brad, do you want to kick off?
Sure. I'll start there. Like, as Dave mentioned, like earlier, like, we can't really speak to our competitors as to what they're up to, but we can talk about what we're doing and how that's being differentiated. And, you know, I think, we're very fortunate that we've been a cloud-first company from the start. We have very little legacy infrastructure, and so by being on a modern tech stack, we're playing in an environment where innovation is happening in this space. And so, we live in AWS, that's our cloud platform. AWS is innovating in AI and developing large language models that we can leverage, and our data is there, protected and safe. And so, we're very fortunate to not have legacy software within our company that is holding us back.
We've built distributed systems, so we don't have big, monolithic sets of software that are hard for us to be nimble and move. We take pieces of stuff we've built, we put it together, and it enables us to accelerate the deliverables. I think this is a big differentiator for us in terms of our speed to market, that we're in the cloud, and how we built things in a modern way enables us to go fast and pivot very quickly.
Yeah, maybe just to add to Brad's piece, going back to my intro that I just did, you know, a little while ago, is a few things. First, we did machine learning for many, many years. This is not something that we started last year when all of us played with ChatGPT over Christmas. That's something that, as Dave explained, and you saw the timeline, that we have actually been doing for a while because we knew that getting scientific insights of very large data sets that we had was not possible to do except if you go into large language models and for the team under Dave's leadership had been building.
And then I think the second piece is it was very clear to us, several years ago, you might remember if you go back into quarterly calls and, on the presentation, we talked about the AI Academy, that Dave was a key part of setting up, with the HR, training team, it was really clear that one of the biggest issue we're gonna have as a company, like most companies, I believe, was a change management problem. Because you had a lot of more younger people that because of what they study in school, were maybe more used to these type of tools, and I'm thinking people outside of data science and computer science, obviously.
But what was very clear to me is that the top 200 people at the company who drive a lot of the capital allocation decision and the culture, many of them had no idea what machine learning was. And so that's how we developed this program, to be able to get employees from the senior leadership down to frontline employees, to go and to just educate themselves on what those tools are. And the example I always used that was really telling in our desire to start this academy and to really invest in the change management process, not only on the technology, was the analogy that I described many times internally, which is, you know, I started working in 1995, and my first job was in Japan.
As you know, in the 90s, the personal computer were getting on everybody's desk. And I was shocked because, again, coming out of college, I used computer all the time, but my boss was reading his email by his secretary printing his email on paper. He will read the paper email, then she'll come to his office, he will dictate to her, she'll take stenography, she will walk back to her office, and she will type the email on her desktop. And of course, I was crazy. But that's the only thing he knew. All his career, he never had a laptop or desktop on his desk. He never took the time. I'm sure, like every human being, he was scared of things he didn't understand and thought he could run the business like he used to run the business.
That analogy really stuck with me very profoundly because I think in most companies in the world, the top 200, 300 leaders who really run the company, make most of the decisions, have the same issues today with machine learning. What we really try to do at Moderna, and I think mChat and the access to Copilot from Microsoft has helped tremendously, is people are able to try. We have a lot of workshops that I need to give, Brice a lot of kudos for doing, he's all over the company all the time. The most powerful things we have is when you have, like we had recently in research, in the platform team, where basically scientists showing their colleagues how they're using it.
So it's not only the digital team sharing the tools and explaining the tools, but it's very powerful when you start to have colleagues using it and realizing that your colleagues are able to be much more productive, have many more ideas, and then you realize that you're not gonna break it. I think it's like with every technology, whether it's the iPhone or the internet when it came, we had the same change management issue, where there was reverse mentoring. If you remember, like, younger people were coaching CEOs how to use the internet, which seems ridiculous today, but that was the case in, you know, the late 90s. And so what we're really pushing hard with the company for the academy, for what Brice is doing and a lot of activities, is talking about it.
Every town hall I go to at the company, we talk about AI, we look at the models. Every business review I do, there needs to be an AI agenda where the teams review what they are doing. And every time we go, we're having so much fun with Brad because we're seeing so many ideas that then we try to replicate across the business.
The thing I would add, so the cultural piece is so important and kind of a foundational element, but then the technology part matters a tremendous amount as well. And, you know, anyone can just hire a bunch of data scientists, but they need the tools, and they need the data to actually be successful and useful. And so we've invested for many years now in digitizing processes such that we have structured data available and invested in data warehouse and data lakes to collect that data, to make it available so that data scientists can actually do that work instead of just managing and munging data all the time. And then we put in place, as Brad talked about at the top, a lot of infrastructure for deploying those models.
So if we have an algorithm, we can write code, we can get that into production with an API ready in minutes. Like, that's not an exaggeration. The platform we've built on the cloud with the scaling lets those things run at scale. I just checked this morning, and we've run over 13 million inferences on that platform to date. That just shows the level of scale that we're trying to operate here. This isn't a onesie, twosie playing around with AI. We've industrialized the way that we deliver it here.
... Great. Thank you very much. Picking up on what you just spoke to, Dave, in terms of how much we've internalized and made investments in AI, there is a question coming in regarding partnerships, and the question is: how are you planning to prioritize partnerships versus in-house investment into AI tools?
Maybe I can start, and then Brad can talk kind of broader about the AI partnerships. I'd say for us, we always think about, you know, what are we uniquely capable of? What unique data sets do we have? What are the things that are critically important to us? We evaluate in the context of that. We're not too proud to say that we have to do everything ourselves. We're really in this for the impact on patients' lives and the fastest way that we can deliver the value to our pipeline. So Brad, why don't you?
Yeah, actually, Brice, you've been managing our partnerships with all of our key technology friends in the industry. It'd be great to hear of the update from you where we're at with our partners.
Yeah, of course. So I want to highlight and piggyback on what was said before, that the momentum effect is huge because the moment when you have—we currently have more than half of the company actively using AI through mChat and various other ways, and that triggers a lot of ideas for them. For instance, if I take an example of our standard operating procedures, we have several thousands of them. It's a humongous corpus of data that has to be dealt with, and it's mainly language, which was hard to leverage with ML, as we stated before.
So very soon we found that there was a way to look at how policies trickle down through the standard operating procedures and help the people who have this huge amount of work to do to leverage AI, to have opinions, and have an insight on how to update standard operating procedures. And then we started to see the coherence among them, because there's a lot of correlations between the various procedures, and then looking at the deviations, and then idea leading to idea, leading to idea, how we could use, for instance, deviations to anticipate what might go wrong with a batch and do right first time, which is the holy grail of manufacturing, and it's going to be very important when you do individualized therapies, for instance. And so every vendor out there sees this, knows this, understands this.
They speak with us, they realize how far ahead we're moving. We're moving not only because we are, we are like specifically, you know, geniuses or anything, but because of the momentum effect of everyone using this in every part of the company. So they proactively came to us very early on. That was Microsoft with the Copilot early adopter program. That was OpenAI with the early pilot. We also had one from Tableau GPT. So almost every tech provider out there has us on their radar as within the field of life science or pharma at large, one of the best trial grounds for their technology.
This gives us, in return, the opportunity, because we have these 2,000 enthusiasts on the Teams channel and a forum who continuously speak about this, to welcome the new technology, to discuss them, to explore them, and then to try to make the best decision in a very fast-moving environment. So the combination of our scale within the size of the company, the fact that everyone knows about this and learned from us as a user how they can improve their products, gives us this slight edge, this slight looking ahead, you know, the around the corner to skate where the puck will be and not where the puck is right now, as I explained a little bit earlier. So the partnerships are first driven by the tech vendors who come to us.
It gives us a lot of optionality, which is really fantastic. Right now, we have invested a lot in the relationship with OpenAI because they are a massive player, and they made some huge announcements again on Monday about where they are going. But we're also in partnership with Google and their Vertex platform to work on the large amount of data and how we do the data permissioning, which is so important. Who has access to the data? And when you work it through AI, where does it go? We are working with Bedrock and Amazon because a lot of our data is there, and we're also looking at startups. We have a startup community that is very thriving at the moment, and we stay in contact with them.
We went to a few startup incubators in the last week to see what's happening out there in the startup community, and they, of course, would love nothing more than to have Moderna try one of their products. So a very thriving community, both with the big tech companies and the small startups, and we are very lucky to be part of every pilot and early adopter program out there, because they know us to be the right company to go to when they have some new technology to experiment.
Great. Thank you for all that color. That was very good. A question for you, Brice. There was a lot of emphasis early on that Stéphane talked to in terms of change management and transformation. A question is coming in regarding: what are the challenges that you've faced in trying to move a whole organization to adopting these new AI tools?
Yes. So, I think we are not very familiar with these kind of topics because I also, like Stéphane, started my career with the computers coming, and my very first job I had, I was given a typewriter, even though I actually had done my studies with a computer. So, I remember my boss, same thing as Stéphane, not wanting to use one, and not because he was afraid, but because he was dismissive, because he felt that this was something for secretaries, because they use typewriter, and this is a better typewriter with a big screen. "But why should I use a typewriter? I'm not a secretary." So there was a notion of status there that hurt the whole migration to computing.
And so as Stéphane mentioned, having the leaders support this and champion this actively, not just accept it, but really embrace it and drive the teams, is huge in a company like ours. And we wanted to make sure very early on that no one had the wrong impression—that this was a status symbol, and that, you know, they, they would be like above, if you will, using something like generative AI, because maybe their children were using it to cheat on their homework or, or to do some images on Discord channels or things like this.
I think we had a great audience to begin with, but we very quickly, with that prompt contest that Brad highlighted, we had 180 concrete submissions from the various teams on concrete things that they were doing, and this is back in June and July, with generative AI on top of machine learning, which powers it, of course. Suddenly we were struck that we just had, in a matter of days, across every team of the company, a variety of examples of how this was making their lives concretely better.
Immediately, the leadership took on and realized that this was something they needed to use for their own leadership, so that they write, you know, messages to their teams, and they have them proofread, their monthly business reviews, quarterly business reviews, and they, they get coaching back from the model on, on, on ways of thinking, organizing their thoughts. I think this is the greatest obstacle, is the moment, that moment where leaders think this is not for them, that sometimes kills the momentum in companies, and we've never had that moment. I want to salute also Stéphane, because on day one, he was on board. We recorded together a podcast for the company, and that was back in May, Stéphane, I think, right when mChat went out.
When the CEO of the company, with someone like me in charge of transformation, record a podcast for everyone to hear, the leaders get the message, they come on board, and then we have this fantastic momentum, both top-down and bottom-up, and that's how we solved this.
I think the peer-to-peer learning is a very important one as well, so absolutely, it-- having lived it here at Moderna so far. So a few more questions on my side before I hand it back to Laura, to see if there are any questions coming in on the audio lines. One on INT again: "How are you training the INT algorithm to improve? Are there outcomes and patient responses that are fed back or integrated into the antigen selection process?" So Dave, I think that's one for you.
Yeah. I don't know how much detail I want to go into the specifics of the algorithm beyond what we've shared there. But I would say from a high level, it's certainly something we're very interested in looking at. What are the actual efficacy responses, which groups are not responding, and why? And then feeding that back into the development process. But we're not at a stage where we're talking about this being some continuous improvement, right? This is a big regulatory question we have to work with our partners in the regulatory agencies. So talk about how we can update that model in a way that's safe and effective and statistically valid for the work that we're doing.
But we're not done, we're not satisfied with where the algorithm is, and we will continue to work and improve them.
Okay, great. One additional question, more on the overall policy in AI: "There's a lot of new tools that are being applied to various business aspects. Does Moderna have a responsible AI policy?
I'll take that one, maybe because we, we just published and I signed the, our Moderna AI Code of Conduct. We've been working on this for the past six months, and really, we've been working on this ever since we existed, to, to make responsible use of AI. We have three core principles, which we have published now. And everyone, by the way, everyone in the champions team, in the teams, in the leadership, has worked on this. It was a huge collective effort to gather together and think, if we are this leader, this forefront player in AI, and that the AI providers come to us to understand the use cases of their technology, we also need to, to lead in how we want to regulate the use cases and to, to make sure we do the right thing.
So we have three core principles: It's integrity, quality, respect. Integrity means we own this. We don't blame others, or the sources, or the models, or whatever. No. We do AI, we leverage technology, but this is ours to own, and we have integrity in how we own this, and we take full responsibility for everything that we do in AI. And we make sure that we have people in charge and people who feel ownership of every way that AI is used in the company. Quality, of course, is that we want to make sure that at every step of the company, we use it with the best level of accuracy, with the best level of explainability, of transparency, of how did we get to that outcome with AI. It's getting harder and harder, especially with generative AI.
So it's a fascinating task to understand how the models work and to have the users understand how it works and what to expect. For instance, if you ask calculation to GenAI, it doesn't do calculation natively, it does predictions. If you say, "I have 50 countries, there are one city in each country, how many cities?" It's gonna say 40, because predicting 40. But if you go through a Python script, it's going to do the calculation. And having that understanding, that explainability of how this works, is it trying to guess or is it trying to calculate? And there are people who realize this, and that they, they drive this correctly, is huge. And the third is respect.
Is to make sure that we don't inherit biases, that we don't inherit ethical issues from the way the models were trained, that we can fine-tune them, that we can human curate them also, and that they are fair, that they're impartial to the users. And this is going to be all the more important, that we, we have a huge duty ahead of us with our medicines, to make them fair, to the people and the patients. And AI is a concrete component of this, which we are very, very intent on making it part of the fairness of the company to every possible patient in the world, regardless of geography or accessibility.
Yeah. And just to add on, you know, classic to Moderna, we learn as we go, and we implement best-of-breed. And so the inspiration of this policy was drawn from multiple sources, like the White House AI Bill of Rights, Microsoft's responsible AI standards, the acceptable AI use policies of other companies like Salesforce, and even some of our competitors, like Pfizer, Novartis, and others. And so, we've learned a lot, and we've implemented our version of what we think great looks like, so that we can ensure that we have a proper code of conduct in how we manage our AI platforms and the capabilities that come with it.
Great. Thank you both for that comprehensive response. I will now hand the call back over to Laura to see if there are additional questions on the audio line.
We currently have no hands raised.
Okay, great. So with that, Stéphane, can you make some closing remarks before we conclude the presentation? Thank you.
Sure. So thank you, Lavina, and thank you, team, for the great presentation, but more importantly, for the work you do every day to really transform the company and make AI just how we run the business. Thank you for all of you for joining us today. I hope you got a good sense of how committed we are to AI and how it's already changing the company. Many of you know our mindsets. There are two mindsets that are already relevant for machine learning and how this was just the right tool for us as a company, culture-wise. One is we obsess about learning, and two is we digitalize everything. And if you think about those two and this tool that's available, this is really an amazing opportunity for us to drive a company forward faster.
As we talked about, speed is really the number one value we see in this tool across the entire value chain and across all G&A function. Again, this is not only for science, but I think in finance, you know, in IR, and all across the company, enormous leverage. The scalability, as we have, you know, this gigantic pipeline, we have no intention to build a company with diseconomies of scales like we see in large companies. But we want to use this unique moment to push really hard, to use AI to allow us, as Brad discussed, to scale the company in term of product, without scaling the company in term of, of people. I mean, of course, the consequence of all that is cost.
We believe we're going to have a great cost leverage across the entire P&L of the company, thanks to, to machine learning. So with that, I wish you a good day. We look forward to sharing... There's been a lot of question about INT, but we look forward to sharing the INT data once we have them, but before the end of this year, for a full year, so I will data. So have a great day. Thank you.