Ladies and gentlemen, thank you for standing by. Welcome to the Moderna Manufacturing & Digital Day webcast call. At this time, all participants are in a listen-only mode. After the speaker presentation, there will be a question- and- answer session. To ask a question during the session, you will need to press star one on your telephone. If you require any further assistance, please press star zero. I would now like to hand the conference over to your speaker, Ms. Lavina Talukdar, Head of Investor Relations. Please go ahead.
Good afternoon, everyone, and welcome to Moderna's first Manufacturing & Digital Day presentation. Before I begin, I would like to remind everyone that this conference call will include forward-looking statements. Please see slide two of the accompanying presentation and our SEC filings for important risk factors that could cause our actual performance and results to differ materially from those expressed or implied in these forward-looking statements. We undertake no obligation to update or revise the information provided on this call as a result of new information or future results or developments. I will now turn the call over to Stéphane.
Thank you, Lavina, and good afternoon, everybody, or good evening if you're calling from Europe. Thank you for joining us. We're excited with my colleagues to welcome you today to this first Manufacturing & Digital Day. As you know, we started the company nine years ago asking the question, could we turn this molecule messenger RNA, the software of life, into a medicine? From our first day, we got excited about the idea that if we could do that, we could unleash a new class of medicines with very exciting features like very large product opportunity, the ability to make secreted protein, transmembrane, intracellular, potentially antibodies, potentially viral proteins, i.e., antigens, and also potentially the ability to combine molecules to make very exciting products for patients. Large product opportunity.
We also got very excited because mRNA is an information molecule, so we thought that we could have, over time, a higher probability of technical success of our drugs. We could accelerate research and development timelines because of the platform nature of what we do, if we were to invest in process development, in robotics and in digital. Over time, a greater capital efficiency, again, because of the platform. If you think about it and double click for a minute on those last three features, those just come down to the science. mRNA is an information molecule. You pick the protein you want to design in the coding region. It's the same formulation for every modality. That's what really got us excited of how do we build the company.
As you know, because of a large opportunity ahead of us, since the very first day, we have been very focused with the team and with the board to manage risk. Four risks that we talk about on a daily basis are technology risk, biology risk, execution risk, and financing risk. As many of you know, we have managed the technology and biology risk by building a portfolio, a portfolio across six different modalities. In each of those modalities to manage biology risk, we decide to not do only one medicine, but do several. By managing this technology and biology risk, what we basically needed to ensure is that we didn't increase execution risk, because it is one thing to take one drug at a clinic at the same time.
It is a different game to take 24 drugs in a clinic at the same time with a new technology as we are building the new company, because you're literally flying the plane as you are building it at the same time. It became very clear to us in the early days that we had no chance to build the company in the right way, focusing on quality, making sure we could run this clinical experiment across 24 drugs without building our own manufacturing facility. If you step back and you think about how we build the company, so the first phase, of course, was get to the clinic safely, invest in the science. As many of you know, we entered the clinic with our first flu vaccine in December 2015.
A few months after, when we got the human data of immunogenicity, and we showed that the antibody that we found in human was neutralizing, we started a discussion with our board and very quickly moved into designing a new facility, a facility of the future that will enable us through a full integration through raw materials, making mRNA, formulation, filling vials, quality control, all in one physical location. That we would be in charge of our destiny, making sure we have the right quality, that we could move very quickly, and we could control our cost, and also know-how. It was very clear to us because it's a new technology, and as many of you have observed, it is very, very difficult to build a biotech company around a new technology. The challenges around technical development, process development, manufacturing, quality, everything has to be invented.
When you do it through contract manufacturers, we believe at Moderna it is a very risky proposition, because the asymmetry of upside and downside between the contract manufacturers and the sponsors are just totally misaligned. We said we have to invest because we are playing a very long game. We have to invest and own our own site. Not only it has to be fully integrated, we said it has to be fully digital. We have to really leverage the ability to have access to information in real time so that we can execute properly and that we can learn very quickly. The next phase of the company was, of course, running those experiments in the clinic and figure out through clinical data where was the mRNA technology of Moderna working best.
As you know, we believe that in the last few months of 2019, we went through a very important inflection point in the company history. We were able to define two modalities that we view as core, where, in our opinion, the technology risk is off the table, and still exploring the modalities where we don't have enough clinical data yet to make such determination positively or negatively. The way we think about the company, as you can see on this slide, is basically we have two businesses now. We have a business on the left of core modalities where we believe the technology risk is off the table.
Where, as we communicated since the JP Morgan conference earlier this year, our strategy is to double down and to leverage the uniqueness of this information mRNA platform to scale the company very quickly, and to go very quickly from a drug concept around the meeting room in our scientist's mind, to a development candidate ready to go into GLP talks and into the clinic. This is around the infectious disease prophylactic vaccine, on the left in light blue, and the IV systemic therapeutics that you can see next to it. On the right side of the slide, the exploratory modalities are basically us waiting for clinical data to decide what we do with those four additional modalities.
As we said before, if we have positive clinical data and we think the technology risk is off the table for any of those four modalities, we will move them as core. Again, we will double down and triple down into many more medicines. If we have negative clinical readouts, we'll have to assess, do we think based on what we've learned, we have a way to recover that modality? If so, we might decide to go back to the drawing board and go back to the clinic with an improvement in the technology. As we said also, we'll be very disciplined with share of our capital. If we believe we've run fully the clinical experiment, and this is not a good use of our time and capital, we will basically shut it down.
That's exactly what we're going to do moving forward as a company. Two businesses on the left, acceleration, investment, focus on BLA, and on the right, being very disciplined and very patient for the clinical readout to tell us what to do next. That's exactly what we are entering as a company. This phase where we're focused on filing multiple BLAs, so that when we get to this point, we'll be able to scale the company, and we are very excited about the next leg on this journey. That's what I wanted to share with you for this introduction. What I would like to do is to review very briefly, the agenda for this afternoon, where my colleagues are going to lead the entire discussion. I'm going to hand over to Juan Andres. Juan runs our clinical, sorry, our technical operations and quality.
He's going to introduce himself in a minute. I've had the chance to work with Juan for now close to 18 years, that edges both of us. We'll do a small Q&A session on the topic that Juan will cover. We're going to move into the digital space. We are very fortunate to have Karim Lakhani from Harvard Business School joining us today, to be able to talk to us about some important learnings and frameworks around Competing in the Age of AI. Our colleague Marcello, who runs all the digital and operational excellence operation in the company, will share with you a little bit of what we are doing there and what we've been doing recently.
Then we'll be very happy to be joined by a member of Marcello's team, Dave Johnson, who is leading all of our AI and analytics across Moderna. After that, we'll do a short 50 minutes Q&A session before a couple thoughts for closing. With this, without any further ado, I'm turning over to Juan.
Good afternoon, good morning, good evening, wherever you are. My name is Juan Andres, and I lead, as Stéphane said, technical development, manufacturing, and quality for Moderna, where I have been almost for the last three years. I have 30 years of experience in the pharmaceutical industry. I started with Eli Lilly for 18 years in a number of different global roles, in a number of different countries, mostly in the technical area. 12 years in Novartis prior to Moderna, where in my last three jobs in Novartis were leading technical development, leading global quality, and leading manufacturing operations for all the different divisions at a global level. Here I am. We are in Moderna. We are a platform. We are a new platform, as Stéphane said, so many things to be learned.
Before I go there, let me give you a glimpse on how our process works. Our starting point is the DNA plasmid, and I would like you to think about the DNA plasmid as a template. That DNA template, together with nucleotides, with the right enzymes, and in the right buffer conditions, form mRNA. This process is done without live cells in an in vitro way. Once the mRNA is formed, we go to downstream purification, typical through chromatographic columns, in order to ensure that the final mRNA product has the right purity and quality conditions. The process does not stop there. Once you have the mRNA, we need to put it under or inside a number of different lipids. That is our lipid nanoparticle process that would enclose, cover that mRNA in order to make it our product that would be injected with our drug product.
Our drug product is an injectable form, so we need to go through the appropriate aseptic operations, and then finally our quality control before the product can be released. As I said, we are a new platform. Two things here. New means we have to do a lot of learning, and platform means we believe that once we know how to make one, we will know how to make many. These two equations work indistinctively. The more products you have, the more learning that you would have. The more learning that you acquire, the more learning that you will be able to apply to a number of different products. This applies into a number of different things. Let's go back into history. At the beginning, we had more questions than answers, and we had to go and answer one of them.
Can we reproduce the product? What is going to be the stability? Can we do this at GMP scale? We have solved most of these questions, but this is in the journey. Every time that we solved a question, we knew that it was not only for the manufacturing process. We needed to make sure that that new process would work in vivo. We all know that making and developing medicine is a team sport. It needs to be having a number of different units, a number of different disciplines, scientists, people with experience in order to make that happen. The last thing, we had many questions, we had a few answers, and we needed to have the patience, as Stéphane said, to go one question at a time and solve them. Where would we start?
The place where we started were with questions associated with what is the structure, what is the level of purity, what is the stability, how to measure things, what is the right size of the LNP for the different products that we are going to be taking into the clinic? What about stability? How to measure from a quality control the properties of all these attributes. The first thing that we covered was around quality. It is very difficult to improve anything that it is variable. We know variability is going to exist with us, especially in a new platform, but we focused on taming that variability. Quality needed to be first. I wanted to highlight this as a control chart, and the first thing that you have to do is get in control first.
Once you are in control and you are in good levels of variability, you can focus in other things such as speed, scale, cost. We knew these problems were engineering problems. We know that these are engineering problems that we will solve as they come, but we needed to focus quality first. This will become evident in a number of different examples that I will go through in a few minutes. Let's go back to Norwood, and let's start on how the process started at the beginning. Remember, plasmid, mRNA, LNP, fill/ finish, and the QC of it. We needed to travel to different parts of the world to make that process happen. From one continent to another, back, another facility, the other side of the U.S. Now we have Norwood. In Norwood, we have vertically integrated all these different parts.
We are capable of making plasmid, mRNA, LNP, and we also have a small fill/ finish operations where we can make aseptic drug product. We have the quality control laboratories and the QA release to be able to do that. Now, we knew because we were a platform that we wanted to do this many times, and we were going to be learning a lot. Our vision from the beginning was that we needed to digitally integrate everything that we did. We needed to go and be paperless. We needed to capture knowledge so that we could fast go from one place to the other. The design of digital and automation was an obsession from the beginning. Norwood is mostly a clinical development site. That is the vision that we had, and that is its main purpose.
However, we'd like to think of it as three engines within this site. The first one is preclinical. So the first, the preclinical engine basically has made, had a lot of automation for this one. So let me go one by one. Let me center your attention on preclinical. Preclinical makes batches that are on milligrams level. To this date, we have made over 23,000 batches of preclinical material. A scientist in the bench enters the code of mRNA, and it is sent electronically. The site at Norwood in the preclinical area receives that code, and within days or weeks, there is a vial back that arise to the patient to do the research or to do some of the development activities. That is the scale. The second scale is the one in the personalized vaccine unit.
You have in this area, we are making one batch per patient that we have in the clinic. It is an area that it is already in the clinical, it follows good manufacturing practices, and the scale is in the hundreds of milligrams. We have made over 90 manufacturing batches in that location. I will go, in one of the case studies that I will review later, is around this area. The third engine is the clinical engine. That is where scale-up starts, and one batch can be used for many patients. In this area, we have a scale that goes from grams, 5 g , to to-date, 75 g, that we could go even higher as our needs for a scale are required. I described the platform, I described Norwood and the different areas. What is what we have here?
We have similar processes, both preclinical, the personalized cancer vaccine unit with personalized medicine, as well as the clinical areas. They make plasmid, they make mRNA, they make LNPs, and they are made in a very similar way, but at a different scale. Similar processes. We also have here the cell-free processes that will enable to have less capital requirements. All of this will impact quality, speed, also our ability to scale, and the cost. Three case studies that or three examples that we have prepared for you today. These are examples that illustrate how much we've learned and we've been able to do in the last few years, but they are very recent. As such, they will also give you an illustration on how we intend to go in the future.
I always say your past and your present is the best indication of your future. Let me start with the first case study. The first one is the personalized cancer vaccine unit. This is one of our programs that we have in partnership with Merck, where the patient gets a sample of their product, and we need to go back and make a vaccine specifically for that patient. We would like to say needle- to- needle, around 40 days. At this moment in time, the program is in phase II, and it is up to 200 patients for proof of concept. These patients, these manufacturing batch patients, need to be done at Norwood. It is not only the speed at which we need to manufacture this clinical trial is designed to coincide with KEYTRUDA dosing regimen.
We need to also land the vaccine on a specific dose for KEYTRUDA. I believe it is the third dose. That is the place where we land our vaccine. The third consideration that I would bring to you is, as it is personalized medicine, you cannot scale up your product. You need to scale it out. We will always have to make, in personalized medicine, as many batches as patients you have. Let me illustrate the learnings that we have had with PCV to date, because our success rate since we have started has improved, our cycle time has been reduced, our yield has been improved, and as a consequence of all of that, our capacity has improved. We have made to date over 90 patient batches of personalized cancer vaccine. Granted, the scale is not as big as a big product.
When you make 90 products, in my experience, by the time you have made 90 products of a normal product that it is commercial, you achieve that scale in your second, third, or fourth year after being commercial. This is the part where we have gained a tremendous amount of experience. Remember, while the process is the same, every batch is different from the code, from the mRNA that we produce. It is very important to assess what is the quality and what is the learning that we are going to be getting with this. Let me start, as I did before, with quality. As you can see here in the slide, the first part, or the first batches that we made, our success rate was okay-ish. Was good, but it was okay. It was around 69%.
As we have been making more and more product, as you would expect as you get more experience, our success rate has improved significantly. You can see there how our experience has made us improve our quality. This is not the end. As we gain more experience, as we make more batches, this is going to improve. Next slide, please. In addition to that success rate, every mRNA is different, but it has the same specification. Think about lot one is one patient, lot two, all the way into lot 90. The specifications are the same. Even with a different mRNA, we were obsessed with having the same specifications, the same quality. They are landing in the same place. They follow exactly the same specifications. That will become important in a number of different things that I will highlight later. I also highlighted speed.
Our speed has improved, and since the beginning, we have gone faster, around 40% since 2018. Why is this important? Think about making many batches at the same time with a schedule in which you need to arrive at a certain point in time to the patient. If you want to make many, many patients at the same time, being shorter in your cycle time will allow you to overlap one manufacturing with the other. Therefore, you are going to be increasing your capacity and ensuring that you're going to have your delivery on time, the shorter you are. In this case, speed is important. Speed, quality, yield will give you cost reductions, and this has been our experience as well since we have arrived. We have had the cost reduction per batch as we have gained in experience. This is case study one.
Let me start with case study two. That is the cytomegalovirus vaccine. We call it CMV. This is a case that when I first saw this with experience that I had in manufacturing, I thought this was going to be science fiction. I was making many batches, many proteins before, and my first reaction was, "This cannot be done." How are you going to put six different mRNAs into the same product and basically inject it, have those six mRNAs arrive into the cell, transcribed into protein by the ribosomes, and five of those six proteins come together into a pentamer, and then another one forming the vaccine? We had, as you can imagine, many questions and things that we needed to resolve from a manufacturing point of view. Lots of challenges ahead of us.
We also knew that the CMV vaccine would be something that would have to scale to many patients, different than the previous case in which you had one patient at a time. In this case, we needed to scale up. We needed to do in a way that stability in refrigerated conditions rather than frozen conditions was going to be a good consideration. We needed to do a number of different things where our creativity coming from platform and technical development, and our ability to make product in Norwood would be very important. These things basically get together in the speed. One of the things that happened with CMV is within a year, we ended with a lyophilized form that basically increases our chances tremendously, is well-known, lyophilization, but it was the first product that we lyophilized in our platform.
We were successful taking this to phase II, where the product is right now. We are at phase II with a lyophilized form that we intend to take all the way into BLA. Okay. In this case, scale is important. Our scale at the beginning was 5 g, and we used it for CMV. 75 g was our next one, but we had never done this with CMV. We had done this with another program, with VEGF, for instance. We used the experience that we had in VEGF, scaling at the 75 g to export that knowledge and experience, our digital platform and our equipment conditions to basically take this to CMV. We were able to go faster. Next phase is going to scale.
What we are thinking at this moment in time is around 150 g, but it could be another scale. I like my chances. In addition to this, obviously as you scale, your cost reduces. That is a factor of your scale and your quality conditions. At this moment in time, our obsession is reduce variability, be able to scale, and then things will fall into place from the rest of the things. Next slide for CMV is, as I said, we have the lyophilized image that we intend for phase III. Norwood can produce the phase III material, and we have the potential to do the scale for a potential commercial launch, and we will do the fill finish at a CMO on this lyo. The image that you have on the screen are actual, real lyo products that we have on phase II. This is CMV lyo product.
This concludes the case number two. Let me take you to our case study number three, coronavirus. Folks, this was unplanned. We didn't expect this. It landed in our lab at the beginning of January, and it required a number of different things that we have learned throughout the years. It needed a convergence of a number of different disciplines, scientists, all the things that we have learned together, in addition to good doses of boldness and a huge amount of collaboration together. We also used all the different parts of the engines that I have talked to you before. The preclinical, the personalized vaccine unit, and a number of different things going forward, we are going to be using all parts of Norwood in this place. We didn't think ever about using the personalized vaccine unit for something different than PCV.
When we saw the challenge of going really fast, we said, "Why don't we use the learning that PCV created on us on the same platform, and we use this to attempt to go to phase I for the coronavirus vaccine very quickly?" Let me take you through what has happened in the last few weeks here. Three days, January 13, February 7, February 23. It started on January 13. We selected, or we were given the amino acid sequence that we would need to use to design the plasmid template that would allow us to make mRNA. We did that in a few days. We have the amino acid sequence. By day two, four, we already had the digital sequence engineering and the DNA design.
Between days five and 17, we had to do what normally takes weeks or months, which is get all the documentation together and all the things that you require in order to make a clinical batch. Due to the fact that we are paperless, that we have the platform that we have, basically, we could do that at a very fast speed. On day 17, we were ready to make the plasmid, make the mRNA, and make the LNP at a clinical level. In parallel, we had used already our preclinical engine to make the mRNA that preclinically was going to be tested in vitro, and we sent several copies of that to NIH as well as to our scientist inside, together with a number of backups. The preclinical engine worked to perfection as well to supply that preclinical material.
On February 7th, we had made our vaccine. The vaccine was done, and it was ready to be tested. All the QC was done in a few days rather than in weeks, but we needed to wait two weeks for the sterility test. We cannot make bugs grow faster, and the test takes two weeks. We needed to take two weeks and wait until the sterility test was done. On February 23rd, we shipped to NIH the product that was meeting the specifications. At this moment in time, the IND is open. There was a huge amount of motivation in the system. This is how we felt in the company. Each one of us. We channeled a huge amount of motivation. People were talking to their families. I have not seen the level of energy anywhere before like this, and that was very motivating.
Hopefully, I've been able to convey the three cases that can tell you how all the learning, the platform, the integration in Norwood, have brought to existing examples, and you can make the connection of what is going to happen in the future. We can make the quality, speed, and scale, and cost that we are going to need as a company across the different products. As I said before, we can do this. I like our chances.
Thank you, Juan. With that, operator, we'll go into the Q&A session for manufacturing.
Thank you. As a reminder, to ask a question, you will need to press star one on your telephone. We ask that you please limit yourself to one question and one follow-up question. You may return to the queue. To withdraw your question, press the pound key. Please stand by while we compile the Q&A roster. Our first question comes from Yasmeen Rahimi with ROTH Capital.
Hi, team. Thank you for taking my questions. I have two questions for you. The first one is, can you walk us through what the purification cut-offs are as we go from preclinical batch to personalized cancer vaccine to clinical batch? What are other selection criteria that make clinical batch more unique? The second question is, can you walk us through how mRNA is combined with LNPs to yield the final drug product? What purification process ensures that we get a homogeneous drug product? Thank you for taking our questions.
Well, the first one, the purification process obviously is different at different scales, and whether you are doing this in a preclinical setting or whether you are doing this in a clinical manufacturing with GMPs. The process is very similar. What I would say is mRNA is a large molecule, and it follows traditional purification steps, chromatographic columns to be able to do that like you do in normal large molecules. For the mRNA to LNP is basically a nanoprecipitation process together with the mRNA. You take the mRNA in an aqueous setting. You take the lipids into alcoholic, where the lipids are in solution, and you bring them together under normal conditions. Through a process with the right sizing to make sure that we end up with the right quality.
Normal scale-up processes are followed in terms of different sizes of the purification or sizing equipment, to end up or to be able to process at the scale that is needed.
Thank you.
Thank you. Again, ladies and gentlemen, if you have a question, please press star then one. Our next question will come from Matthew Harrison with Morgan Stanley.
Great. Juan, thanks for taking the time for this. We appreciate it. I guess what I wanted to ask you is, can you just broadly talk about the scale that Norwood was built for? I think you've addressed that in terms of clinical-stage batches and what you've built there in terms of what you've learned. Is this something that you can scale easily into a larger facility or are there additional steps or things that you would have to work out if you had to scale to a commercial facility? Thanks.
Thank you, Matthew, for the question. I think that you're right. We need to scale up. Our learning in scale has gone faster than we anticipated. I think we are at a scale level that in our ability to go and learn from one product to the other, and the scale that we have achieved is good. As we go into commercial, the scale at which the site is designed will allow us to go into first commercial scale. The processes to go into a bigger facility with bigger commercial things is not going to be different than the scale that we have done from milligrams to grams to hundreds of grams. At this moment in time, we expect no issues in doing the normal scale as you would have. This is less complicated from that point of view than other types of scale-ups.
Great. Thanks very much.
Thank you.
Thank you.
Speakers, I'm showing no further questions at this time. I'll turn the call back over to you.
Great. Thank you. It is now my pleasure to move on into the digital portion of the presentation. I'd like to invite Karim Lakhani, professor at Harvard Business School, to kick it off with his talk on Competing in the Age of AI.
Thank you, Lavina. It's a pleasure to be here with you. I'm going to take some time to sort of walk you through a perspective that I've been developing with Marco Iansiti, also a colleague of mine at the Harvard Business School, on how a new breed of company is emerging that is using data analytics and AI at its core, and that these companies compete in a very different way, they operate in a very different way, and then subsequently also have different types of market perspectives and valuations. This work emerged out of about seven years of research that I've been doing with Marco as we were looking at how the dynamics of the software industry, what was happening in the software industry around both ecosystems and platforms, was basically spilling over into the rest of the economy.
As we began our research, what we started to see around 2015 and 2016 was that this was not just a platform story or a business model story, but really a story of operating models and how changes in the operating model of a company imbued with data and AI would make a huge difference. I'm going to walk you through that thinking, and of course, we'll have some time for Q&A as well. The first thing I want to sort of show you is this image, this painting, which is known as The Next Rembrandt. This is a digital rendering done by AI of a Rembrandt portrait. The amazing thing about this portrait is that this emerged out of sort of mere mortals working in Europe, looking at trying to imagine what Rembrandt would do as his next portrait 400 years later.
What they did is they scanned a bunch of images of Rembrandt paintings. They used some statistical algorithms and then literally generated this painting by itself. Do we have the video? Great. Let me show you how this came about as a way to sort of ground you as to why we think this revolution is going on.
One of his great achievements, one of Rembrandt's great achievements, was to portray human emotions in a much more convincing way than artists had before him, and in many ways, for all time. At ING, we believe in the power of innovation, what it can mean to people. We want to bring this innovative spirit to our sponsorship of Dutch art and culture. We knew that for this challenge, we needed to team up with experts from various fields to make this come to life. We're using a lot of data to improve business life, but we haven't been using data that much in a way that touches the human soul. You could say that we use technology and data like Rembrandt used his paints and his brushes to create something new.
The first step was to study the works of Rembrandt in order to create an extensive database. We gathered the data from his collection of paintings from many different sources, including 3D scans and upscaled images using a deep learning algorithm. Because a significant percentage of Rembrandt paintings were portraits, we analyzed the demography of the faces in these paintings, looking at factors like gender, age, and head direction. The data led us to the conclusion that the subject should be a portrait of a Caucasian male with facial hair between 30 and 40 years old, in dark clothing with a collar, wearing a hat and facing to the right. From there, we started to extract features only with faces that were related to that specific profile.
We had to create a whole painting from just data, and we used statistical analysis and various algorithms to extract the features that make Rembrandt. We took parts of the face and we started to compare them. Based on this, we were able to create a typical Rembrandt eye or nose or mouth or ear. After generating the features, we were focusing on the face proportions. We used an algorithm that can detect over 60 points in a painting. We were able to align the faces and to estimate the distance between the eyes, the nose and the mouth and the ears.
A painting is not a 2D picture. It's 3D. You can see the canvas, you can see the brushes, and that's what makes the painting come alive. A height map is essential to make the painting a painting. We incorporated the height map into the painting and printed on a 3D printer that uses a special paint-based UV ink. It printed many layers, one on top of the other, which resulted in the height and texture of the final painting. It's sometimes a magical moment to see a painting for the first time.
Even if it's computer-generated, for me, it is something special. I would have believed if I would saw it in a museum, that it would have been a real Rembrandt, just one I haven't seen before. It will be interesting to see Rembrandt looking at it. He will be happy that there are people trying to understand him and trying to create something out of that. I think he will be happy. The Next Rembrandt makes you think about where innovation can take us. What's next?
Great. The point of that video is to show you how something magical can come out with a question in mind, then real emphasis on gathering data, doing the analysis, and then using pretty much off-the-shelf algorithms to be able to pull this off. For me, the big realization, and for Marco and I, the big realization was that the ways we think about AI in the popular press are really geared towards what we think about as a strong AI, the Star Trek computer. That you can ask any question and it gives you any answer. That's known in the computer science world as strong AI. Weak AI, which is a definition that's also in computer science, really talks about any activity that computers are able to perform that humans once performed. That's known as weak AI.
What's important about this is that the algorithms that we saw the folks at The Next Rembrandt do are geared towards very specific activities and are able to just generate the next Rembrandt. They're not going to generate the next Renoir. For that, you will have to go back and redo all the data collection and so on and so forth. The notion is, can we industrialize this process in a way so we can do this at scale? The weak AI goes a long way today, and that's the basis of how to start to think about how deployment of weak AI algorithms can actually give us tremendous benefit. One of my favorite examples of this comes from China, JD Digits. They were asked by the Chinese government to come up with products and services that can help the Chinese agricultural economy.
One of the areas they went after was insurance for farmers, and they were focused in on thinking about how do we insure livestock, especially pigs. What they realized is that a farmer may not be able to insure his entire livestock, and they might want just 100 pigs instead of 500 pigs to be insured. This becomes a problem because how do you know which pig has been insured? They went ahead and they used facial recognition for pigs to be able to identify which pigs have actually been insured, and that enabled them to then generate a whole range of insurance and financial services products for farmers in China because they deployed off-the-shelf algorithms with a new problem, got this data, and got going.
What's interesting to me about this example is that JD Digits is not going to launch a pig counting or a pig facial recognition software and get rich off that. What they can do is they can take these algorithms and deploy them in their processes and create brand-new services and products for their customers, the Chinese farmers. Now, you can imagine what else you can do. Once you can do facial recognition, you can also keep track of health of pigs, you can keep track of what they're eating and change diet, and all of those can then become products and services that you can then go to the Chinese farmers with and enable them to succeed. Here's an example again of a brand-new type of a business being generated based on attention to data and taking algorithms to go and solve these problems.
Let me also show you one more example of this as well. The other example I'm going to show you is something that I recently published in JAMA Oncology, on an AI system that we developed for radiation treatment planning. This algorithm basically came about through a crowdsourcing contest for algorithms that we launched last year. In the space of 10 weeks and about $50,000 in prize money, we were able to be as good as an average radiation oncologist at Harvard, but also importantly, better than any commercial software that was already out there. The thing to note is, I'm one of the co-authors on this paper. The team, basically the first author and the last author are radiation oncologists. The rest are physicists and data scientists with zero background in biology, being able to, again, attack this problem from a data-centric perspective.
That allows us to then go at this and within, again, in a matter of weeks, develop algos that are solving a critical problem in radiation treatment planning. The point again here being that these algos are available at scale. Let's now think about this from a much broader perspective, which is what happens when these algorithms are deployed at scale, and how this, in fact, can start to change the nature of the firms. My favorite example here is Ant Financial. Ant Financial, as some of you may know, is a company that has spun out of Alibaba. Initially, it started as a escrow service for payments so that you could build trust between consumers and merchants in the Chinese marketplace.
Since 2013, they became independently spun off, they have grown massively to the point now where they serve 1.2 billion consumers on their platform, and they do this with 10,000 people. If you think about the leverage that they're getting, it's pretty incredible. Ant Financial has not just a payments platform, just like PayPal, but they also have, as you see, other products and services based off on this core platform. They also have Ant Fortune, which is the largest money market fund. They have MYbank, which is their banking service provider. They have a cloud service system, and so on and so forth. The thing to note about some of these capabilities is that MYbank, for example, has a 3:1:0 objective.
3:1:0 says that if you apply for a loan for the bank, it takes you three minutes to apply, one second to approve, and zero human intervention. Right? Compare that to your experiences with banks in the U.S., very different scale. The other cool example I found is that recently they launched in China, mobile phone insurance plans geared towards young women who wear tight jeans. You go, "What? How were they able to do that?" Well, they had enough data on the consumers to note, A, shopping habits, and B, the fact that young women who were buying tight jeans were breaking their phones. They could then use that data to come up with a brand-new product and launch that in their marketplace as a way to say what they can now do with the levels of data that they have.
Ant itself has scaled rapidly. They're currently worth about $150 billion in the private market. They're approaching the scale of the U.S. banks today. Again, the joke I make is, they have 10,000 employees serving 1.2 billion customers. My bank here in Boston would probably need 1.2 billion employees to serve 1.2 billion customers, just given how manual their processes are. Just to put this into a perspective, we want to think about companies like Ant Financial really from an abstract point of view of business models and operating models. Business models are really about your strategy, how you create value, and how you capture value. Operating models are really about how you think about how you achieve scale, how you serve more and more customers. Scope, how you in fact serve them with many different products.
Learning, which is how you innovate and improve on that. The operating model is the value delivery mechanism for your business model, and they go hand in hand. The perspective operating models has been around for quite some time. For example, a traditional operating model that might be at a Ford or Sears or a Toyota is set up in a way such that we are siloed. We're siloed in our data, in our infrastructure, in our capabilities, right? This siloed perspective is very useful because it allows us to focus and deliver value quickly to our users and to our customers. This also has its limits. The limits are that basically over time, we get diminishing returns. These operating models peter out because of complexity in the organization, administrative costs, inertia, that we basically tend to peter out with these systems.
What we see happen instead with a company like Ant and other digitally native firms, is that they are set up in a way to drive increasing returns through scale, scope, and learning by incorporating data and analytics at their core. The core operating model is set up in a way so that the different parts of the platform interact together through their systems, and as they get more data, they get better algorithms. As they get better algorithms, they get better services. As they get better services, they get more usage, and so on and so forth. The thing to note, though, is that what this is telling us is that AI analytics and data are now basically relieving the human-centered bottlenecks that have existed for years on end inside of most organizations.
As Satya Nadella has said, AI is sort of in many ways is the new runtime that shapes all we do in most organizations going forward. One of the concepts we talk about in the book quite a bit is that this operating model at Ant is enabled by an AI factory. The AI factory does three things and does three things at scale. The AI factory produces predictions about some future event or some future customer need or some future activity. The AI factory does classification, so it tells you patterns in data and tells you how those patterns are being achieved. Third, it drives automation. Those are the three things that are the core of any AI factory. The big thing, though, in the AI factory is building your data pipeline.
I can't tell you how many companies fail at not being able to build a data pipeline that can actually inject data from all different sources, normalize it, and put it in place so that we can derive algorithmic predictions as well as have an infrastructure that can actually make this happen. One thing to note here is that the AI factory that you see here is the same, whether that's at Ant Financial or if McDonald's built their own AI factory, right? The hamburger factory is very different from a finance factory, right? The data factory, the AI factory underneath, is just going to be looking the same. That also drives convergence with your operating model. What happens in this kind of an operating model core is that the data is shared across the entire enterprise.
Software components and libraries and APIs drive much of the interactions across all the systems, and then applications sit on top to be able to take advantage of this going forward. Now, the result of this AI factory is that we have two different things going on. Companies like a Google, like an Amazon, like a Facebook, like a Alibaba, have network effects. As the number of users increase in their platforms, the value of the inherent service itself increases, and that we've seen this in many consumer products. Also interestingly, as they get this usage data, and as they get more and more of this data, they can sharpen this value curve even more because of AI. We see both of these two things happening at scale in these companies.
It's no surprise to us today to think about this in the sense of why we see companies like a Microsoft or a Google or Facebook leading the race in AI research as well, because they're faced with immense amounts of data, and they've figured out how to deploy AI to capture even more and more value. What happens when this traditional model of diminishing returns meets the model of increasing returns? Well, we get a collision. What you're seeing is this collision is around both business models, but really around operating models. Thinking about this is very important because what happens is that the traditional business delivers value early, right? Keeps going, and it peters out. In the digital business, you actually take a while for you to grow the network effects and the learning effects, so that takes a while to grow.
A traditional business looking at the digital business will say, "There's no value. I'm not going to invest and change my processes to set up this way." As we see in the marketplace today, these collisions are happening left, right, and center. You think about Marriott versus Airbnb, the auto companies versus Waymo and what Waymo is doing, right, or b anks versus Ant Financial. These are collisions of operating models, companies that are set up in a very different way to drive scale, scope, and learning across enterprises and to supercharge their business models. I want to sort of zoom out and tell you what we think in the book as new rule set that has emerged for companies. The first one really relates to universality, which is universality capabilities.
What we see happening is from a recent sort of software eating the world to actually digital and AI becoming the core of most organizations, right? That's going to be the core of most organizations. Whether you be a platform or not, you will need to be able to plug into platforms and be able to play in this way. What we see is this universal capability around the fact that the digital operating model, when it collides with the traditional operating model, will lead to a collision and in fact, a separation in value creation and value capture for the digital operating model. The scale up times take some time, right? We have to be cognizant of that and figure out how that comes through. Importantly, this skill set is universal, right?
We need this skill set in all of our companies going forward, right? That the most product companies will have to also build a skill set inside of their own organizations. The next thing we see is that there is a ton of opportunity, a ton of opportunity everywhere, as you look through that. What we see is that the model here is to take human labor and management off the critical path, right? We develop rules for decision making, and we develop algorithms to enable us to drive those rules, right? We actually, in fact, have humans as the designers and operators of the algorithms of the factory, but not as the factory workers. The factory workers are actually the algorithms themselves. The application space for all of this is immense and keeps growing.
That's why we sort of see separately all this hype and boom in AI. This hype and boom in AI has come about because, A, the cost of generating algorithms have dropped. You need proprietary data, so you definitely need that. Everybody sees opportunity into how we can change operating models to succeed in this way. The next thing to also note is that we also need new strategies, right? As I sort of showed you that collision curve, we actually have to become smart about thinking about how do platforms and AI actually allow us to change the shape of our value curves, right. The much-maligned WeWork example is one of basically, a leasing business with slightly better economics, right? You can imagine that they were still going to be in this diminishing returns curve.
We add a layer of software and digital, and it moves us to more of this higher dashed line. What the trick for us and for many companies is to say, how do I go from this diminishing returns to this increasing returns curve. What you see is that different companies act in different ways. Uber is one where there are benefits to increasing usage and increasing data. Because Uber faces lots of competition in local markets, right, it's not going to be able to get this massive benefit of a sharply increasing curve, r ight? Already, I heard recently that once again in New York, there's a new competitor coming into the New York market to compete against Uber in ride sharing, right? The barriers to entry are quite low. Facebook, of course, has a slightly sharper value curve.
The interesting story about Facebook is that the networks that deliver value are small. They're about at the scale of 100- 200 people. What they have become really good at is adding a lot of scope in terms of products and services so that they can then build their advertising business from that. Of course, Ant Financial is very sharp because they have sort of become this thin life juggernaut. They've set up themselves in a way to be able to provide a whole range of products and services around financial interactions in the Chinese economy, right? Using that data to then generate new types of products and services for themselves as well. What this means, though, is that this type of a firm also has new sets of value.
This is a very crude measurement, which is market capitalization per employee that I've set up. What this shows you is that depending on the firm, you get different levels of leverage in market cap per employee. When we bring in these digitally native firms, like a Facebook or like an Ant Financial, what you see is almost an order of magnitude increase in valuations per employee because they can do more. They have data and analytics at the core that enables them to scale faster, to increase scope in a new way, and to then continuously keep learning and improving themselves.
All of this is based on a book that I just recently finished up, Competing in the Age of AI. This book really sort of goes through the details around what we see emerging as a new type of a company, and also importantly, what are the sort of the challenges and ethical responsibilities for us as we enter this new age of AI.
Thank you, Karim. We now will transition over to Marcello Damiani.
Thank you, Karim. That was really amazing. I encourage everyone to read the book. Good afternoon, and good morning and good evening to everyone. I'm the Chief Digital Officer and Operational Excellence at Moderna. I have joined Moderna five years ago, and I will be walking you through our digital strategy.
Before joining Moderna, I was the CIO at bioMérieux, a diagnostic company. Prior to this, I worked for large-scale organizations such as Motorola in the high-tech industry. Early on in my career, I was a software engineer supporting the aerospace industry. This is my third industry today. As mentioned by Karim and reflected on here by The Economist, data has become the most valuable resources in the last two decades, and the most successful companies were mainly data-centric ones. Advanced analytics is what provide insight to data, and AI has become the holy grail of analytics as it allow us to analyze large amount of data sets that are impossible to analyze with other techniques.
However, AI requires extremely large, structured, and well-annotated data sets, internal and external data sources that are linked and made accessible, and a digital and data-centric organization to help capture the data from inception in an integrated fashion, whether internal or external. For advanced analytics and AI to take off, digital foundation has to be ready. At Moderna, since day one, we have decided to build a data-centric company, and we have integrated automation and digital technology into everything we do, as you can see here in our 10 year vision. Why building a digital biotech? First, our technology is information-based. We instruct the cells in the body to produce protein, and those instructions are like software code. The very essence of our science is digital in nature. Second, we have a platform approach centered around modalities, and learning insight and across modalities is key.
Third, our business strategy is to progress multiple programs in parallel, internally and with our ecosystem of partnerships. Building Moderna in analog world would have been very tedious, if not impossible. What are Moderna's digital building blocks? First, the cloud. The cloud provide us with the agility to operate cost effectively without the limitation due to long implementation cycle times and the capital investment required for traditional approaches. AWS has been a great partner in delivering those capabilities. Second, this is what Karim mentioned, is really integration. Integration is the glue that brings our processes and data together in a consistent manner, avoiding silos of information. We see many companies struggle with integration due to legacy systems, siloed data and processes, therefore what they do is they build inefficiencies across all the functions, this is exactly what we want to avoid. Growing lean is key to us.
The third piece is the Internet of Things. We view this as the next level of integration with real-time collection of data from instruments. The fourth building block is automation and robotics. Our strategy here is to automate when and only when the processes are mature. We do not want to automate too early as this will reduce our agility to evolve and change the processes, but not too late as we want to reduce any manual work as early as possible. The fifth pillar is analytics. Once you have all the building blocks, you have a wealth of data that provides your business insight to make informed decisions. Finally, AI and machine learning, as previously discussed, are the holy grail of analytics and the ultimate level of our digital strategy, using predictive analytics to help with real-time decision making and predictions.
There are four benefits that we see in our digitization strategy. First, quality and error reduction, which increase our probability of success. Second, speed and the way it accelerates our learning and increase our agility. Third, scalability allow us to accommodate growth and demand way more efficiently and sustainably. Fourth, cost reduction, driving efficiencies, reducing waste, whether it's cycle time reduction, more efficient use of skilled resources, or else. The key enabler of this strategy are first the people. We have a digital culture. Everyone at Moderna embraces it and w e partner with them to ensure that the technology is serving their goals. The second piece is process. We make sure that we build our processes from the ground up for a digital world, We use operational excellence techniques like Lean, Six Sigma or else to build optimal digitized processes. The last piece is technology.
We continuously explore technologies and innovations that help us resolve critical problems, and we fully embrace those new technologies when we see benefits. To achieve our vision, we digitized every function at Moderna: research, technical development, manufacturing, quality, clinical development, and the business functions that I won't go through today. This has created a virtuous cycle of learning, where we run more experiments that generate more data, which help improve our algorithms and learning, which in turn help us develop better messenger RNA medicines, and this cycle continues. Let's dive into each of these areas. I'll start with research, where we advance ideas to development candidates. In research, our scientists start with an idea for a new messenger RNA medicine.
To help them realize their vision, we built the Drug Design Studio, a web portal that allows them to design messenger RNA and protein sequences. To date, our scientists have designed more than 20,000 unique messenger RNA sequences. Once designed, the scientist has the ability to order the sequences and formulation directly from our high throughput preclinical production facility in Norwood. After the order is placed, we run automated algorithms to optimize the sequence for expression and ease of production. The orders are routed to our preclinical production, which is integrated with 30+ robotic platforms, and where we collect data at every step of the process. We use this data to continuously improve the process. For example, we built a machine learning model that can predict yield and inform the operators on the actions needed to meet the demand.
With those apps and automation, we were able to scale the preclinical production from 40 different orders per month to up to 1,000 a month, allowing scientists to do way more experiments. Once the messenger RNA have been produced, scientists use a set of digital apps to run their experiments. For example, the registry and inventory app allow us to create and track any material used and created in research and in preclinical production. This includes DNA, messenger RNA, lipids, cell lines, chemicals, and much more. The app manages consumption, aliquoting, material transfer, and stock alerts. As of today, we tracked around 1 million items. To manage in vivo and in vitro studies, Moderna scientists use the in vivo and in vitro study apps to help them design, schedule, and track progress of their study in real time. The goal is to optimize overall study outcomes, resources, and cycle time.
The apps also capture structured data and detailed study conditions for analysis. Finally, included in the app, over 20 data analysis and visual tools that automate tedious calculation and standardize the analysis so the results are comparable across the platform. The second functional area I'll be covering is technical development. The technical development requires us to capture structured data while providing engineers enough flexibility to mature, optimize, and scale processes. How do we do that? In fact, we built two apps. The first one is the platform editor app that allows tech dev engineers to build digital templates of their processes composed of multiple unit ops, which are reusable building block of a process. Each unit ops is composed of a collection of process parameters, material, equipment, and calculations.
This allows the engineer to capture all the information as the process evolves, with the goal to reduce the cycle time of tech transfer and electronic batch record design. The second app is the development hub. It's a process experiment execution system that captures and document experiments, design, and results. It performs automated real-time calculations, support multi-arm experiments, and generates automated reports. The development hub integrates with our analytic development app to streamline submission. This has allowed us to reduce testing cycle time by 36%. The next function is clinical manufacturing. In our GMP manufacturing facility in Norwood, as Juan mentioned early on, we are able to operate the shop floor in a fully integrated and digital fashion. Paperless execution, electronic batch records, automated material flow, review by exceptions, and we are collecting around 7,000 variables in real time. Integrated electronic batch records instruct operators which raw material to use.
All materials, including product intermediates, are bar-coded, scanned to ensure accurate inventory. All consumable materials are replenished at the click of a button using custom-built inventory replenishment system that leverages AWS IoT buttons. Integrated electronic batch records instruct as well operators which equipment to use, verify that the equipment is ready for use, control operations, and collect and analyze the data real-time. With this fully integrated shop floor, we reduced labor by 40% per suite per shift, and the average cycle time by two days. We built as well visual dashboard of all the suites in Norwood that allow operators and managers on the shop floor to monitor all operations real-time. They can visualize and respond to alarm conditions, check the schedule for each suite, and view equipment calibration status.
In this next section, we will discuss how we have also digitized our quality process from testing to releasing medicines for clinical trials. Quality is designed in every step of our digital strategy. For example, the sampling of product and raw materials testing is done digitally through real-time integration with our laboratory information management system, the LIMS. Sample labels are printed and applied on the shop floor. When samples are dropped at the pickup location, all information required for testing is passed digitally to the LIMS system to enable QC testing. When QC tests are approved, the results required for batch record calculations are passed back automatically. Before releasing a batch, all batch record exceptions are captured digitally and can be reviewed by supervisors and the quality assurance team in real-time. This eliminates the need to review thousands of pages of legacy paper records.
Instead, the quality team reviews and approves exception, significantly reducing cycle time. We reduced manual batch records error by 85%, and the review cycle times from three days to three hours. Data integrity is a key component of a digital strategy in a GMP environment. We implemented for our QC labs a data integrity solution on each workstation that capture equipment data in real-time, sent to the cloud with audit trails and security controls. We have built also paperless environmental monitoring at Norwood. The biggest benefit we have seen is that it was built digital from the start, so all environmental qualification was paperless and audit ready, in addition of being very effective and efficient today. The last section I'll be discussing here is our clinical development function. In order to track the progress of the various development candidates, we have created a set of integrated applications.
Workflows include timelines of regulatory filings, planning for IND- enabling GLP toxicology, and clinical operations management. Our clinical operations applications allow us to track our ongoing trials by accessing clinical operations information in real time from our CROs. It also has multiple tools and analytics to draw key insights. Among the tools we have is enrollment forecasting that track actual subject enrollment relative to plan, a detection of enrollment issues, enrollment trends across the portfolio of actual target and baseline forecast. We provide also a view on the various milestones across all Moderna studies. It help us easily compare key metrics against industry benchmark, publish all type of metrics, categories, performance, quality, and risk, allowing the team to track progress. You can see also here how we monitor site activity, including training and trip reports.
We automatically flag reports that fall outside of contracted windows, and we have a heat map of the clinical sites in the U.S.A. I hope this gave you a quick but comprehensive view of how we are building Moderna as a digital biotech. With this, I'm handing over to Dr. Johnson, who heads our informatics, data science, and AI group, and Dr. Johnson will share with you how as part of our digital strategy, we have progressed advanced analytics and AI.
Thank you, Marcello. It's a pleasure to speak to everyone today. As Marcello mentioned, I'm Dave Johnson. I run informatics, data science, and AI here at Moderna. My role is really to deliver digital and AI solutions to better enable research and clinical development. I have a PhD in Information Physics and over 15 years of software engineering and data science experience. The last 10 years have been exclusively in the pharma and biotech environment. I'm delighted today to talk to you about how we're leveraging AI and advanced analytics at Moderna. We use these all across our platform. What you see here on the slide in the left column is a selection of all the algorithms we're running in production today.
The first two highlighted at the top here were actually leveraged in our novel coronavirus vaccine, the first one to optimize our mRNA sequence, and the second to determine the optimal DNA template construction. The next three algorithms that are highlighted there are used regularly as part of the personalized cancer vaccine workflow that Juan spoke about. We use an AI algorithm to design a personalized mRNA vaccine for an individual patient. Then again, we use our mRNA sequence optimization algorithm. We also use Monte Carlo simulation to plan our complex PCV supply chain. Coming up right behind these in the center column are many new algorithms that are in development or final stages of testing and will soon be out in production. Across all of these algorithms, we leverage a wide variety of computational techniques, from simple rule-based optimization to more sophisticated neural networks.
There's really no one-size-fits-all technique, so it's critical to have a large toolbox at your disposal. What I'd like to do today for the rest of this talk is dive deeper into two of these use cases that are really good examples of the kind of AI work we do at Moderna. The first use case is in Sanger sequencing analysis. This analysis is used in our preclinical production process to QC our DNA templates. This is the very first step of our production process. I'll show you how we built an automated data analysis pipeline using an existing algorithm. I'll then walk you through how we extended that algorithm using a neural network to address a challenge with our poly-A tails. Finally, I'll show you how we built a radically new algorithm from the ground up that achieves even better quality performance.
The second use case on the right here is in our research space, where we're using cryo-EM images to study the lipid nanoparticles in our formulated drug product. Analyzing these images requires a sophisticated three stage analysis using three different algorithms. Let's first dig into the Sanger problem. Sanger sequencing is an assay that allows you to determine the nucleotide sequence of input DNA. While Sanger is an older technology and newer next-gen sequencing tools have emerged, Sanger is still used as the workhorse in most DNA synthesis operations due to its low cost. At the bottom here, you can see an example of a chromatogram called a read generated by a Sanger sequencing instrument. Each peak corresponds to a different nucleotide base in the input DNA.
The relative peak heights aren't very important here, but the color of the peaks are, because each of those corresponds to a different nucleotide base, A, C, G, or T. By just looking at the color of each peak, you can read off the nucleotide sequence. Now, this assay is really important for us because we use it to QC the DNA templates we use to produce our mRNA. If just a single nucleotide base is mutated, an insertion, deletion, a modification, that mutation will be transcribed into the mRNA, which will then be translated into the final protein, resulting in a non-functional protein. It's critical that we catch these kinds of mutations very early on in this first stage of creating the DNA template. In practice, the analysis isn't quite as simple as just reading off the nucleotide sequence.
A Sanger data read is noisy at both ends and will only give you around 700 bases of high-quality data. You need to collect multiple reads across your DNA in both directions and then combine the results. The full analysis requires a complex five step analysis pipeline. We built our high-throughput preclinical production engine about six years ago. We didn't have much Sanger yet, so we didn't have the data we needed to build our own algorithm. Instead, what we did is license an off-the-shelf algorithm, which has been an industry-standard tool for the last 20 years, used by thousands of labs. However, what we did realize is that we'd be generating a lot of Sanger data. It takes on the order of 100 Sanger reads per mRNA construct we produce. What we did is we automated the Sanger analysis pipeline using this algorithm.
The actual Sanger wet lab work is done at an external sequencing vendor, as this work is fairly commoditized. However, at the end of the sequencing analysis, the instruments at the vendor upload the data to our cloud storage environment. The moment that data hits our environment, automated processes kick off and spin up compute servers in AWS's Spot marketplace. These Spot servers are pulled from Amazon's excess capacity, so we pay pennies on the dollar. The servers run the Sanger analysis on the raw data and send the results to a custom-built Sanger app. When all the data has been analyzed, the servers are automatically shut down and discarded, and we only pay for the time we use them. This approach is an extremely cost-effective way to perform large volumes of computational analyses in parallel.
This custom-built Sanger app not only captures the results of the analysis, it also is a powerful interactive visualization tool that allows the operators and scientists to inspect the data, the analysis, and any problems that might have been found. In this example here, you can see a target sequence at the top and a couple aligned reads at the bottom. The data from the bottom read is further along, so the peaks are harder to distinguish and lower quality. The consensus of the reads here showed that there was a single G deleted out of the thousands of nucleotides in this sequence. The analysis found this needle in the haystack, this sample was discarded.
Since we automated this analysis some six years ago, we've now collected over 3 million Sanger reads of data across hundreds of thousands of samples. As we process all these samples, we've unfortunately discovered that there are a number of very rare failure modes that are not detected well by the licensed algorithm. With all this data we've collected, we can now turn around and then build better algorithms to catch these cases and improve the quality. The first major failure mode that we tackled is in our poly-A tails. As you might know, our mRNA sequences end with a long stretch of As that give the mRNA stability. We build these tails into our DNA templates so that they'll be transcribed into the mRNA. We use Sanger to analyze these tails just like the rest of the sequence.
Here at the top, you can see an example of a high good quality tail. You can see the repeated A peaks in green. There's a little noise that's underneath the peaks, but overall, it's a very good signal. On the bottom, on the other hand, is a bad tail. In fact, it's an extremely bad example of one. You can see these large secondary peaks sitting underneath the tail peaks, and what these typically indicate is subpopulations in the sample with truncated tails that are shorter than they should be. The real challenge in this problem is that while these subpeaks are really obvious to all of us who are looking at them, the algorithm we licensed is not able to detect it. It calls that read a little bit lower quality, but it still passes it.
Unfortunately, while we had automated the majority of this whole Sanger analysis process, we still had to have trained operators weigh in at the end and manually review these tails. That's obviously a very tedious and expensive activity. We set out to automate this part of the tail analysis. The first thing we did is we had trained operators review thousands of tail chromatograms in a controlled setting and have them label them as pass or fail. In all, we generated over 20,000 labels. The first things we did with this labeled data is to see how consistent our operators were at making these calls. You can see the results here on this chart. On average, two different operators reviewing the same image multiple times only agree with each other about 70% of the time, and some of these combinations are as low as 42% agreement.
It doesn't stop here. You can see on the diagonal, these boxes, we show the same images to operators more than once so we can determine how consistent they were with themselves. You can see the self-consistency here. On average, operators only agree with themselves about 90% of the time. That means that if you show an operator a tail image from this set 10x , one of those times, they will change their mind. Now, in an analysis that's used for quality control, consistency is paramount, so this is a problem. This next slide here, you can see the individual operator performance relative to the team averages using what's called an ROC graph. On the left, we have the true positive rate on the vertical and the false positive rate on the horizontal.
A perfect performance would be in the very top left, whereas the diagonal line here in the middle is the performance of a random classifier, essentially a coin flip. Each colored point here is a different operator. You can see that in general, the human operators are actually pretty good. They're very far away from that random classifier, but no one is perfect or close to perfect. The distribution that you saw of the variability of operator performance is also apparent here, where they're scattered around here. You can see some operators are just in general poor performers. Some are more strict operators and tend to fail more than average, and that's not a particularly good thing either, because they're throwing away good samples. Some operators are more lenient and let more samples through.
What we did is we took this training data and we built a convolutional neural network, or a CNN, that is predicting this overall team performance. This CNN is the same type of network used in state-of-the-art image analysis. The gray shaded curve there is where we can choose for this algorithm to operate. What we did is we chose a point up in the top there that is a balance between too strict and too lenient. What you'll notice here if you look carefully, is that this algorithm achieves superior performance to any single operator. This is an example of when Karim talked about strong AI, where the AI outperforms the humans. We put this algorithm into production over a year ago, and it has saved countless hours of manual tail review. Beyond that, the algorithm is 100% consistent.
Given the same tail image, it will always return the exact same result. We didn't stop there. In our pursuit of higher and higher quality, we discovered numerous other failure modes, like mixed bases, contamination, mispriming, failed contigs, peak drift, and so on. It turns out that all of these problems are rooted in the same mixed base problem, where these subpeaks are not well handled by the algorithm. If we can solve the mixed base problem, we can solve the rest of these. In this particular example that you see here, you see a significant G peak underneath the C peak here. What that means is that approximately 30% of the DNA in the sample has a mutation of C to G, and that's hiding under the dominant peak.
That means that 30% of the protein that we would make from this DNA template through the mRNA would also be mutated and non-functional, which would be a huge hit to the potency. It's really important that we can detect these failure modes as well. We realized we couldn't just layer another algorithm on top here, because there's a fundamental limitation on how the existing algorithm handles these subpeaks. We decided to build an entirely new Sanger analysis algorithm from scratch. To do this, we reached for Bayesian inference, which is a great tool if you have a good understanding of your data generation model. There are two key innovations here. The first is in the peak detection phase, which is the image on the left here, where you need to locate the center and heights of all of the peaks, including these subpeaks.
Here, we designed a mathematical model based on all the historical data we have and used a Bayesian math analysis to find the optimal fit. In this image, you can see how the solid lines here are the raw data and the dashed lines are the fit to the model. There's a really great correspondence here showing an excellent fit. Once we have accurate peak amplitudes, the next innovation is to determine what nucleotide bases those amplitudes indicate. To do that, we use Bayesian hypothesis testing, which evaluates the probability of all possible bases and all possible mixtures of those bases. Here you can see the same mixture on the left from the previous slide.
With just a single read, the algorithm is going to still call it a C because these kinds of mixtures are so rare, but the faded out color of the C indicates that it would be flagged as a low quality or a warning C. However, a really nice feature of this technique is that it calculates the cumulative evidence. If you see this pattern one or two more times, the algorithm is now convinced that this is indeed a mixture of a C and a G. Furthermore, it's able to quantify the exact percentages of each base. Here it sees that percentage is actually 26.5% of the population with the mutation. Now, these kinds of Bayesian techniques are notoriously computationally intensive. This new algorithm takes about 100x the computational power of the previous one.
With modern compute servers in AWS, the analysis only takes about 10 minutes to run. With Sanger, we looked at using AI for this step at the very beginning of our production process. Now we'll switch to one in the very end of our process, where we're looking in the research space, with cryo-EM. We're trying to see more about our lipid nanoparticles. After creating our mRNA, as Juan mentioned at the very start of his talk, we combine the mRNA with small molecules and lipids to form lipid nanoparticles that encapsulate the mRNA. These LNPs are our final drug products. The exact structure of these LNPs is heavily dependent upon the specific components that go into the particles and the specific process by which we combine them. Cryo-EMs are images from an electron microscope that take pictures of our LNPs.
In this image here, you can see a number of darker circles, each of those corresponding to a single LNP in a sample. Usually these images are just viewed at qualitatively and people make judgments on them. What we're doing here with AI is collecting quantitative numeric measures from these images. This is important first because we want to gain insight into how variations of our LNP components and processes impact the LNP assembly. Second, we want to understand how variations in these assembled LNPs will impact downstream in vivo performance. To date, we've generated over 500,000 of these images in countless process variations. Unlike in the Sanger problem, there's not a preexisting algorithm we can turn to. We have to build our own from scratch. The first step of this analysis is called segmentation.
This is where we identify the parts of the image that correspond to individual particles. This is a great example of Karim's weak AI, because humans are very good at segmentation. Millennia of evolution have resulted in human brains excelling at these kinds of image-related tasks. While you can look at this image here and figure out which components of that are particles or not, it's an immensely tedious job for you to go through and actually draw lines around each of those, and it's not particularly accurate. On the other side, it tends to be a very complicated problem for computers to do. It's only in recent years with modern algorithms that we're able to actually solve this. The solution we use leverages a state-of-the-art convolutional encoder-decoder neural network depicted in the middle.
This network maps out an input image of pixels to a probability map that each pixel belongs to that particle or not. We use that to create an image mask. Here on the right, you can see the output of that, where the pixels shaded in purple correspond to the particles, and those shaded in orange are the background. You can see this model performs extremely well at identifying these regions and saves countless hours of tedious manual work. The next task we approached was classifying the type of these particles. In the top left, you can see another cryo-EM image. In this image, there are nice round particles, and then there are these other irregular particles with blebs. What we want to do is build an algorithm that can detect which particle is which. Again, this is another great example of weak AI.
This is not a task that the previous neural network can do. It actually requires us to build a whole new network to classify this. The distribution here, the histogram, shows what would happen if you tried something naive, like just looking at the major axis length. We see that they're just not well separated. We need a more sophisticated machine learning algorithm. We tried a variety of models that are shown here, and what we landed on was a random forest algorithm that incorporates the intensity of every pixel in each of the particles. This model achieves an incredible 0.98 accuracy and 0.99 precision, which is just unheard of to see rates that high. Once again, through the use of this algorithm, we're saving countless hours of manual effort. Finally, we'd like to extract some metrics from the structure of the individual LNPs themselves.
If you look very closely at this image, you can see a railroad pattern around the edges that indicate a lamellar structure. If we draw a line from the center of the particle to its outside edge and then sweep around, averaging the pixel intensities as we go, we can produce a graph like in the top right here. Each row in the graph is the profile of an individual particle in a cryo-EM image with its outside edge aligned to the right. We average vertically across all these pixels, and we get a profile in the bottom right here for the whole image. What you can see come out of this picture is that these pixel intensity peaks show up that you saw with the naked eye, but additional peaks show up as well that were harder to detect.
Beyond that, we're actually able to quantify in nanometers the exact width of that lamellar layer. These profiles turn out to be very formulation specific. In the bottom right here, you can see what happens if you look at a variety of different LNP components and processes. You see a variety of different profiles that come out. Using the quantitative metrics here, whether it's lamellar width and other metrics like it, we hope to gain insight back into the process that we form these LNPs, and then in the downstream LNP performance. What I'd like to do here is close by highlighting the AI factory that we've built at Moderna that allows us to continuously mine and deliver AI algorithms like the several that I've shown you here.
First, you heard from Marcello about all the great lengths that we go to in order to capture and integrate structured data across the company. I just now spoke at length about the various algorithms that we have in research and development. I also walked you through the automated Sanger analysis pipeline example that illustrates the sophisticated cloud and DevOps infrastructure we have at Moderna, which allows us to rapidly deploy compute power. Combining all these together allows us to do AI model design at scale. We leverage all of these to rapidly design, test, and deploy AI models. Finally, this integration, this infrastructure and processes we built in Moderna allow us to rapidly move completed AI models into a production environment for daily use. This AI factory allows us to push these same goals to even greater heights with improved quality, increased speed, increased scale, and reduction in cost.
With that I'll say thank you.
Thank you, Karim, Marcello, and Dave. With that, operator, we are ready for Q&A for the digital section.
Thank you. As a reminder, to ask a question, you will need to press star one on your telephone. We ask that you please limit yourself to one question and one follow-up question. You may then return to the queue. To withdraw your question, press the pound key. Please stand by while we compile the Q&A roster. Our first question comes from Hartaj Singh with Oppenheimer.
Great. Thank you for the questions, and for this really interesting presentation. On just the digital and the AI side, so having been in drug development for a long time, whenever I think of drugs, I think of preclinical, clinical, and CMC, basically the three kind of the holy trinity of the BLA or the NDA.
I guess my question about the AI and digital that Moderna is developing and utilizing is, can you give some concrete examples, which portions of those attributes, and again, I know you can probably spend a long time, but issues that are relevant today, for example, how they might have helped with coronavirus, the vaccine, or CMV, but could you give us tangible examples of how AI and digital are sort of helping you in interfacing through regulators through these three sections of the IND, and then what ends up being eventually the BLA or the NDA? Then I got a quick follow-up question.
Yeah. Hi, this is Marcello here. Just to set the stage, actually, we started AI based on the historical data that we have. You would see more and more AI at that stage in our research and preclinical space than in our clinical space. As we are progressing and building the digital infrastructure to capture all those data from all those engines that we have, we are building more and more AI. As Dave said, we have many algorithms that are at this moment in training to help us in those new AI models in the different areas.
The learning specifically, the learning that we have gathered on PCV has helped us go very fast on the coronavirus vaccine.
Yeah, it's Stéphane. Just to close what Marcello said. If you think about it, as we built the company from having nothing to being a preclinical company and then being the digital behind preclinical, that's why we have the most data. That's why the team has done a lot of work. The next phase that we did as we built the company was CMC at Norwood, which is why you start to see there's a lot of AI algorithm in the CMC space because we are collecting so much data in Norwood.
The team in the last couple years has been building and is finishing the clinical side of digital, including how to file the IND by basically putting all the pieces together, coming from tech dev, where we have integrated the robotics straight through the IND file, the tox study, and the preclinical data as well as clinical synopsis of the clinical study all coming together through that system. In clinical, we are still at a stage where it's mostly building the digital infrastructure to then generate the data to then enable us to do AI.
If I can comment on coronavirus, we used an algorithm that we have developed for previous construct, lung construct, because coronavirus is a lung construct, to be able to produce rapidly the coronavirus construct. This is based on the learning that we had in research and in preclinical from previous ones.
Great. Thank you. Just, I had a question actually earlier when Juan was presenting, and I just wanted to ask that, if you don't mind. Just as a real-world example, if you can just walk us through, if let's say you had been at the equivalent of potentially an analog company, and the coronavirus essentially a problem had landed on your laps, like it did a few weeks ago. What were the steps you'd have to go through, and would probably still be going through in order to get a candidate to the NIH? Then kind of compare that against. What is it specifically that Moderna did to get it to the NIH in those 42 days? Just broadly speaking, if you could just give us a kind of in that context would also be helpful. Again, thank you for the question.
Thank you. That is a really good question. There is a lot of examples. Of course, we have our own platform and technology. It's very difficult to go and compare. I can give you a couple of anecdotes here. A typical manufacturing batch record in the biotech industry has around 10,000 entries. It is thicker than one of those old telephone books. It would have taken a long time to prepare a batch record only to make the product. Having the digital platform, it allowed us to prepare that documentation in a much faster way. The same thing goes for equipment parameters that would allow us to go and set up the equipment and the automation that would allow to make the product.
I went in the presentation, in the slide, it took us days to set up a process like this. I have done something similar than that. It would take weeks or months to get ready to do something along those lines.
Maybe just a little bit, if you think about it, there's a research process, there's a process development piece to a puzzle, and then manufacturing quality, regardless of how you do it. If you think about what some of the big vaccine manufacturers have announced in the last weeks, J&J has announced it will take them 10-12 months to go into the clinic, and Sanofi has announced it would take them 12-18 months to get into the clinic. As you might have heard from Secretary Azar last night, the FDA opened the IND for mRNA-1273 on Monday. That was less than 10 days after filing the IND. If you read it's less than 60 days from getting the sequence published by the Chinese government.
If you ask the question, how did we do that versus the timeline that I just shared, that have been communicated publicly by J&J and Sanofi, I think it goes really into those three buckets. First is the research front, which is we did not need to have a virus, just a sequence on the computer was enough. Thanks to all the work we have done before, we're able to very quickly, working with the NIH to pick the sequence. If you think about research, instead of spending months and months playing with animal model and different candidates and optimizing, we literally did that on computers without ever having the virus in a matter of a day or so.
On the process development, in a typical process, you need an army of engineers to figure out cell line optimization, and that would have taken literally months. In our case, because an mRNA is an mRNA, the team was able to retrofit very quickly the personalized vaccine unit, those robots that some of you have seen in Norwood have been developed for personalized cancer vaccine, to do for the first time our infectious disease vaccine. It's just a little bit of tweaking to do. Again, it took days and not months, because it was the first time we did an infectious disease vaccine on that kind of robotic system. The manufacturing is another piece where we get a massive advantage. It takes a few days to make mRNA. It takes a few days to formulate the mRNA.
If you think about it in days, just use running a recombinant train, that would take four, five weeks in terms of making the API from a recombinant standpoint. I think it's just all those pieces that help, and then you layer on top of that all the digital, like electronic batch record, quality control that the team has been talking about, and it's how you end up having a cycle time that we had of 42 days from sequence to shipping a product, having successfully passed sterility testing. I even think that this is because it's the first time we have done it. I don't think this is the fastest time we will ever do, just because, and Juan is turning already here next to me.
Just because if you think about it, anytime you do something the first time, you are learning things, you have to optimize things, you have to develop new things. As you're able to do it more and more, as Juan has shown with this personalized cancer vaccine slide, as you do it more and more, you become stronger and stronger as you keep on learning. I think that's really a key theme about how we're wired as a company from a counter-cultural standpoint and how we use the digital backbone that we have built and the access to data. It's all about learning. As we've said in the past, we believe mRNA is the new class of medicine. We believe it is a 15-20 year S-shape kind of learning curve, and we want to be the company that learn the fastest.
We've always said we won't be the smartest, but we will be learning the fastest. This digital and these AI tools is really at the heart of strategy.
Great. Thank you so much. I'll jump back in the queue.
Thank you. Our next question comes from Yasmeen Rahimi with ROTH Capital.
Hi, team. Thank you for an incredibly informative session, especially as you're helping us to really visualize how AI is used in mRNA construct design and LNP formulation and construct. I guess I have a question, two questions for you. Can you help us understand how you can utilize AI technology?
To really help predict transferability from preclinical models to clinical models, can you use AI technology to really identify patient selection in clinical studies? As we go with additional data from CMV upcoming, can we really help us to determine who could be responders versus non-responders? The last question is for Stéphane. Not only is manufacturing AI technology setting you apart versus your competitors, but how much is adding AI increasing the barrier to entry among not only other companies in biotech, but specifically in the mRNA world? Thank you for taking my questions.
Okay. Hi, Yasmeen. This is Marcello here. I'll start with your second question. Will AI help us with patient selections? I would say definitely. At that stage, we're connected to all the CROs. We are pulling data from those CROs. As you know, we started our clinical trials at the end of 2015. We're starting to collecting all this data. As we speak, we are looking at different AI models that can help us. As we progress those, I think we will communicate or we will show our progress in that field. On the translational part, it's the same question, actually, because you're moving from a different part of the engines that we are building. You have the research, the manufacturing, and the clinic.
What I can say as of today is we are collecting data in every step of those processes, and we can trace back the data from clinic to research. The question is how this is going to help us to improve in the clinic will be as we progress the clinic to put some problems to our AI team, and they will help us build the models to resolve those problems.
This is Dave here, just to add a little more color here. As I hope is clear from the slides that I shared, all of these problems are a journey. We start solving these problems with the data that we have, and we continuously evolve and continuously push these forward. Right now, in a lot of these spaces, we're using traditional pharmacometrics and pharmacology techniques. We will continuously evolve those as we get more data and inject more and more AI into those spaces.
Yes. It's Stéphane, for the last question. I mean, as Karim said when he did his presentation, we are big believers that time is the only common denominator across companies in an industry. We believe that being able to learn faster is a key competitive advantage. I think it is even more important as you are going through the S- shape curve that I talked about, compared to companies that are the top of a curve, like I used to be in my previous companies, where when you grow and learn 2%-3% a year, you feel great. It's a great year. If you think about the mRNA technology as we've shared in the past, if you look at the technology we have today in the clinic, we just use the technology we got from the Harvard labs when we started the company.
We have improved 1,000x, 2,000x the performance of the mRNA technology. As you know, and you saw the last time, the last spring, and I welcome you to attend the next one with Stephen Hoge, his team in June in New York, is we keep on learning. I mean, if you look at the tools like the cryo that Dave mentioned. It's just one of many examples. We are equipping all the teams across the company with tools to be able to learn fast. The big change management, I believe, and it is across all companies, and we are driving it with Marcello and Tracey, our Head of HR. This is new tools. AI and machine learning are new tools. The biggest challenge, in my opinion, is change management, because you have most of the leaders in most companies, we have never done AI before.
You have people like Dave and Marcello who understand it and have studied it. Most of the line teams, they have never done AI. What we've tried to do is to first build the infrastructure, the digital tools, the processes to get the data, because without the great data, good luck. What we are layering on top of that is in every group in the company is AI. What we want to do, and it's going to take us a couple of years to really make it 100% part of our DNA everywhere, is we want AI in finance. We want AI in HR. We want AI in manufacturing, in finance, everywhere. That's really how we are building the company.
It doesn't happen overnight, but we believe that if we are well-capitalized like we are, with the scale that we are as a company, if we can really build AI as a key common sort of our culture, it's going to be very hard to compete with us.
Yeah. If I can just add, this is Karim here. My observations generally outside of the pharma business, there is a data advantage, right? You need to sort of think about the data you're accumulating and both your data fusion abilities, right, and proprietary data. I think you can sort of see that in how they were talking about from sort of research to manufacturing to clinical, and that this has to be completely be ingested all the time. I think the second thing is really the industrialization of analytics and AI, right? Most of the time you think about decision-making that happens in companies, right? The tools we use are Excel sheets, right? We have Excel sheets. We call up our buddy to give us more data. We think about the data, we modify the Excel sheets, and we do that, right?
This is a totally different beast. You want to industrialize the decision-making process and just think about all the decisions they have to make, again, from R&D all the way to the clinic. The third part, I think the really important part, is this change management part. Where I see these things die in companies over and over again is because the management team, A, is either not committed or they think it's flavor of the month, and they hope that we'll sprinkle some AI magic and we'll go from there. The change management part, both you have the AI natives showing up now in the economy that they're working with, but there's a huge science component here that also needs to be retrained and shown the benefits of this approach. That's a big mandate.
I think many companies outside of the pharma biotech sector fail miserably at this in this journey. I think those are the ways you can think about how competitive advantage is built.
Yes. It's Stéphane, again, very important last point. Last year, Marcello and I took a trip on the West Coast visiting a few companies just to look at AI and what were best practices and change management in those companies and tools. As you know, you heard several times, we are partnering very strongly with Amazon and AWS. The piece that was extremely exciting to me when I went to Seattle with Marcello is the customer obsession that Amazon has brought into the retail space, and Bezos has talked about it all the time. When you talk to the AWS team and you see the tools they are developing, they are totally customer-obsessed. They are developing tools every day that are available in their cloud so that all teams are able to go and to use those tools.
We will have this setup now, which is very interesting, which is you have the Amazon team building more and more tools to just make them available if you use the cloud. They, when these teams need something that is not available, then they build it. We have this really ability to go very fast with existing tools, and that number just increasing, and more and more powerful tools, and then the team being able to design the tools they need if they cannot find them in the Amazon store.
Thank you for the insightful answers.
Thank you. Our next question comes from Cory Kasimov with JP Morgan.
Hey, guys. This is Matthew on for Cory. Thanks for taking my questions. For the first part of your talk on your personalized cancer vaccine manufacturing success rate, where I believe you said you're currently at around 87%, what are the key reasons for manufacturing failure? How, if at all, do incorrect sequences that you outlined in the digital part of your talk factor into this?
Well, I think it is a combination of factors. A lot of that is just the know-how. I didn't go into the presentation associated with what are the root causes. It is a normal part when you do a scale-up. Obviously, they go around equipment variability. It goes about operators variability. It goes around process variability. There were a number of different things. At the beginning, we even started with a CMO until we were able to bring that into Norwood. Through the obsession in tackling one problem at a time, we started to increase the success rate as we continue doing it. I would have argued that the fact that we are doing personalized cancer vaccine and we are making as many batches as we make, the learning, as Stéphane mentioned before, is going faster because our aim is higher.
We can connect the dots on a number of different root causes at a much faster way. And of course, digital was part of it from the beginning.
Got it. Is there a technical limit to how much you can improve upon this? How do you improve upon the rate even further?
I think the sky is the limit. I think we will get way into high 90s here if we go all the way into commercial, which is the normal thing. We will go into validation. At that moment in time, we would have incredible experience associated with it. I also point, if you remember in the slide, there were different colors. Some of the first colors were basically failures that we found out after the batch. In some or in all of the recent failures that we had, we had more aborts rather than failures, so it was not a surprise.
In these cases, particularly important, because if something goes wrong for whatever reason, and as I said, it could be equipment, as simple as an equipment malfunction, I don't know, a pump fails to produce or just something that is unexpected, you discover that immediately, so it is not a failure later, it is an abort. When you're doing something for a patient in a small period of time, basically it allows you to go and produce fast again and arrive to your destination. Those things are integrated, so even in the failures.
Yes. It's Stéphane, just maybe to add a point that Juan mentioned during his presentation, which I think is really critical, which is the personalized cancer vaccine has been extremely enabling for us because we have made so many lots. As Juan said, it would have taken us years after launch of a CMV vaccine or Zika to get to the same point of learning. The power of this personalized cancer vaccine was it not only it was an extremely important enabler to the corona vaccine, as we discussed earlier, but more important to me for clinical batch. When we make big batch for, again, CMV or Zika or MMA or pick your favorite product in the pipeline.
When the team makes that, all the learnings coming from the ATN that we have in PCV is extremely valuable. If you think about it, if we did not have PCV, it would take us years from now to get to the same place in learning, and that's a company advantage that I think sometimes is not always fully appreciated.
Great. Thanks for taking my question.
Thank you. We have a question from Matthew Harrison with Morgan Stanley.
Hey, great. Thanks for taking the question. I guess what I wanted to ask is, how far do you think you've pushed the limits of the technology and is there a case here where you've tried to use AI and have failed and what sort of evolution do you need to see to be able to push it further? Thanks.
It's Stéphane. Matthew, you're asking about mRNA science, correct?
I was asking about AI and sort of, you gave some examples of some concrete processes where you've had success. I'm wondering, has there been something where you've tried to push AI into a process or a situation and haven't been able to make it work? I guess I was trying to understand, is that a limitation of just technology and you think it'll catch up? Or what are the reasons where maybe you couldn't use it in certain situations?
Sure. This is Dave. I'll take the question. I don't know offhand any kind of large failures that come to mind. What we do when we start AI projects is we focus on ones where there's clear, tangible business value that we can achieve in a realistic timeframe. Again, there's this journey aspect of it where we try to focus on problems we can get a minimal viable model out that's delivering value, and then we run that for a while, and we continuously evolve and improve that. I think this is something that gives us a much larger chance of success rather than going for some large moonshot problem that will take us multiple years. We try to focus on these real smaller achievable problems first.
Okay, thanks. That's helpful.
Thank you. Our next question will come from Salveen Richter with Goldman Sachs.
Good afternoon. Thanks for taking my question. One on AI. I guess if you're integrating it across maybe different verticals, how much do you want to optimize one part before you fully integrate, just because you may have to play with various inputs. Just trying to understand how you do that. The second question is, as you scale up on manufacturing, how do you think that's going to impact COGS going forward?
Okay. I'll take the integration verticals. Actually, we don't look at the verticals. We look overall how and where we need to collect data to be able to improve the processes. If you look at verticals, you're creating silos of information, and this is the thing that we want to avoid, actually. We want to make sure that we're collecting data from all those verticals that in some instances we will use to solve business problems.
Yeah, if I may add, this is Karim. The perspective to take is one of the architecture, and the architecture of these AI-first companies is very different. Even the setup of a silo is going to be antithetical to what you want to achieve, right? This is the notion that the operating model, the way you're structured, is very different than in a traditional company. There's a notion in the management literature called Conway's Law, that the architecture of the technology reflects the architecture of the organization. That's what you're hearing, that it's a completely different architecture. These questions sometimes are not even relevant in this model.
I can give you an example. We are collecting data from one of our vendors, I won't name the vendors. We were building this data just as an integration between us and them. Two years down the road, we saw some issues and problems that we needed to solve. We were figuring out how to do it. What we've done is we looked holistically at all the data that we had available, and we detected patterns that helped us resolve the issue. Shouldn't we have collected all this data and stored it, we weren't able to do anything with this use case or problem that we had.
We were able, in that particular instance, to identify three out of about a dozen instruments at the vendor that were malfunctioning and not configured properly. We were able to take that data for a purpose that we're using entirely separate and identify challenges and tell the vendor to go correct that mistake.
Another example that comes to mind, and I will comment on COGS here in a minute, is things such as predictive maintenance, in-process controls, a number of different things that we would go. We can even go and we have started thinking about going into continuous manufacturing as we move forward. Let me take the COGS piece. One element associated with COGS is obviously the technology, which is not necessarily related with AI. The fact that what we use is the body as the bioreactor rather than the manufacturing process. You're not having big bioreactors. Your capital intensivity, the length and the cycle time of your process associated with the technology is completely different profile, which is going to give you a completely different profile of COGS.
Artificial intelligence and all the digitization applied to manufacturing, the first thought in mind is associated with quality, very similar to what we have. In counter to what some people think, high quality.
Means lower cost, because it means high reliability. That's why we are focusing on reducing that variation, increasing our reliability there. Artificial intelligence and digitization automation applied to yields, to cycle time, as I mentioned, continuous manufacturing. All that would take our costs into one direction, down.
Yes. Salveen, it's Stéphane, a bit further another dimension of manufacturing, which I think is an amazing power of the mRNA technology because it's a platform, is that we do not have to be precise on every launch forecast by product, but across the portfolio. If you think about a wonderful launch that has happened recently is SHINGRIX. SHINGRIX is GSK biggest vaccine, $1 billion first year of launch. GSK has been very open about it that they cannot make enough. They cannot make enough because they never thought they will sell a $1 billion the first year of launch. This is the best vaccine launch ever. If you think about it, the problem with traditional products is that you need a dedicated manufacturing facility for every product. You have to guess way before you have your phase III data, your ramp, your commercial ramp.
Even you don't know your efficacy, it's just a big guess. In our case, what we think is an amazing power of a platform which will play into maximizing supply chain, which when you do a 90% gross margin business, is the most important than the 5% improvement in the cost of goods, is make sure you never miss a sale. We believe by having this platform, we should be able to really manage a portfolio so that we only have to be right at the portfolio and we start to have 24 products like today, and sooner or later, 30 or 40 products. That makes the problem easier and easier to manage. I think this is one of the biggest value.
Juan is going to work really hard with his team to take the cost of good down and of course, scale and volume will help. For me, the piece that's the most remarkable is we do not need to invest additional CapEx for every program. If you look at it, we just announced recently EBV and pediatric RSV. We don't need one more dollar of CapEx t o those products. It's in the same rooms with the same people with the same equipment. That's just fantastic savings. It's on the maximizing sales. If one day we come to you guys and we say we cannot sell enough because we didn't do the forecast right, we are not doing our jobs right.
Thank you.
Thank you. We do have a follow-up from Hartaj Singh with Oppenheimer.
Hey, great. Thank you for the follow-up. I just had one quick follow-up on AI, artificial intelligence. This was, I think maybe for Stéphane or whoever can answer it. In some ways, I guess the holy grail of drug development is understanding the pathology of the disease and finding what the underlying problem is, finding a target, right? Understanding whether it's druggable, and then coming up with a drug for that. With mRNA, you're coming up with a different modality to do that.
I guess my question is in that context, within your five or your six verticals, is there any one that is more amenable for AI to actually help in that process, automate that process so it doesn't become this laborious kind of go into the petri dish and then going out of the petri dish and going into the kind of process or not really? How much time could that happen? Thank you.
Yeah. It's Stéphane, I'll take a shot at this one. I think it goes even beyond AI. I'll go back to basics, which is you think about our technology. We do not invent biology. That's, for me, the most profound thing, which is if you look at traditional medicines, it's a guesswork. In that world, you can see how AI helps you avoid spending too much time in the petri dish or into a mouse or whatever. In our case, we use evolution. If you think about the coronavirus, why could we go so fast? We don't do any biology work. We just do a sequence, drop it into the same cassette as the CMV, and we flew to the clinic. If you think about MMA product, it's the same thing.
We use a sequence that all of us on the call have, which because we don't have MMA disease, and drop it into the same cassette as a chik antibody technology, and you fly to the clinic. The piece that I already think is powerful about our technology that I think sometimes is underappreciated is we don't have to invent biology. We just use evolution and that's the best AI system I know.
Great. Thank you, Stéphane.
Thank you. Speakers, I'm showing no further questions in the queue at this time. I would like to turn the call back over to Stéphane Bancel for any closing remarks.
Thank you. I'll be brief. First of all, I would like to thank our guest, Karim, for his help thinking about the business model of a company and his help today in the presentation. I'd like to thank Juan, Marcello, and Dave for their presentation, but even more important, for the impressive progress that they and their team have achieved in 2019. As you know, we are very excited by the next leg of growth of a company, doubling down on our vaccine, doubling down on our IV systemic products. As you know, we have five new development candidates in just the last 60 days. I'm very excited and looking forward to 2020 because we're going to have a lot of clinical data with us.
As you might have seen last night, we announced that the CMV phase II was fully enrolled ahead of plan, which makes us very comfortable to confirm we'll have data in Q3 of this year, given all the subjects have been dosed now. As I said last night, we had the HHS Secretary Azar, who communicated that the FDA has opened the IND for the corona vaccine 1273, and so we're hoping that soon the first subject will be dosed in the U.S. More than ever, we believe that mRNA is the nucleus of medicine. Our Norwood site is a strategic asset, as you get a good sense for it, and what we have achieved and what we will achieve looking forward could not happen without Norwood.
Digital and AI is becoming day after day more and more part of the DNA of a company. You have my full commitment and the commitment of the team that we will keep pushing it every day so that the company just becomes stronger and stronger. With that, I would like to thank you for taking the time today, and I hope to see many of you on April 14 in New York when we do our Vaccine Day. Thank you.
Ladies and gentlemen, this concludes today's conference call. Thank you for your participation. You may now disconnect.