Ladies and gentlemen, thank you for standing by, and welcome to the Moderna Manufacturing and Digital Day Webcast Call. At this time, all participants are in a listen only mode. After the speaker presentation, there will be a question and answer session. I would now like to hand the conference over to your speaker, Ms. Lavina Talizdar, Head of Investor Relations.
Please go ahead.
Good afternoon, everyone, and welcome to Moderna's first Manufacturing and Digital Day presentation. Before I begin, I would like to remind everyone that this conference call will include forward looking statements. Please see Slide 2 of the accompanying presentation and our SEC filings for important risk factors that could cause our actual performance and results to differ materially from those expressed or implied in these forward looking statements. We undertake no obligation to update or revise the information provided on this call as a result of new information or future results or developments. I will now turn the call over to Stephane.
Thank you, Lavinia, and good afternoon, everybody, or good evening, if you're calling from Europe. Thank you for joining us. We're excited with my colleagues to welcome you today to this first Manufacturing and Digital Day. So as you know, we started the company 9 years ago asking the question, could we turn this molecule messenger RNA, the software of life, into a medicine? From the first day, we got excited about the idea that if we could do that, we could unleash a new class of medicines with very exciting features like very large product opportunity, the ability to make secreted protein, transmembrane, intracellular, potentially antibodies, potentially viral proteins, I.
E. Antigens and also potentially the ability to combine molecules to make very exciting products for patients. So large product opportunity. We also got very excited because mRNA is an information molecule. So we thought that we could have over time a higher probability of taking on success of our drugs.
We could accelerate research development time lines because of the platform nature of what we do, if we were to invest in process development in robotics and in digital, and over time a greater capital efficiency, again, because of the platform. And so if you think about it and double click for a minute on those last three features, those just come down to the science. MRNA is an information molecule. You pick the protein you want to design in the coding region. It's the same formulation for every modality.
And that's what really got us excited of how do we build the company. As you know, because of a large opportunity ahead of us, since the very first day, we have been very focused with the team and with the Board to manage risk. And 4 risks that we talk about on a daily basis are technology risk, biology risk, execution risk and financing risk. And as many of you know, we have managed the technology and biology risk by building a portfolio, a portfolio across 6 different modalities. And in each of those modalities to manage biology risk, we decided to not do only one medicine but do several.
By managing this technology and biology risk, what we basically needed to ensure is that we didn't increase execution risk because it is one thing to take 1 drug at a clinic at the same time. It is a different game to take 24 drugs in a clinic at the same time with a new technology as you are building the new company because you are literally flying the plane as you are building it at the same time. And so it became very clear to us in the early days that we had no chance to build the company in the right way, focusing on quality, making sure we could run this clinical experiment across 24 drugs without building our own manufacturing facility. And so if you step back and you think about how we build the company, so the first phase, of course, was get to the clinic safely, invest in the science. And as many of you know, we entered the clinic with the 1st flu vaccine in December 2015.
A few months after when we got the human data of immunogenicity and we showed that the antibody that we found in human was neutralizing, we started a discussion with our Board and very quickly moved into designing a new facility, a facility of a future that would enable us through a full integration through raw materials, making mRNA, formulation, filling valves, quality control, all in one physical location. So we'll be in charge of our destiny, making sure we have the right quality, that we could move very quickly and we could control our cost and also know how. It was very clear to us because it's a new technology and as many of you have observed, it is very, very difficult to be the biotech company around the new technology. The challenges around technical development, process development, manufacturing, quality, everything has to be invented. And when you do it through contract manufacturers, we believe that Moderna is a very risky proposition because the asymmetry of upside and downside between the contract manufacturers and the sponsors are just totally misaligned.
And so we said we have to invest because we are playing a very long game. We have to invest and own our own site. And not only it has to be fully integrated, we said it has to be fully digital. We have to really leverage the ability to have access to information in real time so that we can execute properly and that we can learn very quickly. So the next phase of the company was, of course, running those experiments in the clinic and figure out through clinical data where was the mRNA technology of Moderna working best.
And as you know, we believe that in the last few months of 2019, we went through a very important inflection point in the company history. We were able to define 2 modalities that we view as core, where in our opinion, the technology risk is off the table and still exploring the formalities where we don't have enough clinical data yet to make such determination positively or negatively. So the way we think about the company, as you can see on this slide, is basically we have 2 businesses now. We have a business on the left of core modalities where we believe the technology risk is on the table and where we communicated since the JPMorgan conference earlier this year, our strategy is to double down and to leverage the uniqueness of this information mRNA platform to scale the company very quickly and to go very quickly from a drug concept around the meeting room in North Sciences' mind to a development candidate ready to go into GLP talks and into the clinic. So this is around the infectious disease prophylactic vaccine on the left in light blue and the IV systemic therapeutics that you can see next to it.
On the right side of the slide, the exploratory modalities are basically us waiting for clinical data to decide what we do with those 4 additional modalities. As we said before, if we have positive clinical data and we think the technology risk is overstable for any of those 4 modalities, we will move them as core. And again, we will double down and triple down into many more medicines. If have negative clinical readouts, we'll have to assess, do we think based on what we've learned, we have a way to recover that modality? And if so, we might decide to go back to the drawing board and go back to the clinic with an improvement in the technology.
But as we said, we'll be very disciplined with shareholder capital. And if we believe we bring fully the clinical experiment and this is not a good use of our time and capital, we will basically shut it down. And that's exactly what we're going to do moving forward as a company. 2 businesses on the left, acceleration investment, focus on BLA and on the right, being very disciplined and very patient for the clinical readout to tell us what to do next. And that's exactly what we are entering as a company, this phase where we're focused on filing multiple BLAs, so that when we get to this point, we'll be able to scale the company, and we are very excited about the next leg on this journey.
So that's what I wanted to
share with you for this introduction. What I would like to do is to review very briefly the agenda for this afternoon, where my colleagues are going to lead the entire discussion. So I'm going to hand over to Juan Andreas. Juan runs our clinical sorry, our technical operations and quality. He's going to introduce himself in a minute.
I've had the chance to work with Juan for now close to 18 years at Aegis, both of us. And then we'll do a small Q and A session on the topic that Juan will cover. And then we're going to move into the digital space. We are very fortunate to have Karim Blakhani from Harvard Business School joining us today to be able to talk to us about some important learnings and frameworks around competing in the edge of AI. And then our colleague, Marciello, who runs all the digital and operational excellence operation in the company, We'll share with you a little bit of what we are doing there and what we have been doing recently.
And then we'll be very happy to be joined by a member of Marcello's team, Dave Johnson, who is leading all of our AI and analytics across Moderna. After that, we'll do a short 15 minutes Q and A session before a couple of thoughts for closing. So with this, we will then introduce you. I'm turning over to Juan.
Good afternoon, good morning, good evening, wherever you are. My name is Juan Andres and I lead, as Stephane said, Technical Development, Manufacturing and Quality for Moderna, where I have been almost for the last 3 years. I have 30 years of experience in the pharmaceutical industry. I started with Eli Lilly for 18 years in a number of different global roles in a number of different countries, mostly in the technical area. Then 12 years in Novartis prior to Moderna, where in my last three jobs in Novartis were leading technical development, leading global quality and leading manufacturing operations for all the different divisions at a global level.
So here I am, we are in Moderna. So we are a platform. We are a new platform, as Stephane said. So many things to be learned. But before I go there, let me give you a glimpse on how our process works.
Our starting point is the DNA plasmid. And I would like you to think about the DNA plasmid as a template. And that DNA template together with nucleotides with the right enzymes and in the right buffer conditions, form mRNA. This process is done without light cells in an in vitro way. Once the mRNA is formed, we go to downstream purification, typical through chromatographic columns in order to ensure that the final mRNA product has the right purity and quality conditions.
The process then does not stop there. Once you have the mRNA, we need to put it under or inside a number of different lipids. That is our lipid nanoparticle process that would enclose cover that mRNA and in order to make it our product that would be injected with our drug product. Our drug product is an injectable form, so we need to go through the appropriate aseptic operations and then finally our quality control before the product can be released. Now as I said, we are a new platform.
So two things here. New means we have to do a lot of learning and platform means we believe that once we know how to make one, we will know how to make many. And these two equations work in distinctively. The more products to have, the more learning that you would have. And the more learning that you acquire, the more learning that you will be able to apply to a number of different products.
This applies into a number of different things. So let's go back into history. At the beginning, we have more questions than answers and we have to go and answer one of them. Can we reproduce the product? What is going to be the stability?
Can we do this at GMP scale? And we have solved most of these questions, but this is in the journey. Every time that we solved a question, we knew that it was not only for the manufacturing process, we needed to make sure that that new process would work in vivo. And we all know that making and developing medicine is a team sport. It needs to be having a number of different units, a number of different disciplines, scientists, people with experience in order to make that happen.
The last thing, we had many questions. We had a few answers. And we needed to have the patience, as Stephane said, to go one question at a time and solve them. So where would we start? The place where we started were with questions associated with what is the structure?
What is the level of purity? What is the stability, how to measure things, what is the right size of the LNP for the different products that we are going to be taking into the clinic, What about stability? How to measure from a quality control the properties of all these attributes? Now the first thing that we covered was around quality. It is very difficult to improve anything that it is variable.
We know variability is going to exist with us, especially in a new platform, but we focused on taming that variability. So quality needed to be first. So I wanted to highlight this as a control chart and the first thing that you have to do is get in control first. Once you are in control and you are in good levels of variability, you can focus in other things such as speed, scale, cost. We knew these problems were engineering problems.
We know that these are engineering problems that we will solve as they come, that we needed to focus quality first. This will become evident in a number of different examples that I will go through in a few minutes. Let's go back to Norwood. And let's start on how the process started at the beginning. So remember, plasmid mRNA LNP fill finish in the QC event.
And we needed to travel to many to different parts of the world to make that process happen. And so from one continent to another back another facility the other side of the U. S. Now we have Norwood. And in Norwood, we have vertically integrated all these different parts.
So we are capable of making plasmid, mRNA, LNP and we also have a small field finish operations where we can make a septic drug product. And then we have the quality control laboratories and the QA release to be able to do that. Now, we knew because we were a platform that we wanted to do these many times and we were going to be learning a lot. So our vision from the beginning was that we needed to digitally integrate everything that we did. We needed to go and be paperless.
We needed to capture knowledge so that we could fast go from one place to the other. So the design of digital and automation was an obsession from the beginning. So Norwood is mostly a clinical development site. That is the vision that we had and that is its main purpose. However, we'd like to think of it as 3 engines within this site.
The first one is preclinical. So the first the preclinical engine basically has made I had a lot of automation for this. So let me go 1 by 1. Let me center your attention on preclinical. Preclinical makes batches that are on milligrams level.
And to this date, we have made over 20 3,000 batches of preclinical material. A scientist in the bench enters the code of mRNA and it is sent electronically, the site at Norwood in the preclinical area receives that code and within days or weeks there is a vial bag that arrives to the patient to do the research or to do some of the development activities. That is the scale. The second scale is the 1 in the personalized vaccine unit. And you have in this area, we are making 1 batch per patient that we have in the clinic.
It is an area that it is already in the clinical. It is it follows good manufacturing practices and the scale is in the 100 of milligrams. We have made over 90 manufacturing batches in that location. I will go in one of the case studies that I will review later is around this area. The 3rd engine is the clinical engine.
That is where scale up starts and one batch can be used for many patients. In this area, we have a scale that goes from grams, 5 grams to, to date, 75 grams that we could go even higher as our needs for scale are required. So I described the platform. I described Norwood in the different areas. So what is what we have here?
We have similar processes both preclinical, the personalized cancer vaccine unit with personalized medicine as well as the clinical areas, they make MR classmates, they make mRNA, they make LMPs and they are made in a very similar way, but at a different scale, so similar processes. So we also have here the sell free processes that will enable to have less capital requirements. All of this will impact quality, speed, also our ability to scale and the cost. So 3 case studies that or three examples that we have prepared for you today. And these are examples that illustrate how much we've learned and we've been able to do in the last few years, but they are very recent.
And as such, they will also give you an illustration on how we intend to go in the future. I always say your past and your present is the best indication of your future. So let me start with the first case study. The first one is the personalized cancer vaccine unit. This is one of our programs that we have in partnership with Merck, where the patient gets a sample of their product and we need to go back and make a vaccine specifically for that patient.
And we would like to say needle to needle around 40 days. At this moment in time, the program is in Phase II and it is up to 200 patients for proof of concept. And these patients, these manufacturing batch patients need to be done at Norwood. It is not only the speed at which we need to manufacture this. This clinical trial is designed to coincide with KEYTRUDA dosing regimen.
So we need to also land the vaccine on a specific dose for KEYTRUDA. I believe it is the 3rd dose. So that is the that is the place where we land our vaccine. The third consideration that I would bring to you is as it is personalized medicine, you cannot scale up your product. You need to scale it up.
So we will always have to make in personalized medicine as many batches as patients you have. So let me illustrate the learnings that we have had with PCB today, because our success rate since we have started has improved, our cycle times has been reduced, our yield has been improved. And as a consequence of all of that, our capacity has improved. So we have made to date over 90 patient batches of personalized cancer vaccine. Granted, the scale is not as big as a big, big product.
But when you make 90 products in my experience, by the time you have made 90 products of a normal product that it is commercial, you achieve that scale in your second, third or fourth year after being commercial. And this is the part where we have gained a tremendous amount of experience. Remember, while the process is the same, every batch is different from the code from the mRNA that we produce. So it is very, very important to assess what is the quality and what is the learning that we are going to be getting with this. Let me start as I did before with quality.
As you can see here in this slide, the first part or the first batches that we made, our success rate was okay, was good, but it was okay. It was around 69%. And as we have been making more and more product as you would expect as you get more experience, our success rate has improved significantly. And you can see there how our experience has made us improve our quality. And this is not the end.
As we gain more experience, as we make more batches, this is going to improve. Next slide, please. In addition to that success rate, every mRNA is different, but it has the same specification. So think about Lot 1 is 1 patient, Lot 2 all the way into LOD90. The specifications are the same.
And even with a different mRNA, we were obsessed with having the same specifications, the same quality and they are landing in the same place. They follow exactly the same specifications. That will become important in a number of different things that I will highlight later. I also highlighted speed. So our speed has improved.
And since the beginning, we have gone faster, around 40% since 2018. Why is this important? Think about many making many batches at the same time with schedule in which you need to arrive at a certain point in time to the patient. If you want to make many, many, many patients at the same time, being shorter in your cycle time will allow you to overlap one manufacturing with the other. Therefore, you are going to be increasing your capacity and ensuring that you're going to have your delivery on time the shorter you are.
So in this case, speed is important. Speed, quality, yield will give you cost reductions and this has been our experience as well since we have arrived. So we have had the cost reduction per batch as we have gained in experience. So this is case study 1. Let me start with case study 2.
That is the cytomegalovirus vaccine, we call it CMV. And this is a case that when I first saw this with experience that I had in manufacturing, I thought this was going to be science fiction. I was making many batches, many proteins before and my first reaction was this cannot be done. How are you going to put 6 different mRNAs into the same product and basically inject it, have those 6 mRNAs arriving to the cell, transcribed into protein by the ribosomes and 5 of those 6 proteins come together into a pentamer and then another one forming the vaccine. And so we had, as you can imagine, many, many questions and things that we needed to resolve from a manufacturing point of view.
So lots of challenges ahead of us. We also knew that the CMV vaccine would be something that would have to scale to many, many patients, different than the previous case in which you had one patient at a time. So in this case, we needed to scale up. And we needed to do in a way that stability in refrigerated conditions rather than frozen conditions was going to be a good consideration. So we needed to do a number of different things where our creativity coming from platform and technical development and our ability to make product in Norwood would be very important.
And these things basically get together in the speed. One of the things that happened with CMV is within a year, we ended with a lyophilized form that basically increases our chances tremendously, is well known lyophilization, but it was the first product that we lyophilized in our platform. And we were successful taking this to Phase 2 where the product is right now. So we are at Phase 2 with a lyophilized form that we intend to take all the way into BLA, okay? So in this case, the scale is important.
So our scale at the beginning was 5 grams and we used it for CMB. Now 75 grams was our next one, but we have never done this with CMV. But we had done this with another program with BEGF for instance. And we used the experience that we had in BEGF scaling at the 75 grams to export that knowledge and experience our digital platform and our equipment conditions to basically take this to CMV. So we were able to go faster.
Next stage is going to scale. What we are thinking at this moment in time is around 150 grams, but it could be another scale. I like my chances. In addition to this, obviously as you scale, your cost reduces. But that is a factor of your scale and your quality conditions.
At this moment in time, our obsession is reduce variability, be able to scale and then things will fall into place from the rest of the things. Next slide for CMV is, as I said, we have the lyophilized image that we intend for Phase 3. Norwood can produce the Phase 3 material And we have the potential to do the scale for a potential commercial launch. And we will do the fill finish at a CMO on this LIO. The image that you have on the screen are actual real LIO products that we have on Phase II.
This is CMV Lyo product. This concludes the case number 2. So let me take you to our case study number 3, coronavirus. Fox, this was unplanned. We didn't expect this.
It landed in our lab at the beginning of January and it required a number of different things that we have learned throughout the years. So it needed the convergence of a number of different disciplines, scientists, all the things that we have learned together in addition to good doses of boldness and a huge amount of collaboration together. We also used all the different parts of the engines that I have talked to you before. And so the preclinical, the personalized vaccine unit in a number of different things going forward are going we are going to be using all parts of Norwood in this place. We didn't think ever about using the personalized vaccine unit for something different than PCV.
But then when we saw the challenge of going really fast, we said, why don't we use the learning that PCB created on us on the same platform and we use this to attempt to go to Phase 1 in for the coronavirus vaccine very, very quickly. So let me take you through what has happened in the last few weeks here. 3 days, January 13, February 7, February 23. It started on January 13. We selected or we were given the amino acid sequence that would that we would need to use to design the plasmid template that would allow us to make mRNA.
And we did that in a few days. We have the select we have the amino acid sequence. By day 2, 4, we already had the digital sequence engineering and the DNA design. Now then between days 5 17, we have to do what normally takes weeks or months, which is get all the documentation together and all the things that you are required in order to make a clinical batch. Due to the fact that we are paperless that we have the platform that we have basically we could do that at a very, very fast speed.
So on day 17, we were ready to make the plasmid, make the mRNA and make the LNP at a clinical level. In parallel, we had used already our preclinical engine to make the mRNA that pre clinically was going to be tested in vitro and we sent several copies of that to NIH as well as to our scientist inside together with a number of backups. So the preclinical engine worked to perfection as well to supply that preclinical material. So on February 7, we had made our vaccine. The vaccine was done and it was ready to be tested.
All the QC was done in a few days rather than in weeks, but we needed to wait 2 weeks for the sterility test. We cannot make VACS grow faster and the test takes 2 weeks. So we needed to take 2 weeks and wait until the sterility test was done. And then on February 23, 23, we shipped to NIH the product that was meeting the specifications. And at this moment in time, the IND is open.
There was a huge amount of motivation in the system. This is how we felt in the company. Each one of us, we channel a huge amount of motivation. People were talking to their families. I have not seen the level of energy anywhere before like this.
And that was very, very motivating. So hopefully, I've been able to convey the 3 cases that can tell you how all the learning, the platform, the integration in Norwood have brought to existing examples and you can make the connection of what is going to happen in the future. We can make the quality, speed and scale and cost that we are going to need as a company across the different products. As I said before, we can do this by like our chances.
Thank you, Juan. And with that, operator, we'll go into the Q and A session for manufacturing. Thank
Our first question comes from Yasmeen Rahimi with ROTH Capital.
Hi, team. Thank you for taking my questions. I have two questions for you. The first one is, can you walk us through what the purification cutoffs are as we go from preclinical batch to personalized canceled vaccine to clinical batch? What are other selection criteria that makes clinical batch more unique?
And then the second question is, can you walk us through how mRNA is combined with LNP to yield the final product? So what purification process ensures that we get a homogeneous drug product? And thank you for taking our questions.
Well, the first one, the verification process obviously is different at different scales. And whether you are doing this in a preclinical setting or whether you're doing this in a clinical manufacturing with GMPs. But the process is very similar. What I would say is mRNA is a large molecule and it follows traditional purification steps, chromatographic columns to be able to do that like you do in normal large molecules. For the mRNA to LNP, it's basically a nano precipitation process together with the mRNA.
You take the mRNA in an equity setting, you take the lipids in a synthetic into alcoholic where the lipids are in solution and you bring them together under normal conditions. And then you go through a process with the right sizing to make sure that we end up into with the right quality. Normal scale up processes are followed in terms of different sizes of the purification or sizing equipment to end up or to be able to process at the scale that you can.
Thank you. Thank you. Our next question will come from Matthew Harrison with Morgan Stanley.
Great. Juan, thanks for taking the time for this. We appreciate it. I guess what I wanted to ask is, can you just broadly talk about the scale that Norwood was built for, I think you've addressed that in terms of clinical stage batches and what you've built there in terms of what you've learned, is this something that you can scale easily into a larger facility? Or are there additional steps or things that you would have to work out if you had to scale to commercial to a commercial facility?
Thanks.
So thank you, Matthew, for the question. I think, Tim, you're right. I mean, we need to scale. We need to scale up. And our learning in the scale has gone faster than we anticipated.
I think we are at a scale level that in our ability to go and learn from 1 product to the other in the scale that we have achieved is good. As we go into commercial, the scale at which the site is designed will allow us to go into 1st commercial scale. The processes to go into a bigger facility with bigger commercial things is not going to be different than the scale that we have done from milligrams to grams to 100 of grams. And at this moment in time, we expect no issues in doing the normal scale as we would do as you would have. This is less complicated from that point of view that other type of scale ups.
Thank
you. Speakers, I'm showing no further questions at this time. I'll turn the call back over to you.
Great. Thank you. It is now my pleasure to move on into the digital portion of the presentation. And I'd like to invite Karim Lathani, professor at Harvard Business School to kick it off with his talk on competing in the age of AI.
Thank you, Lavena. It's a pleasure to be here with you. And I'm going to take some time to sort of walk you through a perspective that I've been developing with Marco Yanceidi, also a colleague of mine at our Business School on how a new breed of company is emerging that is using data analytics and AI at its core and that these companies compete in a very different way, they operate in a very different way and then subsequently also have different types of market perspectives and valuations. This work emerged out of about 7 years of research that I've been doing with Marco as we were looking at how the dynamics of the software industry, what was happening in the software industry around both ecosystems and platforms was basically spilling over into the rest of the economy. And as we began our research, what we started to see around 2015 and 16 was that this was not just a platform story, but really a view platform story or business model story, but really a story of operating models and how changes in the operating model of the company imbued with data and AI would make a huge difference.
So I'm going to walk you through that thinking and of course we'll have some time for Q and A as well. So the first thing I want to sort of show you is this image, this image, this painting, which is known as the next Rembrandt. And this is a digital rendering done by AI of a Rembrandt photo, of a Rembrandt portrait. And the amazing thing about this portrait is that this emerged out of sort of mere mortals working in Europe, looking at trying to imagine what Rembrandt would do as his next portrait 400 years later. And what they did is they scanned a bunch of images of Rembrandt paintings.
They used some statistical algorithms and then literally generated this painting by itself. Do we have the video? Great. So let me show you how this comes up came about as a way to sort of ground you as to why we think this revolution is going on.
One of his great achievements, one of Rembrandt's great achievements was to portray human emotions in a much more convincing way than artists had before him, and in many ways, for all time.
Power of innovation, what it can mean to people. We want to bring this innovative spirit to our sponsorship of Dutch art and culture. We knew that for this challenge, we needed to team up with experts from various fields to make this come to life.
We're using a lot of data to improve business life. But we haven't been using data that much in a way that touches the human soul. You could say that we use technology and data, like Rembrandt used his paints and his brushes to create something new.
The first step was to study the works of Rembrandt in order to create an extensive database. We gathered the data from his collection of paintings from many different sources including 3 d scans and upscale images using a deep learning algorithm. Because a significant percentage of Rembrandt paintings were portraits, we analyzed the demography of the faces in these paintings looking at factors like gender, age and head direction. The data led us to the conclusion that the subject should be a portrait of a Caucasian male with facial hair between 30 40 years old in dark clothing with a collar wearing a hat and facing to the right. From there, we started to extract features only with faces that were related to that specific profile.
And we had to create a whole painting from just data And we used statistical analysis and various algorithms to extract the features that make Rembrandt Rembrandt.
We took parts of the face and we started to compare them. And then based on this we're able to create a typical Rembrandt eye or nose or mouth or ear. After generating the features, we were focusing on the face proportions. We used an algorithm that can detect over 60 points in a painting. We were able to align the faces and to estimate the distance between the eyes, the nose, and the mouth and the ears.
A painting is not a 2 d picture. It's 3 d. You can see the canvas, you can see the process, and that's what makes the painting come alive. A height map is essential to make the painting a painting.
We incorporated the height map into the painting and printed on a 3 d printer that uses a special paint base UV ink. It printed many layers, 1 on top of the other, which resulted in the height and texture of the final painting.
Sometimes a magical moment to see a painting for the first time, even if it's computer generated. For me, it is something special. I would have believed if I would have saw it in a museum that it would have been a real Rembrandt, just one I haven't seen before.
It will be interesting to see Rembrandt looking at it. He will be happy that there are people trying to understand human, trying to create something out of that. So I think he will be happy.
The next Rembrandt makes you think
Great. So the point about videos is to show you how something magical can come out with a question in mind and then real emphasis on gathering data, doing the analysis and then using pretty much off the shelf algorithms to be able to pull this off. And for me, the big realization and for Barco and the big realization was that the ways we think about AI in the popular press are really geared towards what we think about as a strong AI, the Star Trek computer, right, that you can ask any question that gives you any answer. That's known in the computer science world as strong AI. But weak AI, which is a definition that's also in computer science, so he talks about any activity that computers are able to perform that humans once performed.
That's known as weak AI. And what's important about this is that the algorithm that we saw, the folks at the next Rembrand project do are geared towards very specific activities and are able to just generate the next Rembrandt. They're not going to generate the next Renoir, right? For that, you will have to go back and redo all the data collection and so on and so forth. But the notion is, can we industrialize this process in a way so we can do this at scale.
And so the weak AI goes a long way today. And that's the basis of how to start to think about deployment of weak AI algorithms can actually give us tremendous benefit. One of my favorite examples of this comes from China, JD Digits. So they were asked by the Chinese government to come up with products and services that can help the Chinese agricultural economy. And one of the areas that went after was insurance for farmers and they were focused in on thinking about how do we insure livestock, especially pigs.
And so what they realize is that a farmer may not be able to insure the entire livestock of and they might want 100 pigs instead of 500 pigs to be insured. And this becomes a problem because how do you know which pig has been insured. So they went ahead and they used patient recognition for pigs to be able to identify which pigs have actually been insured and that enabled them to then generate a whole range of insurance and financial services products for farmers in China because they deployed off the shelf algorithms with the new problem, got this data and got going. And what's interesting to me about this example is that JD Digits is not going to launch a pick counting or a pick fish recognition software and get rich off that. But what they can do is they can take these algorithms and deploy them in their processes and create brand new services and products for their customers, the Chinese farmers.
Now you can imagine what else you can do. Once you can do facial recognition, you can also keep track of health of pigs, you can keep track of what they're eating and change diet and all those can then become products and services that you can then go to the Chinese farmers with and enable them to succeed. So here's an example again of a brand new type of a business being generated based on attention to data and taking algorithms to go and solve these problems. Now let me also show you one more example of this as
well.
So the other example I'm going to show you is something that I recently published in JAMA Oncology on an AI system that we developed in for radiation treatment planning. And this algorithm basically came about through a crowdsourcing contest for algorithms that we launched in last year. And in the space of 10 weeks and about $50,000 in prize money, we were able to be as good as an average radiation oncologist at Harvard, but also importantly, better than any commercial software that was already out there. Now the thing to note is, I'm one of the co authors on this paper. The team basically the first author and the last author are radiation oncologists.
The rest are physicists and data scientists with 0 background in biology, being able to again attack this problem from a data centric perspective. And that allows us to then go at this and within again in a matter of weeks develop algos that are solving a critical problem in radiation treatment planning. And the point again here being that these algos are available at scale. Let's now think about this from a much broader perspective, which is what happens when these algorithms are deployed at scale and how this in fact can start to change the nature of the firms. And my favorite example here is Ant Financial.
Ant Financial, as some of you may know, is a company that has spun out of Alibaba. Initially, it started as a escrow service for payments so that you could build trust between consumers and merchants in the Chinese marketplace. And then since 2013, when they became and finally spun off, they have grown massively to the point now where they serve 1 point 2,000,000,000 consumers on their platform and they do this with 10,000 people. So if you sort of think about the leverage that they're getting, it's pretty incredible. And Financial has not just a payments platform just like PayPal, but they also have, as you see, other products and services based off on this core platform.
So they also have and Fortune, which is the largest money market fund. They have MyBank, which is their banking service provider. They have a cloud service system and so on and so forth. The things you note about some of these capabilities is that my bank, for example, has a 310 objective. 310 says that they can if you apply for a loan for the bank, it takes you 3 minutes to apply, one second to approve and 0 human intervention, right?
Compare that to your experiences with banks in the U. S, very different scale. The other cool example I found is that recently they launched in China mobile phone insurance plans geared towards young women who wear tight jeans. And you go like, what, how are they able to do that? Well, they had enough data on the consumers to note A, shopping habits and B, the fact that young women who were buying tight jeans were breaking their phones.
And so they could then use that data to come up with a brand new product and launch that in the marketplace as a way to say what they can now do with the levels of data that they have. Now Anne itself has scaled rapidly. They're currently worth about $150,000,000,000 in the private market and they're approaching the scale of the U. S. Banks today.
Now again, the joke I make is, again, they have 10,000 employees serving 1,200,000,000 customers. My bank here in Boston would probably need like 1,200,000,000 employees to serve 1,200,000,000 customers, just given how manual their processes are. So just to put this into a perspective, we want to sort of think about companies like Ant Financial really from a STRAT point of view of business models and operating models. Business models are really about your strategy, how you create value and how you capture value. And operating models are really about how you think about how you achieve scale, how you serve more and more customers, scope, how you in fact serve them as many different products and learning, which is how you innovate and improve on that.
The operating model is the value delivery mechanism for your business model and they go hand in hand. And the prospective operating models has been around for quite some time. So for example, a traditional operating model that might be at a Ford or Sears or Toyota is set up in a way such that we are siloed. We're siloed in our data, in our infrastructure, in our capabilities, right? And so this silo perspective is very useful because it allows us to focus and deliver value quickly to our users and to our customers.
But then this also has its limits. The limits are that basically over time, we get diminishing returns. These operating models peter out because of complexity in the organization, administrative costs, inertia that we basically tend to peter out with these systems. What we see happen instead with a company like ANZ and other digitally native firms is that they are set up in a way to drive increasing returns to scale, scope and learning by incorporating data and analytics at their core. The core operating model is set up in a way so that the different parts of the platform interact together through their systems and as they get more data, they get better algorithms.
As they get better algorithms, they get better services. As they get better services, they get more usage and so on and so forth. And the thing to note though is that what this is telling us is that AI analytics and data are now basically relieving the human centered bottlenecks that have existed for years on end inside of most organizations. And as Satya Nadella has said, AI is sort of in many ways, it's a new runtime that shapes all we do in most organizations going forward. Now one of the concepts we talked about in the book quite a bit is that this operating model at Ant is enabled by an AI factory.
The AI factory does 3 things and the 3 things at scale. The iFactory produces predictions about some future event or some future customer need or some future activity. The AI factory does classifications. So it tells you patterns in data and tells you how those are patterns are being achieved. And third, it drives automation.
So those are the three things that are the core of any AI factory. The big thing though in the AI factory is building your data pipeline. I can't tell you how many companies scale at not being able to build a data pipeline that can actually ingest data from all different sources, normalize it and put it in place so that we can drive algorithmic predictions as well as have an infrastructure that can actually make this happen. Now one thing to note here is that the AI factory that you see here is the same whether that's in Ant Financial or if McDonald's put their own AI factory, right? Because the Hamburger factory is very different from a finance factory, right?
But the data factory, the AI factory underneath is going to be looking the same. And that also drives convergence with your operating model. What happens in this kind of an operating model core is that the data is shared across the entire enterprise, software components and libraries and APIs drive much of the interactions across all the systems and then applications sit on top to be able to take advantage of this going forward. Now the result of this AI factory is that we have 2 different things going on. Companies like Google, like Amazon, like Facebook, like Alibaba have network effects.
So as the number of users increase in their platforms, the value of the inherent service itself increases and that we've seen this in many consumer products. But also interestingly, as they get this usage data and as they get more and more of this data, they can sharpen this value curve even more because of AI. And we see both of these two things happening at scale in these companies. And it's no surprise to us today to think about this in the sense of why we see companies like Microsoft or Google or Facebook leading the race in AI research as well, because they're faced with immense amounts of data and they've figured out how to deploy AI to capture even more and more value. Now what happens when this traditional model of diminishing returns meets the model of increasing returns?
Well, we get a collision. And what you're seeing is this collision is around both business models, but really around operating models. And thinking about this is very important because what happens is that the traditional business delivers value early, right, and keeps going and it peters out. In the digital business, you actually take a while for you to grow the network effects and the learning effects. So that takes a while to grow.
And so a traditional business looking at the digital business will say, there's no value, there's no value. I'm not going to invest in change to sit up this way. But as we see in the marketplace today, these collisions are happening left, right and center. You think about Marriott versus Airbnb, the auto companies versus Waymo and what Waymo is doing, right, or banks versus and financial. And these are collisions of operating models, companies that are set up in a very different way to drive scale, scope and learning zoom out and sort of tell you what we think in the book as sort of the new rule set that has emerged for companies.
The first one really relates to universality, which is universality capabilities. What we see happening is from Andreessen's sort of software eating the world to actually digital and AI becoming the core of most organizations, right? That's going to be the core of most organizations. But whether you be a platform or not, you will need to be able to plug into platforms and be able to play in this way. Then what we see is this universal capability around the fact that the digital operating model when it collides with the traditional operating model will lead to a collision and in fact a separation and value creation and value capture for the digital operating model.
But the scale of times take some time, right? And so we have to be cognizant of that and figure out how that comes through. But importantly, this skill set is universal, right? We need this skill set in all of our companies going forward, right? And that most product companies will have to also build a skill set inside of their own organizations.
The next thing we see is that there is a ton of opportunity, a ton of opportunity everywhere as you look through that. And what we see is that the model here is to take human labor and management off the critical path, right? We develop rules for decision making and we develop algorithms to enable us to drive those rules, right. But we actually in fact have humans as the designers and operators of the algorithms of the factory, but not as the factory workers. The factory workers are actually the algorithms themselves.
And the application space for all of this is immense and keeps growing. And that's why we sort of see separately sort of all the type and boom in AI. The type and boom in AI has come about because, a, the cost of generating algorithms have dropped. You need proprietary data, so you definitely need that. But everybody sees opportunity into how we can change operating models to succeed in this way.
The next thing to also note is that we also need new strategies, right. So as I sort of showed you that collision curve, we actually have to become smart about thinking about how do platforms and AI actually allow us to change the shape of our value curves, right? So the much maligned rework example is one of basically leasing business with slightly better economics, right? So you can imagine that they were still going to be in this diminishing returns curve, we add a layer of software and digital and it moves us to more of this higher dash line. What's the trick for us and for many companies is to say, how do I go from this diminishing returns to this increasing returns curve?
And what you see is that different companies act in different ways. Uber is one where there are benefits to increasing usage and increasing data, but because Uber faces lots of competition in local markets, right, it's not going to be able to get this massive benefit of a sharply increasing curve, right. Already I heard recently that once again in New York, there's a new competitor coming into the New York market to compete against Uber in ride sharing, right? And so the barriers to entry are quite low. Facebook, of course, has a slightly sharper value curve.
But the interesting story about Facebook is that the networks that deliver value are small there, but at the scale of 100 to 200 people. And what they have become really good at is adding a lot of scope in terms of products and services so that they can then build their advertising business from that. And of course, Ant Financial very sharp because they have sort of become this thin life juggernaut. They set up themselves in a way to be able to provide a whole range of products and services around financial interactions in the Chinese economy, right, and then using that data to then generate new types of products and services for themselves as well. So what this means though is that this type of a firm also has new sets of value.
So this is a very crude measurement, which is market capitalization for employee that I've set up. And what this shows you is that depending on the firm, you get different levels of leverage in market cap per employee. But when we bring in these digitally data firms like a Facebook or like an Ant Financial, what you see is almost an order of magnitude increase in valuations per employee because they can do more. They have data and analytics at the core that enables them to scale faster, to increase scope in a new way and to then continuously keep learning and improving themselves. So this all of this is based on a book that industry finished up, it's Competing the Age of AI.
And this book really sort of goes through the details around what we see emerging as a new type of a company. And also importantly, what are the sort of the challenges and ethical responsibilities for us as we enter this new age of AI.
Thank you, Karim. And we now will transition over to Marcello Damiani.
Thank you, Karim. That was really amazing. I encourage everyone to read the book. Good afternoon and good morning and good evening to everyone. I'm the Chief Digital Officer and Operational Excellence at Moderna.
I have joined Moderna 5 years ago, and I will be walking you through our digital strategy. Before joining Moderna, I was the CIO at BioMarieu, a diagnostic company. And prior to this, I worked for large scale organizations such as Motorola and the high-tech industry. And early on in my career, I was software engineer supporting the aerospace industry. So this is my 3rd industry today.
As mentioned by Karim and reflected on here by The Economist, data has become the most valuable resources in the last 2 decades. And the most successful companies were mainly data centric Advanced analytics is what provide insight to data and AI has become the holy grail of analytics as it allow us to analyze large amount of data sets that are impossible to analyze with other techniques. However, AI requires extremely large structured and well annotated data sets. Internal and external data sources that are linked and made accessible, and a digital and data centric organization to help capture the data from inception in an integrated fashion, whether internal or external. For advanced analytics and AI to take off, digital foundation has to be ready.
At Moderna, since day 1, we have decided to build a data centric company and we have integrated automation and digital technology into everything we do, as you can see here in our 10 year vision. Why building a digital biotech? First, our technology is information based. We instruct the cells in the body to produce protein and those instructions are like software code. So the very essence of our science is digital in nature.
2nd, we have a platform approach centered around modalities and learning inside and across modalities is key. 3rd, our business strategy is to progress multiple programs in parallel internally and with our ecosystem of partnerships. So building Moderna in analog world would have been very tedious, if not impossible. What are Moderna's digital building blocks? 1st, the cloud.
The cloud provide us with the agility to operate cost effectively without the limitation due to long implementation cycle times and the capital investment required for traditional approaches. AWS has been a great partner in delivering those capabilities. 2nd, this is what Kareem mentioned is really integration. Integration is the glue that bring our processes and data together in a consistent manner, avoiding silos of information. We see many companies struggle with integration due to legacy systems, silo data and processes.
And therefore, what they do is they build inefficiencies across all the functions. And this is exactly what we want to avoid. Growing lean is keen to us. The 3rd piece is the Internet of Things. We view this as the next level of integration with real time collection of data from instruments.
The 4th building block is automation and robotics. Our strategy here is to automate when and only when the processes are mature. We do not want to automate too early as this will reduce our agility to evolve and change the processes, but not too late as we want to reduce any manual work as early as possible. The 5th pillar is analytics. Once you have all the building blocks, you have a wealth of data that provides you your business insight to make informed decisions.
Finally, AI and machine learning, as previously discussed, are the Holy Grail of analytics and the ultimate level of our digital strategy, using predictive analytics to help with real time decision making and predictions. There are 4 benefits that we see in our digitization strategy. 1st, quality and error reduction, which increase our probability of success. 2nd, speed and the way it accelerates our learning and increase our agility. 3rd, scalability allow us to accommodate growth and demand way more efficiently and sustainably.
And 4th, cost reduction, driving efficiencies, reducing waste, whether it's cycle time reduction, more efficient use of skilled resources or else. The key neighbor of this strategy are first the people. We have a digital culture. Everyone at Moderna embraces it and we partner with them to ensure that the technology is serving their goals. The second piece is process.
We make sure that we build our processes from the ground up for our digital world and we use operational excellence techniques like Lean, 6 Sigma or else to build optimal digitized processes. The last piece is technology. We continuously explore technologies and innovations that help us resolve critical problems and we fully embrace those new technologies when we
see benefits.
To achieve our vision, we digitized every function at Moderna, research, technical development, manufacturing, quality, clinical development and the business functions that I won't go through today. This has created a virtuous cycle of learning where we run more experiments that generate small data, which help improve our algorithms and learning, which in turn help us develop better messenger RNA medicines and the cycle continues. Now let's dive into each of these areas. I'll start with research, where we advance ideas from development candidates sorry, advance ideas to development candidates. In research, our scientists start with an idea for a new messenger RNA medicine.
To help them realize their vision, we built the drug design studio, a web portal that allows them to design messenger RNA and protein sequences. To date, our scientists have designed more than 20,000 unique messenger RNA sequences. Once designed, the scientist has the ability to order the sequences and formulation directly from our high throughput preclinical production facility in Norwood. After the order is placed, we run automated algorithms to optimize the sequence for expression and ease of production. The orders are routed to our preclinical production, which is integrated with 30 plus robotic platforms and where we collect data at every step of the process.
We use this data to continuously improve the process. For example, we have built a machine learning model that can predict yield and inform the operators on the actions needed to meet the demand. With those apps and automation, we were able to scale in preclinical production from 40 different orders per month to up to 1,000 a month, allowing scientists to do way more experiments. Once the messenger RNA have been produced, scientists use a set of digital apps to run their experiments. For example, the registry and inventory app allow us to create and track any material used and created in research and in preclinical production.
This includes DNA, messenger RNA, lipids, cell lines, chemical and much more. The app manages consumption, aliquoting, material transfer and stock alerts. As of today, we tracked around 1,000,000 items. To manage in vivo and in vitro studies, Moderna scientists use the in vivo and in vitro study apps to help them design, schedule and track progress of their study in real time. The goal is to optimize overall study outcomes, resources and cycle time.
The ass also captures structured data and detailed study condition for analysis. Finally, included in the app over 20 data analysis and visual tools that automate tedious calculation and standardize the analysis so the results are comparable across the platform. The second functional area I'll be covering is technical development. The technical development requires us to capture structured data while providing engineers enough flexibility to mature, optimize and scale processes. So how do we do that?
In fact, we built 2 apps. The first one is the platform editor app that allows Tech Dev Engineers to build digital templates of their processes composed of multiple unit ops, which are reusable building block of a process. Each unit ops is composed of a collection of process parameters, material, equipment and calculations. This allows the engineer to capture all the information as the process evolves with the goal to reduce the cycle time of tech transfer and electronic batch record design. The second add is the Development Hub.
It's a process experiment execution system that captures and document experiments, design and results. It performs automated real time calculations, support multi arm experiments and generate automated reports. The development hub integrates with our analytic development app to streamline submission. This has allowed us to reduce testing cycle time by 36%. The next function is clinical manufacturing.
In our GMP manufacturing facility in NOL, as Juan mentioned early on, we are able to operate the shop floor in a fully integrated and digital fashion. Paypal's execution, electronic batch records, automated material flow, review by exceptions, and we are collecting around 7,000 variables in real time. Integrated electronic batch records instruct operators which raw material to use. All materials, including product intermediates are barcoded scanned to ensure accurate inventory. All consumable materials are replenished at the click of a button using custom built inventory replenishment system that leverages AWS IoT buttons.
Integrated electronic batch records instruct as well operators which equipment to use, verify that the equipment is ready for use, control operations and collect and analyze the data real time. With this fully integrated shop floor, we reduced labor by 40% per suite per shift and the average cycle time by 2 days. We built as well visual dashboard of all the suites in Norwood that allow operators and managers on the shop floor to monitor all operations real time. They can visualize and respond to alarm conditions, check the schedule for each suite and view equipment calibration status. In this next section, we will discuss how we have also digitized our quality process from testing to releasing medicines for clinical trials.
Quality is designed in every step of our digital strategy. For example, the sampling of product and raw materials testing is done digitally through real time integration with our laboratory information management system, the Lens. Sample labels are printed and applied on the shop floor. When samples are dropped at the pickup location, all information required for testing is passed digitally to the LIMS system to enable QC testing. When QC tests are approved, the results required for batch record calculations are passed back automatically.
Before releasing a batch, all batch record exceptions are captured digitally and can be reviewed by supervisors and the quality assurance team in real time. This eliminates the need to review thousands of pages legacy paper records. Instead, the quality team reviews and approves exception significantly reducing cycle time. We reduced manual batch records error by 85% and the review cycle times from 3 days to 3 hours. Data integrity is a key component of a digital strategy in a GMP environment.
Therefore, we implemented for our QC Labs a data integrity solution on each workstation that capture equipment data in real time sent to the cloud with audit trails and security controls. We have built also paperless environmental digital from the start. So all environmental qualification was paperless and audit ready, in addition of being very effective and efficient today. The last section I'll be discussing here is our clinical development function. In order to track the progress of the various development candidates, we have created a set of integrated applications.
Workflows include timelines of regulatory filings, planning for IND enabling GLP toxicology and clinical operations management. Our clinical operations applications allow us to track our ongoing trials by accessing clinical operations information in real time from our CROs. It also has multiple tools and analytics to draw key insights. Among the tools we have is enrollment forecasting that track actual subject enrollment relative to plan, a detection of enrollment issues, enrollment trends across the portfolio of actual target and baseline It help us easily It help us easily compare key metrics against industry benchmark, publish all type of metrics, categories, performance, quality and risk, allowing the team to track progress. You can see also here how we monitor site activity, including training and trip reports.
We automatically flag reports that fall outside of contracted windows and we have a heat map of the clinical sites in the USA. I hope this gave you a quick but comprehensive view of how we are building Moderna as a digital biotech. And with this, I'm handing over to Doctor. Johnson, who heads our Informatics, Data Science and AI Group. And Doctor.
Johnson will share with you how as part of our digital strategy, we have progressed advanced analytics and AI.
Thank you, Marcello. It's a pleasure to speak to everyone today. As Marcello mentioned, I'm Dave Johnson and I run informatics, data science and AI here at Moderna. My role is really to deliver digital and AI solutions to better enable research and clinical development. I have a PhD in information physics and over 15 years of software engineering and data science experience.
The last 10 years have been exclusively in the pharma and biotech environment. So I'm delighted today to talk to you about how we're leveraging AI and advanced analytics at Moderna. We use these all across our platform. What you see here on the slide in the left column is a selection of all the algorithms we're running in production today. The first two highlighted at the top here were actually leveraged in our novel coronavirus vaccine.
The first one to optimize our mRNA sequence and the second to determine the optimal DNA template construction. The next three algorithms that are highlighted there are used regularly as part of the personalized cancer vaccine workflow that Juan spoke about. We use an AI algorithm to design a personalized mRNA vaccine for an individual patient and then again use our mRNA sequence optimization algorithm. We also use Monte Carlo simulation to plan our complex PCB supply chain. Coming up right behind these in the center column are many new algorithms that are in development or final stages of testing and will soon be out in production.
Across all of these algorithms, we leverage a wide variety of computational techniques from simple rule based optimization to more sophisticated neural networks. There's really no one size fits all techniques, so it's critical to have a large toolbox at your disposal. So what I'd like to do today is for the rest of this talk is dive deeper into 2 of these use cases that are really good examples of the kind of AI work we do at Moderna. The first use case is in Sanger sequencing analysis. This analysis is used in our preclinical production process to QC our DNA templates.
This is the very first step of our production process. I'll show you how we built an automated data analysis pipeline using an existing algorithm. I'll then walk you through how we extended that algorithm using a neural network to address the challenge of their poly tails. Finally, I'll show you how we build a radically new algorithm from the ground up that achieves even better quality performance. The second use case on the right here is in our research space.
We're using cryo EM images to study the lipid nanoparticles in our formulated drug product. Analyzing these images requires a sophisticated 3 stage analysis using 3 different algorithms. Let's first dig into the Sanger problem. Sanger sequencing is an assay that allows you to determine the nucleotide sequence of input DNA. While Sanger is an older technology and newer next gen sequencing tools have emerged, Sanger still use it as the workhorse in most DNA synthesis operations due to its low cost.
At the bottom here, you can see an example of a chromatogram called a read generated by a Sanger sequencing instrument. Each peak corresponds to a different nucleotide base in the input DNA. The relative peak heights aren't very important here, but the color of the peaks are, as each of those corresponds to a different nucleotide base A, C, G or T. So we're just looking at the color of each peak, you can read off the nucleotide sequence. And this assay is really important for us because we use it to QC the DNA templates we use to produce our mRNA.
If just a single nucleotide base is mutated, insertion, deletion or modification, that mutation will be transcribed into the mRNA, which will then be translated into the final protein, resulting in a non functional protein. So it's critical that we catch these kinds of mutations very early on in this first stage of creating the DNA template. In practice, the analysis isn't quite as simple as just reading off the nucleotide sequence. The Sanger data read is noisy at both ends and will only give you around 700 basis of high quality data. So you need to collect multiple reads across your DNA in both directions and then combine the results.
The full analysis requires a complex 5 step analysis pipeline. We built our high throughput preclinical reduction engine about 6 years ago. We didn't have much Sanger yet, so we didn't have the data we needed to build our own algorithm. Instead, what we did is license an off shelf algorithm, which has been an industry standard tool for the last 20 years used by thousands of labs. However, what we did realize is that we'd be generating a lot of Sanger data.
It takes on the order of 100 Sanger reads per mRNA construct we produce. So what we did is we automated the Sanger analysis pipeline using this algorithm. The actual Sanger wet lab work is done at an external sequencing vendor as this work is fairly commoditized. However, at the end of the sequencing analysis, the instruments at the vendor upload the data to our cloud storage environment. The moment that data hits our environment, automated processes kick off and spin up compute servers in AWS' Spot marketplace.
These Spot servers are pulled from Amazon's excess capacity, so we pay pennies on the dollar. The servers run the Sanger analysis on the raw data and send the results to a custom built Sanger app. When all the data has been analyzed, the servers are automatically shut down and discarded and we only pay for the time we use them. This approach is an extremely cost effective way to perform large volumes of computational analyses in parallel. This custom built Sanger app not only captures the results of the analysis and also is a powerful interactive visualization tool that allows the operators and scientists to inspect the data, the analysis and any problems that might have been found.
In this example here, you can see a target sequence at the top and a couple of aligned reads at the bottom. See the data from the bottom read is further along, so the peaks are harder to distinguish in lower quality. The consensus of the reads here shows that there was a single G deleted out of the thousands of nucleotides in this sequence. The analysis found this needle in the haystack and so this sample was discarded. Since we automated this analysis some 6 years ago, we've now collected over 3,000,000 Sanger reads of data across hundreds of thousands of samples.
As we process all these samples, we've unfortunately discovered that there are a number of very rare failure modes that are not detected well by the license algorithm. With all this data we've collected, we can now turn around and then build better algorithms to catch these cases and improve the quality. The first major failure mode that we tackled was in our poly A tails. As you might know, our mRNA sequences end with a long stretch of As that give the mRNA stability. We build these tails into our DNA templates that will be transcribed into the mRNA.
We use Sanger to analyze these tails just like the rest of the sequence. Here at the top, you can see an example of a high good quality tail. You can see the repeated A peaks in green. There's a little noise that's underneath the peaks, but overall it's a very good signal. On the bottom, on the other hand, is a bad tail.
In fact, it's an extremely bad example of 1. You can see these large secondary peaks sitting underneath the tail peaks and what these typically indicate is subpopulations in the sample with truncated tails that are shorter than they should be. Now the real challenge in this problem is that the while these sub peaks are really obvious to all of us who are looking at them, the algorithm we license is not able to detect it. Calls that read a little bit lower quality, but it still passes it. So unfortunately, while we had automated the majority of this whole Sanger analysis process, we still had to have trained operators weigh in at the end and manually review these tails.
That's obviously a very tedious and expensive activity. So we set out to automate this part of the tail analysis. The first thing we did is we had trained operators review thousands of tail chromatograms in a controlled setting and as a label in this past or failed. In all, we generated over 20,000 labels. And the first things we did with this label data is see how consistent our operators were at making these calls.
You can see the results here in this chart. On average, 2 different operators reviewing the same image multiple times only agree with each other about 70% of the times and some of these combinations are as low as 42% agreement. But it doesn't stop here. You can see on the diagonal these boxes, we show the same images to operators more than once, so we can determine how consistent they were with themselves. You can see the self consistency here.
On average, operators only agree with themselves about 90% of the time. That means that if you show an operator tail image from this set 10 times, one of those times they will change their mind. Now in an analysis that's used for quality control, consistency is paramount. So this is a problem. So this next slide here, you can see the individual operator performance relative to the team averages using what's called an ROC graph.
On the left, we have the true positive rate on the vertical and the false positive rate on the horizontal. A perfect performance would be in the very top left, whereas the diagonal line here in the middle is the performance of a random classifier, essentially a coin flip. Each colored point here is a different operator. You can see that in general, the human operators are actually pretty good. They're very far away from that random classifier, but no one is perfect or close to perfect.
But the distribution that you saw the variability of operator performance is also apparent here where they're scattered around here. You can see some operators are just in general core performers, some are more strict operators and tend to fail more than average. That's not a particularly good thing either because they're throwing good samples. And then some operators are more lenient and let more samples through. So we did is we took this training data and we built a convolutional neural network or a CNN that is predicting this overall team performance.
This CNN is the same type of network used in state of the art image analysis. The gray shaded curve there is where we can choose for this algorithm to operate. What we did is we chose a point up in the top there that is a balance between too strict and too lenient. Now what you'll notice if you look carefully is that this algorithm achieves superior performance any single operator. So this is an example of when Karim talked about strong AI where the AI outperforms the humans.
We put this algorithm into production over a year ago and have saved countless hours of manual tail review. But beyond that, the algorithm is 100% consistent. Given the same tail image, it will always return the exact same result. But we didn't stop there. In our pursuit of higher and higher quality, we discovered numerous other failure modes like mixed spaces, contamination, mispriming, failed contigs, peak drift and so on.
It turns out that all of these problems are rooted in the same mixed space problem where these sub peaks are not well handled by the algorithm. If we can solve the mix based problem, we can solve the rest of these. In this particular example that you see here, you see a significant G peak underneath the C peak here. And what that means is that approximately 30% of the DNA in the sample has a mutation of C2G and that's hiding under the dominant peak. Now that means that 30% of the protein that we would make from this DNA template through the mRNA would also be mutated and non functional, which would be huge hits the potency.
So it's really important that we can detect these failure modes as well. We realized we couldn't just layer another algorithm on top here because there's a fundamental limitation on how the existing algorithm handles these sub peaks. So we decided to build an entirely new Sanger analysis algorithm from scratch. To do this, we reached for Bayesian inference, which is a great tool if you have a good understanding of your data generation model. There are 2 key innovations here.
The first is in the peak detection phase, which is the image on the left here, where you need to locate the sensor and heights of all of the peaks, including these sub peaks. Here we designed a mathematical model based on all the historical data we have and used a Bayesian math analysis to find the optimal fit. In this image you can see how the solid lines here are the raw data and the dashed lines are the fit to the model. There's a really great correspondence here showing an excellent fit. Once we have accurate peak amplitudes, the next innovation is to determine what nucleotide basis those amplitudes indicate.
To do that, we use Bayesian hypothesis testing, which evaluates the probability of all possible bases and all possible mixtures of those bases. Here you can see the same mixture on the left from the previous slide. With just a single read, the algorithm is going to still call it a C, because these kinds of mixtures are so rare, but the faded out color of the C indicates that it would be flagged as a low quality or a warning C. However, a really nice feature of this technique is that it calculates the cumulative evidence. So if you see this pattern 1 or 2 more times, the algorithm is now convinced that this is indeed a mixture of a C and a G.
And then furthermore, it's able to quantify the exact percentages of each base. So here it sees that that percentage is actually 26.5 percent of the population with the mutation. These kinds of basing techniques are notoriously computationally intensive. This new algorithm takes about 100 times the computational power of the previous one. But with modern compute servers and AWS, the analysis only takes
about 10 minutes to run.
So with Sanger, we looked at using AI for this step at the very beginning of our production process. Now switch to 1 in the very end of our process, where we're looking in the research space with cryo EM. We're trying to see more about our lipid nanoparticles. After creating our mRNA, as Juan mentioned at the very start of his talk, we combine the mRNA with small molecules and lipids to form lipid nanoparticles that encapsulate the mRNA. These LNPs are our final drug products.
The exact structure of these LNPs is heavily dependent upon the specific components that go into the particles and the specific process by which we combine them. Cryo EMs are images from an electron microscope that take pictures of our LMPs. In this image here, you can see a number of darker circles, each of those corresponding to a single LMP in a sample. Now usually these images are just viewed at qualitatively and people make judgments on them. But we're doing here with AI is collecting quantitative numeric measures from these images.
This is important first because we want to gain insight into how variations of our LNP components and processes impact the LNP assembly. But second, we want to understand how variations in these assembled LMPs will impact downstream in vivo performance. To date, we've generated over 500,000 of these images in countless process variations. But unlike in the Sanger problem, there's not a pre existing algorithm we can turn to. We have to build our own from scratch.
The first step of this analysis is called segmentation. This is where we identify the parts of the image that correspond to individual particles. Now this is a great example of frames weak AI because humans are very good at segmentation. Millennia of evolution have resulted in human brains excelling at these kinds of image related tasks. But while you can look at this image here and figure out which components of that are part of particles or not, it's an immensely tedious job for you to go through and actually draw lines around each of those and it's not particularly accurate.
But on the other side, it tends to be a very complicated problem for computers to do. It's only in recent years with modern algorithms that we're able to actually solve this. The solution we use, leverages the state of the art convolutional encoder decoder neural network depicted in the middle. This network maps out an input of input image of pixels to a probability mass that each pixel belongs to that particle or not. Then we use that to create an image mass.
And here on the right, you can see the output of that, where the pixels shaded in purple correspond to the particles and those shaded in orange on the background. So you see this model performs extremely well at identifying these regions and saves countless hours of tedious manual work. The next task we approached was classifying the type of these particles. In the top left, you can see another cryo EM images image. In this image, there are nice round particles and then there are these other irregular particles with blebs.
We want to do is build an algorithm that can detect which particle is which. Again, this is another great example of weak AI. This is not a task that the previous neural network can do. It actually requires us to build a whole new network to classify them. The distribution here, the histogram shows what would happen if you tried something naive like just looking at the major access length.
And we see that they're just not well separated. We need a more sophisticated machine learning algorithm. We tried a variety of models that are shown here and what we landed on was a random forced algorithm that incorporates the intensity of every pixel in each of the particles. This model achieved an incredible 0.98 accuracy and 0.99 precision, which is just unheard of to see rates that high. And once again, the views of this algorithm, we're saving countless hours of manual effort.
Finally, we'd like to extract some metrics from the structure of the individual L and Ps themselves. If you look very closely at this image, you can see a railroad pattern around the edges that indicate a lamellar structure. If we draw a line from the center of the particle to its outside edge and then sweep around averaging the pixel intensities as we go, we can produce a graph like in the top right here. Each row in the graph is the profile of an individual particle in a cryo EM image with its outside edge aligned to the right. Then we average vertically across all these pixels and we have a profile in the bottom right here for the whole image, which you can see come out of this picture is that these pixel intensity peaks show up that you saw in the naked eye, but additional peak show up as well that were harder to detect.
But then beyond that, we're actually able to quantify in nanometers the exact width of that lamellar layer. These profiles turn out to be very formulation specific. In the bottom right here, you can see what happens if you look at a variety of different LNP components and processes. You see a variety of different profiles that come out. So using the quantitative metrics here on this Lamella width and other metrics like it, we hope to gain insight back into the process that we form these LMPs and then in the downstream LMP performance.
So what I'd like to do here is close by highlighting the AI factory that we've built at Moderna that allows us to continuously mine and deliver AI algorithms like the several that I've shown you here. First, you heard from Marcello about all the great lengths that we go to in order to capture and integrate structured data across the company. I just now spoke at length about the various algorithms that we have in research and development. I also walked you through the automated Sanger analysis pipeline example that illustrates the sophisticated cloud and DevOps infrastructure we have at Moderna, which allows us to rapidly deploy compute power. Combining all these together allows us to do AI model design at scale.
We leverage all of these to rapidly design, test and deploy AI models. Finally, this integration, this infrastructure and processes we built in Moderna allow us to rapidly move completed AI models into a production environment for data use. This AI factory allows us to push these same goals to even greater heights with improved quality, increased speed, increased scale and reduction in costs. That'll say thank you.
Thank you, Karim, Marcello and David. And with that, operator, we are ready for Q and A for the digital section.
Thank Our first question comes from Hartaj Singh with Oppenheimer.
Great. Thank you for the questions and for this really interesting presentation. On just the digital and the AI side, so having been in drug development for a long time, whenever I think of drugs, I think clinical, clinical and CMC, basically the 3 kind of the holy trinity of the BLA or the NDA. So I guess my question about the AI and digital that Moderna is developing and utilizing is can you give some concrete examples what which portions of those attributes? And again, I know you can probably spend a long time, but issues that are relevant today, for example, how they might have helped with coronavirus, the vaccine or CMV, but could you give us tangible examples of how AI and digital are sort of helping you in kind of in interfacing through regulators through these three sections of the IND and then what ends up being eventually the BLA or the NDA?
And then I got a quick follow-up question.
Yes. Hi. This is Marcello here. Just to set the stage, actually we started AI based on the historical data that we have. So you would see more and more AI at that stage in our research and preclinical space than in our clinical space.
But as we are progressing and building the digital infrastructure to capture all those data from all those engines that we have, we are building more and more AI. And as Dave we have many algorithms that are at this moment in training to help us in those new AI models in the different areas.
The learning specifically the learning that we have gathered on PCB has helped us go very, very fast on the coronavirus vaccine. Yes. It's Stephane. Just to close what Marcelo said, I mean, if
you think about it, as we build the company from having nothing to being a preclinical company and then being the digital and being beyond preclinical, that's why we have the most data. That's why the team has done all our work. The next phase that we did as we build the company was no CMC and Norwood, which is why we start to see there's a lot of AI algorithm in the CMC space because they are collecting so much data in Norwood. And the team in the last couple of years has been building and is finishing in the clinical side of digital, including how to file BIN by basically putting our pieces together coming from Techdev where we have integrated the robotics straight to the IND file, the TOK study and the preclinical data as well as clinical synopsis of the clinical study all coming together through that system. So in clinical, but still at the stage, it's mostly building the digital infrastructure and then generate the data to then enable us to
do And
if I can comment on coronavirus, we used an algorithm that we have developed for previous construct, long construct, because coronavirus is a long construct to be able to produce rapidly the coronavirus construct. This is based on the learning that we had in research and in preclinical from previous ones.
Great. Thank you. And then just I had a question actually earlier when Han was presenting and I just want to ask that if you don't mind. So just as a real world example, if you can just walk us through, let's say you had been at the equivalent of potentially an analog company and the coronavirus essentially a problem had landed on your laps like it did a few weeks ago. What would the steps you'd have to go through and would probably still be going through in order to get a candidate to the NIH?
And then then kind of compare that against what is it specifically that Moderna did to get it to the NIH in those 42 days? Just broadly speaking, if you could just give us a kind of in that context would also be helpful. Again, thank you for the question.
Thank you. That is a real good question. And so there is a lot of examples.
And of course, I mean, we
have our own platform and technology, so it's very, very difficult to go and compare. But I can give you a couple of anecdotes here. So a typical manufacturing batch record in the biotech industry record in the biotech industry has around 10,000 entries. And it is thicker than one of those old telephone books. And so just it would have taken a long time to prepare a batch record only to make the product.
And having the digital platform, it allowed us to prepare that documentation in a much, much faster way. The same thing goes for equipment, equipment parameters that would allow us to go and set up the equipment and the automation that would allow to make us to make the products. I went in the presentation in the slide. It took us days to set up a process like this. I have done something similar than that.
It would take weeks or months to get ready to do something along those lines.
Yes. And maybe just a little bit, if
you think about it as
a research process, there's a process development piece to a further and then manufacturing quality, regardless
of how you do it.
If you think about what some of the big vaccine manufacturers have announced in the last weeks, J and J has announced it would take them 10 to 12 months to go into the clinic and Sanofi has announced it would take them 12 to 18 months to get into the clinic. I mean, as you might have heard from Secretary Azar last night, the FDA opened the IND for amount of $12.73 on Monday. That was less than 10 days after filing the IND. And if you read it, it's less than 60 days from getting the sequence published by the Chinese government. And so if you ask the question, how did we do that versus the time line that I just shared that have been communicated perfectly by J and J and Sanofi, I think it goes right into those 3 buckets.
First is the research front, which is we did not need to have a virus, just a sequence on the computer was enough. And thanks to all those we have done before, we're able to very quickly working with the NIH to pick the sequence. So if you think about the research, it's not spending even more 10 months playing well in one model and different candidates and optimizing. We literally do that on computers without evaluating the virus in a matter of a day or so. Then on the process development, in a typical process, you need an army of engineers to figure out the cell line optimization and that will take literally months.
In our case, because an mRNA is an mRNA, the team was able to retrofit very quickly the personalized vaccine unit. Those robots that some of you have seen in our route have been developed for personalized cancer vaccine to do for the first time our infectious disease vaccine. So it's just a bit of tweaking to do. Again, it took days and not months because it was the first time we did an infectious disease vaccine on that kind of robotic system. And then the manufacturing is another piece where we get the massive advantage.
It takes a few days to make mRNA. It takes a few days to formulate the mRNA. So if you think about it in days, just use running a recombinant train, that would take you to 4, 5 weeks in terms of making the API from a recombinant standpoint. So I think it's just all those pieces that help and then you layer on top of that all the digital like electronic batch record, quality control that the team has been talking about And it's how you end up having a cycle time that we had in our 42 days from sequence to shipping a product having successfully passed sterility testing. And I even think that this is because it's the first time we have done it.
I don't think this is the fastest time we will ever do just because and Ron is turning already on next to me. Just because if you think about it, anytime you do something the first time, you are learning things, you have to optimize things, you have to develop new things. And as you're able to do it more and more, one has shown in the previous personal cancer vaccine slide, as you do it more and more, you become stronger and stronger as you keep on learning. And I think that's really a key theme about how we're wired as a company from a kind of culture standpoint and how we use the digital backbone that we have built and the access to data. It's all about learning.
As we've said in the past, and we believe mRNA is a new class of medicine. We believe it is a 15 to 20 year S shaped kind of learning curve, and we want to be the company that learn the fastest. We always said we won't be the smartest, but we will be learning the fastest. And this digital and this AI tools is really at the heart of our strategy.
Great. Thank you, Samad. I'll jump back in the queue.
Thank you. Our next question comes from Yassine Rahmani with Roth Capital.
Hi, team. Thank you for an incredibly informative session, especially as you're helping us to really visualize how AI is used in mRNA construct design and LNP formulation and construct. I guess I have a question, two questions for you. Can you help us understand how you can utilize AI technology to really help predict traceability from preclinical models to clinical models? Can you use AI technology to really identify patient selection and clinical studies?
And as we go with additional data from CMV upcoming, can we really help us to determine who could be responders versus non responders? And then the last question is for Stephane. Not only is manufacturing AI technology setting you apart versus your competitors, but how much is adding AI increasing the barrier to entry among not only other companies in biotech, but specifically in the mRNA world? And thank you for taking my questions.
Okay. Hi, Usmeen. This is Marcelo here. I'll start with your second question. Will AI help us with patient selections?
I would say definitely. At that stage, we are connected to all the CROs and we are pulling data from those CROs. And as you know, we started our clinical trials at the end of 2015 and we are starting to collecting all this data. And as we speak, we are looking at different AI models that can help us. And as we progress those, I think we will communicate or we will show our progress in that field.
On the translational part, it's the same question actually, because you're moving from a different part of the engines that we're building. So you have the research, the manufacturing and the clinic. What I can say as of today is we are collecting data in every step of those processes and we can trace back the data from clinic to research. So the question is, how this is going to help us to improve in the clinic will be as we progress the clinic to put some problems to our AI team and they will help us build the models to resolve those problems.
This is David. Just to add a little more color here. As I hope it's clear from the slides that I shared, all of these problems are a journey. We start solving these problems with the data that we have and we continuously evolve and continuously push these forward. Right now in a lot of these spaces, we're using traditional pharmacometrics, pharmacology techniques.
We will continuously evolve those as we get more data and inject more
and more AI into those spaces. Yes. And Stefan for the last question. I mean, as Karri said when the EBITDA presentation, we are big believers that time is the only common denominator across companies in an industry. And so we believe that being able to learn faster is a key competitive advantage.
And I think it is even more important as you are going through the S shaped curve that I talked about compared to companies that are the top of the curve like I used to be in my previous companies where when you grow and learn 2% to 3% a year, you feel great. It's a great year. If you think about the MRN technology as we've shared in the past, if you look at the technology we have today in the clinic versus the technology we got from the Harvard Labs when we started the company, we have improved a 1000x to 1000x the performance of the mRNA technology. And as you know and you saw at the last time, they last spring, and I welcome you to send the next one with Stephen Hogan, his team in June in New York is we keep on learning. I mean, if you look at the tools like the cryo that Dave mentioned, it's just one of many examples.
We are equipping all the teams across the company with tools to be able to learn fast. The big change in management, I believe, and it is across all companies and we are driving it with Marcello and Tracy, our Head of HR, is this is new tools. AI and machine learning are new tools. And the biggest challenge, in my opinion, is change management because you have most of the leaders in most companies, we have never done AI before. You have people like Dave and Marcelo who understand it and have studied it.
And but most of the line teams, they have never done AI. So what we've tried to do is to first build the infrastructure, the digital tools, the processes to get the data because we have the great data, good luck. And then what we are layering on top of that is in every group in the company is AI. And what we want to do and it's going to take us a couple of years to really make it 100% part of our DNA everywhere is we want AI in science, we want AI in HR, we want AI in manufacturing, in finance everywhere. And that's really how we are building the company.
So it doesn't happen overnight, but we believe that if we are well capitalized like we are, with the scale as well as the company, if we can really build AI as a key commerce of our culture, it's going to be really, really hard to compete with us.
And thanks, Ed. This is Karim here. My observations generally outside of the Pharma business, right, there is a data advantage, right? You need to sort of think about the data you're accumulating and both your data fusion abilities, right, and proprietary data. And I think you can sort of see that in how they were talking about from sort of research to manufacturing to clinical.
And that this has to be completely be ingested all the time. I think the second thing is, look at the industrialization of analytics and AI, right? Most of the time you think about decision making the habits in companies, right? The tools we use are Excel sheets, right? We've had Excel sheets, we call up our buddy to give us more data, we think about the data, we modify the Excel sheets and do that, right?
This is a totally different beast. You want to industrialize the decision making process and just think about all the decisions they have to make, again, from R and D all the way to the clinic. And the 3rd part, I think the really important part is this change management part. Where I see these things die in companies over and over again is because management team, A, is either not committed or they think it's play over the month and they hope that we'll sprinkle some AI magic and we'll go from there. And the change management part, both like you have the sort of the AI data showing up now in the economy that they're working with, But there's a huge science component here that also needs to be retrained and shown the benefits of this approach.
And that's a big mandate. And I think many companies outside of sort of the pharma, biotech sector fail miserably at this in this journey. So I think those are the ways you sort of can think about how competitive advantage is built.
Yes. And Stephane, again, it's worth the last point. Last year, Marcelo and I took a trip on the West Coast, meeting a few companies just to look at AI and what our best practice is and change management in those companies and tools. And as you know, you've heard several times, we are partnering very strongly with Amazon and AWS. And the piece that was extremely exciting to me when I went to Seattle with Marchetto is the customer obsession that Amazon has brought into the retail space, and Bezos has talked about it all the time.
When you talk to the AWS team and you see the tools they are developing, they are totally customer obsessed. They are developing tools every day that are available in their cloud so that all teams are able to go and to use those tools. And so we really have this set up now, which is very interesting, which is you have the Amazon team building more and more tools to just make them available if you use the cloud. And then Dave and his teams, when they need something that is not available, then they build it. And so we have this really ability to go very fast with existing tools and that number just increasing and more and more powerful tools.
And then the team being able to design the tools they need, they cannot find them in the Amazon store.
Thank you
for the insights fully answered.
Thank you. Our next question comes from Cory Kasimov with JPMorgan.
Hey, guys. This is Matthew on for Cory. Thanks for taking my questions. So for the first part of your talk on your personalized cancer vaccine manufacturing success rate, where I believe you said you're currently at around 87%. What are the key reasons for manufacturing failure?
And then how, if at all, do incorrect sequences that you outlined in the digital part of your talk factor into this?
Well, I think it is a combination of factors. A lot of that is just the know how. I didn't go into the presentation associated with what are the root causes. But it is a normal part when you do a scale up. Obviously, they go around equipment variability, it goes about operators variability, it goes around process variability.
So there were a number of different things. At the beginning, we even started with the CMO until we were able to bring that into Norwood. So through the obsession in tackling one product one problem at a time, we started to increase the success rate as we continue doing it. I would have argued that the fact that we are doing personalized cancer vaccine and we are making as many batches as we make, the learning, as Stephane mentioned before, is going faster because our end is higher. And so we can connect the dots on a number of different root causes at a much faster way.
And of course, digital was part of it from the beginning.
Got it. And then, I mean, is there a technical limit to like how much you can improve upon this or how do you improve upon the rate even further?
I think the sky is the limit. And I think we will get way, way, way into high 90s here when if we go all the way into commercial, which is the normal thing, we will go into validation. At that moment in time, we would have incredible experience associated with it. I also point, if you remember in the slide, there were different colors. Some of the first colors were basically failures that we needed to that we found out after the batch in some or in all of the recent failures that we had, we had more aborts rather than failures.
So it was not a surprise. In this case, it's particularly important because if something goes wrong for whatever reason, and as I said, it could be equipment as simple as an equipment malfunction. I don't know, a pump fails to produce or just something that is unexpected. You discover that immediately. So it is not a failure later, it is an abort.
When you're doing something for a patient in a small period of time, basically it allows you to go and produce fast again and arrive to your destination. So those things are integrated, so even in the failures. Yes. And it's Stephane,
just maybe to add a point that Juan mentioned during his presentation, which I think is really critical, which is the personalized cancer vaccine has been extremely enabling for us because we have made so many lots. As Juan said, it will have take us years after launch of a CMV vaccine or Zika to get to the same point of learning. And so the power of this personalized cancer vaccine was it not only it was an extremely important enabler to the corona vaccine as we discussed earlier, But more important to me for clinical batch. When we make big batch for, again, CMV or Zika or MMA or Pick Your Favorites product in the pipeline. When the team makes that all the learnings coming from the ATN that we have in PCV is extremely valuable.
And if you think about it, if we did not have PCV, it will take us years from now to get to the same place in learning. And that's a complete advantage that I think sometimes is not always fully appreciated.
Great. Thanks for taking my question.
Thank you. We have a question from Matthew Harrison with Morgan Stanley.
Hey, great. Thanks for taking the question. I guess what I wanted to ask is, how far do you think you've pushed the limits of the technology? And is there a case here where you've tried to use AI and have failed? And what sort of evolution do you need to see to be able to push it further?
Thanks.
It's Stefaan, you're asking about mRNA science, correct?
No, I was asking about AI and sort of where you you gave some examples of some concrete processes where you've had success. I'm wondering, has there been something where you've tried to push AI into a process or a situation and haven't been able to make it work? And I guess I was trying to understand, is that a limitation of just technology and you think it will catch up or what are the reasons where maybe you couldn't use it in certain situations?
Sure. This is Dave. I'll take
the question. So I don't know offhand any kind of large failures that come to mind. What we do when we start AI projects is we focus on ones where there's clear tangible business value that we can achieve in a realistic timeframe. And again, there's this journey aspect of it where we try to focus on problems we can get a minimal viable model out that's delivering value. And then we run that for a while and we continuously evolve and improve that.
And I think this is something that gives us a much larger chance of success rather than going for some large moonshot problem that will take us multiple years. So we try to focus on these real smaller achievable problems first.
Okay, thanks. That's helpful.
Thank you. Our next question will come from Salveen Richter with Goldman Sachs.
Good afternoon. Thanks for taking my questions. So one on AI, I guess if you're integrating it across maybe different verticals, how much do you want to optimize one part before you fully integrate just because you may have to play with various inputs? So just trying to understand how you do that. And then the second question is as you scale up on manufacturing, how do you think that's going to impact COGS going forward?
I'll take the integration
verticals.
Actually, we don't look at the verticals. We look overall how and where we need to collect data to be able to improve the processes. If you look at verticals, you're creating silos information and this is the thing that we want to avoid actually. We want to make sure that we're collecting data from all those verticals that in some instances, we will use to solve business problems.
Yes. If I may add, I think this is Karim. The perspective to take is one of the architecture. And the architecture of these AI first companies is very different. Like even the setup of a silo is going to be and it's ethical to what you want to achieve, right?
And so this is the notion that the operating model, the way you're structured is very different than in the traditional company. And there's a notion in the management literature called Conway's Law that the architecture of the technology reflects the architecture of the organization. And that's what you're hearing, that it's a completely different architecture. So these questions sometimes don't even are not even relevant in this model.
I can give you an example. We are collecting data from 1 of our vendors. I won't name the vendors. And we were building this data for just as an integration between us and them. And 2 years down the road, we saw some issues that we needed and problems that we needed to solve.
And we were figuring out how to do it. So what we've done is we looked at holistically at all the data that we had available and we detected patterns that helped us resolve the issue. Shouldn't we have collected all this data and started, we weren't able to do anything with this use case or problem that we had. Yes, we were able in that particular instance to identify 3 out of about a dozen instruments at
the vendor that were malfunctioning and not configured properly. So we're able to take that data for a purpose that we're using entirely separate and identify challenges and tell the vendor to go correct that mistake.
Another example that comes to
mind and I will comment on COGS here in a minute, is things such as predictive maintenance, in process controls, a number of different things that we would go. I mean, we can even go and we have started thinking about going into continuous manufacturing as we move forward. So let me take the COGS piece. One element associated with COGS is obviously the technology, which is not necessarily related with AI. But the fact that what we use is the body as the bioreactor rather than the manufacturing process, you're not having big bioreactors, your capital intensity, the length and the cycle time of your process associated with the technology is completely different profile, which is going to give you something a completely different profile of COGS.
Then artificial intelligence and all the digitization applied to manufacturing, the first thought in mind is to sit it with quality, very similar to what we have. And counter to what some people think, high quality means lower cost, because it means high reliability. That's why we are focusing on reducing that variation, increasing our reliability there. Then, artificial intelligence and digitization automation applied to yields, to cycle time, as I mentioned, continuous manufacturing. All that would take our COGS into one direction down.
Yes. And Salveen, it's Stefan. Maybe it was another dimension of manufacturing, which I think is an amazing power of the mRNA technology because it's a platform is that we do not have to be precise on every launch forecast by product across the portfolio. If you think about a wonderful launch that has happened recently is Shingrix. Shingrix is GSK's biggest vaccine, dollars 1,000,000,000 first year of launch.
But GSK has been very open about it that they cannot make enough. And they cannot make enough because whenever they thought they will sell $1,000,000,000 the 1st year of launch. This is the best taxi launch ever. And if you think about it, the prime traditional product is that you need a dedicated manufacturing facility for every product. And so you have to guess way before you have your Phase III data, your ramp, your commercial ramp.
And even you don't know your efficacy, it's just a big guess. And so in our case, what we think is an amazing power of a platform, which we play into maximizing top line, which when you do a 90% gross margin business, is the most important than the 5% improvement in the cost of goods is make sure you never miss a sale. And we believe by having this platform, we should be able to really manage the portfolio so that we only have to be right at the portfolio and we start to have 24 products like today and sooner I have 30 or 40 products. That makes the product easier and easier to manage. And I think this is one of the biggest value.
Juan is going to work really hard with his team to take the cost of goods down. And of course, scale and volume will help. But for me, the piece that's the most remarkable is we do not need to invest additional CapEx for every program. If you look at it, we just announced recently EBV and pediatric RSV. We don't need 1 more dollar of CapEx to those products.
It's in the same rooms, with the same people, with the same equipment. So that's just fantastic savings. And then it's on the maximizing sales. If one day we'll come to you guys and we say we cannot sell enough because we didn't do the forecast right, we are not doing options right.
Thank you.
Thank you. And we do have a follow-up from Hartaj Singh with Oppenheimer.
Hey, great. Thank you for the follow-up. I just had one quick follow-up on artificial on AI artificial intelligence. This was I think maybe for Stephen or whoever can answer it. In some ways, I guess the holy grail of drug development is finding understanding pathology of the disease and finding what the underlying problem is, finding a target, right, then understanding whether it's druggable and then coming up with a drug for that.
With mRNA, you're coming up with a different modality to do that. So I guess my question is in that context, within your 5 or your 6 verticals, is there anyone that is more amenable for AI to actually help in that process, automate that process, so it doesn't become this laborious kind of going to the petri dish and then going out of the petri dish and going into the kind of process or not really? And how much time could that happen? Thank you.
Yes. So it's Stefan. I'll take a shot at this one. I think it goes even beyond AI, I'll go back to basics, which is you think about our technology, we do not invent biology. And that's for me the most profound thing, which is if you look at traditional medicines, it's a guesswork.
And so in that world, you can see our AI helps you avoid spending too much time in the petri dish or into a mouth or whatever. But in our case, we use evolution. If you think about the coronavirus, why could we go so fast? We don't do any biology work. We just took a sequence, dropped it into the same cassette as the CMV and we flew to the clinic.
If you think about the VMMA product, it's the same thing. We use a sequence that all of us on the call have, which because we don't have MMA disease and drop it into the same cassette as the chick antibody technology and you fly to the clinic. And so the piece that I really think is powerful about our technology that I think sometimes is underappreciated is we do not have to invent biology. We just use Evolution and that's the best AI system I know.
Great. Thank you, Sussan. Thank you. Speakers, I'm showing no further questions in the queue at this time. I would like to turn the call back over to Stephane Vinsel for any closing remarks.
Thank you. I'll be brief. So first of all, I would like to thank our guest, Karim, for his help thinking about the business model of the company and his help today in the presentation. I'd like to thank Juan, Marcelo and Dave for their presentation, but even more important for the impressive progress that they and their team have achieved in 2019. As you know, we are very excited by the next leg of growth of the company, doubling down on our vaccine, doubling down on IV systemic products.
As you know, we have 5 new development candidates in just the last 60 days. I'm very excited and looking forward to 2020 because we're going to have a lot of clinical data with us. As you might have seen last night, we announced that the CMV Phase 2 was fully enrolled ahead of plan, which makes us very comfortable to confirm which we'll have data in Q3 of this year, given all the subjects have been dosed now. And as I said last night, we had the HHS secretary, Azar, who communicated that the FDA has opened the IND for the corona vaccine 1273. And so we're hoping that soon the first subject will be dosed in the U.
S. More than ever, we believe that mRNA is a nucleosome medicine. Our Norwood site is a strategic asset as you get a good sense for it. And what we have achieved and what we will achieve looking forward could not happen without Norwood. Digital and AI is becoming day after day more and more part of the DNA of the company.
And you have my full commitment and the commitment of the team that we will keep pushing it every day so that the company just becomes stronger and stronger. So with that, I would like to thank you for taking the time today. And I hope to see many of you on April 14 in New York when we do our vaccine day. Thank you.
Ladies and gentlemen, this concludes today's conference call. Thank you for your participation. You may now disconnect.