Good day, ladies and gentlemen, and welcome to the Artificial Intelligence and Cognitive Sensing at the Endpoint conference call hosted by QuickLogic. At this time, all participants are in a listen-only mode. Later, we will conduct a question and answer session. If you would like to ask a question at that time, you may submit your question online at any time during the conference using the Ask a Question tab on your webcast screen. If anyone should require operator assistance during the conference, please press star then zero on your telephone keypad. As a reminder, today's conference may be recorded. I would now like to turn the call over to Mr. Brian Faith, CEO of QuickLogic. Sir, you may begin.
Good morning, everyone. I'm excited to kick off the agenda today. Following me is Malik Saadi, VP Strategic Technologies of ABI Research. He will cover what he sees as growth and drivers in the AI and cognitive computing space. After Malik, our CTO and I will return to share what QuickLogic is doing in this exciting space, followed by each one of the companies involved in the QuickAI ecosystem. Guy Paillet, CEO of General Vision, Jungho Ahn, SVP of Nepes, and Chris Rogers, CEO and founder of SensiML. Malik, I will turn it over to you.
Hey, good morning. Good morning, and thank you for inviting me to this webinar, Brian. You asked me to provide a global overview on cognitive computing, artificial intelligence, market drivers, and opportunities with some focus on industry applications. Next slide. Before getting there, let me give a brief introduction to artificial intelligence and how we see it here at ABI Research. Artificial intelligence is one of the hottest and versatile technology breakthrough in modern time. The technology will enable many industries to increase their productivity and improve operation efficiency through implementation of intelligent process automation. In fact, artificial intelligence is nothing else than a new computing paradigm that allows machines to perform intelligent tasks with low to no assistance from humans. All start from gathering relevant data from sensors, devices, and applications surrounding us.
Those data are then sent to training engine to be structured, classified, and filtered to identify patterns, behaviors, and trends created collectively from all data point gathers. Once those patterns are identified and grasped, they are sent to the inference engine, which is responsible for recognizing the patterns, making sense of them, and then taking special actions, commonly called inference in the AI jargon. Let me give you an example. For example, let's take a faulty machine making an unusual noise. That noise could be a pattern exploited by AI engine to help identifying and localizing a given anomaly within the machine. The engine, the artificial intelligence, could potentially prescribe and recommend procedures to prevent potential failures of the machine. Traditionally, both training and inference require significant data processing resources, and this has prevented AI from taking off for years.
With the improvement of both processing and communication technologies, AI is now intruding into many industries, including professional services, consumer electronics, social networking, and industrial automation. Next slide. In terms of market size, ABI Research estimates that 3.2 billion devices shipped in 2023 will be actively using AI in some way or shape. Smartphones and consumer devices and desktops will continue to have the lion's share from this volume. Industrial application will be a major area of growth within the next 5-10 years. Although the focus of this presentation is industrial application, ABI Research tracks 58 use cases for artificial intelligence across 11 major verticals. That includes mobile devices, consumer electronics, computing devices, smart home, retail, industrial automation, robotics, and smart building. Next slide.
Before jumping to industrial applications, let's have a look into how AI is implemented in our lives. There are three possible scenarios for executing artificial intelligence algorithms, and this slide illustrates just this. Let's start with scenario 1, shown in the upper part of this chart, which describes a situation where all AI processing, including training and inference, resides in the cloud. Actually, this is the most popular option for many industries that use cloud source data, including professional services, retail services, social networking, and others. This scenario is highly dependent on always-on communication, unlike scenario 2 and 3 that are less reliant on always-on communication. Now for scenario 3, described in the middle part of that chart.
It shows the situation where the big part of the training resides in the cloud, while the inference is dealt with at the edge of the network, with edge here describes either an on-premise server, a gateway or the end device itself. This type of scenario is typically used for certain process-constrained applications that require immediate inference, but offer a certain tolerance to reliability of decision-making. In scenario 3, which is described in the lower part of the chart, it shows a situation where all AI functions, including training and inference, are executed at the edge of the network, either that be on the on-premise server, on the application gateway, or on the end node itself. This scenario is often used for mission-critical applications that require near real-time performance. Otherwise, that execution could be affected by latent communication or slow process execution.
Typical application for scenario 3 includes industrial automation, robotics, and certain automotive applications. Slide 4. Let's see the benefits and costs for executing certain AI or cognitive computing function into the cloud versus the edge of the network. Cloud AI offers many benefits, including high processor resources capable of dealing with a large volume of data used for training. This data could originate from a variety of sources, which could make the training more efficient and more reliable. Cloud AI is not designed for mission-critical applications where low latency is an important element in the operation of those machine-critical devices. Data privacy and security could also be an inhibitor for cloud AI, maybe for some specific industries, including industrial manufacturing, automotive, or network automation. Let's now jump to edge AI.
Edge AI, on the other hand, brings processing capabilities closer to the end node, which means AI functions are executed with a very low latency and far less reliant on connectivity with the outside world. Not only this minimizes the risk for data to fall into the wrong hands, but this could potentially lead to lower AI implementation costs compared with cloud AI. ABI Research has identified a clear trend towards edge AI, initially driven by the migration of AI inference within consumer electronics and computing devices. This trend is increasingly driven by AI adoption in the industrial segments, including video applications, manufacturing, automotive, and robotics. These trends are creating tremendous opportunities for chipset suppliers like QuickLogic, NVIDIA, Qualcomm, and Intel, who are now preparing their processors and chipsets to tap into these opportunities. Slide five.
This slide focus on market opportunities for cognitive computing within the industrial market. As you can see here, our forecast indicate there will be some 365 million industrial applications that could potentially rely on cognitive computing by 2023. As you can see from the pie chart above, this market is still largely driven by video application, including set monitoring, personal facial recognition, camera surveillance, and traffic flow analytics. However, major areas of growth will be smart manufacturing that will see its market share jumping from 2% growth in 2017 to above 10% anticipated for 2023. Smart retail store devices is also a market expected to grow within the next six years, with shipments of capable devices expected to exceed 50 million devices by 2023.
Overall, we're expecting industrial AI applications to rely massively on edge computing, with only 25% of those applications are anticipated to use the cloud for both training and inference. Let us zoom in on edge computing and edge AI to see how computing is distributed across different device types. As you can see from this slide, as far as industrial manufacturing is concerned, the majority of the AI training and inference occurs in the on-premise server. Manufacturers generally prefer on-premise servers over clouds for many reasons, including compliance and interoperability with legacy automation platforms that are set on the premise of the manufacturing site. There is a security and privacy concerns when sensitive data is shared with the cloud in the case where cloud AI is used. Obviously, there are also concerns around costs and time to integrate AI with existing IT platforms.
As you can see from this figure, inference will move from on-premise to the end devices as those devices become more intelligent and powered by adequate process resources. This specifically is the case for machine vision and predictive maintenance, where decisions needs to be made in real time and as close to the end node as possible. We see training to continue to be residing on-premise servers, we don't anticipate this to change drastically within the coming years. I have come to the end of my presentation here. Enjoy your day, and I hope the Webex will help you in making some informed decisions. I hand it over to Brian. Thank you very much for your attention. Thank you.
Thank you, Malik, for that insightful presentation. When most people talk about AI today, they're referring to the AI that takes place in the cloud of the data center. It's what has driven growth for GPUs and FPGAs in particular. There are good reasons to implement deep learning in the cloud, where there's almost infinite compute capability and substantial power budgets. As Malik said earlier, there are clear reasons for wanting to move some of the AI or cognitive computing capability to the far edge where endpoint devices reside, essentially moving a lot of the intelligence to the device with the sensor. In viewing AI as a processing spectrum from cloud to endpoint, we can see how a hybrid approach of cloud and endpoint processing are complementary. Let's first look at a cloud-dominated architecture.
Imagine a network of sensors sending their raw data back to a cloud service for the compute and decision-making, particularly if there are other contextual data required to make the decision. While this works well in many applications, there are trade-offs. In this diagram, the red circles represent all of the raw data, while the green triangle represents the data that the system is looking for to be actionable. First, sending back all raw data to the cloud for processing adds in a latency that might be unacceptable in some use cases. If it is a simple thermostat sampling temperature, latency wouldn't be much of an issue. If it is for a voice-enabled application, even a couple seconds of latency would be viewed as unacceptable. Second, let's think about power consumption.
While power consumption of processing in the cloud is not viewed much as an issue, the energy consumed to send all of the raw data back to the cloud may severely impact battery life in some applications. Lastly, data privacy can be an issue, and we'll get into an example on that later. On the other end of the spectrum is to deploy more of the computing to the endpoint. By pushing some of the always-on intelligence out to the endpoint, one can reduce the system latency of communicating with the cloud, save power consumption by not transmitting all of the raw data back to the cloud, and enable more data privacy by intelligently choosing what and when to send.
We believe that the synergy between endpoint and cloud can provide a better system approach to AI, where endpoint is a key component and enables cloud AI to be optimized. Let's use Amazon Alexa as an example where we believe they implemented a balance of cloud and endpoint processing the right way. Would you feel comfortable if your Amazon Echo sent each and every sound to the cloud for processing? The incremental electricity bill may not bother you, but you probably wouldn't feel comfortable with Amazon listening to every sound and word spoken in your room. Moreover, what happens if your available internet bandwidth drops while your teenager is watching that latest YouTube video? By moving the AI for local command processing, the detection of the keyword Alexa to the endpoint device, you now have a low latency and more private way of accessing Amazon's cloud.
Your actual intended request for Amazon is the only thing that is transmitted. Let's carry that example a little further. What about hearable devices that want to enable the same Alexa experience? If you were to stream most of the microphone data to the cloud or a smartphone, you would likely have battery life issues from the power consumed in continuously transmitting that data. Contrast that with being able to do more local processing of the microphone data to pick up spoken commands before opening a communication link to the cloud. The point here is that intelligent system partitioning is necessary to deploy AI in a way that results in a more positive ROI for the system OEM and a positive user experience of the consumer.
All of this probably sounds intuitive. Let's get to some of the challenges we see in actually realizing more AI and cognitive computing into endpoint devices. If you're a large platform company like Amazon or Google or Fitbit, you can invest in hundreds of algorithm engineers and data scientists to understand the data, know how to model it, and invest in the software needed to partition the system appropriately. Unfortunately, not everyone can afford that investment, and IoT is too fragmented a market for most companies to invest such resources in. We have always believed that an ecosystem of companies with domain expertise can provide a solution, and in this case, a platform solution that can be used by OEMs to deploy AI at scale.
Today, we are announcing QuickAI, a new collection of companies who bring deep domain expertise in the field of AI and cognitive computing. You'll hear more about each company, and I hope by the end of this webinar, you will understand why we are so excited to be part of this strong ecosystem. Now, I will turn the call over to our CTO and Senior VP of Engineering, Dr. Timothy Saxe, to share more about how we see solutions being developed based on the QuickAI platform.
Thank you, Brian. I'm going to talk about two applications and our hardware development kit platform. The two applications of vision inspection. Vision has a number of challenges. In particular, you can go discrete, which might mean looking at a fruit, for example, or a continuous surface kind of inspection. Discrete is a good case for endpoint because you might actually move it into the field and be looking at items before you pick them to see if they're suitable for picking. Surface creates the problem of speed. If you're printing packaging material, you might want to be inspecting it to make sure it's good, and you might want to stop the line quickly if you find defects in it so you can repair them before too much bad material is created.
In terms of performance, moving to an endpoint means typically a small MCU for power reasons, and small MCUs don't typically have the performance required to manage pixel data from a camera. We find FPGA is a good way to do that. You can use an FPGA to interface to the camera and to massage the data stream in a way that's compatible with the neurons. For instance, you might be choosing a region of interest, or you might be doing subsampling or histograms, and then presenting the data to the neurons in order to get their output. The second example is predictive maintenance. As Malik mentioned, people like to check for vibration or listen to sounds on machinery. One of the things about industrial machinery is it tends to be very different.
You might have a similar piece of equipment, let's say an air conditioner that you want to monitor, and that air conditioner might be mounted on the roof of a concrete building, which is a very rigid roof, or it might be mounted on the top of a warehouse with a flexible roof made out of wood or something like that. The way that air conditioner will respond varies based on the conditions. Because of that, you need to train it in situ. You can't train it offline, which creates this need for endpoint training. The system would learn then from either vibration or audio in order to work what's normal, and then when something's not right, it would tell you that the thing is not right. The data rates are kind of high for pure software, you might be a few kilohertz sampling rates.
Audio is definitely high for pure software. Again, the FPGA turns out to be useful in this instance because with the FPGA, you can compute various things like FFTs at somewhere like a one-tenth to one-quarter of the power that you would normally do in software. Then another device that's very useful in our solution is the thing we call the Flexible Fusion Engine. It's specifically designed for managing accelerometers and gyros, it is perfect for offloading the MCU handling the accelerometer and gyro that you need for the vibration analysis. The final piece is the QuickAI AI HDK platform, which is designed to do the multiple needs of endpoint AI.
It's designed to be low power, it's designed to do data collection because you need to get the data in order to analyze it in the first place, it's designed to be small enough, about the size of a business card, you can easily deploy it on proof of concept applications. To support all of this, it's a rich platform. It has the EOS device, which has an M4 processor, the FPGA for doing that kind of processing, the FFE for doing the accelerometer kind of processing, and it has the accelerometer, gyro, and magnetometer on it for all of the motion applications. It has two microphones, it can do the audio applications that people need. It has BLE for communicating data, either during data capture or results when it's doing inferencing.
It has two of the NM500 neuron chips, each of which has 576 neurons. It's 1,000 neurons there. It also has an expansion connector, which lets us connect on-camera modules and more neurons if you have an application that needs either more neurons or video. If you need other sensors, you can build adapter boards that plug onto that connector and add in any kind of sensor that you need for your endpoint application. I would like to turn the presentation over to Guy Paillet, who is the CEO of General Vision.
I'm going to introduce a different way of making machine learning, which is lifelong learning in real-world. Basically, our neural network has been in continuous operation since 2003 on more than 50 systems aboard fisherman vessels in Iceland and Norway. This has been saving about $2 million per boat, it's not theoretical. It has been trained in deep sea water by the fishermen themselves, they are definitely not PhD. Well, maybe some are. They don't have any cloud access. They have a lot of cloud about them, but they don't have any cloud access. In terms of we believe that we have the most extensive fielding of chips.
What we can see here is also a Pulnix Zcam, which is the first neural network camera, which was designed actually very close to QuickLogic on the other side of the street, by Pulnix, and has 312 neurons. These neurons were actually designed by IBM, with whom I was a partner at the time starting 1993. It's definitely the longest AI available for now. Basically, NeuroMem technology was called initially ZISC, which stands for Zero Instruction Set Computer, and was invented in IBM France in Paris. General Vision has continuously improved the technology since 1993, and made the NeuroMem, which stands for neuromorphic memory. This IP was used on both ASIC and FPGA, obviously we see a very big synergy with FPGA as well. We have two chips. The first chip was the CM1K, 1,000 neurons.
The second one is NM500, which is now manufactured by Nepes, a Korean company. This IP actually translated also into a chip, Intel Quark SE, which is still marketed by Intel. It has a mere 128 neurons, it's very small comparatively to the 1,000. Obviously, it was also included into the Intel Curie module. There is also a large ecosystem. Lots of companies are developing knowledge builder applications, software development kits, and so on around this technology. There is a big difference with deep learning because obviously the training can be done real-time by the user itself with few milliwatts. We can see on this slide the part of the data sheet of Intel Quark SE, which has this 128 neurons with 128 components per neuron. On this video actually shows a real-time learning of different small characters on the paper path.
This was actually a FPGA IP implemented into a FPGA. This was demonstrated at the CES two years ago with quite good success. Obviously, it's possible to turn that into a real toy. We never did that. We can also have obviously always-on intelligence and you see that this application work on a small battery. Learning on the go. We can close the loop on the sensor because as a human eye is actually directed by the brain, it's possible also to modify the exposure and the shutter speed and so on. Again directly by the neural network. Possible to make sensor fusion. Some university in India has make both combine face recognition together with voice recognition . Totally autonomous. Does not depend to the cloud. Obviously can be connected as well, but can live without it.
Also important thing, it's possible to see the knowledge which is inside of the neuron, each neuron being a memory. Therefore, to trace back all the knowledge has been created where other method of learning just have coefficient, which is make difficult to understand the magic behind it. One thing we can do is also selective transmission. Transmit only novelties when something is strange or new. Also selective storage, which is something store on the novelty or when something is of interest. An example here is, well, it's in metric, but 37.5 is barely the temperature of the brain. In this application, we show that we have a few thousand neurons actually, which is in that case close to 25,000 neurons. They are running without any cooling, no fan, directly into the plastic box.
The good news is the plastic box didn't melt. That's an interesting part of it, which clearly show that we are much lower power. On the right of the screen, we can see a pattern that General Vision own with one of the largest glass company in the world, AGC Inc., for what we call intelliGlass, because the chip are so low power that we can put it into the thickness of the glass as well. In that case, you can see the sensor data presentation logic, which is the best of the data presentation logic is to be a FPGA, because it give us the flexibility. Then the neuron and the local decision logic also, which can be eFPGA ultimately as well.
In this video, we demonstrate that by clicking on the bus, we can actually track the bus and relearn as we go, because when the bus clarity is fading, we can relearn real-time, in order to continuously track the bus. This is same thing for an aircraft where we can relearn change of attitude as the distance or similarity range is actually diminishing. Here is a presentation of the neuron. They are just all connected together, and we have a broadcast mode, which is basically we solve the memory bottleneck. This is similar to the biological brain because we broadcast an information to all the neurons, and all the neurons will compete, to find which one has the best response. Neuron cell is very simple. Obviously, there is a memory, because intelligence is memory.
There is a category which give us what is the class or category of the recognized object. Also, we have the actual active inference field, which is a similarity domain and some other context. Obviously, the key ingredient is the learn and recognition logic. It's possible to cascade the neuron inside of a chip, but also outside of a chip, with only electric limitation, not architectural limitation. Obviously, as I mentioned before, there is a very big synergy between NeuroMem neurons and the FPGAs, because we have the need of conditioning the data and to put them into pattern, which are being able to be learned and recognized by the neurons. Actually, a neuron itself, it's a piece of memory with a bunch of logic gates around, and the FPGA side is about the same thing with a different wiring.
With that, it's possible to actually make real intelligence. As well, it's possible to make what we call cognitive storage, which is either selectively store interesting things into like an SD card or any kind of small storage device. Also possibly make a very high speed search based on semantic, on the content of this storage. Now, coming to a very interesting embedded application. One of the very important application is real-time condition monitoring, for any mechanical device like a jet engine, which is a good idea, or ball bearing, but also part or any kind of either biological or mechanical or electromechanical device, and detect when something is out of the normal operation range. It's very easy to learn a normal operation and detect abnormal operation, like abnormal heartbeat or abnormal vibration for ball bearing. One of the big application also is inspection, visual inspection.
Image recognition has remained elusive. Everybody's trying, but so far does not work very well. We believe that the combination of FPGA together with neuron can greatly improve things like obviously navigation for drones, also have the capability of making very large surface inspection, satellite imaging. We need to extract some features from the images, and this is where the FPGA or flexible kind of logic is very well appropriate for this kind of process. Also, we need to be able to make real-time relearning. Again, it can be reasoning where we can use FPGA for reasoning and telling the neuron to relearn something. These are very simple application. Here it's a very different application. It's cybersecurity. This application, the NeuroTube itself, has been delivered to a defense contractor for securing uplink for the very large drones like Predators and detecting tokens.
We can detect, validate a token, reaching the drones in about 400 nanosecond. Therefore, we don't need to reduce the speed of the communication. As well, the consumption is very small because we got about 2.6 teraoperation equivalent per second with less than 12 watts. There are a lot of application in also in cybersecurity, in detecting behavior of connection for denial-of-service attack. Various application. Also, there are application on text as well, for latent semantic analytics, understanding what sentiment are in the text. All these application has been prototyped already and used with the hardware. Another application here is what we call adaptive control. On the left, you can see a pendulum, inverted pendulum for adaptive control.
On the right, you can see an actual application where the gentleman has been implanted with electrode, and we plan to use a neuron to rebalance and generate the proper stimulus for him to be able to walk naturally. Right now, he can walk, but just one leg at a time. The promise of this application is to have this gentleman, Mark, walking normally. Thank you, and I'm going to turn it over to Jungho.
Hi. I'm James, and it is my great pleasure to be a part of this ecosystem. I personally thank Brian and other great people at QuickLogic to prepare this event. For the next 15 minutes, I'm going to introduce about Nepes Corporation, since many of the audiences may not be familiar with who we are and what we are doing. I will introduce what is NM500 and its application, the challenges, and our expectation of this ecosystem. The Nepes Corporation is South Korean public company doing advanced packaging foundry services. Technology services we provide are flip-chip bumping, wafer level package, fan-out wafer level package, and system in package or SiP modules . We have been doing this business for over 28 years, and we have top-tier customers in consumer electronics, automotive, mobile, and wearables around the world.
In a nutshell, what we're very good at is making smallest packages IC with more components in a very cost-effective way. How smaller can you make? I'll show you one example in the next slide. On your left-hand side, orange color board called Orangepip, which is an Arduino clone. On your right-hand side, the tiny chip a person hold is called a Datduino. We put every component on the Orangepip and integrate it into the Datduino package using fan-out SiP technologies. They are 100% compatible to each other, and even greater thing is there are still some empty rooms in Datduino so that you can put more components later time. I guess now you get the idea. In the next slide. What is NM500? NM500 is a neuromorphic chip.
General Vision, Nepes has been working closely for the last couple of years to design and manufacture it. It has 576 identical cells called neuron. A neuron is consisted of logic gates and memory, and each neuron is parallelly connected to the other neurons. It is designed from scratch for true parallel computing and mass scalability. Each NM500 has 576 neurons. What this number 576 means to you? It means it can process and identify 576 different objects or patterns from the data. They are already more than enough for IoT, wearable, hearable, or even many vision applications. We started mass producing NM500 from the last year, and it has certified for the industrial use. To use the chip, we developed evaluation board called a NeuroShield, which follows Arduino or mbed form factor.
In case you want to expand number of neurons, we also have a NeuroBrick. You can stack it on the NeuroShield. For those who want to develop intelligent applications quickly, we have a NeuroStick. You can insert it into any standard USB port of your computer and instantly begin developing. Prodigy board is for those who want to develop vision applications. When you get the NM500, it is like a baby. It does not understand the world, so you need to train with the data. We provide generic knowledge building software called a Knowledge Studio. It is cross-platform software, runs on Windows, Macintosh, and Linux. We use show and tell method. Only thing you need is just point and click the data you want to train. Main purpose of this software is create, evaluate, and verify the knowledge for the NM500.
In the next slide, is a spec sheet. NM500 is small. It has a dimension of 4.5 mm, runs in 36 MHz, with core runs in 1.46 mW. Average power consumption is 135 mW in active mode, and package was done with a 64-pin Chip Scale Package and 500 micro-thickness. Next page. The application field. NM500 can be applied to really a variety of different industries. You can use it with image recognition, sound signal recognition, video data collection, text and packet recognition, and et cetera. The biggest industry requirement comes from the vision recognition, and that is what we are focused more on. We are working with many automotive companies, video surveillance companies, toys, and education markets as well. Today, obviously, because we are in the semiconductor industry, I would like to focus on semiconductor equipment market.
From the last year, market research shows that this year, semiconductor equipment market grows continuously, and especially the visual inspection grows up to $4.4 billion out of $50 billion market. As you know, every manufacturing site, always the last processing line is visual inspection. If we can horizontally expand with NM500 visual inspection solutions, it is going to be a huge market. I will show you one live example. The ILB, or in-line bonding equipment, is to attach ICs on a thin film type substrate. The upper left-hand picture is actual ILB equipment installed in our factory. The upper right-hand picture is we installed camera and the lighting to prevent the light noise, and we use one NM500 for this project.
While it can detect correctly bonding chip in green, which has no problem, but any error, such as wrong chip position, double bonding, or a flying chip, and et cetera, can be detected in red. You can see it working in the lower side pictures. It has been running 24/7, and the result is absolutely phenomenal. Record shows it has detected 100% errors so far. Even greater thing is now equipment engineers learn how to train and use this solution at the field. If there is any new devices coming, they know how to apply and deal with it. It is so-called a field trainability, a very important NM500's feature, which you can train the data at the field you are deploying and use it right away. Next slide. After working with many customers, we found out that there are some challenges in AI adaptation.
We all know that now the industry has transitioned from mass production to small quantity batch production era. That means your production should have more flexibility than ever. Technology is evolving every day, so does number of products you are making. Here are some of the challenges I listen from our customers when they are adapting intelligence in their business. The first one is vulnerability of network. If you are using deep learning, chances are you probably upload your data to the cloud for training. Some companies are flexible on that, but many are still hesitating to do that because the data may contain very sensitive information. NM500 can be configured to use with the cloud, but its nature is local execution, so you can be free from security concern. Second is data is keep changing.
For example, in our factory, the device, the wafer that we're processing is changing almost every week. This means training data set changes every week. How are you going to make knowledge or so to speak, a training out of them in time? When you make training for a new device and later old device come back again, what are you going to do with that? Train again? This is a big challenge. With NM500, adaptive field training will solve the problem. Third, there is a time when you encounter a problem, the problem that rarely happen. You can fix it easily if you know the problem beforehand. But when it happens without warning, it causes serious problem. Because it rarely happen, it is very hard to get abnormal condition data. Without data, you cannot train AI. AI is all about data with algorithm. How to solve this problem?
For example, a lot of machinery uses motors. Every motors, before it break down, the vibration gets different. With NM500, you only train the motor's vibration with normal condition data and detect any different patterns for analyzing its status later. Simple, but very effective solution. Hard to train problem is very well-known problem for deep learning. Because neural network it creates are so complex when there's a problem with the results, it is so hard or almost impossible to trace back which data causes the problem. NM500 provide knowledge model. You can trace back and pinpoint exactly which data causes the problem. This auditability is important for the companies when you encounter problem with your intelligence system. Last but not least is edge versus cloud. Using only cloud system also works well, but it has its drawbacks too.
If you're running sensor network, you know how massive network traffic occurs? If your sensor network can upload only the trained data you specify, overall network cost dramatically decreases. In case of mission-critical applications, you may want both edge and the cloud side, same intelligent capability in case of network connection loss. All of these real-world challenges are coming from very flexible environment, and it can be solved with NM500 in a very cost-efficient way. The next slide. What we expect from this QuickAI ecosystem. I'm very happy to be a part of QuickAI ecosystem. I believe this ecosystem will make powerful synergies. You know, AI needs combination of different technologies.
Especially if you are in embedded edge AI field, you need a semiconductor IC, you need hardware development kit, you need software development kit, you need data science platform, and of course, you need a live test bed environment for quick development as well. You cannot do it all alone. I have this high expectations of this whole ecosystem. Especially the new MERSIT platform that QuickLogic just released today, I believe QuickLogic hit the right spot with it. With its ultra-low power and the always-on capability with sound and sensor signal processing, combined with NM500, will open up wide variety of applications. On top of that, top-notch data science team of SensiML will integrate their robust knowledge processing platform with the MERSIT platform. Users can develop their own knowledge pack seamlessly working with the MERSIT platform.
It can be easily applied to AI speakers, predictive maintenance, smart toys, sound surveillance, or even automotives. I cannot wait for combining MERSIT platform with our vision solution. In the next slide, I'm going to show you one good example. We are currently developing 2-stage authentication for Android devices. It uses face and voice authentication and needs 100% accuracy. Face alone may not achieve the accuracy level, but voice combined, there are many papers out there already proved it is achievable. Applications like mobile, tablet, smart TV, signage, door locks are only a few customer requests for these applications already. I'm very exciting about the future of this ecosystem, and I hope this will give audience some idea about where we're going together. Thank you very much.
Thank you. Hello, my name is Chris Rogers. I am CEO and founder of SensiML. I'm excited to be part of QuickLogic's AI cognitive sensing for endpoints initiative, and welcome the opportunity to provide an overview of SensiML's product and fit within the QuickLogic ecosystem partner product plans for AI. SensiML is a transformative software toolkit enabling the rapid development and ongoing learning of smart sensor algorithms for endpoint devices. QuickLogic's QuickAI HDK is a very compelling hardware platform that we're very excited about, with multiple AI accelerator cores that provides a great opportunity for us to maximize power and performance optimization of sensor algorithms as generated by the SensiML toolkit. Before I get into the details of the toolkit itself, we'll take a step back and look at the overall solution as a whole. Next slide.
If we look at a traditional IoT solution, we typically have analytics being performed in three domains. I think we're all familiar with the cloud domain, where deep learning takes place. More recently, we've started to see the emergence of edge compute with things like fog computing initiative that is taking a distributed approach to AI, recognizing some of the scaling issues of centralized approach. The missing link in all of this has been the endpoint device. While there are sophisticated and evolved tools for AI, for cloud that we're all familiar with, things like Google TensorFlow, Caffe, Hadoop, Microsoft Azure machine learning platforms, Spark, and others. There are starting to be similar platforms at the edge.
What's surprising is that the toolkits available for developing algorithms for embedded endpoints is largely the same as it was 25 years ago when I was a practicing engineer developing embedded solutions myself. I find that very surprising. I think there's a great opportunity here that we can greatly improve the developers' process of creating scalable algorithms in a resource and time-efficient manner, begin to approach the kind of sophistication in software tools that we see elsewhere in the Internet of Things network. A bit of background on SensiML. The SensiML toolkit started life about three and a half years ago. At the time, we were known as the Intel Curie Knowledge Builder.
The little picture you see at the bottom left of the screen is actually a picture of Intel CEO, Brian Krzanich, presenting in Intel's big Intel Developer Forum back in 2016, the advent of this toolkit as a game changer for creating algorithms for sensing analytics at the edge for the Intel Curie device at that time. In the interim, we have spun out the Intel Curie Knowledge Builder as an independent software vendor. It is a wholly autonomous company that now provides a hardware-agnostic solution that supports over 30% of the Internet of Things endpoint devices that are available out in the market today. There was not insignificant investment that had been made in the Intel days. I think the cumulative R&D that was spent over the three years amounted to $18 million worth of development.
We have now a very mature toolkit that is ready to be applied to current and next-generation solutions. Notably, what we have with the QuickAI HDK is a very capable device that we're very excited about as a next-gen endpoint with a great deal of opportunities to address the accelerated cores that it makes available. Next slide. When we talk about developing for endpoints, certainly in the market today, there are a great number of devices that bill themselves as smart endpoints or smart devices, right? Whether it's a consumer space, or in the industrial or commercial space, we've got any number of devices that would add smart to the name, smart thermostats, smart predictive motion and maintenance, sensors for industrial. We even have a smart egg crate or egg tray device out there for consumer.
What I think we find with these smart devices is they tend to fall into two camps. The first camp is actually truly smart devices that have been built to be intelligent and to provide a great deal of analytics at the edge itself, but at the cost of significant development team size and cost and time spent hand-coding algorithms that work well for that device. Then you've got sort of the other bucket of devices which aren't so much really smart as they are just connected devices, where they take raw sensor data and then transmit that somewhere else within the network, like in the previous slide, to provide analysis either at the edge, on a smartphone application, or in the cloud. With all the caveats that come with that others on this call have already covered quite well.
As a device developer who's considering building a smart device faces the prospects of how to implement this, they're stuck between the need to invest significantly and with great risk because there's no assurance upfront that the algorithms that they would need to implement their application can fit a given piece of hardware using the tool sets as they exist today. The other is that they can suffer through some of the typical use case restrictions that come with cloud compute or computing on an edge device where you've got latency, security issues, and the bandwidth limitations over the network that don't support mobile applications, don't support battery-powered devices, as well as a truly smart device. Next slide.
What SensiML aims to do is to bring the sophistication that you see with other AI solutions elsewhere in the network to the smart device, and to do so in a way that doesn't require that the developer has to have an extremely large team and a multidisciplinary team that has data scientists, DSP engineers, test and data collection technicians, firmware engineers, and app developers, along with the domain experts for the given application, all figuring out how to collaborate, though they don't necessarily speak each other's language. The analogy I like to use is that before there were applications like WordPress that brought a high level of automation to website authoring. To build a really compelling website, you had to hire experts who really understood their way around HTML, CSS, and JavaScript.
Nowadays, you can put a tool like WordPress in the hands of somebody who understands what they want to build from a content standpoint, but not necessarily be a web expert, and they can build compelling websites all on their own. That's what we aim to do with SensiML, to be able to provide the ability for the long tail of IoT application developers, without heavy expertise, to be able to quickly build algorithms that can make sense of sensor data and transform that into meaningful events, and do so without having to have the significant investment that would otherwise be needed. Next slide. What makes a sensor truly smart? This flow here kind of shows going from the physical property to be measured by a sensor of some flavor, all the way to taking that to some meaningful event that's of interest for a given application.
The example we use here is for predictive maintenance, and I'm looking to find some discrete classes of machine anomalies, such as excessive vibration from a loose motor mount or a flange bearing failure, or some kind of a obstruction of a fan blade on a motor frame. Right? A connected sensor in sort of a conventional sensor application would take the sensor itself. It would capture the data and sample it at a given frequency. You might do some simple transforms and signal conditioning, and maybe some simple compression in order to make it somewhat more efficient in its transfer of that data elsewhere in the network. It certainly doesn't go to the extent of providing any meaningful insights.
A smart sensor takes that many steps further and applies the expertise of a domain expert who understands various failure mechanisms within predictive maintenance, and can label segments of data that can then be used by a tool like SensiML to go through a population of data that has been labeled and create algorithms that will classify accordingly, so that the device that's been now programmed with this algorithm can autonomously detect and flag such meaningful events, as opposed to just send lots of redundant data over the network to be handled elsewhere. How we do this at a 40,000-foot level?
This slide shows an example of the various components of the SensiML solution, starting from what we call SensiML Data Capture Lab, is a PC and mobile application variants that we have available that allow a developer to collect in the field or import existing data such that they can easily label. This can be a fairly mundane and time-intensive task for supervised machine learning, is just the actual capture of the data and labeling of that. We seek to make that as painless as possible by automating a lot of the process by having tools within that particular application that help you with, if you can label a couple of examples, it can infer by example what it believes to be other examples, and then that's a matter of just confirming whether that, in fact, is consistent with your understanding.
At the point that you've got labeled data, then you pass it up to what's called the SensiML Analytics Engine, which is a cloud-based tool that then traverses all of the various event segmenter algorithms. It performs the feature engineering, looking over the entire space of available feature extractors that are on-hand. We have over 100 of these feature extractors going from very simple downsampling to much more sophisticated MFCC and FFT-type feature extractors that really can take advantage of the FPGA that's available in the QuickAI. When this analytics engine is given a set of constraints in terms of how accurate does my model need to be, how much memory do I have available, and by the way, most of these parameters are understood by the fact that the target platform is the QuickAI.
It knows what the parameters of that device are, it can already have a fairly good understanding of how to optimize for that particular target hardware. It becomes a matter of providing it with labeled train and test data, as well as basic parameters for the expectations of the model, and then the tool comes back with what we call a knowledge pack. A knowledge pack is our term for either a library level or binary code that's now firmware compatible with the end device so that you can now quickly flash that device with the generated model and then go empirically test that in the field and confirm that it does what you think it does.
To close the loop further, once that model has been generated, any subsequent events that match patterns that it is familiar with would just return back the event itself, thereby greatly reducing the bandwidth required over the network and giving you extremely low latency and the ability to control and/or get feedback directly from the device. For things that it isn't aware of, let's say you have some new novel class that hasn't been identified in your development phase, you can optionally configure the knowledge pack such that it'll report back either the raw data or some interim feature vectors so that that can be used for ongoing learning. That learning mechanism allows you to close the loop and then have sort of a continuous learning process from the device, even after it's been deployed.
If you compare and contrast that to how things are done today, as I mentioned on this next slide, typically today you would have five different domains of expertise ranging from data scientist to DSP engineer, firmware developer, app coder, and a domain expert, all doing custom code work, often within a statistical modeling tool that can come up with a theoretical model that can work on a very fat client device. Now you have the challenge of how do you optimize that to be power and performance friendly for an embedded platform? That is often a matter of custom coding within a compiler environment, and can be iterative and can be fraught with risk, as there's no deterministic way to know whether that model will in fact fit the hardware or not.
That process, we've talked to many different developers, can range from $500K or more, and takes six to nine months of development time alone. That can frequently mean that the algorithm becomes the bottleneck or critical path in the overall deployment of a new device. Minimizing the risk, lowering the cost, and making it scalable so that developers can come up with algorithms that can be quickly adapted and learning over time is what the goal of our tool is. If we can track that on the next slide, we take an approach where it's an automated learning using the Analytics Studio. At the very minimum, you may need only an app coder and a domain expert if you don't have a lot of additional code to add and firmware above and beyond what comes out of the classifier itself.
We've done applications where this has been done in as fast as four to six weeks. I'll note that that four to six weeks can be a majority of the time spent with test technicians just doing data collection. Whereas you spend a lot of time with data science, DSP engineers, and high-value resources in the case of generating algorithms in a more automated approach, this job can often be done with data collection test technicians properly labeling data sets. On the next slide, just some examples of some applications we've done in the past. When this was the Intel Knowledge Builder Toolkit, we did a POC that's public with Honeywell for a next-generation first responder. First responders have these devices called PASS devices that are built into their rescue breathing apparatus.
Basically, it looks to see if there's any motion, and if it detects any lack of motion for 30 seconds, it sends off this 120 dB alarm to notify somebody nearby that you have a man down. It's not a terribly sophisticated device, but yet an opportunity to make something that can do certainly that use case plus a whole lot more. Honeywell had asked us if we could create a smart wearable that could give a field commander contextual awareness of what's going on within an emergency situation. He's got a number of first responders in a building responding. You may not necessarily know what's going on at any given time. They use a common radio channel to talk, and it's a noisy environment, and they're talking over each other. Often it can be very difficult to just understand what's happening.
In the case of this POC, we created two wearable devices that allowed them to do gesture recognition for quick status indication, and another that was a body-worn wearable that had a library of different activities of interest, like climbing a ladder, walking, running, laying prone that it would detect and provide a dashboard for the incident commander to say, "Of all the responders I have on-site, here's what's going on," right? Give them better awareness. We did another application for Intel in its own fabs from a manufacturing environment to best understand whether they had any imminent machine failures. This was really a predictive maintenance application. Rotating machinery was involved. We utilized the same classifier that we've been discussing in that context to see if we can't detect machine anomalies before they became issues that required a lines down situation within the fab. Then, next slide.
This is an architectural view showing how the solution integrates with the QuickLogic QuickAI SDK. As I mentioned, we have a number of different applications within our tool chain, the end result being the creation of what we call this knowledge pack. What I'm very excited about with this particular hardware platform is that there are a number of different accelerator opportunities here that couple quite nicely with the capabilities of the software toolkit. The first being event detection and feature generation, which can utilize the resources of the FPGA fabric as well as the Flexible Fusion Engine with DSP functions that can allow us to do very power-efficient feature extraction on the front end of the tool chain.
Finally, the classifier that we had run on the Curie as Guy had mentioned in those days, we had used a fairly modest 128 neurons that were available on that chip. Now we have over 1,000 neurons available, and with hardware acceleration, then we can offload the classifier portion of the work to that NM500 silicon so that we have the full use of almost 10x the number of neurons we had from the Intel Curie days. This could lead to some very significant applications both in terms of power and battery life, as well as the complexity of sensor algorithms that we can actually implement in this kind of a device. We're very excited by that and look forward to explaining more, I thank you very much.
Thank you, Chris, Malik, Guy, and Jungho. Thank you to the audience for your time today. We hope you found the insight from all these great companies both interesting and informative. As you can imagine, we are big believers in the promise of AI and cognitive computing at the endpoint and enthusiastic about the capability and domain expertise of General Vision, Nepes, and SensiML enabling it. We will now open the call for questions.
As a reminder, ladies and gentlemen, if you have a question at this time, you may submit your question online by clicking on the Ask a Question tab on the webcast screen. Thank you.
Yeah. You have to read the question.
Okay. Our first question is why did you choose to work with General Vision's AI hardware platform? What are the metrics for power, performance, et cetera, that lead you to believe they are better?
The thing we like about General Vision's platform is it's oriented towards endpoints in the sense that there's a piece of hardware there that gives you good power performance as opposed to the deep learning, which is very power-consuming, high compute, big IoT devices. Basically, power, performance, two aspects of that. The second point that I would make is that this AI is particularly well-tuned for training in the field, which is unlike the classical TensorFlow kind of AI. The applications we see are ones that need training in the field.
Another side effect, the fact that they've already been used in the Intel processor, Intel Quark and other chips. I think there's a lot of AI innovations that are happening out there, none of which or some of which are commercialized and some aren't, but clearly General Vision has done a good job in getting into the market, into real devices, it sort of de-risks that for other people. I think those are the primary reasons. That was QuickLogic's perspective, but I think it'd be interesting to get, James, your perspective from Nepes on why you chose General Vision as well, since you have that already deployed in your NM500.
Yes. If you are comparing with the other deep learning solutions, they are using so much power in that field. NM500 is already proven, and when you are manufacturing it uses the most stable processing lines in the semiconductors. It is a very much proven technology. That is the first impression that we get, we decide to manufacture the NM500. It was a great job.
Okay. The next question that we see here is did QuickLogic and partners come up with a solution with a specific customer application in mind? I'll start with our perspective and then open that up to Guy, Chris, or James for their perspective also. The way that we're approaching this is that the solution is the platform. You can see that with the HDK, that it actually has a lot of different sensors on the HDK and connectors that we can add additional sensors. We're looking at this more from a set of applications. I think everybody that's familiar with QuickLogic knows that we have our patented FFE, which is really good at interfacing with sensors and lightweight processing for motion and biometric sensors.
We know the FPGA has a lot of vision use cases that we've done long ago in the past that we're probably going to start bringing back to bear on this market. This is a flexible platform designed to address those types of applications, and that's where we see working with these companies like Nepes, SensiML, and General Vision, especially around the areas of vision and time domain series data. No to a specific customer, but yes to a set of customers and a set of applications that we think is going to be very important in this IoT space. Guy, James, or Chris, do you want to add anything to that?
This is Chris. I can say that like you said, Brian, we didn't target a particular application per se. We look at it as there continues to be sort of an advancement on sensor technology and price and performance of MEMS sensors particularly have led to a variety of sensor types that can provide very rich data that on the one hand is a great opportunity for much better contextual insight of what's happening, but on the other, floods networks with lots of potential raw sensor data that needs to be processed. We see there being opportunities to provide this kind of analytics across a long tail of IoT applications, whether it be time series data from accelerometer and gyro data for motion analytics or vibration sensing or pressure sensing or the vision applications that we talked about.
When we developed a toolkit with SensiML, our goal was to try to make a general-purpose tool that could be readily adapted by developers to a variety of different use cases and streamline their process greatly.
Yes. This is James. Yes, I would like to add 2 more applications. The number 1 is the predictive maintenance for the semiconductor equipments or their factory airflow machines. Because while Nepes is more focused on the vision areas, but actually for the predictive maintenance, they need a different kind of sensors. Those kind of areas, I think that QuickLogic and the SensiML has more knowledge and expertise in that area. If we can work together, then first we can go into that market immediately, and the second one is the current applications that we are making as a 2-stage authentications. Face recognition plus voice recognition at the same time. The face recognition for those mobile or tablets, we can actually progressing a lot. In the voice applications, well, that is not actually our expertise.
In this ecosystem, we can combine it all together, and we can actually go into that applications as well.
Maybe, this is Guy Paillet. Maybe I can add something. This technology, as I mentioned, has been started in 1993 with what was labeled ZISC at the time, zero instruction set computer, that I co-invented as a partner with IBM France. IBM developed back in 1993, the ZISC36 had only 36 neurons. The beauty with the semiconductor company technologies right now is that we can go to 500 neurons with 110 nanometers technology, whereas the ZISC36 was a one micrometer technology. It was a big chip with 36 neurons. We are just at the beginning of the roadmap of the, obviously, putting more neurons. Putting more neurons does not involve rethinking the architectures. IBM, for example, in the 1990s had made a lot of applications.
They made a product called Neuroscope that they use in their plants all over the world to make condition monitoring and predictive maintenance. Something similar to what Nepes is doing now in inspection, but also ball bearing monitoring, something like that. Right now with the cooperation with QuickLogic and Nepes and SensiML, I think we have the whole ecosystem to deploy this solution on the very wide scale, especially with the fact that it's very easy to increase the number of neurons, and also to package that in the obviously, a small setting similar to what the Intel Curie was with Intel. I think it's just the beginning of the road for all of us.
Thanks, Guy Paillet, Chris, and James. The next question, Guy Paillet, this probably is appropriate for you to take. It's what resolution and frame rate of camera input can the device handle? I would say it's probably the technology handle first, and then maybe James, you could discuss the device specifically with the NM500. Frame rate and resolution of the vision applications.
Well, it's a very interesting question, actually. When I was in France and cooperating with the French military, one of the applications I did is depleted uranium shell tracking and detection. We were handling stereoscopic sensors. This was back in 1994 at 1,500 frames per second and for detecting two pixel singularities. This was done with actually the ZISC78, which was the quarter micron technology. Definitely it's possible to go to a very sophisticated and very demanding application. The chip itself or the neurons themselves, actually, the main features is the fact that we can match one pattern versus any numbers. Any numbers so far we have been up to 1 million. What has been delivered, for example, to the defense contractor here in U.S. is one pattern versus 64,000 in 400 nanoseconds once the pattern is entered.
Definitely it's possible to go to very high frame rate by putting more chips and so on. The technology itself is not the limitation in terms of frame rate, is really sensor-depending.
Yes. I operate to Guy. The Prodigy board that we're producing is actually comes with the cameras. First, the version of the camera that we're choosing was the five megapixel 30 frames per second. Actually, the camera itself is not very important. You can actually go even higher resolution, higher the faster frame rates, but it's only how you configure with the NM500.
Okay, our next question is regarding the availability of all these elements of the solution. I think I can let General Vision, SensiML, and Nepes share their own availability. What I'll say is consistent with what we had on our press release today, which is for us, this is all based on our existing EOS S3 platform, which is available now. It's in mass production. The QuickAI HDK that was shared in our slides is sampling now, and it'll be generally available by the end of this quarter, Q2. The SensiML Analytics Toolkit will be ported to the excuse me, the QuickAI HDK platform, and that'll be available at some point during Q3. Guy, Chris, or James, if you wanted to share any availability on the call, please go ahead now. Yeah, this is Chris.
From SensiML standpoint, there is an existing release for the supported devices such as the Curie. If customers want to familiarize themselves with the tool flow and how the basic operation works, that's available today. As Brian mentioned, in Q3, we'll have the successful port with the hardware optimizations available for the QuickAI.
Let me see. NM500 was already available from the last year or so.
Yeah.
In terms of General Vision, as our name suggests it, we are focusing on vision. We have image knowledge builder, which has been dealing with multiple chips, including the Physis 78 and the CM1J and now the, obviously, the NM500 that James mentioned is in production. Therefore, all these products are available as well as software development kits, which allow for easy interface of existing device to the NM500.
Okay. There's been a couple of questions submitted to us that are more investor related, which I'll try to cover on our earnings call next week on May 9th. We'll wait for another few seconds here to see if there's any other questions coming in.
As a reminder, ladies and gentlemen, if you have a question at this time, you may submit your question online by clicking on the Ask a Question tab on the webcast screen. Thank you.
Okay, looks like no more questions are coming in. I'd like to thank everybody for joining us today. I'd like to thank all the presenting companies for their participation. We're very excited about this ecosystem, as I said earlier, and stay tuned for progress from QuickLogic and press releases, our blog and our earnings calls, and we're looking forward to doing some great things together with this ecosystem. Thank you.
Yeah. Thank you very much.
Ladies and gentlemen, thank you for participating in today's conference. This does conclude the program, and you may all disconnect. Everyone have a great day.