Good day, ladies and gentlemen, and welcome to the Artificial Intelligence and Cognitive Sensing at the Endpoint Conference Call hosted by QuickLogic. At this time, all participants are in a listen only mode. Later, we will conduct a question and answer As a reminder, today's conference may be recorded. I would now like to turn the call over to Mr. Brian Faith, CEO of QuickLogic.
Sir, you may begin.
Good morning, everyone. I'm excited to kick off the agenda today. Following me is Malik Saadi, VP Strategic Technologies of ABI Research. He will cover what he sees as growth and drivers in the AI and cognitive computing space. After Malik, our CTO and I will return to share what QuickLogic is doing in this exciting space, followed by each one of the companies involved in the QuickAI ecosystem.
Guy Pye, CEO of General Vision Jung Ho An, SVP of Nepis and Chris Rogers, CEO and Founder of SensiML. Malik, I will turn it over to you.
Hey, good morning and thank you for inviting me to this webinar, Ryan. So you asked me to provide a global overview on cognitive computing artificial intelligence, market drivers and opportunities with some focus on industry applications. Before getting there, let me give a brief introduction to artificial intelligence and how we do see it here at ABR Research. Artificial intelligence is one of the hottest and diversified technology breakthrough in modern time. The technology will enable many industries to increase their productivity and improve operation efficiency through implementation of intelligent process automation.
In fact, artificial intelligence is nothing else than a new computing paradigm that allows machines to perform intelligent tasks with low to no assistance from humans. So all start from gathering relevant data from sensors, devices and applications surrounding us. Those data are then sent to training engine to be structured, classified and filtered to identify patterns, behaviors and trends created collectively from all data point gatherers. Once those patents are identified and grasped, they are sent to the inference engine, which is responsible for recognizing the patents, making sense of them and then taking special actions commonly called inference in the AI jargon. Let me give you an example.
For example, let's take a faulty machine making an unusual noise. So that noise could be a pattern exploited by AI engine to help identifying and vocalizing a given anomaly within the machine. And the machine and the engine, the artificial intelligence could potentially prescribe and recommend procedures to prevent potential failures of the machine. So traditionally, both training and inference require significant data processing resources and this has prevented AI from taking off for years. However, with the improvement of both processing and communication technologies, AI is now entrenched into many industries including professional service, consumer electronics, social networking and industrial automation.
Next slide. In terms of market size, ABI Research estimates that 3,200,000,000 devices shipped in 2023 will be actively using AI in some way or shape. Smartphones and consumer devices and desktops will continue to have the lion's share from this volume, but industrial application will be a major area of growth within the next five to ten years. Although the focus of this presentation is industrial application, ABR Research tracks 58 use cases for artificial intelligence across 11 major verticals. That includes mobile devices, consumer electronics, computing devices, smart home, retail, industrial automation, robotics and smart building.
Next slide. Before jumping to industrial applications, let's have a look into how AI is implemented in
this slide.
There are three possible scenarios for executing artificial intelligence algorithms, and this slide illustrates just this. So let's start with scenario one shown in the upper part of this chart, which describes situation where all AI processing, including training and inference, resides in the cloud. Actually, this is the most popular option for many industries that use cloud sourced data, including professional services, retail services, social networking and others. This scenario is highly dependent on always on communication. Unlike scenario two and three that are less reliant on always on communication.
So now for scenario three, described in the middle part of the chart, it shows the situation where the big part of the training resides in the cloud while the inference is dealt with at the edge of the network. With edge here describes either an on premise server, a gateway or the end device itself. This type of scenario is typically used for certain process constrained applications that require immediate inference but offer a certain tolerance to reliability of decision making. In scenario three, we describe which is described in the lower part of the chart, it shows a situation where all AI functions, including training and inference, are executed at the edge of the network, either that be on the premise server, on the application gateway or on the end mode itself. This scenario is often used for mission critical applications that require near real time performance.
Otherwise, that execution could be affected by latent communication or slow process execution. Typical application for scenario three includes industrial automation, robotics and certain automotive applications. Slide four. Now let's see the benefits and costs for executing certain AI or cognitive computing function into the cloud versus the edge of the network. Cloud AI offers many benefits, including high processor resources capable of dealing with a large volume of data used for training.
Those data could originate from a variety of sources, which could make the training more efficient and more reliable. However, Cloud AI is not designed for mission critical applications where low latency is an important element in the operation of those machine critical devices. Data privacy and security could also be an inhibitor for cloud AI, mainly for some specific industries, including industrial manufacturing, automotive or network automation. Let's now jump to AI. Edge AI, on the other hand, brings processing capabilities closer to the end mode, which means AI functions are executed with a fairly low latency and far less reliant on connectivity with the outside world.
Not only this minimizes the risks for data to fall into the wrong hands, but this could potentially lead to lower AI implementation costs compared with cloud AI. API research has identified the clear trends towards HII initially driven by the migration of AI inference within consumer electronics and computing devices. But this trend is increasingly driven by AI adoption in the industrial segment, including video applications, manufacturing, automotive and robotics. And two strengths are creating tremendous opportunities for chipset supplier like QuickLogic, NVIDIA, Qualcomm and Intel, who are now preparing their processors and chipsets to tap into these opportunities. Slide five.
This slide focuses on market opportunities for cognitive computing within the industrial market. As you can see here, our forecast indicator will be some three sixty five million industrial applications that could potentially rely on cognitive computing by 2023. As you could see from the pie chart above, this market is still largely driven by video application, including set monitoring, personal facial recognition, camera surveillance and traffic flow analytics. However, major areas of growth will be smart manufacturing that will see its market share jumping from 2% scored in in 2017 to above 10% anticipated for 2023. Smart retail store devices is also a market expected to grow within the next six years, with shipments of capable devices expected to exceed 50,000,000 devices by 2023.
Overall, we expect industrial AI applications to rely massively on edge computing, with only 25% of those applications are anticipated to use the cloud for both training and inference. Now let us zoom in on edge computing and edge AI to see how computing is distributed across different device types. As you can see from this slide and as far as industrial manufacturing is concerned, the majority of the AI training and inference occurs in the on premise server. Manufacturers generally prefer on premise servers over cloud for many reasons, including compliance and interoperability with legacy automation platforms that are set on the premise of the manufacturing site. There is a security and privacy concerns when sensitive data is shared with the cloud in the case where cloud AI is used.
And obviously, there are also concerns around costs and time to integrate II with existing IT platforms. As you can see from this figure, inference will move from on premise to the end devices as those devices become more intelligent and powered by adequate process resources. This specifically is the case for machine vision and predictive maintenance where decisions needs to be made in real time and as close to the end node However, we see training to continue to be resizing on premise servers, and we don't anticipate this to change drastically within the coming years. I have come to the end of my presentation here.
So enjoy your day, and I hope the WebEx will help you in making some informed decisions. Now I hand it over to Brian. Thank you very much for your attention. Thank you.
Thank you, Malik, for that insightful presentation. When most people talk about AI today, they're referring to the AI that takes place in the cloud or the data center. It's what has driven growth for GPUs and FPGAs in particular. And there are good reasons to implement deep learning in the cloud where there is almost infinite compute capability and substantial power budgets. And as Malik said earlier, there are clear reasons for wanting to move some of the AI or cognitive computing capability to the far edge where endpoint devices reside, essentially moving a lot of the intelligence to the device with the sensor.
In viewing AI as a processing spectrum from cloud to endpoint, we can see how a hybrid approach of cloud and endpoint processing are complementary. Let's first look at a cloud dominated architecture. Imagine a network of sensors sending the raw data back to a cloud service for compute and decision making, particularly if there are other contextual data required to make the decision. While this works well in many applications, there are trade offs. In this diagram, the red circles represent all of the raw data, while the green triangle represents the data that the system is looking for to be actionable.
First, sending back all raw data to the cloud for processing adds in a latency that might be unacceptable in some use cases. If it is a simple thermostat sampling temperature, latency wouldn't be much of an issue. But if it is for a voice enabled application, even a couple seconds of latency would be viewed as unacceptable. Second, let's think about power consumption. While power consumption of processing in the cloud is not viewed much as an issue, the energy consumed to send all of the raw data back to the cloud may severely impact battery life in some applications.
And lastly, data privacy can be an issue and we'll get into an example on that later. On the other end of the spectrum is to deploy more of the computing to the endpoint. By pushing some of the always on intelligence out to the endpoint, one can reduce the system latency of communicating with the cloud, save power consumption by not transmitting all of the raw data back to the cloud and enable more data privacy by intelligently choosing what and when to send. We believe that the synergy between endpoint and cloud can provide a better system approach to AI where endpoint is a key component and enables cloud AI to be optimized. Let's use Amazon Alexa as an example where we believe they implemented a balance of cloud and endpoint processing the right way.
Would you feel comfortable if your Amazon Echo sent each and every sound of the cloud for processing? The incremental electricity bill may not bother you, but you probably wouldn't feel comfortable with Amazon listening to every sound and word spoken in your room. Moreover, what happens if your available Internet bandwidth drops while your teenager is watching that latest YouTube video? By moving the AI for local command processing, the detection of the keyword Alexa to the endpoint device, you now have a low latency and more private way of accessing Amazon's cloud. Then your actual intended request for Amazon is the only thing that is transmitted.
Now let's carry that example a little further. What about hearable devices that want to enable the same Alexa experience? If you were to stream most of microphone data to the cloud or smartphone, you would likely have battery life issues from the power consumed and continuously transmitting that data. Contrast that with being able to do more local processing of the microphone data to pick up spoken commands before opening a communication link to the cloud. The point here is that intelligent system partitioning is necessary to deploy AI in a way that results in a more positive ROI for the system OEM and a positive user experience of the consumer.
All of this probably sounds intuitive, so let's get to some of the challenges we see in actually realizing more AI and cognitive computing into endpoint devices. If you're a large platform company like Amazon or Google or Fitbit, you can invest in hundreds of algorithm engineers and data scientists to understand the data, know how to model it and invest in the software needed to partition the system appropriately. Unfortunately, not everyone can afford that investment and IoT is too fragmented a market for most companies to invest such resources in. We have always believed that an ecosystem of companies with domain expertise can provide a solution and in this case a platform solution that can be used by OEMs to deploy AI at scale. And so today we are announcing QuickAI, a new collection of companies who bring deep domain expertise in the field of AI and cognitive computing.
You'll hear more about each company and I hope by the end of this webinar you will understand why we are so excited to be part of this strong ecosystem. Now I will turn the call over to our CTO and Senior VP of Engineering, Doctor. Tim Sachs to share more about how we see solutions being developed based on the QuickAI platform.
Thank you, Brian. I'm going to talk about two applications and then our hardware development kit platform. So the two applications of vision inspection, vision has a number of challenges, in particular that you can go discrete, which might mean looking at a fruit, for example, or continuous surface kind of inspection. Discrete is a good case endpoint because you might actually move it into the field and be looking at items before you pick them to see if they're suitable for picking. Surface creates the problem of speed.
If you're printing packaging material, you might want to be inspecting it to make sure it's good and you might want to stop the line quickly if you find defects in it so you can repair them before too much bad material is created. In terms of performance, moving to an endpoint means typically a small MCU for power reasons and small MCUs don't typically have performance required to manage pixel data from a camera. So we find FPGA is a good way to do that. You can use an FPGA to interface to the camera and to massage the data stream in a way that's compatible with the neurons. So for instance, you might be choosing a region of interest or you might be doing subsampling or histograms and then presenting the data to the neurons in order to get their output.
The second example is predictive maintenance. As Malik mentioned, people like to check for vibration or listen to sounds on machinery. And one of the things about industrial machinery is it tends to be very different. So you might have a similar piece of equipment, let's say an air conditioner that you want to monitor. And that air conditioner might be mounted on the roof of a concrete building, which is a very rigid roof or it might be mounted on the top of a warehouse with a flexible roof made out of wood or something like that.
So the way that air conditioner will respond varies based on the conditions. Because of you need to train it in situ, you can't train it offline, which creates this need for endpoint training. And the system would learn then from either vibration or audio in order to work what's normal and then when something's not right, it would tell you that the thing is not right. The data rates are kind of high for pure software. So you might be a few kilohertz sampling rates.
Audio is definitely kind of high for pure software. So again, the FPGA turns out to be useful in this instance because with the FPGA, you can compute various things like FFTs at somewhere like a oneten to onefour of the power that you would normally do in software. And then another device that's very useful in our solution is the thing we call the flexible fusion engine. It's specifically designed for managing accelerometers and gyros. And so it is perfect for offloading the MCU handling the accelerometer and gyro that you need for the vibration analysis.
So the final piece is the QuickAI HDK platform, which is designed to do the multiple needs of endpoint AI. It's designed to be low power. It's designed to do data collection because you need to get the data in order to analyze in the first place. And then it's designed to be small enough about the size of a business card, so you can easily deploy it on proof of concept applications. And to support all of this, it's a rich platform.
It has the EOS device, which has an M4 processor, processing, the FFE for doing the accelerometer kind of processing. And it has the accelerometer, gyro and magnetometer on it for all of the motion applications. It has two microphones, so it can do the audio applications that people need. It has BLE for communicating data either during data capture or results when it's doing inferencing. And it has two of the NM500 neuron chips, each of which has five seventy
It's 1,000 neurons there. It also has an expansion connector, which lets us connect on camera modules and more neurons if you have an application that needs either more neurons or video. And then if you need other sensors, you can build adapter boards that plug onto that connector and add in any kind of sensor that you need for your endpoint application. Now I would like to turn the presentation over to Guy Paier, who is with the CEO of General Vision. Okay.
So I'm going
to introduce a different way of making machine learning, which is lifelong learning in real world. Basically, our neural network has been in continuous operation since 2003 on more than 50 system aboard of fisherman vessel in Iceland and Norway. And this has been saving about So it's not theoretical. And it has been trained in deep sea water by the fishermen themselves.
So they are definitely not PhD, well, maybe some are. And but they don't have any cloud access. They have a lot of cloud about them, but they don't have any cloud access. So in term of we believe that we have the most extensive fielding of chips. And what we can see here is also PLNICS Z CAM, which is the first neural network camera, which was designed actually very close to QuickLogic on the other side of the street by PLNICS and three twelve neurons.
These neurons were actually designed by IBM with whom I was a partner at the time in starting 1993. So it's definitely the longest AI available for now. Okay. So basically, the Neural MEM technology was called initially ZISC, which stands for Zero On Screw Sunset Computer and was invented in IBM, France in Paris. And then General Vision has continuously improved the technology since 1993 and make the Neural MEM, which stands for Neuromorphic Memory.
And this IP was used on both AirSeq and FPGA. And obviously, we see a very big synergy with FPGA as well. We have two chips. The first chip was the CM1K1000 neuron and the second one is NM500, which
is now
manufactured by company. This IP actually translated also into a chip named Quark SE, which is still marketed by Intel. It has a mere 128 neurons, so it's very small comparatively to the 1,000. But obviously, it was also included into the Qiari module. There is also a large ecosystem, a lot of companies are developing knowledge builder application, software development kits and so on around this technology.
And there is a big difference with deep learning because obviously the training can be done real time by the user itself with few milliwatts. So we can see on this slide, part of the data sheet of Intel Quark SE, which has this 128 neurons with 128 component per neurons. So on this video is actually show a real time learning of different small character on the paper patch. And this was actually FPGA IP on the implanted into FPGA. This was demonstrated at the CES two years years ago with quite good success.
And obviously, it's possible to turn that into a real toy. We never did that. And we can also have obviously, always on intelligence, and you see that this application work on a battery, a small battery. Learning on the go, we can close the loop on the sensor because as the human eye is actually directed by the brand, it's possible also to modify the exposure and the shutter speed and so on, and again, directly by the neural network. Possible to make sensor fusion, some university in India has made both combine face recognition together with voice recognition.
Totally autonomous, does not depend to the cloud, obviously can be connected as well, but can live without it. And also important thing, it's possible to see the knowledge, which is inside of the neuron, each neuron being a memory. And therefore to trace back, all the knowledge has been created where our method of learning just have coefficient, which is make difficult to understand the magic behind it. So one thing we can do is also selective transmission, transmit only novelties when something is strange or new and also selective storage, which is same thing store only novelty or when something is of interest. So an example here is, well, it's in metric, but 37.5 is barely the comparator of the brain.
And in this application, we show that we have a few thousand neurons actually, which is in that case close to 25,000 neurons, and they are running without any cooling, no fan, directly into the plastic box. And the good news is the plastic box didn't melt. So that's an interesting part of it, which clearly show that we are much lower power. And on the right of the screen, we can see a patent that General Vision own with one of the largest company in the world, Assai Glass, for what we call ancillary glass because the chips are so low for where that we can put it into the thickness of the glass as well. And in that case, you can see the sensor data presentation logic, which is the best of the data presentation logic is to be FPGA, because it gives us the flexibility.
And then the neuron and local decision logic also, which can be eFPGA ultimately as well. So in this video, we demonstrate that by clicking on the bus, we can actually track the bus and relearn as we go, because once the bus clarity is fading, we can relearn real time in order to continuously track the bus. And this is same thing for an aircraft where we can relearn change of attitude as the distance or similarity around this is actually diminishing. So here is a presentation of the neuron, they are just all connected together. And we have a broadcast mode, which is basically we sold the memory bottleneck.
And this is similar to the biological brand because we broadcast an information to all the neurons and all the neurons will compete to find which one has the best response. Neuroncell is very simple. Obviously, there is a memory because intelligence is memory. There is a category which give us what is the class or category of the recognized object. Also, we have the actual active and field, which is similarity domain and some other context.
And obviously, it's a key ingredient, it's a learn and recognition logic. And it's possible to cascade the neuron inside of a chip, but also outside of a chip with only electric limitation, not architectural limitation. So obviously, as I mentioned before, there is a very big synergy between MEMS neuro MEMS neurons and the FPGAs because we have the need of conditioning the data and to put them into pattern, which are being able to be learned and recognized by the neurons. And actually, the neuron itself, it's a piece of memory with a bunch of logic gets around and FPGAs is about the same thing with a different wiring. So with that, it's possible to actually make real intelligence.
As well, it's possible to make what we call cognitive storage, which is either selectively store interesting things into like an SD card or any kind of storage small storage device or also possibly make a very high speed search based on semantic on the content of this storage. And now coming to a very interesting embedded application, one of the very important application is real time condition monitoring for any mechanical device like a jet engine, which is a good idea, or ball bearing, but also ARP or any kind of either biological or mechanical or electromechanical device and detect when something is out of the normal operation range. It's very easy to learn normal operation and detect abnormal operation like abnormal heartbeat or abnormal vibration for ball bearing. One of the big application also is inspection, visual inspection. Image recognition has remained elusive.
Is trying, but so far does not work very well. And we believe that the combination of FPGA together with NurOwn can greatly improve things like obviously navigation for drones, also have the capability of making very large surface inspection, satellite imaging. And so in term of we need to extract some features from the images, and this is where the FPGA or flexible kind of logic is very well appropriate for this kind of process. Also, we need to be able to make real time relearning. And again, it can be reasoning where we can use FPGA for reasoning and telling the neuron to something.
Neuron So these are very simple application. Here, it's a very different application. It's cybersecurity. This application, the NeuroQube itself has been delivered to a defense contractor for securing uplink for the very large drones like Predator and detecting tokens. We can detect validate token reaching reaching the drones in about four hundred nanosecond.
And therefore, we don't need to reduce the speed of the communication as well as consumption is very small because we got about 2.6 terawatt operation equivalent per second with less than 12 watts. So there are a lot of application in also in cybersecurity in detecting behavior of connection for denial of service attack, various application also via application on text as well for semantic Latin semantic analytics, understanding what sentiment are in the text. So all these application has been prototyped already and used with the hardware. Another application here is what we call adaptive control. On the left, you can see a pendulum, inverted pendulum for adaptive control.
And on the right, you can see an actual application where the gentleman has been implanted with electrode and we plan to use the neuron to boom balance and generate the proper stimulus for him to be able to walk naturally. Right now, he can walk, but just one leg at a time. But the promise promise of this application is to have this gentleman, Mark, working normally. Thank you. And I'm going to turn it over to John Ho.
Hi, I'm James, and it is my great pleasure to be a part of this ecosystem. I personally thank Brian and other great people at QuickLogic to prepare this event. For the next fifteen minutes, I'm going to introduce about Nepes Corporation. Since many of the audiences may not be familiar with who we are and what we are doing. Then I will introduce what is NN500 and its application, the challenges and our expectation of this ecosystem.
Okay. Now the Nethys Corporation is South Korean public company doing advanced packaging foundry services. Technology services we provide are flip chip bumping, wafer level package, then our wafer level package and system in package or same modules. We have been doing this business for over twenty eight years and we have a top tier customers in consumer electronics, automotive, mobile and wearables around the world. So, in a nutshell, what we're very good at is making smallest package IC with more components in a very cost effective way.
So how smaller, can you make? I'll show you one example in the next slide. On your left hand side, orange color board called orange board, which is an Arduino clone. On your right hand side, the tiny chip a person holds is called a Dodduino. We put every components on the Orange Board and integrate it into the Dodduino package using Fan Out SiP technologies.
So they are 100% compatible to each other and even greater thing is there are still some empty rooms in Dartuino so that you can put more components later time. I guess, now you get the idea. So in the next slide, now, what is NM500? NM500 is a neuromorphic chip. General Vision and Nephew has been working closely for the last couple of years to design and manufacture it.
It has five seventy six identical cells called neuron. A neuron is consists of logic gates and memory and each neuron is parallelly connected to the other neurons. It is designed from the scratch for true parallel computing and mass scalability. Each NF-five 100 has five seventy six neurons now. What this number five seventy six means to you?
It means it can process and identify five seventy six different objects or patterns from the data. So they are already more than enough for IoT, wearable, hearable or even many digital applications. We started mass producing NF-five 100 from the last year and it has certified for the industrial use. So, to use the chip, we developed evaluation board called a NeuralShield, which follows Arduino or embed form factor. And in case you want to expand number of neurons, we also have a NeuralBrick.
You can inspect it on the on the NeuralShield. And for those who want to develop intelligent applications quickly, we have a Blended, NeuroStick. You can insert it into any standard USB port of your computer and instantly begin developing. Prodigi Board is for those who want to develop vision applications. Now, when you get the NF-five 100, it is like a baby.
It doesn't understand the world. So you need to train with the data. We provide genetic knowledge building software called a Knowledge Studio. It is a cross platform software, runs on Windows, Macintosh, and Linux. We use show and tell method.
Only thing you need is just point and click the data you want to train. Main purpose of this software is create, evaluate, and verify the knowledge for the NF 500. So in the next slide, suspect ship. NF-five 100 is a small. It has a dimension of 4.5 millimeter, runs in 36 megahertz with core runs in 1.46 milliwatts.
Average power consumption is 135 milliwatts in active mode and package was done with a sixty fourteen chip scale package and 500 micro thickness. Next page. Now the application field. NM500 can be applied to really a variety of different industries. You can use it with image recognition, sound signal recognition, video, data collection, text and packet recognition, and etcetera.
Still the biggest industry requirement comes from the vision recognition and that is what we are focused more on. We are working with many automotive companies, video surveillance companies, toys and education markets as well. But today, obviously, because we are in the semiconductor industry, I'd like to focus on semiconductor equipment market. From the last year, market research shows that this year semiconductor equipment market grows continuously and especially the visual inspection is grow up to $4,400,000,000 out of $50,000,000,000 market. As you know, manufacturing site always the last processing line is visual inspection.
So if we can horizontally expand within 500 visual inspection solutions, it's going to be a huge market. I'll show you a one live example. The ILB or Inner Lab Bonding Equipment is to attach ICs on a thin film type substrate. The upper left hand picture is actual ILB equipment installed in our factory. The upper right hand picture is we installed camera and the lighting to prevent the light noise and we use one NM500 for this project.
Well, it can detect correctly bonding chip in green, which has no problem. But any error such as wrong chip position, double bonding or, flying chip and etcetera can be detected in red. You can see it working in the lower side pictures. It has been running 20 fourseven and the result is absolutely phenomenal. The result record shows it has detected 100% errors so far.
Even greater thing is now equipment engineers learn how to train and use this solution at the field. So if there's any new devices coming, they know how to apply and deal with it. It is so called a field trainability, a very important NF 500 feature, which you can train the data at the field you are deploying and use it right away. And the next slide. After working with many customers, we found out that there are some challenges in AI adoption.
We all know that now the industry has transitioned from mass production to small quality batch production era. That means your production should have more flexibility than ever. Technology is evolving every day, so does number of products you are making. So here are some of the challenges I listen from our customers when they are adapting intelligence in their business. The first one is vulnerability of network.
If you are using deep learning, chances are you probably upload your data to the cloud for training. Some companies are flexible on that, but many are still hesitating to do that because the data may contain, very sensitive information. NF-five 100 can be configured to use with the cloud, but its nature, is local execution, so you can be free from security concern. Second is, data is keep changing. For example, in our factory, the device, the wafer that we're processing is changing almost every week.
This means training data set changes every week. How are you going to make knowledge or so to speak, training, out of them in time? When you are making, you are when you are making training for a new device and later old device come back again, what are you going to do with that? Training again. This is a big challenge.
With NF 500, adaptive field training will solve the problem. Third, there is a time when you encounter a problem. The problem that rarely happens. You can fix it easily if you know the problem beforehand, but when it happens without warning, it causes serious problem. Because it rarely happens, it is very hard to get abnormal condition data.
Without data, you cannot train AI. AI is all about data with algorithm. How to solve this problem? For example, a lot of machinery uses motors, and every motors before you break down, the vibration gets different. So with the NF 500, you only train the motors vibration with normal condition data and detect any different patterns for analyzing its status later.
Simple, but very effective solution. Hard to train problem is very well known problem for deep learning because neural network it creates are so complex when there's a problem with the results, it is so hard or almost impossible to trace back which data causes the problem. NM500 provides knowledge model. You can trace back and pinpoint exactly which data causes the problem. This auditability is important for the companies when you encounter problem with your intelligence system.
Last but not least is edge versus cloud. Using only cloud system also works well, but it has its drawbacks too. If you're running sensor network, you know how massive, network traffic occurs? If your sensor network can upload only the trained data you'd specify, overall network cost dramatically decreases. And in case of mission critical applications, you may want both edge and the cloud side, same intelligent capability in case of network connection loss.
All of these real world challenges are coming from very flexible environment And it can be solved with the M500 in a very cost efficient way. Now the next slide, what we expect from this, Qig AI ecosystem. Now I'm very happy to be a part of QuickAI ecosystem. I do believe this ecosystem will make powerful synergies. As you know, AI needs combination of different technologies, especially if you are in embedded edge AI field.
You need a semiconductor IC, you need hardware development kit, you need software development kit, you need data science platform, and of course, you need a light test bed environment for quick development as well. You cannot do it all alone. So I have this high expectations of this whole ecosystem. And especially the new Mercedes platform that QuickLogic has released today, I believe QuickLogic, hit the right spot with it. With this ultra low power and all its own capability with sound and sensor signal processing combined with an N500 will open up wide variety of applications.
On top of that, top notch data science team of SenseML will integrate their robust knowledge processing platform with the Mercy platform. The users can develop their own knowledge pack seamlessly working with the Mercy platform. It can be easily applied to like AI speakers, predictive maintenance, smart toys, sound surveillance, or even automobiles. So I cannot wait for combining Mercy platform with our vision solution. In the next slide, I'm going to show you the one good example.
We are currently developing two stage authentication for Android devices. It uses face and voice authentication and needs 100% accuracy. Face alone may not achieve the accuracy level, but voice combined, there are many papers out there already proved it is achievable. Applications like mobile, tablet, smart TV, signage, door locks are only a few customer requests for these applications already. So I'm very excited about the future of this ecosystem and I hope this will give audience some idea about where we're going together.
Thanks very much.
Thank you. And hello, my name is Chris Rogers. I am CEO and Founder of SensiML. I'm excited to be part of QuickLogic's AI Cognitive Sensing for Endpoints initiative and welcome the opportunity to provide an overview of Sensimol's product and fit within the QuickLogic ecosystem partner product plans for AI. Sensimol is transformative software toolkit enabling the rapid development and ongoing learning of smart sensor algorithms for endpoint devices.
QuickLogic's QuickAI HDK is a very compelling hardware platform that we're very excited about with multiple AI accelerator cores that provides a great opportunity for us to maximize power and performance optimization of sensor algorithms as generated by the SensiML toolkit. But before I get into the details of the toolkit itself, we'll take a step back look at the overall solution as a whole. Next slide. So if we look at a traditional IoT solution, we typically have analytics performed in three domains. I think we're all familiar with the cloud domain where deep learning takes place.
More recently, we've started to see the emergence of edge compute with things like FOG computing initiative that is taking a distributed approach to AI, recognizing some of the scaling issues of centralized approach. But the missing link in all of this has been the endpoint device. And while there are sophisticated and evolved tools for AI, for cloud that we're all familiar with, things like Google TensorFlow, Caffe, Hadoop, Microsoft Azure machine learning platform, Spark and others. And they're starting to be similar platforms at the edge. What's surprising is that the toolkits available for developing algorithms endpoints is largely the same as it was twenty five years ago when I was a practicing engineer developing embedded solutions myself.
And I find that very surprising. I think there's a great opportunity here that we can greatly improve the developer's process of creating scalable algorithms and a resource and time efficient manner and begin to approach the kind of sophistication and software tools that we see elsewhere in the IoT network. So a bit of background on SensiML. The SensiML Toolkit started life about three point five years ago. At the time, we were known as the Intel Curie Knowledge Builder.
The little picture you see at the bottom left of the screen is actually a picture of Intel's CEO, Brian Krzanich, presenting in Intel's big Intel Developer Conference back in 2016, the advent of this toolkit is a game changer for creating algorithms for sensing analytics at the edge for the Curie device at that time. In the interim, we have spun out the Curie Knowledge Builder as an independent software vendor. It is a wholly autonomous company that now provides hardware agnostic solution that supports over 30% of the IoT endpoint devices that are available out in the market today. There was not insignificant investment that had been made in the Intel days. I think the cumulative R and D that was spent over the three years amounted to $18,000,000 worth of development.
So we have now a very mature toolkit that is ready to be applied to current and next generation solutions. And notably, what we have with QuickAI XTK is a very capable device that we're very excited about as a next gen endpoint with a great deal of opportunities to address accelerated course that it makes available. Next slide. So when we talk about developing for endpoints, certainly in the market today, are a great number of devices that build themselves as smart endpoints or smart devices, right? Whether it's a consumer space or in the industrial or commercial space, we've got any number of devices that would add smart to the name, thermostats, smart predictive motion and maintenance, sensors for industrial.
We even have a smart egg crate or egg tray device out there for consumer. But what I think we find with these smart devices is they tend to fall into two camps. The first camp is actually truly smart devices that have been built to be intelligent and to provide a great deal of analytics at the edge itself. But at the cost of significant development team size and cost and time spent hand coding algorithms that work well for that device. And then you've got sort of the other bucket of devices, which aren't so much really smart as they are just connected devices, where they take raw sensor data and then transmit that somewhere else within the network like in the previous slide to provide analysis either at the edge, on a smartphone application or in the cloud.
And with all the caveats that come with that, that others on this call have already covered quite well. So as a device developer who's considering building a smart device faces the prospects of how to implement this, they're stuck between the need to invest significantly and with great risk because there's no assurance upfront that the algorithms that they would need to implement their application can fit a given piece of hardware using the tool sets that they exist today. The other is that they can suffer through some of the typical use case restrictions that come with cloud compute or computing on an edge device where you've got latency, security issues and the bandwidth limitations over the network that don't support mobile applications, don't support battery powered devices as well as a truly smart device. Next slide. So what SensiML aims to do is to bring the sophistication that you see with other AI solutions elsewhere in the network to the smart device and to do so in a way that doesn't require that the developer has to have extremely large team and a multidisciplinary team that has data scientists, DSP engineers, test and data collection technicians, firmware engineers and app developers, along with the domain experts for the given application, all figuring out how to collaborate, though they don't necessarily speak each other's language.
The analogy I like to use is that before there were applications like WordPress that brought a high level of automation to website authoring. To build a really compelling website, you had to hire experts who really understood their way around HTML, CSS and JavaScript. Nowadays, you can put a tool like WordPress in the hands of somebody who understands what they want to build from a content standpoint, but not necessarily be a compelling websites all on their own. That's what we aim to do with SensiML to be able to provide the ability for the long tail of IoT application developers without heavy expertise to be able to quickly build algorithms that can make sense of sensor data and transform that into meaningful events and do so without having to have the significant investment that would otherwise be needed. Next slide.
So what makes the sensor truly smart? This flow here kind of shows going from the physical property to be measured by sensor of some flavor, all the way to taking that to some meaningful event that's of interest for a given application. And the example we use here is for predictive maintenance. And I'm looking to find some discrete classes of machine anomalies, such as excessive vibration from a loose motor mount or a flange bearing failure or some kind of obstruction of a fan blade on a motor frame, right. So connected sensor in sort of a conventional sensor application would take the sensor itself, it would capture the data and sample it at a given frequency.
You might do some simple transforms and signal conditioning and maybe some simple compression in order to make it somewhat more efficient in its transfer of that data elsewhere in the network. But it certainly doesn't doesn't go to the extent of providing any meaningful insights. Smart sensor takes up many steps further and applies the expertise of a domain expert who understands various failure mechanisms within predictive maintenance and can label segments of data that can then be used by a tool like SensiML to go through a population of data that has been labeled and create algorithms that will classify accordingly. So that the device that's been now programmed with this algorithm can autonomously detect and flag such meaningful events as opposed to just send lots of redundant data over the network to be handled elsewhere. So how we do this?
It's a 40,000 foot level. This slide shows an example of the various components of the SensiML solution, starting from what we call SensiML Data Capture Lab is a PC and mobile application variance that we have available that allow a developer to collect in the field or import existing data such that they can easily label. This can be a fairly mundane and time intensive task for supervised machine learning is just the actual capture of the data and labeling of that. So we seek to make that as painless as possible by automating a lot of
the
process, by having tools within that particular application that help you with if you can label a couple of examples, it can infer by example what it believes to be other examples. And then that's a matter of just confirming whether that in fact is consistent with your understanding. At the point that you've got labeled data, then you pass it up to called a Sensible Analytics Engine, which is a cloud based tool that then traverses all of various event segmenter algorithms, performs the feature engineering, looking over the entire space of available feature extractors that are on hand. We have over 100 of these feature extractors going from very simple down sampling to much more sophisticated MFCC and FFP type feature extractors that really can take advantage of the FPGA that's available in QuickAI. When this analytics engine is given a set of constraints in terms of how accurate does my model need to be, how much memory do I have available.
And by the way, most of these parameters are understood by the fact that the target platform is the QuickAI AI. It knows what the parameters of that device are. So we can already have a fairly good understanding of how to optimize for that particular target hardware. So it becomes a matter of providing it with labeled train and test data as well as basic parameters for the expectations of the model. And then the tool comes back with what we call knowledge pack.
Knowledge pack is our term for either library level or binary code that's now firmware compatible with the end device, so that you can now quickly flash that device with the generated model and then go empirically test that in the field and confirm that it does what you think it does. To close the loop further, once that model has been generated, any subsequent events that match patterns that it is familiar with would just return back the event itself, thereby greatly reducing the bandwidth required over the network and giving you extremely low latency and the ability to control and or get feedback directly from the device. For things that it isn't aware of, let's say you have some new novel class that hasn't been identified in your development phase, you can optionally configure the knowledge pack such that it will report back either the raw data or some interim feature vectors, so that that can be used for ongoing that learning mechanism allows you to close the loop and then have sort of a continuous learning process from the device even after it's been deployed. So if you compare and contrast that to how things are done today, as I mentioned on this next slide, typically today you would have five different domains of expertise ranging from data scientists to DSP engineer, firmware developer, app coder and domain expert, all doing custom code work often within a statistical modeling tool that can come up with theoretical model that can work on a very fat client device.
Now you have the challenge of how do you optimize that to be power and performance friendly for an embedded platform. And that is often a matter of custom coding within a compiler environment and can be iterative and can be brought with risk as there's no deterministic way to know whether that model will in fact fit the hardware or not. That process, we've talked to many different developers can range from $500,000 or more and take six to nine months of development time alone. So that can frequently mean that the algorithm becomes the bottleneck or critical path in the overall deployment of a new device. Minimizing the risk, lowering the cost and making it scalable so that developers can come up with algorithms that can be quickly adapted and learning over time is what the goal of our tool is.
So if we can track that on the next slide, we take an approach where it's an automated learning using the analytics studio. It's a very minimum. You may need only an app coder and a domain expert if you don't have a lot of additional code to add and firmware above and beyond what comes out of the classifier itself. We've done applications where this has been done in as fast as four to six weeks. And I'll note that that four to six weeks can be a majority of the time spent with test technicians just doing data collection.
So whereas you spend a lot of time with data science, DSP engineers and high value resources in the case of generating algorithms in a more automated approach. This job can often be done with data collection test technicians, properly labeling datasets. On the next slide, just some examples of some applications we've done in the past. When this was the Intel Knowledge Builder Toolkit, we did a POC that's public with Honeywell for a next generation first responder. First responders have these devices called pass devices that are built into their rescue breathing apparatus.
And basically it looks to see if there's any motion and if it detects any lack of motion for thirty seconds, it sends off this 120 dB alarm to notify somebody nearby that you have a man down. It's not a terribly sophisticated device, but yet an opportunity to make something that can do certainly that use case plus a whole lot more. So Honeywell had asked us if we could create a smart wearable that could give field commander contextual awareness of what's going on within an emergency situation. He's got a number of first responders in a building responding, you may not necessarily know what's going on at any given time. They use a common radio channel to talk and it's a noisy environment and they're talking over each other.
So often it would be very difficult to just understand what's happening. In the case of this POC, we created two wearable devices that allowed them to do gesture recognition for quick status indication and another that was a body worn wearable that had a library of different activities of interest like climbing a ladder, running, laying prone, that it would detect and provide a dashboard for the incident commander to say, all the responders I have on-site, here's what's going on, right, and give them better awareness. We did another application for Intel in its own fabs from a manufacturing environment to best understand whether they had any imminent machine failures. So this was really a predictive maintenance application, rotating machinery was involved and we utilized the same classifier that we've been discussing in that context to see if we can't detect machine anomalies before they became issues that required a lines down situation within the fab. And then next slide, this is an architectural view showing how the solution integrates with the QuickLogic QuickAI HDK.
As I mentioned, we have a number of different applications within our tool chain, the end result being the creation of what we call this knowledge pack. And what I'm very excited about with this particular hardware platform is that there are a number of different accelerator opportunities here that couple quite nicely with the capabilities of the software toolkit. The first being event detection and feature generation, which can utilize the resources of the FPGA fabric as well as the flexible fusion engine with DSP functions that can allow us to do a very power efficient feature extraction on the front end of the tool chain. And finally, the classifier that we had run on the Curie, as Guy had mentioned, in those days, we had used a fairly modest 128 neurons that were available on that chip. Now we have over 1,000 neurons available and with hardware acceleration, then we can offload the classifier portion of the work to that NM500 silicon, so that we have full use of almost 10x the number of neurons we had from the Intel Curie days.
And so this could lead to some very significant applications and both in terms of power and battery life as well as the complexity of sensor algorithms that we can actually implement in this kind of a device. So we're very excited by that and look forward to explaining more. And I thank you very much.
Thank you, Chris, Malik, Guy and Zhong Ho. And thank you to the audience for your time today. We hope you found the insight from all these great companies both interesting and informative. As you can imagine, we are big believers in the promise of AI and cognitive computing at the endpoint and enthusiastic about the capability and domain expertise of General Vision, NEPIS and SensiML enabling it. We will now open the call for questions.
Okay. Our first question is, why did you choose to work with General Vision's AI hardware platform? What are the metrics for power performance, etcetera, that lead you to believe they are better? Well,
the thing we like about General Vision's platform is it's kind of oriented towards endpoints in the sense that there's a piece of hardware there that gives you good power performance as opposed to the deep learning, which is very power consuming, high compute, big kind of devices. So basically power performance, two aspects of that. And then the second point that I would make is that this AI is particularly well tuned for training in the field, which is unlike the classical TensorFlow kind of AI. And the applications we see are ones that need training in the field.
And I'll just add to the fact that they've already been used in the Intel processor, Intel Quark and other chips. I think there's a lot of AI innovations that are happening out there, none of which or some of which are commercialized and some aren't, but clearly General Vision has done a good job in getting into the market into real devices. And so it sort of de risk that for other people. So I think those are the primary reasons. And that was QuickLogic's perspective, but I think it'd be interesting to get James your perspective from Nepus on why you chose General Vision as well since you have that already deployed in your NM500.
Yes, because if you are comparing with the other deep learning solutions, they are using so much power in that field. But NM500 is already proven and when you are manufacturing it, it uses the most stable processing lines in the semiconductors. So it is very much a proven technologies. That is the first impression that we get. We decide to manufacture the NM500.
It was a great job.
Okay. The next question that we see here is did QuickLogic and partners come up with a solution with a specific customer application in mind? So I'll start with our perspective and then open that up to Guy, Chris or James for their perspective also. The way that we're approaching this is that the solution is the platform. You can see that with the HDK that it actually has a lot of different sensors on the HDK and connectors so that we can add additional sensors.
So we're looking at this more from a set of applications. I think everybody is familiar with TriClosic knows that we have our patented FFE, which is really good at interfacing with sensors and lightweight processing for motion and biometric sensors. We know the FPGA has a lot of vision use cases that we've done long ago in the past that we're probably going to start bringing back to bear on this market. And so this is a flexible platform just to designed to address those types of applications. And that's where we see working with these companies like Nepis, SensiML and General Vision for especially on the areas of vision and time domain series data.
So no to a specific customer, but yes to a set of customers and a set of applications that we think was going to be very important in this IoT space. So Guy, James or Chris, do you want to add anything to that?
This is Chris. I can say that, like you said, Brian, we didn't target application per se. We look at it as there continues to be sort of an advancement on sensor technology and price and performance of MEMS sensors particularly have led to a variety of sensor types that can provide very rich data that on the one hand is a great opportunity for much better contextual insight of what's happening, but on the other floods networks with lots of potential raw sensor data So we see there being opportunities to provide this kind of analytics across a long tail of IoT applications, whether it be time series data from accelerometer and gyro data for motion analytics or vibration sensing or pressure sensing or the vision applications that we talked about. So when we developed the toolkit with SensiML, our goal was to try to make a general purpose tool that could be readily adapted by developers to a variety of different use cases and streamline their process clearly.
Yes, this is James. And yes, I would like to add two more applications. The number one is the predictive maintenance for the semiconductor equipments or their factory airflow machines. Because NetPath is more focused on the vision areas, but actually for the predictive maintenance, they need a different kind of sensors. And those kind of areas, I think the QuickLogic and the SensiML has more knowledge and expertise in that area.
So if we can work together, then first, we can go into that market immediately. And the second one is the current applications that we are making. It's a two stage authentications. It's face recognition plus a voice recognition at the same time. The face recognition for those mobile or tablets, we can actually progressing a lot, but in the voice applications, well, that is not actually our expertise.
So in this ecosystem, so we can combine it all together and we can actually go into that applications as well.
Maybe this is Guy. Maybe I can add something because this technology has been, as I mentioned, has been started in 1993 with what was labeled Zisk at the time, zero construction set computer that I co invented as a partner with IBM France. And IBM developed back in 'ninety three, the Zisk 36 has only 36 neurons. So the beauty with the semiconductor company technologies right now is that we can go to 500 neurons with 110 nanometer technology, whereas the ZISC 36 was a one micrometer technology, was a big chip with 36 neurons. And we are just at the beginning of the road map of the obviously, putting more neurons.
And putting more neurons does not involve rethinking the architectures. And IBM, for example, in the 1990s, they had made a lot of applications. They made a product called Neuroscope that they use in their plants, all over the world, to make condition monitoring and predictive maintenance, something similar to what NEPES is doing now in inspection, but also ball bearing monitoring and things like that. So right now, with the cooperation with QuickLogic and NetEase and Sanciano, I think we have the whole ecosystem to deploy this solution on the very, very wide scale, especially with the fact that it's very easy to increase the number of neurons and also to package that in the like, obviously, a small setting similar to what the Curie was with Intel. So, I think there is it's just the beginning of the road for all of us.
Thanks, Guy, Chris and James. The next question, Guy, this probably is appropriate for you to take. It's the what resolution and frame rate of camera input can the device handle? I would say it's probably the technology handle first and then maybe James, you could discuss the device specifically with the NM500. So frame rate and resolution of the vision applications.
Well, it's a very interesting question actually. When I was in France and cooperating with the French military, one of the application I did is a detector beryllium shell tracking and detection. We were handling stereoscopic sensors. This was back in 1994 at 1,500 frames per second and for detecting two pixel singularities. So in term of and this was done with actually the Zisk78, which was the quarter micron technology.
So definitely, it's possible to go to a very sophisticated and very demanding application. So the chip itself or the neuron themselves, actually, the main feature is the fact that we can match one pattern versus any numbers. And any numbers so far, have been up to 1,000,000. What has been delivered, for example, to the defense contractor here in U. S.
Is one pattern versus 64,000 in 400 nanosecond once the pattern is entered. So definitely, it's possible to have to go to very, very high fan rate by putting more chips and so on. So the technology itself is not the limitation in terms of frame rate, it's really sensor depending.
Yes. I'll hand it to Guy. The Protege board that we are producing is actually comes with the cameras. First, the version of the camera that we're choosing was the five megapixel 30 frames per second. But actually the camera itself is not very important.
So you can actually go even higher resolution, higher the faster frame rates, but it's only the how you configure with the NM500.
Okay. Our next question is regarding the availability of all these elements of the solution. So I think I can let General Vision, SensiML and Nephew share their own availability. What I'll say is consistent with what we had in our press release today, which is for us, this is all based on our existing EOS S3 platform, which is available now. It's in mass production.
The QuickAI HDK that was shared in our slides is sampling now and it will be generally available by the end of this quarter, Q2. And then the SensiML Analytics Toolkit will be ported to the QuickAI HDK platform and that will be available at some point during Q3. So Guy, Chris or James, if you wanted to share any availability on the call, please go ahead now.
Yes, this is Chris. From SensiML standpoint, there is an existing release for the supported devices such as the Curie. So if customers want to familiarize themselves with the tool flow and how the basic operation works, that's available today. And then as Brian mentioned, in Q3, we'll have the successful port with the hardware optimizations available for the QuickAI.
And then 500 was already available from the last year.
In terms of general vision, as our name suggested, we are focusing on vision. So we have image knowledge builder, which has been dealing with multiple chip, including first is 78 and same one k. And now obviously, the M500 that James mentioned is in production. So therefore, all these product are stable as well as software development kits, which allow for easy interface of existing device to the NM500.
There's been a couple of questions submitted to us that are more investor related, which I'll try to cover on our earnings call next week on May 9. We'll wait for another few seconds here to see if there's any other questions coming in. Okay. It looks like no more questions are coming in. So I'd like to thank everybody for joining us today.
I'd like to thank all of the presenting companies for their participation. We're very excited about this ecosystem, as I said earlier. And stay tuned for progress from QuickLogic and press releases, our blog and our earnings calls. And we're looking forward to doing some great things together with this ecosystem. Thank you.
Thank you very much.
Ladies and gentlemen, thank you for participating in today's conference. This does conclude the program and you may all disconnect. Everyone have a great day.