Service Launch

Sep 25, 2018

Slides

Hello and welcome to today's webinar, Robotic Process Automation and Internal Audit. Are you ready? It's September 25, 2018 and we are very happy to have a very large audience with us today on today's webinar. We're going to go through a few administrative items before we get started with the content portion of the presentation. And I do want to let everybody know it is best to use Google Chrome or Firefox to view today's webinar. That's the best, those are the best internet browsers to, that are compatible with ON24, but we also will be showing a video during today's webinar to show you how a bot actually works. So it will be a bot in action. So if you are not logged into the webinar using on Google Chrome or Firefox, we do recommend that you do so at this point in the webinar so that you are able to view the slides moving and then also see the video in action. We are recording today's webcast, so please do refer your colleagues to this webinar if you do enjoy it. We will launch the video through media player. So one thing to remind them is it's best to watch the recorded webinar on Google Chrome or Firefox. We will be administering for CPE polling questions throughout today's webinar. So we will be talking to you for the next 60 minutes and we will offer 1.2 CPE credits. And in order to earn the CPE credits, you will need to answer at least 3 out of the 4 polling questions and we will deliver the CPE certificates within 60 days of the webinar date via email. They will not be available for download through that directly after the webinar. So we will remind you about that throughout the webinar. We also have set up a separate phone line for you to use if you are not able to hear the audio through your computer. So the number is on your screen right now. It is also in the PDF version of the slides that are available in your resources area. And if you need it later on in the webinar, please send us a question through the Q and A box and we will send it to you directly. So I'm going to go ahead and turn it over to our first presenter today, Sandra Stuttis Kennedy, who will introduce the rest of the speakers and then start off the webinar. Fantastic. Thanks, Lark, and I guess good day, everyone. This is Andrew Struthers Kendy speaking, with Protiviti and I lead our technology audit and advisory practice on a global basis. As you can see on screen, I'm joined today by both David Malcolm and Angela Policakos, who will be covering some of the topics of today's webinar, including a case study and a video that Lark mentioned, which I think will be a tremendous interest to all the attendees. David is the Managing Director within Accenture's internal audit organization and leads their global IT audit function, was heavily involved in the project that we'll be profiling. And then Angelo is a Director in Protiviti's Technology Audit and Advisory Practice and leads many of our internal audit related RPA activities and broader strategy. So I'm very glad to be joined by David and Angelo as they will no doubt bring significant experience and expertise and great insight to the discussion today. So I think it's safe to speak for everyone on our side and say we're really thrilled that there are so many folks interested and on this webinar to talk about an area that frankly is among the hottest and most hyped in the market at the moment, robotic process automation. As we were preparing for the webinar and trying to put together an hour's worth of content, I think we quickly realized there was much more to say about RPA than we could cover in an hour. So this webinar will likely serve as a starting point and perhaps the first in a series where we'll continue the discussion around the topic. So I would say just stay tuned for future webinars on the topic to extend the conversation. That said, during the next hour, I think we're going to be covering the topics you can see on screen. So provide an overview of RPA, what it is, what it isn't, discuss how internal audit can get started on their own RPA journey, discuss the value that can be delivered by talking through a case study. And of course, we'll kind of leave some time for Q and A. So let's kind of get into it here. We hosted a webinar last month on the results of our recently released SOX benchmarking survey and kind of from that a couple of things stood out that I just wanted to comment on. Really, the first is that sort of technology adoption, in particular for emerging technologies, remains relatively low, although organizations do appear to be committed to increased use. And the second really is that there was tremendous interest in RPA coming through the Q and A of that webinar. And candidly, many of the questions were asking around the fundamentals of RPA and we just weren't able to fully address them in the time we had in that webinar. So I want to start things off by providing a bit of background. So kind of in a nutshell, RPA is software that allows for the mimicking of human interactions with systems and data, And often it's programmed to follow the exact steps the user would follow. So logging into a system, navigating menu parts, entering data, running reports, sending emails, those sorts of things. So I would say think of RPA as a virtual workforce. Automation, of course, kind of not a new concept or nor a new capability, things like macros, workflow tools, screen scraping, testing tools used by technology departments, all examples of legitimate examples of automation, many have been around for quite some time. I think what makes RPA different is that these package solutions that are now available are much more accessible, more broadly capable and relatively lightweight to implement. So I guess what do I kind of mean by that? RPA solutions can be deployed without the need for the technical coding expertise. I kind of emphasize can there because I think in many situations you want that level of expertise when you're building automation solutions that will have an ongoing role in the operations of your business or in transaction processing, but it can be deployed in the right instances by individuals with a much lower level of technical capability. So that covers the accessibility piece. They're also more broadly capable than I think previous automation solutions. So operating with limited kind of a little restriction in terms of the systems they can interact with, the actions they can undertake. And then finally, they kind of generally operate as a layer sitting on top of existing systems. So in that sense, they do not require heavy technical integration. Let me also just define a couple of terms that are pretty commonly thrown around in the marketplace around RPA, and those are the concepts of attended versus unattended bots. So essentially think of an attended bot as those that are user initiated. They can reside on a desktop, they can reside in a central location and be initiated a bit like an advanced version of the macro that you have in the spreadsheet that you click play and it's again user initiated. And then unattended bots are those that are generally stored in a central location in some kind of orchestration or central solution and they are triggered by either a schedule or the occurrence of an event. So hopefully that's helped with some level set. I've already mentioned the results of our SOC survey indicating those kind of limited technology adoption. What we're seeing here on the screen is the results of our most recent internal audit capabilities and need survey, where we had a specific focus on a section that's focused specifically on analytics and technologies. And I think clear in that publication as well is that there's a strong desire from audit leaders to increase both the focus and competency around RPA, both the auditing of RPA solutions and the adoption of RPA within the term audit. So it is a clear voice here coming from a broad range of stakeholders across those 2 surveys and publications. That's a couple of 1,000, thousands of stakeholders provided their input. So there's a clear voice coming that there's a need to increase skills and capabilities as well as start or progress the exploration and adoption of how emerging technologies can drive increased efficiency, increased effectiveness across business operations, certainly including internal audit. So let's maybe spend a minute on, I guess, what we call here the intelligence continuum When it comes to technology, at present, RPA represents the low end of the spectrum. But I would say that bots are the workhorse solution and kind of do the tireless work that's repetitive, that's routine and they get through kind of transaction processing in a highly efficient, highly effective way. For the most part, they have to follow strictly program rules. And while there are some AI or intelligent components being actively explored by many of the vendors, There are very few true intelligent automation solutions that reside in the RPA realm currently available. So then you kind of go up a level, pretty big step. You've got cognitive automation. That takes us to the point where we can start to program machines to process data and react a bit more like a human. I'm trying to use the word think, but probably react. They have an increased ability for things like image and character recognition, the ability to work with unstructured data, the replication of some judgment based tasks and then some basic learning capabilities that allow them to drive some improvement in quality and speed of processing. And then sort of the top end of the spectrum here, you've got true AI, artificial intelligence, which allows the things like advanced natural language recognition and processing, dealing with large diverse sets of unstructured data, things like hypothesis based predictive analysis. So that's where it's using the information it's had in the past to predict what's going to come next. True self learning rules that kind of continuously rewritten to improve performance over time. So I think we're more I'll touch on this a bit more, but I think it's when automation core RPA starts to get combined with some of these more intelligent capabilities that will really see a significant steepening of the value curve. Again, this is an area that many vendors are actively investing in and exploring in the RPA space. So I just want to spend a couple of minutes talking about the benefits of automation to internal audit. And we've got a number of things on the slide here. So I'll just pick a couple of areas that I think are particularly worthy of emphasis. Top left, which I think sometimes gets overlooked is increased visibility and also credibility within the organization. Perhaps a read between the lines item here is the benefit that it has for the sense of well-being, the energy, the enthusiasm that members of internal audit teams derive from being asked to or volunteering to get involved in innovation or automation activities. We as an organization have seen just tremendous uptake and interest in things like innovation challenges, hackathons and other activities. And I think that we really can't overemphasize the benefit even if it doesn't manifest itself in measurable ROI that you'll have around just sort of well-being and sense of interest and enthusiasm that you can get from adopting these types of tools and initiatives. Over on the sort of the far right, I guess, 3 o'clock in the clock face, you've got making the human work less repetitive and analytical well-being. If you can take the robotic, the routine tasks out of the auditor and put them into the realm of the robot, allow the auditor to focus on the more intellectually challenging judgment based tasks. I think you not only get increased efficiency and reduced error rates, you also again enhance the sort of the sense of satisfaction that folks get from what they spend their time on. And then the last point I'll probably emphasize here is right at the bottom, sort of 6 o'clock there is increased breadth of coverage. If we really step back and think about what risk assurance we're providing when we test a sample of 5 or 10 or even 25 in a broader population that may include many thousands, tens of thousands, hundreds of thousands of transactions, it's very low. And that's why we're often trying to respond to issues that are identified and we get the question back, why didn't testing find it, what it wasn't in our sample. So what RPA allows us to do is do much more with the same effort or even less effort and either move towards true to statistical based sampling, confidence and error rates or even full population sampling. So there are a number of benefits here that I haven't touched upon, but I think those are 3 that I wanted to kind of particularly emphasize. So we're going to move to the polling questions and I'll just sort of remind everyone of what Locke said in the introduction. You are required to respond to these to get CPE credits. So please take some time to respond to this polling question. And I think while this is on screen and folks are responding, I'll just maybe take a minute to kind of talk about sort of what internal audit can and should be doing with respect to RPA. And there's 4 or 5 points that I want to make. I think the first is the business is just topic of the question we're asking if the business is looking to adopt RPA solutions and assisting the company with the identification of risks associated with that initiative, providing guidance around control design enhancements and testing approach, helping the company identify or recommend process or controls that might be well suited in turn law that is kind of in a unique position in many organizations of having the breadth of knowledge and the depth of understanding of the process and activities that they can really provide that short list of good candidates, assisting the company in evaluating ROI and efficiencies gained. And then last but certainly not least, I think considering how RPA can be used to drive enhanced delivery of the internal activities themselves. So let's move on from here. And I guess I'll just comment briefly and then I'll hand it on to Angelo who's going to take on the next section. So I think we're seeing 65% roughly of organizations that have either not started or don't have anything in production yet, but maybe doing some exploration. And this is again sort of what the organization is doing. I think that's roughly consistent with what we see. I think there's just again a lot of interest and hype. If we were to ask this question, I think in 3, 6, 12 months time, we're going to see a lot of change, lot of change in that point of view and just a very small percentage there that consider themselves to have well established programs with numerous spots in production. So, Angelo, I'm going to hand over to you to take us through the next section. Sounds good. Thanks, Andrew. So, we will be talking about how internal audit functions could leverage RPA to improve their effectiveness and efficiency. But first, the topic we all love, we got to think about RPA and the risks that it presents. So there's a host of risks that need to be considered as an organization embarks on their RPA journey. So just like with any digital transformation, there are many traditional risks that need to be considered, whether that be around the confidentiality, integrity or availability of any new IT platforms that are being introduced, as well as the program's governance. And of course, many of these traditional risks are often addressed with your traditional controls around having a documented strategy, having policies, well defined roles and responsibilities and of course our favorite IT general controls. But aside from the traditional risks, RPA does bring many new twists to risk that need to be considered. So for example, I mentioned IT general controls. We're not just thinking about applying your standard set of ITGCs around security change and operations around the whole RPA platform. And I guess when I say RPA platform, I'm referring to products like Automation Anywhere, UiPath, LUPRISM, etcetera. But in addition, there are specific IT controls that need to be considered at an individual bot level and this will require internal audit teams to dive much deeper into the design of the bots, how decisions are made, what triggers alerts, what gets logged, how credentials get managed, etcetera. And just taking security as an example, you're going to see there is multiple layers of IT controls that need to be considered. Think about credential management, permission management. You have now the credentials and the permission to the overall RPA platform and that's the platform that lets you control and manage the bots. You also have credentials and permissions to the virtual machines that are running the actual bots. And then finally, you have the credentials and permissions to the actual application that the bot may be interacting with. So aside from IT general controls and maybe more strategically speaking, you now also have risks around the benefits of the RPA program as a whole. A big question that's going to come up is, are we getting the benefits? And especially when you start considering the annual licensing costs. And typically, not necessarily for all RPA vendors, you have annual costs that apply for each bot. Now you also have the time to develop the bots, the time to maintain them and all the technical and people infrastructure that is needed to support the RPA environment. So now I'm going to dive a little bit deeper into some more specifics with respect to some of the risks. A big one is change management. So when you compare RPA to true IT integration, it is a relatively fragile technology. And what I mean by that is if a system that a bot interacts with were to change, let's say the system gets upgraded, well then any underlying bot that interacts with that system may not necessarily function anymore. So there needs to be good coordination between both IT and RPA teams to ensure that the impact to any BOSS is considered during the change management process. And then we'll cover this in a bit, but this is also why you'll find that one of the key criteria for selecting a good automation candidate is simply to avoid automating a process that is going to be going through a major change in the short term or maybe a process that relies on systems that frequently undergo program changes. I'll cover a couple of more risks on this slide. Another one that often is not thought about is the downstream impact to the IT environment. So compared to us, humans, bots can interact with applications significantly faster. And while maybe one bot is not going to place a significant load on a system, consider the downstream impact of 10 bots, all working on a queue, all attempting to grab transactions, simultaneously log into systems, extract data, process data, etcetera. And that's certainly not normal. It's something that probably would have never been anticipated during traditional performance testing on the system. And so while yes, RPA is technology agnostic. The beauty of it is it's going to be able to work on these legacy platforms. RPA teams will definitely want to be testing below the box place on the IT environment, maybe not if it's just a handful of bots, but especially if you're going to have dozens of bots operating a process simultaneously. And then I'll finally touch on one more risk, something that is not also considered and maybe it's not relevant yet, but think about the potential loss of human capital and knowledge of a process. So over time, what I imagine is going to happen is bots will become the new normal. And we may forget how these processes actually function before bots were put in place. So I guess the good news is any good bot development methodology does require fairly extensive documentation around how the process to be automated is actually executed manually. And we also need to make sure that only because we're automating something, this isn't like a set it and forget it type of thing where now the IT or the RPA team has to worry about the process. We still need to have true business process ownership of that process regardless if a bot performs it. So I guess the documentation that I was referring to not only becomes critical from a business continuity perspective, but it's also what I believe most external auditors will likely be interested in seeing, especially as organizations start thinking about automating various control activities or financial processes. And that is the detailed documentation around the automated process, the input, the output associated with the process and the rules behind any decision making. So let's now switch gears a little bit and probably talk a little bit about more of the fun side of RPA And that is how can we with an internal audit leverage this technology to bring us just more efficiencies. And probably what I'll say is, it's a lot more about effectiveness than it may necessarily be about efficiency. We could think about it from a few different lenses. We could partner with the business to help them automate the actual control activity. We could also look at it maybe from a more traditional lens of can we automate the underlying testing. And the case that we're going to be covering later is going to walk through a case just like that where we partnered with the internal auto organization of Accenture to help automate the testing, and we'll go through that here in a bit. But then in addition, it's not just strictly controls, it's strictly testing. There's a whole host of administrative activities that we as internal audit often perform. So that could be around managing artifact requests, managing escalations. Oftentimes, we use these GRC systems and we have to upload results to them. We have to download results from them. So it could be a whole host of internal audit administrative activities that tend to be those things that are high volume and probably better candidates for our PA. So we'll go through just a high level methodology of how we can start on this journey. And you could look at this from the lenses I previously mentioned. It could be from the perspective of automating controls, automating the testing or simply automating administrative activities. So the best way, I think we have found to determine where our PA can deliver the best value, the best ROI is to really take a structured approach to identify, evaluate, categorize and prioritize automation candidates. So for example, suitable tasks for automation are those that are going to be routine, they're manually intensive, they're prone to human error and then they're rules driven. So they're not overly subjective. You could almost process flow things out. And oftentimes, it's very much a if then if a certain condition, then do something if another condition, do something else. So it's really those if then type statements. And most importantly, and this is typically what we found to be the challenge, the supporting data, it's got to be readily available, it's got to be electronic and ideally readable. So it's Excel, it's Word, it's e mail, it's readable PDFs, etcetera. And it's not to say that if it's semi structured, you can't it's not a good RPA Automation candidate. But if you're starting off on this journey, you definitely want to make sure you're meeting kind of the typical criteria of an RPA candidate. So this doesn't have to be a drawn out process. It's as simple as getting the internal audit group together, brainstorming on what routine administrative activities or stakeholder facing activities meet the traditional RPA criteria. And then using RPA to automate the control activity itself or perhaps the actual testing, those are typically the obvious areas to explore. But then when you think about it, those aren't necessarily high volume. If you're going to automate the testing, it's got to be for other reasons beyond doing it in less time. And what we'll find is usually the reasons are getting more coverage, getting deeper insights. We find that simply having an automation opportunity brainstorming session, that's often what yields kind of your good first initial list. And then really the next step is you got to look at this list a little bit more deeply. This could be accomplished by going through every single item on that list and then think about it in terms of technical feasibility and then the expected value that automation can deliver. So value with a lot of the RPA vendors are focused on giving hours back to the business. But from an internal audit perspective, it may be less tangible metrics, right? So it could be increased insight, it could be improved breadth and depth of coverage or an ability just to focus on more value add activities. And I know we're finding this often too, a lot with our newer consultants, they don't want to be doing the repetitive tasks that could be very mentally taxing. So I guess once the candidates have been identified, they've been outlining which controls or activities will be automated first, outlining which controls or activities will be automated first and in what order. And I guess one word of caution, it's rare that you're going to be able to automate something as is. There's almost always going to be some tweaking, some streamlining that's going to be needed prior to that automation. So one golden rule I've heard and I like is don't automate a broken process, don't automate something that's poorly designed or anything that's continuously changing. So I think now we'll probably just shift gears to another polling question. And there's a little bit of a twist to this question. What progress has your organization's internal audit function made with RPA? So just notice here we're talking more specifically about the internal audit function or if your company doesn't have an internal audit function maybe think about it from a controls or compliance lens. Angela, great. Thanks for that. I'm just going to jump in while people respond or our audience response to the questions and kind of maybe address some of the long list of stuff we've had come through the Q and A. So, there is a couple of questions here that but the way that we're using the term is, but the way that we're using the term is basically to define an automation solution for a specific activity or series of activities I. E. A process. The technical definition of bot sort of varies slightly by vendor, but that's more about how they sort of deal with licensing and processing cycles than is around the automation of specific activities or process. And then there was a related question about is a bot different from a model or is it a type of model? I will say that sort of the thing that generally accepted definition of model is that it's inherently inaccurate to some degree because it's trying to model an outcome based on inputs and other assumptions, whereas a bot for the most part is strictly rules based. So what you put in, it proceeds, it puts through the rules engine and again it comes up with the results. So in that respect, I think they are different. And then one last question that I'll address now, we can save some more for the end, was kind of a couple of questions in around increased coverage during full population testing and basically is that dangerous for us to do that because that might reveal things that we've been able to avoid through limited sample testing. Well, I'll probably answer a question with a question, which is ask your executive management and your audit committee whether they would like to know what's out there and whether they want true risk assurance or whether they want to maybe have a positive security that some of our current testing methods produce. So I think it's a legitimate question. If I was sitting in that seat, I would want to know. So great. I think we can probably move to the next slide. That's Angelo. Maybe let me comment here. And I think if we remember back, I think the assessment of the business on a not started response was around 30 something percent. So 15% to 20% difference between where the business is and where order is. And I would offer that as we're about to get into, there's no real reason that to be the case. So I think this offers an opportunity or demonstrates that there is an opportunity for audit to take its own steps to evaluate, pilot, adopt and then operate ops for its own purposes, many use cases, which Angela has kind of covered in the preceding slides. All right. So and now for my favorite part of today's webinar and hopefully yours too, we'll talk about how we partnered with Accenture's internal audit department and leveraged our PA to improve the effectiveness of controls testing. And here I'm joined by David Malcolm, who leads the IT Audit Services globally for Accenture. So I'll let Dave provide some initial background on the objectives and then I'll be diving into details around the project including showing a video for 1 of the bots that we jointly created together. So Dave? Thanks, Angelo. So just a little bit about our organization. So Accenture, we work with clients in over 120 countries and 40 different industries to help them become higher performing organizations. And a lot of the work we do is helping them leverage new technologies, including digital, mobility and cloud to improve their operations. And part of the work that we deliver for our clients now and also helping our clients on their own roadmaps to implementing technologies including RPA. In addition to the work that we're doing with our clients internally at Accenture, we're also looking for opportunities to leverage RPA to operate more efficiently and effectively across our internal business. So given the increasing use of and importance of RPA in Accenture's operations, our IT internal audit plan this past year included a review where we assess the controls and governance around our organization's development of RPA. At the conclusion of this audit, this had us thinking about how we, an internal audit, could leverage an RPA to help ourselves become more efficient and effective in our own operations. So to this point, we entered a partnership with Protiviti, who we worked with for a number of years, to identify areas of our audit plan where we potentially could leverage RPA in place of the manual testing that we do today. For a lot of the reasons that Angelo previously covered, we targeted our IT SOX general controls that we test annually on behalf of Accenture to support our 404 attestation. The reasons we tested these controls or targeted these controls for our proof of concept are these are very mature controls that we've been testing for a number of years. They're very stable, not many changes to them. They're very repetitive. We do the same testing year over year. The evidence has remained the same year over year and what we review as part of our testing efforts. So that makes them really good candidates to target for automation. Additionally, like Angela mentioned, this is testing that to our team really isn't the most glamorous. So it's not doing testing of new technologies. It's not emerging technologies. It's pretty basic general controls testing that we've been doing for the last 15 years. So to the extent that we're able to remove that from our employees' audit plans and let them focus on more interesting higher value areas, it helps improve the engagement of your staff as well. So Protiviti has helped us do our testing of our IT controls for at least the last 10 years. So they have a very intimate knowledge of our environment and the systems and the controls. So it was natural to partner with them to bring their deep understanding of our environment as well as to RPA with our team. So what I looked at was if we could ultimately automate our SOX controls that we test each year, this would allow me to replace those hours that we spend on that testing with at least another 3 audits of higher value, higher risk areas to Accenture. Additionally, our control owners, so the business that has to pull all the evidence that we review to do our testing, they also would free up the time because we would deploy automation capabilities that would pull this evidence for us. And finally, we could also improve our coverage by expanding the samples that we select if we chose to without any increased time to do the testing as the bot could increase the samples really at the same do the testing at the same amount of time as it would for the regular sample sizes that we originally would select and choose. So for that reason, we worked with Protiviti. We engagement about 6 to 8 months ago and Angelo will get into some more of the details on what we were able to deliver. Perfect. Thank you, Dave. So this 4 step approach aligns very nicely with really what we what I mentioned earlier. We knew we wanted to align on the IT general controls because we have been performing the testing for several years, we had pretty good familiarity not just with how the controls are executed, but more importantly, the underlying evidence in place to support each control. So we took an approach of assessing the controls, categorizing them. And once we aligned on what seemed to be the best automation candidates, we performed fairly deep dive walk throughs with those control owners and eventually worked to develop the bots. So what I'll be doing now is walking through that process we followed with slightly with a little bit more granularity. So in terms of assessing the controls and you kind of have a few nice visuals here. Remember the RPA assessment framework or the automation framework. So we're not just thinking about the control frequency. We're taking into consideration, is this a control test that takes us a considerable amount of time to test? Is this a higher risk control that could benefit from some form of more periodic and continuous auditing or monitoring? When you start thinking about the artifacts that we request of the IT organization, the more painful requests, the ones that take them quite a number of hours to actually pull down the documentation. So we're taking into consideration the candidacy of those control tests that we would select to automate. And then what you see here on the right hand side is we have all the criteria and we're really going through the library of controls and evaluating which ones lend themselves best to automation. Once we have performed that evaluation, we are in a position to really plot them out, both from a value perspective as well as an RPA complexity standpoint. Our controls that rely on evidence that is coming from standardized systems, so ticketing systems, systems that have more structured evidence, readable evidence, those end up being lower on the RPA complexity curve. Whereas higher risk controls, controls that occur more frequently, controls where if there was a breakdown, we would be concerned. From a value perspective, those are the types of controls we care more to automate. It wouldn't make sense to necessarily automate a lower risk control. We rather automate a higher risk control because in essence, we're now building a form of continuous auditing and monitoring. So we ultimately aligned on 3 controls. And for those three controls, we followed just like you have a a system development life cycle methodology for new system implementations, we followed a bot development life cycle methodology. And this is what you should see if you were auditing the RPA program. Process definition, so we documented at a fairly granular level of detail exactly how the artifacts get generated and how the testing is performed. Oftentimes, we recorded these sessions because we don't want to miss any details. We need to know exactly the access needed to generate the artifacts, whether there was any variation in the process, etcetera. So once that was done, we then worked with IT to gain all the requisite access. In this case, all the access we needed was read only access. So we weren't as concerned necessarily from a security standpoint. Also, we weren't as concerned because we knew that we would be running these bots in an attended manner. And in other words, the auditor would be able to trigger the bot and have full control of the bot because given the nature of us automating the testing, I think we still want to have some manual QA of those results. So after documenting the process and the various requirements, we work to develop the design of the bot. That took into consideration what exactly will get logged, what exceptions will get triggered and alerted, etcetera. We did your traditional testing of the bot to make sure that what will the bot do if it runs into a piece of unstructured evidence. In our case, we just have the bot skip over that and flag that for manual review. But what happens if the credentials that the bot is passing into the system have expired? How should the bot handle that? So testing was certainly required. And I guess now we will show the video that was a video that shows 1 of the bots that was made as part of this project. So, first, we're going to have just a quick overview, and this is somewhat of a recap of what we did. What you're going to be seeing here is one of the bots, it's focused on a change management process. We talked a little bit about the approach that we took in terms of assessing the controls, categorizing the controls, performing the walk throughs, etcetera. The bot itself is really running on its own. But however, at points during the video, we've sped up the video just because no one likes to see an auditor in action and really the bot could only run as fast as the system is able to handle. So this specific video that you're going to see relates to the change management process. Anytime there's a change, it needs to be documented, it needs to be approved, there needs to be evidence of testing before the change gets migrated into production. So the bot follows a path of identifying the population, extracting the artifacts, performing the testing, but then the 4th step, which is something we just assume that there's always going to be that manual QA and review. So this is just now a snapshot of what the bot development looks like in UiPath and now the bot is running. So it's on a SharePoint site. It's actually running a query to extract the population of changes. It's saving that query because we know all the external auditors love to see evidence of the query. Now what it's going to do is it's going to log in to an SAP ticketing system where it could go through each and every change ticket that was referenced in the population and start pulling down all of that all of the documentation. So it's pulling down approvals, it's pulling down evidence of testing, it's looking for the existence of information and it's also renaming all of this information and organizing it all into folders. If it runs into something that doesn't look standard, it's just going to skip over that and flag that later for manual review. The bot is actually going into e mails and grabbing the name of the approver, the title of the approver and making sure that's appropriate. Now it's starting to log all of the information that it captured through those attributes. So general information about the change and then eventually just some the typical testing attributes. Was the change approved? Was there evidence of testing? Was there anything unusual found or at any point did the bot run into some non standard input? So this in case populated the lead sheet, but it also organized all of the work papers that would normally be used in support of the testing that was completed. We configured the bot so that it could look at a sample, so it could look at a sample of 25, but we also added a configuration that doesn't necessarily require changing the bot code. A flag can be changed just so that the bot can look at the full population. But clearly, the bot just runs. The bot runs behind the scenes. So before where you not only had time from the auditor, you also had the time it took IT to generate this information. Now you, in essence, have a bot that addresses all of the artifact gathering and pulling everything down, etcetera. So it allows them to focus more on their business as usual activities. So to summarize, we have value delivered not just from an accuracy standpoint. What I really think when you think about how we would leverage Deepbox, it's reduced the whole cycle time for automation. We don't have to have requests, delays, sample requests, perhaps more delays, and then do the testing, which takes a few weeks and then have the closed meeting. At the end of the day, we go from a 2 month audit cycle to a more instantaneous audit cycle. I really think the coverage and the insights, that's really why you want to do this. It's great that you could save some time to replace the auditor and assuming you do testing a couple of times a year, it's true that the bot is can do that a lot more efficiently, but you also got to consider the time it took us to develop the bot and to QA the bot. So the reasons got to be have to be greater than just looking for time savings. So that really summarizes what I would say some of the key points that we work to deliver as part of this engagement. I just want to pass it on back to Dave. Maybe he could talk a little bit about where we go next from this. Thanks, Angelo. So as we've been going through the results of the proof of concept, both with our internal stakeholders, my leadership and internal audit as well as our external stakeholders, so our IT organization as well as our internal controls function, the results, I think, have been very well received. And it's something we definitely want to move forward and see value in moving forward and expanding into other areas of our operations. So today, we're working with our IT organization. There were some other controls that we looked at and evaluated to potentially automate, but they weren't the best candidates. And there could be either a little tweak to the way the control runs or the way that evidence is captured or retained that would make it a better candidate for automation. So we're working with our IT group to see if there's changes they can make to reengineer their processes and controls that would then help us move in RPA to do the testing. Because really what we saw and when we presented the benefits and results, there was actually more efficiency gained on the side of the time it takes for our IT group to pull the evidence, respond to our request than it was for us to actually do the testing. So there was value for them in being invested in the process as well because it frees up time on their end too that they were spending supporting the testing process in the past. We're also working with other groups within Accenture. So on our finance and operations side for some of the business process controls and helping them identify where there are opportunities to leverage RPA to automate some of the testing and evidence gathering on controls on the business process side. And then finally, across internal audit, we're going to take a broader initiative to review all of our audit procedures that we execute on our standard audits on our audit plan as well as some of our administrative procedures. So a lot of the reporting and scorecarding that we do to communicate with our stakeholders to determine where we have processes that would be good candidates to automate. So we think that we think across our audit plans and some of the steps that we do on our standard audit, they align to the controls that we reviewed as part of our proof of concept. So they should also be good candidates to implement RPA capabilities there as well. And then from an administration standpoint, a lot of the reporting that we do today is highly manual. So we're looking for opportunities where we can replace those manual steps where we're getting data from multiple systems to create single reports or single scorecards and use bots to automate the process to generate and communicate those out to our stakeholders. So before we get too far ahead of ourselves, we want to go through all this evaluation, but at the same time, we want to make sure that we retain the same governance and control that we set up for the proof of concept. So Angelo mentioned all the documentation that we retained that showed how we built out the capabilities, tested the capabilities. So if anyone ever came back to us and had questions about how we ran our bot, that documentation is still available. And we have a formal methodology now that we follow for any future development and testing that we do and make sure we do that in a centralized manner here in internal audit. So it might be something setting up that something that resembles like a center of excellence here, where we have one central team responsible for the creation and maintenance and management of our automation capabilities that we deploy within our department. So we were extremely happy with the results of the proof of concept. A year ago, have you asked me this is something that I thought would add value to internal audit or something that internal audit was ready for? We're usually not known for being on the cutting edge of embracing new technologies in audit. But I think this was a good opportunity or a good example of how audit can use some of the same technologies the rest of our business is using to become more efficient and more effective. So with that, thank you very much, David. It was probably by far one of the most exciting projects I've worked on as an internal auditor. It's rare we're able to work on something that we just feel like we're truly delivering not just time savings, but added effectiveness. I think using technology that's pretty exciting and I'm hopeful that it's here to stay for the near term. All right. So first of all, David, Angelo, thanks so much for that detailed commentary and insight. I think really just fantastic. So we have 2 more following questions before we get into kind of wrap up in Q and A. One that hopefully folks are responding to now if your organization has deployed bots, what role does internal audit play today? So if you just kind of take a few seconds to respond to that, I'll maybe try and address a couple of the questions in the queue. So there's a theme of questions around the bots capabilities to read unstructured data or even scrape data from documents like PDFs, Word, etcetera. So I think that truly digitized documents, digitized PDFs, how highly effective at doing that. PDFs that are sort of scanned images of hard copy with handwriting signatures, then it's largely dependent on the capabilities of optical character recognition software that is a third party software or somewhat integrated into the RPA solution. So, maybe that address the part of the question that was asked around, I guess, data scraping around the ability to deal with unstructured data there, it can do it. You need to kind of program or develop rules to guide the bot where to look, what to look for, what keywords are phrases to search for and pick up. So it does start to run into some limitations when you start to talk about large volumes of distributed and unstructured data. So I think we're probably ready to review the results of the polling question. So what basically what role has audit played? I think probably we've got 40 odd percent that have done something or plan to do something. Probably the surprise to me there is 30% that say no current plans. Perhaps if we ask that question to you in a month's time or after you've got through your planning and risk assessment, you may have a different point of view as you maybe understand the RPA space and the associated risks with a bit more clarity. So one more following question. If your organization has made progress with the use of RPA, which are the following areas well, which of the following has been your principal area of focus? So select the one that has been the principal focus if you have made progress with RPA. And while that's happening, I will take a look at the queue and see if there are other questions here to ask. So I think there's just a couple of questions around the level of skill or capability needed to develop these solutions. So I think there's a continuum that there's a continuum. I think if you're looking at productivity solutions that aren't going to have a role in transaction processing and the ongoing operations of your environment, then I think there's an opportunity for people without a technical background to certainly learn and develop bots in a pretty safe environment. If you're talking about things that are going to be kind of a core part of the way that the business runs, then I think you're going to you need to and want to look at professional grade sort of development capabilities and practices to build and deploy those types of bots. But it's absolutely possible for people without technical background to go through training, learn and become capable in developing kind of simple to modestly complex RPA solutions. Yes. And I'll just add, Andrew, like what's great is in the RPA marketplace, a lot of the major RPA players, they're actually making their academies or universities available. You could download their tool. You could download trial versions. They have pretty well defined curriculum because they recognize people these days, they want to learn through YouTube videos. So I know one of them specifically has a fairly extensive RPA bot development training course that you could take that really walks you through. I think it's about 40 to 60 hours of information, where you go through guided exercises, you watch some videos, you read some information, but it really positions you on how to develop a bot. And I think our experience has been, yes, if you're non technical, you could definitely learn how to build a bot, one that may not have too many twists and turns. But as you're looking to do this at an enterprise level and have some form of consistency with how you're developing a bot, definitely having some of the background in computer science and prior programming languages to do more advanced things or if you run into any bumps, not having some of that foundational background will challenge you. So I would say, if you're doing this in an experimental like way, by all means, if you want to make a career out of this, you certainly don't need to be non technical. We'll probably just need to apply yourself a lot harder. But I think most organizations are finding that typically the folks with more stronger technical aptitude, maybe a computer science background, computer engineer, they're the ones that lend themselves best to the actual bot development. At the same time, you need business analysts. And the traditional internal audit capability that around documenting process, documenting controls, that is key. The business analyst is going to document not only the to be process, but also understand the rules associated with the process and then would work with the developer to actually perform the automation. And I guess that's just a couple of the roles. Organizations, I need to take this seriously. They need to have governance around the prioritization of bots, project management. You probably want some overarching solution architect that's looking at how the bots are being developed and making sure that we're developing them in an object oriented fashion so that we can leverage components and not have to update 10 bots if there was a simple change to a system. Instead, we can just update a small component. Yes. Angelo, thanks very much for kind of the detailed elaboration there. We'll move on and just review the results and then I think we're going to wrap up. We've addressed a number of questions already. There's a long list. So we'll probably have to forego the Q and A at the end. But just a few things as we do wrap up here. So the question around if RPA has been deployed in the business, and I think there's roughly 30%, 40% answer that question. And what role what role is audit kind of played and we're looking at control testing being that the one of the primary areas, the catch all other kind of bucket obviously popular, but we don't have a lot of detail there. And then the review of kind of administrative and aspect gathering exercises tend to be manual and heavily labor intensive, I think again, good candidates. So let's kind of take some time here to wrap up and I think just a few takeaways from what we've discussed and obviously they're on the screen, but I think take steps to increase the knowledge and develop capabilities within the audit team so that the team can both be effective in delivering on all this assurance mandate as the business exploring and adopting these solutions, but also assist in determining ways in which RPA could be used to support internal activities. I think from there, look for opportunities to deploy RPA to drive efficiency and effectiveness in internalized own activities. And Angelo covered a number of those earlier on the use case slide and then obviously the case study. And then finally, I think keep abreast of other emerging technologies and ever evolving technology landscape. I mean, not just from a risk perspective, but also to identify opportunities for adoption. There's so much more out there that internal needs to understand. You've got the process mining, you've got artificial artificial intelligence, machine learning, natural language processing, augmented and virtual reality blockchain. I think that things, it's sort of alphabet soup and the list goes on. But making sure that order is remaining current and capable around those things, I think, is very important. And then really just as we wrap up here, I wanted to cover kind of a point of view that we've developed around the future of audit or kind of what we're calling next generation audit and goes broader than technology enablement, which you can see on the kind of the left hand side. I think with the rapid pace of innovation in 10, what it really does face a growing challenge of adapting to change while continuing to deliver on its core mission. Many we're seeing many leading audit functions pursue transformation opportunities kind of with the objective of establishing this the version of their vision of next gen. And again, enabling technology is really just one part. So along with the evolution of other foundational elements around covenants and methodology, which address things like talent and resource alignment and utilization, risk assessment, execution reporting. We're really seeing all the organizations take steps to transform themselves and stay current in an increasingly digital and innovative world. So I'd say on this point, just stay tuned for more from Protiviti on this and our vision of next gen order, including a webinar. And then just again, as we start to wrap up here, as I mentioned at the top of the webinar, this is expected to be planned to be the 1st in a series. We'll be back with more on this topic. We'll frame the discussions that we have in future webinars based on the large number of great questions that we received during today's discussion. So So with that, I think that's really all we have time for. David, Angelo, thanks again for the really high quality insights that you've shared over the last hour. As always, thank you for keeping us on track. To all of the attendees, thanks for all the interest in the topic. I think certainly feel free to reach out to us with any follow-up questions or comments and we'd be glad to kind of speak with you. So really that's it today. Thanks for your time. Good luck with everything. Hopefully the information we've shared will help you and your companies kind of make progress along your automation and transformation journeys, find the right balance between speed and the control and of course face the future with confidence.