Measuring the ROI of an AI Virtual Assistant for your organisation’s L&D
August 8, 2018
There’s an old adage that the only thing more expensive than training is not training. While we recognise the need for training, we find it difficult to measure its impact on efficiency and performance. Then there are the hard financial questions: what should we spend and what return will we see on our investment?
Ways of measuring ROI
There have been many attempts to quantify the impact of training on employees and validate the important role played by training. Perhaps the most comprehensive approach is the Kirkpatrick/Phillips evaluation model.
As the table below shows Kirkpatrick and Phillips identify five different levels of effectiveness ranging from ‘satisfaction’ with training (those ‘happy sheets’ handed in after a training session) to a profit-and-loss calculation of its financial benefit.
While the Kirkpatrick/Phillips model indicates how we can measure the success of training programmes, it’s not prescriptive. It doesn’t tell you what to do to improve training and reinforce learning. It simply focuses on where the gaps may be.
Moreover, the model’s static. It applies to each discrete piece of training. In the modern working environment, where instant access to information is becoming ever more critical, the notion of training as an episodic event no longer holds. The model shows us where to focus, but we need something more dynamic and responsive to help today’s L&D.
The limits of training
The Kirkpatrick/Phillips model presumes a traditional approach to training in organisations. Simply put, that means you have the training and assessment and then you’re deemed competent enough to perform the task. If the task or process changes, there’ll be more training in the new task or process – and so on.
There are problems with this approach. It presumes training and work are distinct, discrete areas of activity. It tends to involve training that’s away from the desk: taking employees out of work to attend classes or making them do e-learning courses at their desks. And, most important of all, it ignores the uncomfortable fact, first described by Ebbinghaus in the nineteenth century, that we forget much of what we’ve learned very shortly after we’ve learned it.
The modern learner
The other striking aspect ignored by traditional training is the needs and behaviour of the learner. Modern learners have access to vast sources of information immediately, round the clock. For anyone with a mobile or smart device the answer (or many of them) is just a couple of taps away. Ease of access and constant connectivity are changing learner behaviour.
The way people access information socially is becoming the way people expect to access information at work. If you want to know something today what do you do? Wait for L&D to run a training class or e-learning module or go to Google, YouTube, Pinterest, or wiki sites? And how can L&D compete with Google?
With such resources at their fingertips, modern learners feel in control of their learning. This is a powerful motivator. Handled correctly, it can make training more effective because learners invest in it and they identify their training needs and how best to meet them.
The advantages of AI Virtual Assistants
Training is often removed from the environment in which it’s to be applied. It’s confined to the classroom, trapped in the LMS, or accessible only when timetabled. Imagine, though, that training in your organisation was as readily available and as digestible as the information you access on Google, YouTube, or Wikipedia.
This is where AI Virtual Assistants can come in. We already use them in the home (Apple’s Siri or Amazon’s Alexa are common examples), but they’re increasingly being utilised in the workplace. Properly integrated and embedded in the workflow, VAs can make a real difference to the way training works and bring an obvious, quantifiable ROI for your investment in L&D.
A VA can be given access to your organisations’ training resources and information on work practices. It can be placed inside work management tools like Slack and Trello. Like the apps we use at home, VAs can provide instant responses offering a wealth of information but, critically, it’s not just any information: it’s targeted, contextualised and validated by L&D so that it’s the right information in the right place at the right time.
VAs in the workflow
Let’s look at a couple of examples of where VAs can extend training into the workspace.
Firstly, a Virtual Assistant can make a difference in the way you support and train new employees in the onboarding process. In the past, this has meant days, even weeks of training in the basics of their job and an introduction to HR procedures.
An AI Virtual Assistant can guide new employees through all the processes and information they need at their own pace when they need it. A VA offers instant responses to questions ranging from how to order office supplies, to holiday entitlements, to health and safety compliance and so on. Instead of being overwhelmed with too much information in one go, a new employee can access information just when needed.
Similarly, the VA can act as a personal assistant as you work. If you encounter a technical problem or need to know a procedure, you can just consult your Virtual Assistant. The VA can act as a personal organiser giving you reminders and access to resources for daily and weekly tasks. VAs can also make recommendations based on your requests (think of the similar services offered by Amazon or Netflix based on your browsing habits). If a piece of training is relevant, your VA can point you in that direction.
VAs help to manage learning, but also empower learners to take control of and responsibility for their own training. Having a VA always on means there’s no reason to forget. They break the costly cycle of learning and forgetting that undermines all formal training. By being there in the workflow acting as a mentor, VAs make learning a continuous process, rather than an episodic event. AI Virtual Assistants offer just-in-time learning at the point of need.
Replace trainers with VAs?
Despite the popular narrative of artificial intelligence replacing human beings in the workforce, VAs aren’t about to replace trainers. Rather they’re going to redefine the role of L&D and give it greater, rather than less prominence within an organisation.
For VAs to be effective they need to have information in a format that’s accessible and usable by them. This means L&D producing bite-size forms of learning and making content resources available for intelligent searching. They need to liberate the content from the classroom or the LMS.
You also need to train the VA, just as you would a real employee. Over time, however, a VA is capable of training itself by recognising patterns of enquiry and adapting to the requests it receives.
A new role for L&D
Integrating a Virtual Assistant in the workflow makes training more accessible, but it isn’t the complete answer. You need to move L&D into the workflow too and remove the barrier that often lies between training and the workplace. With L&D within the workflow trainers and learning designers can see more clearly where their intervention can make a difference.
The VA, in turn, isn’t just providing vital information to the employee, it’s also about delivering useful, quantifiable, hard data on learners’ requirements back to the learning designer. The data gives learning designers clearer insight into how training is applied. With this feedback, a learning designer can produce what learners are really asking for, instead of designing according to some notional idea of what learners ought to learn. With this approach, learners are at the centre and learning designers move closer to where actual learning takes place.
This shift towards training in the workflow allows a virtuous circle to be established whereby learners learn when they need to, and learning is reinforced. The circle is completed as the learning designers learn from the learner.
Instructional design models like ADDIE (Analysis, Design, Development, Implementation, and Evaluation) emphasise the need for constant evaluation, but that final element is often taken for granted or ignored as too hard to measure. With a VA in the workflow, you can directly measure the impact and effectiveness of L&D based on the evidence of employees’ on-the-job needs. Also, VAs offer the opportunity to re-use and re-purpose pre-existing content in a way that gives it more impact.
Both of these outcomes are direct evidence of ROI. Evaluation becomes more than responses recorded on ‘happy sheets’. Learners are listened to and L&D responds effectively.
So, if you have doubts about whether L&D is making the right impact in your organisation, add an AI Virtual Assistant to your team and reap the benefits.
Got a learning problem to solve?
Get in touch to discover how we can help
