How impactful is your distance training program?

Since Covid, more and more organisations are rolling out mobile training programs, sometimes replacing face-to-face training sessions. While these programs, when based on lightweight and inclusive formats, help firms improve training continuity at scale — they are left with a big challenge: how to measure their effective impact.

What is Impact?

“Impact” can be many things, but it is always what we pay training for. This is why impact definition and measurement is so important for distance training programs: it could an improvement in beneficiaries’ financial performance, a boost in staff’s productivity, an advancement in service delivery, a better product, or simply a higher satisfaction rate from beneficiaries, partners, or funders. Bottom line here is that impact is (1) a positive outcome of our action (2) that needs to be measured and proven.

Why set up Impact Measurement when rolling out distance training programs?

Mobile training programs convey educational materials to beneficiaries, with the purpose of equipping them with activable skills and knowledge. This means that implementation of a training program is generally not enough of proof of its effectiveness, even with traction. It must also show specific factors of success that can be retention, behavioural change, income generation, or many more — and success should be a combination of several factors, broken down into measurable KPIs, and defined according to your objectives. But what do measuring impact this way achieve?

1. Measurement content quality & assessing delivery

Training materials and delivery are the two pillars of a successful training program. They can both make or break the outcomes of your training program, and for this reason need constant attention, and often serious iterations to make materials format and delivery mode frictionless.

Impact measurement is particularly critical for NGOs and firms in the Health sector

2. Cost benefit analysis & ROI

Return on Investment — often a literal dollar figure attached to each program— is an increasingly popular and strategic metric amongst nonprofits, often under scrutiny from donors, governments or partners. In this context, collecting and delivering meaningful training impact data is essential, as it is the primary material from which reporting and advocacy campaigns can be generated.

3. Reporting to partners, sponsors & stakeholders

Advocacy is a continuous effort for an organisation to retain its partners support and engagement into its actions. It is essential when ROI and long term impact can take years or decades to measure while budgets are decided every year.

Mobile technologies offer unprecedented — and mostly untapped — tracking and reporting abilities

What tools can be used to assess the impact of training programs?

Mobile technologies offer unprecedented — and mostly untapped — tracking and reporting abilities, generally highly cost-efficient, and easily scalable. Here are a few generic examples of tools you can leverage to track and measure relevant KPIs:

Mobile-based Surveys

Surveys are very effective tools for evaluating mobile training programs. They provide a means to collate both quantitative and qualitative data, at scale, with A/B testing and randomisation options. Beneficiaries can be asked questions in a variety of ways such as multiple-choice or numerical evaluation questions that can be compiled and processed automatically, or open-ended and ranking questions that give an opportunity to collect first-hand insights, at scale and seamlessly: firms can roll out these surveys at a much higher frequency than if they have to send field surveyors.

Trainees’ Usage Data

Usage data is an extraordinary and still widely untapped statistical tool to ascertain course interaction and user performance. With mobiles, firms can measure with unprecedented precision which percentage of trainees actually utilised the course content, what topics they struggled with, how much time they spent in average or individually on each page or exercice, at which time they are the most activable and efficient; and breakdown this data per age group, gender, region, language, device used, and so many more. This tracking can even be pursued for trainees that are offline, thanks to asynchronous analytics technologies.

Evaluation (Quizzes)

Assessments, quizzes and exercices are excellent tools to fulfil two goals: they provide an opportunity to both the trainee and the trainer to assess the trainee’s performance and progression ; but they also generate interaction and calls to action highly beneficial to trainees attention and motivation. Mobile training programs should have assessments and quizzes regularly spread throughout their materials, and because they are automated, adding more of them does not surcharge the trainers.

Eye-Test

After all, this is the ultimate test: are trainees applying their learnings in real life? Are they mobilising acquired information at the right time and changing their behaviour accordingly? Quizzes and assessments can be one way to measure knowledge retention, as well as post-surveys taken months or years after training ; but the best indicator of success remains unprompted action taking from the trainee. With mobile technologies, trainees can be empowered to share their stories and relate their successes — also a measurable KPI for impact measurement. Still want a literal eye-test? They can photograph their achievements.

About Chalkboard Education

Chalkboard Education provides mobile-based distance training solutions tailored for NGOs. Lightweight, offline-first and complete with full analytics capabilities, Chalkboard Education is designed for rural and hard-to-reach community training. Currently used in 10+ countries in Africa and South America, Chalkboard Education is available worldwide.

Educational technology in Africa and emerging markets https://www.chalkboard.education/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store