How to Measure the Success of Cyber Awareness Training

Cybersecurity awareness training has become nearly ubiquitous in organizations as CISOs try to limit the damage from phishing attacks and other threats that can only do harm if workers are careless. Such learning can take many forms, from signs posted around an office to mandatory training modules for departments and individuals. It only takes one bit of malware, let in by one employee, to imperil a whole organization. Boards of directors know this, and it’s why cybersecurity awareness training — in one form or another — typically ends up in the budget.

But are organizations getting what they should get from their cybersecurity awareness training programs?

The only way to know for sure is to measure the efficacy of training programs that an organization already has up and running. This is a much less common practice than having the training in the first place. However, if organizations don’t do it, they can’t maximize the value of their cyber awareness investments.

Start With Quantifiable Goals

The first thing that organizations should do in measuring the performance of their cybersecurity awareness training programs, is to establish high-level goals. Three common objectives of cybersecurity awareness training programs are: 

• Risk Reduction: Whether an organization has experienced a breach itself or has become aware of a breach in its industry, every company that has a cybersecurity awareness training program in place is looking to mitigate some level of cyber risk. 
• Changes in Workforce Behavior: Reducing the frequency with which employees click on phishing emails or are taken in by other types of social engineering are examples of quantifiable goals in this category. Another objective might be to reduce the incidence of employees downloading and opening infected files. 
• Reputation Protection and Other Cost Avoidance: A breach that is widely reported and that involves customer data can have an immediate impact on an organization’s operations. Once public, a breach can negatively impact a company’s reputation. It’s even worse, of course, if the breach results in a locked system or stolen data and is followed by a demand for payment. But even breaches that don’t involve ransomware can tie up the CISO, other executives, and the cybersecurity team in days or weeks of recovery work. 

Specific Things to Measure

There are many key performance indictors (KPIs) that organizations can implement to make sure their cybersecurity awareness training programs are reaching the right employees and to figure out where more cybersecurity training is needed. While organizations may have specific awareness concerns that they want to probe for, the following metrics (in one form or another) are of interest to most CISOs.

• Participation Rates: This is the most basic KPI of all. If employees aren’t taking part in the training program, they’re not going to understand the risks they’re creating for their organization and their colleagues. It’s imperative for CISOs to have a view of who has gotten the training and who hasn’t.
• Knowledge: Having data on which people have participated in training doesn’t tell you what they’ve gotten out of it, though. To get to this deeper level of insight, companies can use periodic tests and quizzes. The tests may use an online multiple choice model and require employees that don’t achieve a certain score to retake training modules.
• Phishing Simulations: Another potentially insightful type of knowledge test, used routinely these days, is to create fake phishing campaigns to see whether employees detect the attempt to fool them. In a larger company especially, these phishing simulations can yield valuable insights.
• Attack Detection: Of course, once your cybersecurity awareness training program is in place, there are also ways of gathering performance insights that are more integrated into the daily flow of work at your company. For instance, a CIO can make it easy for employees to report suspicious emails to their office. This can be done through an email plug-in and can give a sense of the extent to which the cybersecurity imperative is being taken up by people whose main jobs lie outside the IT realm. That says a lot about the quality of a cyber awareness program.
• Overall Cybersecurity Health: Here we move to a metric that may not be of the cybersecurity awareness training program directly but that could well shed light on its performance. The idea is to get a sense of how your organization stacks up, from a security preparedness standpoint, against other organizations of your size or in your industry. The metrics can be very basic, like comparing your organization’s training participation rates to those of comparable organizations. The metrics can also be more sophisticated, showing the relative number of breaches you have had and what they have cost you. These benchmarks can give you insights into changes that you might want to make in a cybersecurity awareness training program.

Metrics’ Role in Budgetary Negotiations

From an organizational standpoint, the data on cybersecurity continues to get more alarming. In Mimecast’s 2023 State of Email Security report, virtually all respondents (97%) said their organizations had been on the receiving end of a phishing attack in the last year. The threat is especially high for organizations with more than 10,000 people; roughly three-quarters of these large organizations say the number of phishing attacks is rising.

Phishing, of course, is only one type of cybersecurity threat that capitalizes on human error. There are others, such as password attacks and eavesdropping attacks (which can result from the use of public WIFI networks). All of these risks can be mitigated with the help of a good cybersecurity awareness training program.

The right data can back up a CISO’s argument for more cybersecurity spending generally and for a greater cyber awareness budget specifically. CISOs who can demonstrate improvement over time in key cybersecurity awareness training program metrics will be in the best position to get the funds — and enhance the safeguards — that will make their organizations more secure.

The Bottom Line

While there is wide agreement within organizations on the need for cybersecurity awareness training, there is often limited knowledge about the performance of such training programs. CISOs who use KPIs to measure the results of those programs over time put themselves in the best position to get the resources they need to keep the organization learning at a high level. Read how Mimecast can help your organization benchmark its awareness program and help you improve it.