Essential Types of Metrics to Boost Support for Your Cybersecurity Learning Program – Part 2

Jul 31, 2023

OffSec

OffSec

Content Team

In the first blog post on this topic we discussed the importance of cybersecurity learning metrics, outlined metrics to optimize training activities, and explored how metrics can measure the impact of learning on cyber preparedness and the organization’s business performance.

In this post we will examine metrics to track individual development and performance and to assess job satisfaction and improve employee retention.

3. Metrics to Track Individual Development and Performance

CISOs and cybersecurity managers have a huge stake in the personal development and job performance levels of their staff. And because of constant changes in the threat landscape and the huge demand and very limited supply of professionals with advanced cybersecurity skills (and, for that matter, basic cybersecurity skills), they often place great value on being able to move existing employees to more responsible levels in the same role or to other high-priority cybersecurity roles (e.g., cloud security, application security, penetration testing, forensics, threat hunting, security architecture, and security administration). 

Sometimes training activities can be tied directly to existing productivity statistics such as vulnerabilities remediated or average time-to-identification of attacks. However, even when this is not possible, other metrics can supply strong indicators of improved job performance. (See table)

How these metrics are used:

These metrics let cybersecurity managers:

• Document improved individual productivity
• Track the achievement of departmental staffing goals such as filling key cybersecurity roles with internal candidates 

Where organizations already collect productivity data, it should not be difficult to document the effect of training. For example, if throughout their enrollment in a training program, a SOC analyst triages more alerts in fewer hours, you can calculate a hard dollar cost reduction attributable to the training based on the hourly burdened cost of that analyst.

Improved management and peer evaluations can serve as strong indicators of better performance, even when no precise productivity statistics are available.

In the same way, newly acquired skills can provide a meaningful proxy for improved performance, as well as represent a type of measurable progress toward the goal of filling gaps in cybersecurity capabilities.

Finally, when upskilling employees leads to promotions and lateral movement into high-priority cybersecurity roles, you can point to hard savings from eliminating the need to recruit and onboard external candidates. 

How these metrics are collected:

Job productivity statistics can often be acquired from existing cybersecurity operations platforms and applications. They then need to be correlated with individuals’ training times, to show the difference between pre-training and post-training productivity.

Job performance evaluations, as well as data on promotions and job mobility, should be available from human resources systems (but may need to be anonymized to comply with privacy and security regulations).

Skills acquisition data may be available from your organization’s LMS and HR records.

4. Metrics to Assess Job Satisfaction and Improve Employee Retention

Employee retention is a critical issue for most cybersecurity groups. It is extremely costly to recruit and onboard cybersecurity professionals. For some organizations, people with key skills may be effectively unobtainable. While you aren’t likely to get accurate information by asking employees outright “does this training program increase the chance that you will stay with our organization,” there are questions that will provide good indicators, particularly if responses change over time. (See table)

How these metrics are used:

These metrics can help cybersecurity and HR managers assess:

• The impact of training on learners’ job satisfaction 
• The learners’ own opinion of the impact of training activities on their job productivity
• How retention patterns differ between employees and groups who have received training and those who haven’t.

It is important to note that an employee might be satisfied with a program, course, or module but not feel that it made them more satisfied with their job or better prepared to do it (the topic could be interesting but not relevant). Conversely, an employee could give a training activity a low or medium rating but still feel that it will have a positive impact on job performance (perhaps the material was exceptionally difficult but still very informative and useful). That’s why it is important to have separate metrics for learner satisfaction with the activity, the impact of the activity on job satisfaction, and the impact of the activity on job performance.

Obviously, cybersecurity professionals who give strongly positive responses to questions about the impact of training on job satisfaction and job enablement are more likely to stay with their employer than those who give weak responses or who haven’t had training. Also, changes in these metrics can provide early indicators of positive or negative trends in retention. 

Similarly, net promoter score (NPS) and employee net promoter score (eNPS) metrics (responses to “would you recommend” questions) have been proven to be strong predictors of customer loyalty and employee retention, respectively.

Finally, although it may take time to accumulate enough data to draw conclusions, you should be able to compare retention rates between employees who have had access to training and those who haven’t. It may also be possible to segment the analysis further into those who receive no training, a base level of training, and high levels of training.

How these metrics are collected:

Job satisfaction and enablement ratings and eNPS metrics can be gathered from surveys given throughout the learning process or when certain objectives are achieved, to ensure continuous tracking. 

Alternatively, if your organization already conducts employee satisfaction surveys, these types of questions can be added, and you can compare the results for cybersecurity professionals who have received different levels of training. 

Finally, retention data may be available from HR records.

The Bottom Line

To summarize our discussion: CISOs and security leaders can gain a lot by investing a little imagination and effort in expanding the range of metrics they track. 

Certainly, continue to use data on enrollments, completions, achievement, and engagement to fine-tune training offerings. 

But go a little farther. Compile data that connects training with job productivity, the acquisition of key skills, and the organization’s ability to fill high-priority cybersecurity positions through upskilling existing employees.

After that, it probably won’t be too hard to start tracking the impact of training on job satisfaction, and ultimately the retention of skilled, hard-to-replace cybersecurity veterans.

By then you will be ready to arrive at the holy grail of business metrics – connecting your activities to cyber readiness and through that to business objectives. And you don’t have to invent new metrics to do that; just show how training is affecting existing KPIs and OKRs.

To address the specific learning and skills development needs of the enterprise, we offer a Learn Enterprise subscription. As OffSec’s answer to cybersecurity skills development at scale, Learn Enterprise can help you attract, assess, develop, and retain top cybersecurity talents through continuous skills development and knowledge acquisition.

Book a meeting with an OffSec’s learning and skills development expert to get started with Learn Enterprise.