Measures and metrics: How to report on more than just the SOC

You can’t improve something unless you measure it. Security teams often report SOC metrics like the number of threats blocked or alerts received, which provide little value. These numbers don’t accurately reflect the success of the security function.

A good security function will collect and report measures for all their selected security controls and report the absence of any additional required controls.

Everyone wants to do this, but it’s not easy. How do you collect metrics on your security posture that truly matter and provide actionable insights? How can we go beyond surface-level statistics and develop a comprehensive measurement system that drives continuous improvement?

In this talk, I will explore the following key points, focusing on practical examples aligned with NIST SP 800-53 and ISO 27002:

1. Identifying Meaningful Metrics:

• Align security metrics with organisational goals and risk management strategies.
• Provide examples of meaningful metrics beyond the SOC, such as time to detect/respond to incidents, vulnerability remediation rates, and user behaviour analytics.
• Prioritise metrics that offer insight into the effectiveness and efficiency of security controls and processes.

2. Practical Examples from Fictional Companies:

• Present case studies from fictional companies across various industries and maturity levels.
• Demonstrate how to apply metrics in different organisational contexts, emphasising the adaptability of NIST SP 800-53 and ISO 27002 standards.
• Provide specific examples of metrics aligned with these standards, tailored to the unique needs of each fictional company.

3. Guidance from NIST SP 800-55:

• Introduce the NIST SP 800-55 Measurement Guide for Information Security, focusing on Volume 1 (Identifying and Selecting Measures) and Volume 2 (Developing an Information Security Measurement Program).
• Discuss how to identify and select metrics that align with organisational goals and security requirements.

Highlight four types of metrics:

• Implementation Metrics: Track progress of specific controls and their deployment status.
• Effectiveness Metrics: Evaluate how well the controls are working and achieving their objectives.
• Efficiency Metrics: Assess the timeliness and resource utilisation of processes.
• Impact Metrics: Determine the broader impact of the controls on the organisation.

4. Data Collection and Analysis:

• Explore methodologies for collecting accurate and relevant data from various sources within the organisation.
• Discuss the use of automated tools and platforms that can aggregate and analyse security data.
• Highlight the importance of context in data analysis to avoid misinterpretation of metrics.

5. Reporting and Communication:

• Offer strategies for presenting security metrics to different stakeholders, including executives, technical teams, and non-technical staff.
• Discuss the role of visualisation tools in making complex data understandable and actionable.
• Emphasise the need for regular reporting cycles and adapting reports based on evolving threats and organisational changes.

By the end of this talk, attendees will understand how to develop and implement a robust metrics program that goes beyond traditional SOC reporting. They will be equipped with the knowledge to collect, analyse, and communicate meaningful security metrics that reflect their true security posture and drive improvements and better risk management.