Gunnar Assmy - Fotolia

Information security metrics: What to collect in the cloud

Threat-related metrics that CISOs find useful often differ from what the C-suite wants to know. Here's how to communicate risk -- and return -- on cloud security investments.

CISOs need an accurate picture of security performance in the cloud. As C-level executives and boards of directors pepper senior security management with questions about risk exposure -- "How much security do we need? How close are we to meeting our compliance objectives?" -- CISOs are looking for ways to effectively measure cloud controls against strategic objectives and report these findings.

How can safe operations in the cloud be measured without a dashboard that drills down into risk versus costs and other information security metrics that executives actually care about?

Slowly but surely, IT departments are adapting their security controls (documented processes) and architecture models to fit information systems in public and private cloud environments. As preventive, detective and responsive controls are transitioned into hybrid cloud models, CISOs now have to consider governance in the cloud and new information security metrics, both internal tracking and service-level agreements (SLAs). 

With some monitoring activities, information security metrics are fundamentally the same in the internal data center and cloud. Chances are, security tools that have been ported to cloud environments will largely capture the same data and provide any information security metrics currently gathered. Virtual firewall appliances log dropped or blocked connections, for instance; cloud-based vulnerability scanners report on the number of infrastructure-as-a-service or platform-as-a-service systems detected, and their patch and exposure status; and host-based security monitoring tools record files accessed and configuration deviations.

New InfoSec cloud metrics

CISOs have to focus on more performance indicators and SLA-related metrics in the cloud, however. That means security teams need to look into what new metrics they can gather that are related to cloud security posture (security operations). If automated provisioning is performed with orchestration tools like Chef and Puppet, logs of instance generation and posture assessment can be collected. Events pertaining to administrative activity, cryptographic tool and key use and access, and deviations from approved configurations can also be monitored to determine the cloud provider's security posture. Important information security metrics include the following:

  • Number of administrative logons to the cloud-provider console.
  • Number of "orphan" accounts that are inactive in cloud environments over a specified period of time.
  • Number of systems started versus those running over a defined time period (in other words, those still running a certain time later).
  • Number of systems that have approved configuration postures versus those that do not.

Technical controls not enough

Security senior management also needs to monitor their cloud providers' contractual obligations, both legal and supply chain. In every contract with providers, there are usually a number of SLAs that range from standard operational availability (uptime and performance) to security-related requirements (incident response times and legal or forensic requests). Some cloud providers may have obligations to meet data lifecycle mandates such as data retention, legal holds for email messages and responses to forensic chain-of-custody requirements for equipment and evidence.

These contractual obligations should all be tracked closely and reported to both operational and executive management. Any changes to cloud provider audit and attestation reports should also be carefully monitored and reported. If a material change to a control statement in the provider's SOC 2 report is relevant, it should be disclosed and assessed by both the security and legal teams.

What are the major issues or concerns related to your cloud computing model?

While cloud service costs are usually more related to operations and development teams, most mature deployments now include a variety of security-related costs. Expenses range from overhead caused by security technology to licensing related to "in cloud" tools and products to new services like cloud access security brokers. Costs may also include identity and encryption services and any other cloud services that are in place solely to provide controls coverage for these environments. Security teams cannot overlook financial and budgeting aspects of cloud use, as information security controls and services are now an integral part of deployments and ongoing operations. Cloud metrics may include costs and budget changes over time, unforeseen changes (both positive and negative) and percentage of overall security budget spent in the cloud versus in-house.

Still waiting on maturity models

Another important area to track is the overall maturity of the cloud security program. All organizations want to know how they are performing compared to others in their industry. And while there is a definitive lack of benchmarks and baselines in this category, we've got to start somewhere.

Most security teams use controls frameworks like National Institute of Standards and Technology (NIST) 800-53, the NIST Cybersecurity Framework, and the Cloud Security Alliance Cloud Controls Matrix in conjunction with a traditional maturity model like the Common Maturity Model. This isn't perfect by any means, but at least organizations can compare the maturity of in-house controls to those in the cloud and look for areas of improvement as they emerge. Some controls are just too immature in the cloud at the moment. It will take time to see cloud vendors improving their capabilities and natively integrating more readily into cloud environments.

Whatever cloud metrics you choose, be sure to gather feedback and find out whether any of this is actually useful to stakeholders. Security senior management has a tendency to report on information security metrics that are incredibly complex or just flat-out not useful to business executives. This is a trend we can stop as we progress into the cloud.

Focus on what matters: improving the state of security controls, discovering process or policy weaknesses and fixing them, reporting on costs and meeting compliance and maturity goals. With those areas of focus in mind, there should be plenty to talk about. 

Next Steps

How to handle cloud migration challenges

What metrics to include in security reports

Dig Deeper on Cloud Network Security Trends and Tactics