Amazon CloudTrail: How it could change cloud logging

The new Amazon CloudTrail cloud logging service could make event management much simpler for security teams. Expert Dave Shackleford explains how.

Logging and event management has long been one of the cloud's biggest security obstacles. Security teams have had numerous questions about logging best practices for the cloud, most which have gone without simple -- or any -- answers:

  • Can cloud IaaS, PaaS and SaaS environments support detailed log generation similar to today's internal requirements?
  • Should log management platforms and log collectors be placed in the same cloud network as my systems? Can they even be placed in the cloud?
  • How is log data stored securely, and how does this fit into our data lifecycle requirements?
  • What about application programming interface logging (API) logging and more application-oriented logs?

This list merely scratches the surface of questions security professionals may have; rarely is a security team able to approximate its internal logging architectures and requirements in a cloud provider's network. Recently, however, Amazon Web Services released a new service called CloudTrail that seeks to change the way cloud logging is performed and managed over time.

In this tip, we'll explain what CloudTrail is, how it works, and why it has the potential to make cloud event management easier in AWS environments.

Amazon CloudTrail explained

The first major addition to Amazon's security-oriented services in some time, CloudTrail is a new logging service that records any API calls made to AWS.

CloudTrail could solve one of the most challenging issues security teams face when migrating IT resources into Amazon.

The service captures an extensive amount of data important to security professionals, including the identity of the API caller, the time of the API call, the source IP address of the API caller, the request parameters and the response elements returned by the AWS service. CloudTrail logging captures all requests made from the standard AWS management console, command-line tools, any AWS software development kits (SDKs) and other AWS services.

CloudTrail could solve one of the most challenging issues security teams face when migrating IT resources into Amazon: the capture and maintenance of event data that can be fed into existing log management and security information and event management (SIEM) platforms. Many enterprises have made significant investments in log management and SIEM systems in order to aggregate and correlate security events across a variety of systems, but many security teams have struggled to find a way to pull in log data from cloud environments.

To that end, CloudTrail leverages Amazon S3 buckets for storage of log data, allowing security teams to leverage the same APIs to access data quickly and easily for correlation and aggregation internally. Log data can also be automatically deleted after a certain period of time, or archived to internal storage or additional Amazon services like Glacier for longer-term retention. Aggregation of log data across accounts and regions is possible, as is automated alerting and notification when certain events are registered.

Security benefits of CloudTrail

Recently, AWS released a whitepaper describing the security benefits of CloudTrail, as well as the potential compliance-related controls and requirements that may be met while using the service. Access to CloudTrail logs -- stored in S3 buckets -- can be restricted using the same role-based AWS IAM policies an organization is already using, and can also take advantage of multifactor authentication.

Amazon also added an alerting feature to CloudTrail that can notify administrators when log archives are generated. These alerts can and should be generated when logging is misconfigured or when logs fail to generate properly. Subsequent actions can be taken based on the "failed logging" or "logging misconfiguration" alerts, which provide S3 bucket addresses and links to additional event data for follow up.

CloudTrail logs are encrypted by default using Amazon's S3 Server Side Encryption (SSE) before they are placed into an S3 bucket. One feature that will help many organizations meet compliance requirements is the ability to control the AWS log data lifecycle and retention. Log files can have an expiration date or can be stored indefinitely.

Finally, log reporting can be highly customized. Log reports can include authentication information, authorization information related to roles and privilege use or misuse, and extensive details on all aspects of API calls (accounts, objects, events, etc.).

CloudTrail is an innovative solution to a thorny problem: logging events in a cloud environment and storing and managing those logs in a simple way. Many organizations have extensive logging infrastructure running internally, and getting logs from a cloud service back to the primary logging platforms was a difficult chore. With CloudTrail, logs can be generated, stored and archived more easily, allowing event management in the cloud to finally integrate with existing log management and SIEM systems.

About the author:
Dave Shackleford is the owner and principal consultant of Voodoo Security, Lead Faculty at IANS, and a SANS analyst, senior instructor and course author. He has consulted with hundreds of organizations in the areas of security, regulatory compliance, and network architecture and engineering, and is a VMware vExpert with extensive experience designing and configuring secure virtualized infrastructures. He has previously worked as CSO for Configuresoft, CTO for the Center for Internet Security, and as a security architect, analyst, and manager for several Fortune 500 companies. Dave is the author of the Sybex book Virtualization Security: Protecting Virtualized Environments, as well as the co-author of Hands-On Information Security from Course Technology. Recently, Dave co-authored the first published course on virtualization security for the SANS Institute. He currently serves on the board of directors at the SANS Technology Institute and helps lead the Atlanta chapter of the Cloud Security Alliance.

Dig Deeper on Evaluating Cloud Computing Providers