momius - Fotolia

Can virtual machine introspection improve cloud security?

What is virtual machine introspection, and can it help improve cloud security? Expert Dan Sullivan explains techniques behind VM introspection and how it can boost security in the cloud.

I read that virtual machine introspection can help boost security in cloud environments. What is virtual machine...

introspection, and how would it improve cloud security?

Virtual machine introspection is a set of techniques for monitoring the state of application software, guest operating systems and virtual machines running on a physical server. VM introspection is implemented in the hypervisor.

Usually, one virtual machine is used to monitor the state of another virtual machine. This avoids the need to load an agent within the target VM after the operating system is started. It also avoids potential problems with introspection code interfering with the state of the virtual machine being inspected.

VM introspection techniques have been applied to cloud security, cloud access control and intrusion detection. Researchers have reported success with identifying rootkits on virtual machines that otherwise would not have been detected.

VM introspections are designed to have minimal effect on the target virtual machine and the hypervisor. The virtual machine introspection operations should also be transparent to the hypervisor and not cause any side effects. Introspection can occur in different ways -- for example, by doing memory introspection or by doing system events introspection, looking at things such as system calls, interrupts and I/O device events, as well as live processes introspection.

VM introspection can help improve cloud security in a number of ways. Virtual machine introspection techniques can detect malware actions that are designed to avoid operating system or resident antimalware detection by observing operating system behavior from the outside. Malicious software sometimes masquerades as legitimate operating system processes, effectively hiding itself among the processes that might otherwise detect its malicious activity. Again, VM introspections can detect malicious patterns that may be hidden from cohosted processes on the target VM. Since introspection functions execute in the hypervisor, it is possible to trace and monitor every interaction between a guest operating system or application and the underlying hardware.

These are all useful techniques for improving the security of cloud services, but there are drawbacks. Perhaps the most significant is that cloud providers may market their services along with the assurance that they would never "look inside" a customer's virtual machine. As with other security controls, users will need to decide how to balance the protection of advance security measures with the risk of allowing unwanted access to their data and processes.

Next Steps

Learn about performance issues when using virtual machines

Find out about picking the right virtual machine for the private cloud

Read about cloud stack security and understanding VM risks

Dig Deeper on Cloud Network Security Trends and Tactics