Innovative Technology Weblog

Last week I was in session about virtualization and one question kept running through my mind. “What will be the impact of Virtualization on Monitoring?� At first you would think that not much will change if you monitor virtual machines instead of physical machines. But the way virtualization has taken off and the more virtualization is used this must change the way we monitor today. Let starts with some of my experiences with virtualization.

Own Experiences with Virtualization

I’m using virtualization a lot nowadays mostly for testing  purposes. There are two kinds of virtualization, operating system and application virtualization. I’m using both by testing complete systems like OpsMgr 2007 RC2 with VMware Server and applications like PowerGadgets with Altiris Software Virtualization. By placing applications and data into managed units called Virtual Software Packages, SVS allows you to instantly activate, deactivate or reset applications and to completely avoid conflicts between applications, without altering the base Windows installation. I wish I had known this great tool earlier before installing all those Vista builds and installing all my software afterwards. To start using SVS have a look at The World’s Greatest SVS Demo video (15 mins). I even virtualized the OpsMgr 2007 User Interfaces on my Vista Laptop.

By buying Softricity Microsoft also has an application virtualization solution. To be clear, SoftGrid’s application virtualization is different than machine virtualization (such as Microsoft Virtual Server or Vmware (ESX) Server), which virtualizes the machine on which the operating system (and applications) are installed. Machine virtualization provides an abstraction layer between the hardware and the operating system that’s running on top of it. It also allows managing and simultaneously operating multiple environments on a single machine.

The abstraction layer created by SoftGrid lies between the operating system and the applications that run within it. By virtualizing all the aspects of an application, it doesn’t affect the operating system or other applications running on that machine. I’ve not used SoftGrid yet but colleagues tell me it’s a great solution. Ruben Spruijt wrote an article in the Dutch Lanvision magazine discussing application virtualization. In the article, he compares Microsoft SoftGrid, Altiris SVS and Citrix Streaming Server.

I’ll limit myself to operating system virtualization. Why? If you want to know if an application is available the underlying operating system is just as important. If there are issues with the operating system this can have an impact on the application.

More info on Virtualization

Overview and Roadmap for Microsoft Virtual Server can be found here (powerpoint) (source: Techlog.org).  More information about Virtualization can be found on Virtualization.info, they even got an Industry Virtualization Roadmap. This is really a great website for keeping track on Virtualization Developments.

What will be the impact of Virtualization on Monitoring?

Back to my original question “What will be the impact of Virtualization on Monitoring?�  In most cases monitoring is done with some kind of agent which sends status information to a central monitoring server. The status info is retrieved via different methods, like SNMP, WMI, Eventlog, Syslog and many more. Just have a look at all the different providers MOM/OpsMgr has. This is the same for almost all other monitoring solutions out there, be it Open Source like Nagios or System Center Operations Manager 2007 or even HP Openview.

But why would you monitor if you can restores systems easily with virtualization techniques like VMWare’s VMotion or other High Availability techniques? VMotion allows users to automatically optimize and allocate entire pools of resources for maximum hardware utilization, flexibility and availability. Or perform hardware maintenance without scheduled downtime and proactively migrate virtual machines away from failing or underperforming servers. And other High Availability solutions which enable an automatic restart of virtual machines with VMware HA.  Why monitor?

I think there is still a need for monitoring with virtualization because monitoring is about being  pro-active which hopefully prevents any downtime at all. And what about troubleshooting issues? If there is an issue with your virtual machine and you restart a copy of that same virtual machine it will also crash because the cause of the problem is replicated to the copy of the virtual machine. Virtualization will not help you solve issues in your IT infrastructure it will only help you keep the downtime to a minimum. But virtualization will change the possibilities for monitoring. How?

Self-Healing systems.

With the help of virtualization it will be easier to come to self-healing systems what is the ultimate goal of Microsoft Dynamic Systems Initiative. Microsoft’s blueprint for the Dynamic Systems Initiative has three parts: virtualized infrastructure, design for operations, and knowledge-driven management. Again one could think why monitor if systems are self-healing? But to become a self-healing system a system has to know that there are issues which can have an impact on its availability. This can only be done by monitoring the health of the system. The question is if this is an external monitoring agent (like a MOM 2005 agent now) or an internal monitoring agent part of the self-healing system. Or if we look at virtualization will the monitoring agent be part of the Hypervisor? Who knows? Microsoft’s new hypervisor software will be built directly into Windows. The thin hypervisor  sits on top of the hardware and virtualizes resources like CPU and memory, so it has the ability to create OS sessions. Will Microsoft build a monitoring agent in the hypervisor? What will other vendors build in their hypervisor? Their own monitoring? Adding monitoring to the hypervisor seems as a logical step towards self-healing systems. A standard about monitoring would prove to be very important as more and more monitoring solutions need to cooperate. That’s why WS- Management is an important initiative for standardization. WS-Management addresses the cost and complexity of IT management by providing a common way for systems to access and exchange management information across the entire IT infrastructure. By using Web services to manage IT systems, deployments that support WS-Management will enable IT managers to remotely access devices on their networks – everything from silicon components and handheld devices to PCs, servers and large-scale data centers.

Provisioning

Virtualization makes  Server Provisioning easy. Server Provisioning is selecting a server from a pool of available servers; loading the appropriate software (operating system, device drivers, middleware, and applications); appropriately customizing and configuring the system, software to create or change a boot image for this server, and change its parameters, e.g. IP address, IP gateway, to find associated network and storage resources – sometimes separated as resource provisioning – audit the system, i.e. ensuring OVAL compliance to limit vulnerability or ensure compliance or install patches, then finally starting the server and its newly-loaded software (source Wikipedia). It’s clear that virtualization is a great way for server provisioning and monitoring could play an important role in Server Provisioning, by detecting the need for provisioning by watching the general health of the machine or application and selecting a server from the resource pool if more or new computing power is needed.
In most monitoring solutions you can configure (automatic) responses on alerts. Be it scripts or other sort of responses that can change the state/health of a monitored device. Think of restarting the service if the service is down or by killing a process that consumes to much memory of CPU power. But with virtualization you can extend your responses even further. You can, by the use of the API’s, change the hardware configuration of your (virtual) server. An application that is consuming to much memory? Why not dynamically enlarge the memory of the (virtual) machine? This can be done already but now it’s done from within the virtualization software. The question is if the monitor in the virtual system is not better capable of determining the state of the (virtual) machine? And should therefore decide if a machine needs more hardware or even less if it’s idle?

There has to be cooperation between the virtualization management software and the monitoring software in the virtual machine to prevent conflicting actions. If for example the virtualization management software sees that there are several virtual machines are having performance issues the steps to correct this must be the same as the steps from the responses from the (virtual) monitoring agent.

Consequences of Virtualization for current monitoring tools.

Finally virtualization is having consequences for the current monitoring tools that need consideration.  The first consequence of using virtualization is an extra operating system or systems to monitor.

1. More OS’s to monitor
If you use virtualization you have extra OS’s to monitor. So besides monitoring the OS’s (and applications) on the virtual machines you also need to monitor the OS of the Host. Some Virtualization solutions have their own monitoring tools like VMware’s Virtual Center which monitors the performance and utilization of physical servers and the virtual machines they are running with detailed reporting of CPU, memory and I/O performance. For MOM 2005 and OpsMgr 2007 there special Management Packs which help you monitor Hosts for Virtual Machines like VMware ESX or Virtual Server. Microsoft offers a Management Pack for monitoring physical computers running the Virtual Server service and the virtual machines that the service hosts.

2. Service-Oriented Monitoring
With the implementation of Service-Oriented Monitoring in OpsMgr 2007 Microsoft made a great improvement to the current way of application or device monitoring. But if you use virtualization will the virtualization layer be one of the components of your service? Or must it be considered as the ‘normal’ hardware in a physical environment? I would say it must be part of the service you are configuring/designing in the Distributed Application Designer of OpsMgr 2007. It’s an extra layer added to your environment which can break and influence your Service availability.

3. Licenses
With the rise of virtualization the licensing model would also change. If we look at the license model for MOM 2005 you must acquire and assign a management license to each device that your instances of the server software manage. A hardware partition or blade is considered to be a separate device (source  Product Use Rights – December 2005). So if I understand it correctly you don’t have to buy a MOM 2005 license for each virtual machine running on a physical server. But that will change with the next version of MOM 2005 Operations Manager 2007. Not for the better ;-(  I don’t know the licensing model of other monitoring solutions like BMC Performance Manager (Formerly PATROL) for virtual machines but I assume they will treat virtual machines the same as physical machines and you should monitor the number of virtual machines created and the monitor licenses needed. Especially if the deployment of new machines is that easy and you automated your deployment of monitoring agents. Another question that springs to mind is do you need a monitoring license for powered-off virtual machines?

Conclusion

If I look in the future, I would expect that monitoring will become very important to virtualization. Not only for the new virtual infrastructure that has to be monitored, but also for the communication between the monitoring agent on the virtual machine and the virtualization management tooling will be more sophisticated. With the help of virtualization it will be easier to come to self-healing systems what is the ultimate goal of Microsoft Dynamic Systems Initiative. With infrastructure virtualization, businesses will attain greater flexibility and agility with the ability to draw upon resources, as needed, from a virtual service pool. Server consolidations, rapid provisioning, and live migrations will reduce incremental costs while providing a level of business readiness and scalability unattainable within a purely physical infrastructure. Monitoring of that virtual environment will become very important.

Please let me know if you agree or disagree.  I really like to receive your comments!

Regards,
Stefan Stranger
http://weblog.stranger.nl