Hyper-V – latoga labs

A customer of mine today asked about the the results from a recently run benchmark of hypervisors published by Virtualization Review in which ESX, Hyper-V, and XenServer were compared. There is a post on the VMware blog questioning the configuration of the benchmark environment, and thus the results and conclusions from the benchmark. I wanted to share both of these links for those who may hve seen only the report and were scratching their heads as well.

~ ~ ~ ~ ~

Since the benchmark was highly based on SQL server running in a VM, this seems a good time to also share additional research recently done on SQL Server and shared at VMworld 2009 in France. This performance research sheds some light on that fact that SQL Server Performance Problems are Not Due to VMware. After hearing lots of customer complaints about poor SQL server performance last year at VMworld in Las Vegas, VMware’s performance team spent three months looking at every way increased performance could be sqeezed out of SQL Server by tweaking ESX, the guest OS, and SQL Server. The net result was that most performance issues seen in running SQL Server virtualized on ESX come from mis-configurations in the other components and not from ESX.

Over the years of playing the Technology Vendor game, I have learned not to give too much attention to IT Analysts statistics. This comes from understanding of how the Analysts’ game is usually played from dealing with them from both the end user client and IT vendor perspective. So when I come across statistics, I usually glance at them and move along. This morning, however, when I came across IDC’s Worldwide Quarterly Server Virtualization Tracker press release (thanks to virtualization.com’s recent post) I had to stop and take a deeper look.

The single line that made me stop was (emphasis mine):

However, in its first quarter of general availability Microsoft Hyper-V delivered a strong showing, and when combined with Virtual Server 2005, Microsoft’s market share is 23% of new shipments.

It was the two phrases in red that makes me call Shenanigans on this data.

First, this report is for the second calendar quarter of 2008 (April, May, June). Hyper-V was release just a few days before the end of June; see Microsoft’s Hyper-V press release announcing availability date on June 26. My first reaction to this point was one of disbelief that any company, even Microsoft, could go from zero to 23% market share in three business days (Thursday June 26, Friday June 27th, and Monday June 30th).

Second, the nuance of what products were included shed light on how that amazing growth happened. The report didn’t just count Hyper-V shipments, but also Virtual Server 2005. It also didn’t indicate if the Hyper-V shipments that were counted included betas that would have been in use well before the release of the product. This artificially inflated market share numbers skews the perception of this part of the report.

This leads me to wonder why the numbers were skewed this way in the first place. Was IDC just looking for some big bang to be able to report on to drive sales of the report and their own corporate visibility? Or was Microsoft involved in some way in the generation of this report and influenced the skewing of these numbers?

Wouldn’t it be nice to have this level of transparency from an Analyst with regards to their reports? Why not show us the break down numbers of just ESX and Hyper-V shipments?

(Disclaimer: I am currently employed at VMware as a Solutions Consultant)

Recent Hypervisor Benchmark Publication & Questions

Calling Shenanigans on IDC Numbers