About a month ago I had a friend contact me asking about some issues he had while buidling a low cost ESX server for his home lab. He was having difficulty with regards to one small piece of hardware, and he didn’t know about the hardware compatibility list for ESX. While it’s a bit late for my friend Dave, here is some great instructions on how to build a $500 ESX Server. Also check out the VM Help site for an unofficial whitebox HCL list; this is not hardware offially supported by VMware but apparently tested by the internet VMware community…your milage may vary.
Today I came across Mike DiPetrillo’s article on Virtual Server Sprawl and found it very reminiscent of issues that I see in my enterprise customers while they are undergoing server consolidation. Virtualization is almost always part of any server consolidation effort today. Yet, there are some companies who, for various reasons, do not think about virtualization during consolidation…they think of it after consolidation. While virtualization during consolidation takes more effort and planning than consolidation alone, there ends up being more waste in time, resources, and capital when virtualization is done after. Unless you’re undergoing a huge enterprise wide consolidation or consolidation of Tier 1 applications, think consolidation and virtualization together.
When you do think about both, make sure that you take into consideration the cultural or social changes in your organization that virtualization may make. If your pulling developers systems out from under their desk and giving them VMs, there will be some resistance just because they feel like they are loosing something.
But, once the benefits of virtualization is discovered, make sure you don’t short change yourself on two fronts:
- Storage: Many an IT manager has short changed themselves on storage during a virtualization effort. Either by purchasing lower grade storage that costs less at the expense of performance or by not buying enough storage. With all the VMs running off of a shared storage device performance needs to be kept in the forefront of your planning. Also, once the users start to realize the benefits of virtualization the potential for VM sprawl starts as does the depletion of your storage.
- Workflow: One of the often overlooked areas of a virtualization deployment is the process and workflow around requesting and authorizing new VMs. This is what leads to VM sprawl and quickly depleted resources. This is also exactly where VMware Lifecycle Manager (LCM) comes into play. You need to think about how groups will request new VMs and the authorization process needed to approve them. LCM will help with all that.
Just a few items to keep in mind while planning a consolidation or a virtualization effort.
Last week, before the end of VMworld, we had a session with one of my customer’s to discuss ESX performance. This discussion was lead by one of VMware’s performance gurus Scott Drummonds. Scott works as the Manager of VMware’s Performance Marketing team, working with the VMware field teams and customers to provide and advise on product performance issues.
During this conversation, I wrote down some key quotes from Scott that I wanted to share. For those that have been around VMTN for a while, you might already know some or all of these performance related suggestions. However, they are key enough that I wanted to draw attention to them.
- “Administration work on the LUN has an impact on performance more than the number of hosts.”
When admin work is done on the LUN is the only time that SCSI reservation locking is used. So placing more VMDKs on the same LUN doesn’t directly impact the performance. See Scalable Storage Performance with VMware ESX Server 3.5 Vroom! blog post for additional details.
- “Put Windows VMs and Linux VMs in sperate clusters because they can share memory more efficiently…”
Part of the memory optimization doen within the ESX hyperadvisor is to share common memory used by multiple VMs. So if you are running 10 Windows VM, the memory used to store the majority of the Windows OS is shared amonst all 10 VMs since the memory contents are the same and do not change. This is what enables memory overcommitting. If you start to mix and match different OS VMs on the same host, this advantage can be minimized.
- “High storage latencies is the largest source of performance problems that I see…”
You can monitor the storage latencies within VC by changing the stats level. See Scott’s Understanding VirtualCenter Performance Statistics performance community doc for more details.
- “Once your swapping, you’re in trouble…”
Seeing swapping within your VM means that you are not allocating enough memory to the VM for the applications that are running. Watch this closely and add memory or migrate VMs when and where needed.
- “RHEL5 does 1000 interrupts per second for “greater precision” (versus most OS’ which only do 100) which can add up to un-wanted overhead in the OS…”
This is an issue with the Linux Timer Rates that Scott talks about. There is a small configuration change that you can make in RHEL5 SMP that will provide an across the board increase in system performance.
Check out all of Scott’s VMTN articles, there is a wealth of information in them.