latoga labs

Technology of Business & Business of Technology

© 2006–2023 · Log in

ESX Performance Discussion and Tips

Leave a Comment

Last week, before the end of VMworld, we had a session with one of my customer’s to discuss ESX performance.  This discussion was lead by one of VMware’s performance gurus Scott Drummonds.  Scott works as the Manager of VMware’s Performance Marketing team, working with the VMware field teams and customers to provide and advise on product performance issues.

During this conversation, I wrote down some key quotes from Scott that I wanted to share.  For those that have been around VMTN for a while, you might already know some or all of these performance related suggestions.  However, they are key enough that I wanted to draw attention to them.

  • “Administration work on the LUN has an impact on performance more than the number of hosts.”

When admin work is done on the LUN is the only time that SCSI reservation locking is used.  So placing more VMDKs on the same LUN doesn’t directly impact the performance.  See Scalable Storage Performance with VMware ESX Server 3.5 Vroom! blog post for additional details.

  • “Put Windows VMs and Linux VMs in sperate clusters because they can share memory more efficiently…”

Part of the memory optimization doen within the ESX hyperadvisor is to share common memory used by multiple VMs.  So if you are running 10 Windows VM, the memory used to store the majority of the Windows OS is shared amonst all 10 VMs since the memory contents are the same and do not change.  This is what enables memory overcommitting.  If you start to mix and match different OS VMs on the same host, this advantage can be minimized.

  • “High storage latencies is the largest source of performance problems that I see…”

You can monitor the storage latencies within VC by changing the stats level.  See Scott’s Understanding VirtualCenter Performance Statistics performance community doc for more details.

  • “Once your swapping, you’re in trouble…”

Seeing swapping within your VM means that you are not allocating enough memory to the VM for the applications that are running.  Watch this closely and add memory or migrate VMs when and where needed.

  • “RHEL5 does 1000 interrupts per second for “greater precision” (versus most OS’ which only do 100) which can add up to un-wanted overhead in the OS…”

This is an issue with the Linux Timer Rates that Scott talks about.  There is a small configuration change that you can make in RHEL5 SMP that will provide an across the board increase in system performance.

Check out all of Scott’s VMTN articles, there is a wealth of information in them.

VMware ESX & VM Server Differences Explained

2 Comments

One item that I have seen come up again and again recently with my customers is a mis-understanding of ESX and VM Server.  Particularly the performance aspects of both.  These are two entirely different products with entirely different goals that need to be kept in mind when comparing the two.

VM Server is the free server virtualization product from VMware.  It’s purpose was to help seed the market and get people more confortable and familiar with virtualization concepts on the server.  What better way to do that than give something away for free for people to play with!  VM Server is similar to VM Workstation/Fusion in that it runs on top of an OS; so from the ground up you have:

server hardware > operating system > VM Server > virtual machine.

This means VM Server inherets the same performance issues that the OS has.  It also means that it has to rely on the OS for a lot for the management of resources.  Net result:  the performance of VM Server will be impacted by the OS you are running it on top of.

ESX is the data center grade hyperadvisor from VMware.  It’s purpose is to provide virtualization for  business critical data centers.  ESX essentially doesn’t run on top of an OS.  You can think of it as a data center OS as it is the base layer for the Virtual Infrastructure set of products. Similarily, from the ground up you have:

server hardware > ESX > virtual machines

You can see immediately that ESX has eliminated the operating system level, this is the source of the performance gain that ESX provides.  It is running on top of the raw hardware.  It is truely a specialized OS with the sole pupose of virtualization.

One thing that VM Server and ESX share, is the engine to run a virtual machine.  This engine has lots of tricks built into it to manage the physical resources of the box and to distribute them efficiently to the virutal machines that need them.  Of couse, you still suffer from that extra piece of the base operating system under VM Server.  This is why you can have VM Server running with no VMs and still see it utilizing resources.  It’s doing the work of interfacing with the base OS for resource management even when no one is using the resources.

So, if you hear somone complain about their VMs are not performing very well ask them which product they are using.  If they are using VM Server, point them here.  If they are running ESX, then it’s time to dive into the typical operating system and application stack tuning/debugging routine.

(Disclosure: I currently work at VMware as a Solutions Consultant)

Green Scene Hits Your Code

2 Comments

Since my mind is on efficiency this morning (see previous post about Facebook) I wanted to share an interesting blog posting I’ve had open in a browser for a few weeks now.  Steve Sounders, web performance guru from Google and previously Yahoo, posted some interesting thoughts on how green is your web page?

Steve did a quick mental experiment of calculating the CO2 emissions caused by bad code on a large website, he used wikipedia as his example.  I find this a bit interesting on the cyclical nature of the topic.  I might be showing my age a bit here, but back when I was a lad learning how to code up on the frozen tundra, we actually took into consideration efficiency and the cost of operations (maybe it was our proximity to Cray Research that drove this…).  I find it interesting that the green movement is causing this topic to be thought of again but in a different way.

I have been doing a little fun project like this myself at home.  A friend loaned my a device called Kill A Watz, which you plug into a power outlet and then plug other electrical devices into the Kill A Watz. The Kill A Watz then measurs how much electicity you are using on that one outlet.  It can track over time and give you the KW over a time period as well as real time watt usage.  I am using this on our home entertainment center to measure how much electricity is uses when it’s in standby mode.  Watch for a posting on that next week.  (I will give a teaser and let you know that a flat panel plasma TV uses twice the electricity when displaying a bright scene than when displaying a dark one…)