Dave, AKA The VMguy, made a post about BIOS recommendations for IBM servers. I thought I’d make a few comments on his recommendations:
Dave recommends turning off “PowerNow” or “Cool’N'Quiet” and points to a KB article about how this may affect timekeeping. However, the KB article does state that this does not apply to ESX server, but only to the various hosted products such as VMware (GSX) Server, Workstation, etc.
I would recommend doing the opposite, at least for HP servers. I have seen situations in our lab where a server with power management turned off will use more power than necessary. For example, with HP Dynamic Power Savings off, one of my test blades showed this:
However, with DPS on, my blade was down to almost half that power usage:
That’s a pretty substantial power savings right there! I’ve also never seen a timekeeping caused by this setting. With IBM servers, your mileage may vary, however.
The last point I would add is that while Dave says you should turn on the “No Execute” or “Execute Disable” (the NX or XD bit, depending if you’re in AMD or Intel land) just to fix an issue with a specific BIOS revision of an IBM server, I would argue that one should always turn on NX or XD on your systems. Why? Because Enhanced VMotion Compatibility requries it!
You must ensure the BIOS settings for these processors enable Hardware Virtualization (if available) and Execute Protection. Default BIOS settings may not always enable these features. Hardware Virtualization is Intel VT on Intel processors and AMD-V on (supported) AMD processors. Execute Protection is Intel eXecute Disable (XD) on Intel processors and AMD No eXecute (NX) on AMD processors.
Another setting that wasn’t mentioned was that on AMD systems, there’s generally an option involving NUMA called “Node Interleaving”. Here’s where it is on an HP system:
On multi-socket AMD systems, each socket has its own path to memory, and each processor has its own memory bank which it can access the fastest. This means that Opteron systems have a Non-Uniform Memory Access. ESX is NUMA-aware, and so will intelligently place a VM’s memory on the same node as the processor its running on (if possible). Enabling Node Interleaving makes the whole memory space appear as a single uniform memory space (even though it isn’t) which can negatively affect performance. So make sure to leave Node Interleaving off.
Good info – I’ll try it out.
Comment by symbolik — February 10, 2009 @ 8:08 am