Additional Tidbits on Virtualization Best Practices

With the latest version of hypervisors;  if the right hardware is purchased (using hardware compatibility lists), the infrastructure is properly tuned (best practices are followed), and the virtual infrastructure is designed properly (referenced architectures) most environments will run with 6% overhead or less.  Most database servers do not run with 90% utilization of CPU and memory, so virtualization should be able to run 80 - 90% of the database servers in the world.

DBAs that make a blanket statement that virtualization has too much overhead, can easily be proven wrong so don't say that.  Not with today's hypervisors.  At the same time, I'm not saying every database, business application or Hadoop environment can be virtualized.  What I can say for a fact is that DBAs that are properly aligned with the business and can have a significant impact on reducing CapEX/OPex and increase operational efficiency through virtualization and leveraging the cloud are worth their weight in gold.  :)

There will be a slight increase in CPU due to the hypervisor allocating resources.  Disk latency is minimal if best practices are followed.  Be aware that you should always test these settings to make sure they increase performance and reduce overhead for your specific environment.

• VMware uses binary translation and paravirtualzation.  Using the right network drivers is very important.
• Consolidation through virtualization allows you to leverage your CAPEx and OPEx but never over commit production or SLA sensitive environments. Get the most of what you have to leverage your hardware just be careful with overcommitment.
• Leverage Hardware Assist with VT and memory management.
• Set Static High Performance.
• Consider disabling C-states.
• Latest versions of OS, network drivers, virtual tools, CPUs, memory all improve performance and reduce overhead in virtualization.
• Try to stay away from CPU affinity.
• Stay away form Direct Path I/O unless you absolutely need the additional few percentages performance it gives you.
• Consider interrupt coalescing.  Adaptive coalescing is usually something to avoid with latency sensitive environments.
• Try to keep all NUMA access within one socket.  Disable Node Interweaving in BIOS.
• Horizontal scaling can be effective with virtual infrastructures and Hadoop.
• Hadoop master servers can significantly leverage virtualization.  However, thoroughly test and make sure you have the skill set to run Hadoop in a virtualized environment. 
• If virtualizing Hadoop data servers make sure you have throughly tested the HVE patches and they are working correctly in your Hadoop distribution.