A DBA's Journey into the Cloud and Big Data: VMware

Showing posts with label VMware. Show all posts

Tuesday, April 9, 2013

Key Links for Virtualizing Oracle

Key Links for Best Practices

Lot of the best practices for virtualizing Oracle applies to hypervisors in general, whether it's VMware or Oracle VM. Obviously, there are specific best practices when it comes to features that are specific to either of the products. For example, we need to create separate interfaces on the VM host (ESXi host or Oracle VM Server) to segment off management related network traffic (i.e. management related traffic to maintain a network heartbeat or the traffic to perform live migrations (vMotion in VMware)). At a minimum, each physical host needs to have 4 physical network interface cards. 6 Network interface cards will be highly recommended. We will create a bonded network interfaces for the following network workloads:

1. 2 NICs bonded for the public network for all oracle database related traffic

2. 2 NICs bonded for oracle private network between the RAC clusters

3. 2 NICs bonded for communication between the ESXi or Oracle VM Server host machines

All the best practices that are applicable at the VM Guest level apply to both VMware and Oracle VM. For example, we want to enable jumbo frames on the Guest VM. We also want to setup hugepages and disable NUMA at the Guest VM level.

In general, we also do not want to over-commit memory or CPUs for production environments. For databases that fit well for consolidation, we can consider over-committing memory or CPUs.

For additional information for best practices for VMware, please read the following articles.

Four key documents for virtualizing Oracle

DBA Best Practices

http://info.vmware.com/content/12581_VirtApps_index?src=socmed_BCAblog&xyz=

Oracle Databases on VMware – RAC Workload Characterization Study http://www.vmware.com/files/pdf/partners/oracle/Oracle_Databases_on_VMware_-_Workload_Characterization_Study.pdf

Oracle Databases on VMware – RAC Deployment Guide http://www.vmware.com/files/pdf/partners/oracle/vmware-oracle-rac-deploy-guide.pdf

High Availability Guide

http://www.vmware.com/files/pdf/partners/oracle/Oracle_Databases_on_VMware_-_High_Availability_Guidelines.pdf

vCloud Suite and vCloud Networking and Security

vCloud Editions

http://www.vmware.com/products/datacenter-virtualization/vcloud-suite/compare.html

vCloud Networking and Security

http://www.vmware.com/products/datacenter-virtualization/vcloud-network-security/compare-editions.html

vCenter Operations

http://www.vmware.com/products/datacenter-virtualization/vcenter-operations-management/compare-editions.html

VMware Tech Resource Center (Videos, Whitepapers, Docs)

http://www.vmware.com/products/datacenter-virtualization/vsphere/resources.html

Miscellaneous

A high level whitepaper on virtualizing Business Critical Apps on VMware

http://www.vmware.com/files/pdf/solutions/VMware-Virtualizing-Business-Critical-Apps-on-VMware_en-wp.pdf

Deployment Guide, Reference Architecture, Customer case studies and white papers

http://www.vmware.com/solutions/partners/alliances/oracle-database.html

Oracle Databases on VMware – Understanding Support and License : http://www.vmware.com/files/pdf/techpaper/vmw-understanding-oracle-certification-supportlicensing-environments.pdf

VMware Network I/O Control: Architecture, Performance and Best Practices http://www.vmware.com/files/pdf/techpaper/VMW_Netioc_BestPractices.pdf

Esxtop and vscsiStats

http://communities.vmware.com/docs/DOC-5490

http://communities.vmware.com/docs/DOC-10095

Memory Management vSphere 5

http://www.vmware.com/resources/techresources/10206

Resource Mgmt vSphere 5

www.vmware.com/files/pdf/mem_mgmt_perf_vsphere5.pdf

Achieving a Million IOPS in a single VM with vSphere5

www.vmware.com/files/pdf/1M-iops-perf-vsphere5.pdf

VMXNET3 was designed with improving performance in mind. See, VMware KB 1001805: http://kb.vmware.com/selfservice/documentLinkInt.do?micrositeID=null&externalID=1001805

Performance Evaluation of VMXNET3 Virtual Network Device can be found at: http://www.vmware.com/pdf/vsp_4_vmxnet3_perf.pdf

Network I/O Latency in vSphere5http://www.vmware.com/resources/techresources/10256

Preferred BIOS settings (always double check with hardware vendor, http://www.vmware.com/pdf/Perf_Best_Practices_vSphere4.1.pdf

Oracle Database on vSphere Deployment Tips - http://www.vmware.com/files/pdf/Oracle_Databases_on_vSphere_Deployment_Tips.pdf

SCSI Queue Depth - Controlling LUN queue depth throttling in VMware ESX/ESXi

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008113

Monitor disk latency at three distinct layers of the device or HBA, the kernel or ESX hypervisor and the guest or virtual machine.

http://communities.vmware.com/docs/DOC-9279

PVSCSI Storage Performance

http://www.vmware.com/pdf/vsp_4_pvscsi_perf.pdf)

Snapshot limitations and best practices to minimize problems http://kb.vmware.com/kb/1025279

Jumbo frames VMXNET3

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1015556

1Gb shortcomings: http://www.vmware.com/files/pdf/techpaper/VMW_Netioc_BestPractices.pdf.

The vSphere 4 CPU scheduler

www.vmware.com/files/pdf/perf-vsphere-cpu_scheduler.pdf

Some excellent storage links from Chris Sakac (EMC) and Vaughn Stewart (NetApp)

VNX and vSphere Techbook

http://www.emc.com/collateral/hardware/technical-documentation/h8229-vnx-vmware-tb.pdf

VMAX and vSphere Techbook

http://www.emc.com/collateral/hardware/solution-overview/h2529-vmware-esx-svr-w-symmetrix-wp-ldv.pdf

Isilon and vSphere Best Practices Guide

http://www.emc.com/collateral/hardware/white-papers/h10522-bpg-isilon-and-vmware-vsphere5.pdf

http://www.vmware.com/files/pdf/techpaper/VMW-vSphere41-SIOC.pdf

http://virtualgeek.typepad.com/virtual_geek/2010/08/drs-for-storage.html

http://www.yellow-bricks.com/2010/09/29/storage-io-fairness/

Sunday, April 7, 2013

Additional Tidbits on Virtualization Best Practices

With the latest version of hypervisors; if the right hardware is purchased (using hardware compatibility lists), the infrastructure is properly tuned (best practices are followed), and the virtual infrastructure is designed properly (referenced architectures) most environments will run with 6% overhead or less. Most database servers do not run with 90% utilization of CPU and memory, so virtualization should be able to run 80 - 90% of the database servers in the world.

DBAs that make a blanket statement that virtualization has too much overhead, can easily be proven wrong so don't say that. Not with today's hypervisors. At the same time, I'm not saying every database, business application or Hadoop environment can be virtualized. What I can say for a fact is that DBAs that are properly aligned with the business and can have a significant impact on reducing CapEX/OPex and increase operational efficiency through virtualization and leveraging the cloud are worth their weight in gold. :)

There will be a slight increase in CPU due to the hypervisor allocating resources. Disk latency is minimal if best practices are followed. Be aware that you should always test these settings to make sure they increase performance and reduce overhead for your specific environment.

VMware uses binary translation and paravirtualzation. Using the right network drivers is very important.
Consolidation through virtualization allows you to leverage your CAPEx and OPEx but never over commit production or SLA sensitive environments. Get the most of what you have to leverage your hardware just be careful with overcommitment.
Leverage Hardware Assist with VT and memory management.
Set Static High Performance.
Consider disabling C-states.
Latest versions of OS, network drivers, virtual tools, CPUs, memory all improve performance and reduce overhead in virtualization.
Try to stay away from CPU affinity.
Stay away form Direct Path I/O unless you absolutely need the additional few percentages performance it gives you.
Consider interrupt coalescing. Adaptive coalescing is usually something to avoid with latency sensitive environments.
Try to keep all NUMA access within one socket. Disable Node Interweaving in BIOS.
Horizontal scaling can be effective with virtual infrastructures and Hadoop.
Hadoop master servers can significantly leverage virtualization. However, thoroughly test and make sure you have the skill set to run Hadoop in a virtualized environment.
If virtualizing Hadoop data servers make sure you have throughly tested the HVE patches and they are working correctly in your Hadoop distribution.

Tuesday, March 20, 2012

VMware Best Practices

Below is a link to a number of VMware best practices around Virtualizing Oracle, Exchange, SQL Server, SAP and Enterprise Java. Virtual infrastructures like anything else needs to be properly designed and configured for the workloads that need to be supported.

It is very important that VMware best practices are followed. As a Tier One Specialist at VMware, I see three areas where best practices need to be followed:

VMware best practices: Oracle, SAP, Peoplesoft, Exchange, Oracle Fusion Applications, etc. do not understand they are in a VM. The tuning and configuration has to be on the hosts and clusters supporting the VMs.
Storage venders best practices for virtualization: 80% of the issues around virtual infrastructures are due to the design, configuration and implementation of storage. Get your storage right.
Virtual infrastructure team best practices: Properly design and set up tiering, processes, provisioning, monitoring, etc.

From a virtualizing Oracle on VMware perspective there are four key documents you need to know inside and out:

Best Practices Guide
RAC Deployment Guide
Workload Characterization Study
High Availability Guide

This link will take you to each of the best practice sections:

http://info.vmware.com/content/12581_VirtApps_index?src=socmed_BCAblog&xyz=

Saturday, December 17, 2011

The vRAC DBA will be one of the highest in demand skill sets in 2012

The acceleration of the virtualization of production database servers and Business Critical Applications (BCA) has exceeded everyone's projections. In talking to technology leaders at Oracle and Microsoft they are also very surprised by how rapidly the movement to virtualization is going. Virtualization is also the foundation that makes cloud technology and products work, so Cloud projects are accelerating the virtualization industry as well. The growth of the cloud market is predicted to reach $241 billion by 2020 according to analysts at Forrester Research.

Virtualization and the Cloud are the next paradigm shift the industry is moving to after Mainframes, Client/Server and Distributed Systems. The virtualization and cloud markets are projected to be factors higher than the database market. There is an exploding demand for DBAs who understand virtualization and the Cloud. This demand is very different than the demand for DBAs that have occurred previously. During the dot com boom, if you were breathing and could spell Oracle you could get a very good paying job as a DBA. The demand was so high a lot of people who did not have the experience or background to be DBAs were given great opportunities and a new generation of DBAs were born. The DBA needed to handle virtualization projects requires a much stronger skill set.

The DBA needed today in virtualization infrastructures is a DBA that:

Understands virtualization best practices, reference architectures and successful processes and methods.
Can validate a scalable virtual infrastructure design.
Can implement databases and business critical applications in virtual infrastructures.
Has a solid background in storage and networking.
Understands the critical pieces in migrating databases and BCAs from physical to virtual infrastructures.
Has a strong background in Oracle RAC, ASM and the entire Oracle database server technology stack.
If migrating SQL Server has a strong background in the SQL Server technology stack and Microsoft clustering.
Understands hardware, processors, IOPS and how the entire infrastructure must generate the proper throughput.
Has a strong background in data architecture, data models and application scaling for databases.

This new DBA has been profiled by an internal center of excellence team at VMware as the vRAC DBA. We believe the vRAC DBA is going to occupy the preeminent position in modern IT environments. The profile of a vRAC DBA has been growing for over a year. Industry experts from CA, owners of database consulting firms, Oracle experts and VMware experts have all been discussing the importance of this emerging role in technical infrastructures.

The ideal profile for this person is a very strong Oracle RAC DBA. This DBA has all the infrastructure and technology expertise. That is why a very strong Oracle RAC DBA is the person targeted as an Oracle Exadata DBA because strong technology experience and expertise in what surrounds the database such as storage and networking. That is the same profile as someone that will be a vRAC DBA, with the addition of the vRAC DBA must also be a virtualization expert.

There is a very strong and expanding demand for vRAC DBAs. With VMware controlling over 80% of the virtualization market, the major percentage of Oracle database server and BCA migrations to virtual servers are going to occur on VMware in the next year. If you want to get in on the wave of where the IT industry is going and be in great demand then acquiring the skill set of vRAC DBA should be in your New Years resolutions. Oracle and SQL Server DBAs that know VMware virtualization are going to be at the top of the IT food chain and in great demand in the upcoming years.

The vRAC DBA was pre-announced at the NYOUG event on December 13. The vRAC DBA will be discussed in more detail on January 19, 2012 at the VMUGs in Boston and New York. More detail will follow at the VMware Partner Exchange in February in Las Vegas and at Collaborate 2012 during the IOUG Virtualization track.

Friday, November 4, 2011

Customers not willing to change hypervisors

In attending multiple technical conferences and local user group meetings surrounding virtualization and the cloud, I heard one consistent message. We are not changing hypervisors. This is good news for VMware which dominates the virtualization x86 market. Reasons why customers told me they will not switch off of the VMware hypervisor:

We are already 30 - 40 virtualized on VMware. We are looking at going to 70% virtualization. We'd be crazy to switch hypervisors now as we are about to go to the next level of virtualization.
With VMware leading the market for years, there is more VMware expertise than any other hypervisor. It is too hard to find a lot of expertise with large scale experience in other hypervisors.
No other hypervisor has all the tooling and infrastructure we need for our enterprise. VMware's management and infrastructure is the only proven enterprise solution.
We are running thousands of VMware VMs. We know there are lots of other companies running thousands of VMs. No other hypervisor has the level of architects, engineers and designers that have the experience found in the VMware eco-system.
We tried other hypervisors that were either free or we were given greatly reduced licensing to use. All of the free and low priced hypervisors have a tremendously higher total cost of ownership. All the tooling and management around VMware is required for our business needs.

Wednesday, May 18, 2011

Preparing for VCP: Free iSCSI SAN Software to use with VMware

There are some nice free and open-source iSCSI servers that you can use as your SAN storage with VMware ESXi. These iSCSI servers are also great for getting a free SAN platform for preparing for VCP. Two free SAN software solutions include:

OpenFiler - a free open-source iSCSI server than can be used as your SAN storage for VMware ESXi and can leverage advanced ESXi features.

Supports NFS, SMB (Windows), iSCSI and HTTP file sharing.
Modified version of Linux that provides an iSCSI target for iSCSI initiators.

StarWind ia an iSCSI target that converts any WIndows server into a SAN.

Provides features such as deduplication, snapshots thin provisioning and supports CDP (continuous Data Protection) and VSS.

Wednesday, May 11, 2011

VMForum in New York City

The VMForum in New York City today has been outstanding. Presentations are standing room only, hands-on labs and vendor booth area have been packed all day.

Michelle Bailey - IDC
Michelle Bailey from the IDC gave a good presentation on cloud computing and the direction of virtualization. A few highlights from her presentation:

We are moving from a product oriented environment to a services oriented environment.
Virtualization is the killer app for the data center.

Cloud 2.0 is about who. Virtualization is foundation for cloud computing.

CC2CC - Customer Cloud to Customer Cloud
Blended - Business Cloud to Customer Cloud
Enterprise Cloud

By 2013 there will be 85 million servers and most will be running in a Virtual Machine (VM). Customers using private clouds are running more efficient organizations than traditional environments. Private cloud enviroments are able to manage more servers per administrator compared to traditional IT infrastructures. Private clouds are running 25% more efficient which is point where industry will move there to get this efficiency.

Cost is not the driving force of virtualization. Its the agility, provisioning, flexibility, reduced down time, etc. that are key factors customers are leveraging with virtualization.

The cloud is also a great opportunity for organizations to reevaluate their operations and organization.

Conclusion
Talked to a lot of VMware customers that are moving to a virtualization first policy. VMForum was a very well attended environment and has a lot of positive energy around it. I'd recommend attending one of these events when they come to your city.

Friday, October 22, 2010

Virtualization Abstracts the Hardware Layer

One of the great features of virtualization is the ability to abstract the operating system and the applications from the underlying hardware. A virtual machine (VM) is a software machine that runs an operating system and its environment. By running an operating system and its applications inside of a software machine instead of a physical machine provides a lot of flexibility and speed in deploying operating systems and applications.

Features in VMware software that leverage hardware abstraction include:

vMotion - ability to move a VM and its environment live to another physical host without any down time.
vCenter Converter - ability to move the operating system and environment from a physical machine to a virtual machine running on a different host.
HA - High availability can start up a VM on a different physical server after its original server fails.
Distributed Resource Scheduler (DRS) - Ability to load balance VMs and make decisions when to move a VM from one host to another with no down time.
DRS Cluster - VMs can be spread across multiple host servers. So operating systems and their applications become completely independent of the hardware they run on.
Distributed Power Management (DPM) - Ability to move VMs from one host to another and power down host servers that are not being utilized and then powering the host servers back up when there is sufficient utilization.
Fault Tolerance (FT) - Ability to mirror VM on a separate host, so if the original environment goes down, then the mirrored environment becomes the primary and a new secondary environment is created.

Treating host servers, VMs and Virtual Appliances (multiple VMs running together) as software components in a virtualized data center allows:

VMs and their applications to be moved transparently to different host servers. This not only abstracts the hardware from VMs (operating systems and their applications) but even what hardware VMs run on.

Thursday, August 19, 2010

Preparing for the VMware Certified Professional Exam

The VMware Certified Professional Exam (VCP) is one of the hottest and also one of the hardest certifications to get in the IT industry today. I'm always impressed when I see that 80% of the students in my classes are looking at VCP certification. I am always being asked how to prepare for the VCP certification, so I thought I'd share some perspective.

Ways to Study
It's important you put together a good game plan for studying the test. The key is to find a method that works for you.

A source that I really like is to use Flashcards Deluxe and www.quizlet.com.

Flashcards Deluxe - an app you can download on your iPhone, Blackberry, etc. You can download flash cards from different sites directly into the app. You can also build your own study cards that you can download into the app. Download prebuilt flashcards on vSphere from Quizlet.
www.quizlet.com - a great source for flash cards. There are a number of prebuilt flash cards for studying vSphere 4, like the Configuration Maximums Guide.

There are products like Garage Band on Mac and lots of free software for audio recording. A number of candidates have found success by using their audio recordings while driving to work, being on the bus/train or riding a stationary bike or elliptical.

Things you need to study

Configuration Maximums Guide: You need to understand limits.
VMware ICM 4.1 Class: Unless you are getting a lot of hands on work and mentoring in your job, you've got to take the VMware ICM 4.1 class. There is just to much topic matter to learn on your own.
Additional Books: Mastering VMware vSphere 4, VCP VMware Certified Professional Study Guide , Maximum vSphere and the Sybex VCP Study Guide are excellent complements to the VMware courseware. There are other excellent books out there, go through them and find the one that is right for you. I am not trying to recommend or not recommend any books.
Hands on Practice: You've got to find a way to get hands on practice. You can install the ESXi 4.1 server on a host based system like VMware Fusion (Mac OS) or VMWare Workstation. You're going to want an extra 4GB of memory but that's more cost effective than buying the hardware to create a test environment.
VMware Documentation: You need to use the VMware documentation as reference materials.

Here is the URL you need to use to find the main documentation set to get you started:

http://www.vmware.com/support/pubs/vs_pages/vsp_pubs_esx41_vc41.html
This documentation set has some key guides you need to be looking at: Introduction to VMware vSphere, Getting Started with ESX, Virtual Machine Administrator's Guide, Resource Management Guide and ESX Configuration Guide.

Once you feel you have a solid foundation you can start prepping for the VCP exam. You absolutely need to follow the Exam Blueprint Guide. This guide will help you focus on the right areas.

Avoid Common Test-Taking Mistakes
Don't make common test-taking mistakes on the exam. Candidates typically miss 5 - 10 questions on the test due to not reading the questions carefully. First of all, read the question carefully. Then do the following before selecting an answer(s).

Make sure you understand what type of question it is and what the question is looking for.
Check to see if the question is looking for two, three or multiple answers.
Read all the answers before making your selection.
Eliminate all obvious wrong answers.
Are there words like "all that apply" or "all that do not apply" or "All of the above" or "None of the Above". Don't let the way the question is written lead you astray.

Pearson VUE
When you are ready to take the test, the VMware Certified Professional exam is administered by Pearson VUE. Pearson VUE has over 3500 authorized testing centers worldwide. To register visit www.pearsonvue.com/vmware and select the VCP4 Certification Exam. The exam code is VCP410.

Good luck! :)

Saturday, August 7, 2010

Some Good Blogs on Virtualization

The emphasis on Cloud Computing is increasing significantly. Since it also has tremendous growth in the market, you are seeing the top vendors putting a lot attention on this space. Virtualization plays a key role in Cloud Computing so I wanted to share some articles on Cloud Computing and Virtualization. I will add to this list over time.

VMware Networking Blog