VMware VVOLs scalability

I'm personally a big fan of VMware Virtual Volumes concept. If you are not familiar with VVOLs check this blog post with the recording of VMworld session and read VMware KB Understanding Virtual Volumes (VVols) in VMware vSphere 6.0

We all know that the devil is always in details. The same is true with VVOLs. VMware prepared the conceptual framework but implementation always depends on storage vendors thus it varies around storage products.

Recently, I have had VVOLs discussion with one of my customers and he was claiming that their particular storage vendor supports a very small number of VVOLs. That discussion inspired me to do some research.

Please, note that numbers below are valid at the moment of writing this article. You should always check current status with your particular storage vendor.

Vendor / Storage Array	Maximum VVOLs / Snapshots or Clones
DELL / Compellent SC 8000	2,000 / TBD
EMC / Unity 300	9,000 / TBD
EMC / Unity 400	9,000 / TBD
EMC / Unity 500	13,500 / TBD
EMC / Unity 600	30,000 / TBD
EMC / VMAX 3	64,000 / TBD
HPE / 3PAR	I have been told that it depends on specific model and firmware versions but it is roughly about 10,000 (the info gathered at November 2018)
Hitachi / VSP G200	2,000 / 100,000
Hitachi / VSP G400	4,000 / 100,000
Hitachi / VSP G600	4,000 / 100,000
Hitachi / VSP G800	16,000 / 100,000
Hitachi / VSP G1000	64,000 / 1,000,000

Numbers above are very important because single VM have minimally 3 VVOLs (home, data, swap) and usually even more (snapshot) or more data disks. If you will assume 10 VVOls for single VM you will end up with just 200 VMs on Dell Compellent or Hitachi VSP G200. On the other hand, EMC Unity 600 would give you up to 3,000 VMs which is not bad and enterprise storage systems (EMC VMAX and Hitachi G1000) would give you up to 6,400 VMs which is IMHO very good scalability.

So as always, it really depends on what storage system do you have or planning to buy.

If you know numbers for other storage systems, please share it in comments below this blog post.

Keywords: vvol, vvols, virtual volumes 

Why use VMware VM virtual disk IOPS limit?

What is VM IOPS limit? Here is explanation from VMware documentation ....

When you allocate storage I/O resources, you can limit the IOPS that are allowed for a virtual machine. By default, these are unlimited. If a virtual machine has more than one virtual disk, you must set the limit on all of its virtual disks. Otherwise, the limit will not be enforced for the virtual machine. In this case, the limit on the virtual machine is the aggregation of the limits for all virtual disks.

I really like this feature because VM vDisk IOPS limit is excellent mechanism to protect physical storage back-end against overloading by some disk intensive VMs and allows to set up some fair user policy for storage performance. Somebody can argue with usage of VM disk share mechanism. Yes, that's of course possible as well and it can be complementary. However, with shares fair user policy your users will get high performance at the beginning when back-end storage has lot of available performance but their performance will decrease later during time when more VMs will use this particular datastore. It means that performance is not predictive and users can complain.

Let's do simple IOPS limit example. You have datastores provisioned on storage pool with automated storage tiering which can serve up to 25,000 IOPS and you have there 100 virtual disks (vDisks). Setting 250 IOPS limit to each virtual disk ensures that if all VMs will use all their IOPSes back-end datastores will not be overloaded. I agree it is very strict limitation and VMs cannot use more IOPS when performance is available in physical storage. But this is business problem and best vDisk limiting policy depends on your business model and company strategy. Below are listed two business models for virtual disk performance limits I've already used on some my vSphere projects:

• Service catalog strategy
• Capacity/performance ratio strategy

Service catalog strategy allows customers (internal or external) increase or decrease vDisk IOPSes as needed and of course pay for it appropriately.

Capacity/performance ratio strategy approach is to calculate ratio between physical storage capacity and performance and use same ratio for vDisks. So if you have storage having 50 TB with 25,000 front-end performance you have 1 GB with 0.5 IOPS. You should define and apply some overbooking ratio because you use shared storage. Let's use ratio 3:1 and we will have 150 IOPSes for 100GB vDisk.

To be honest I prefer service catalog strategy as it is what real world need because each workload is different and service catalog gives better way how to define vDisks to match workloads in your particular environment.

Summary
VM vDisk IOPS limit approach is useful in environments where you want to have guaranteed and long term predictable storage performance (response time) for VMs vDisks. Please, be aware that even this approach is not totally fair because IOPS reality is much more complex and total number of IOPSes on back-end storage is not static number as we use in our example. In real physical storage, the number of front-end IOPSes you can get from storage is function of several parameters like IO size, read/write ratio, RAID type, workload type (sequence or random), cache hit, automated storage algorithm, etc ...

I hope VMware VVOLs will move this approach to the next level in future. However vDisk IOPS limit is technology we can use today.