Friday, March 01, 2019

VMware vSphere Memory Hot Add scalability limitation

VMware vSphere Hot Add CPU/Memory feature has specific requirements and limits. To mention some
  • Virtual machines minimum hardware is version 7.
  • It is not compatible with Fault Tolerance
  • vSphere Enterprise Plus license
  • Hot Remove is not supported
  • Hot-Add/Hot-plug must be supported by the Guest operating system (check at http://vmware.com/go/hcl)
  • Guest-OS technical and licensing limitations had to be taken into consideration.
However, it is good to know about another scalability limitation.

VMware has set a maximum value for hot add memory. By default, this value is 16 times the amount of memory assigned to the virtual machine. For example, if the virtual machine memory is 2 GB, the maximum value for hot add memory is 32GB (2x16).

Actually, this is a good safety mechanism and here is the reason for such restriction ...

When hot memory is enabled, the guest operating system uses a huge amount of kernel memory space in the PFN database. Windows operating system does not have dynamic PFN allocation. When adding memory to the virtual machine, to make it visible to the guest operating system, the PFN database needs to be dynamic as Windows lacks this feature.

Do you want to know more about "Page Frame Number (PFN) database"? Read this article.

This topic is documented in VMware KB https://kb.vmware.com/kb/2020846

Now there is another question. Does this limitation apply only to MS Windows or it applies to Linux OS as well? The short answer is yes it applies to Linux as well. However, for Linux OS there is another limitation. If you are running WM with Linux OS having less then 3GB RAM you can change the memory only up to 3GB RAM in total. If you need more. You have to power off VM, increase memory to for example 4 GB RAM and power on again. When you are running linux with more than 3GB you can use hot memory add but again with a limit to increasing it maximally 16 times.

Hope this is informative.

Memory Hot Add related VMware KBs:

1 comment:

Anonymous said...

it's also useful to check vM config for HotPlugMemoryLimit where there are issues with adding memory while hotplug is on.