I recently conducted a quick analysis of a VMware vSphere–based virtual datacenter for a customer, and here’s what I found.
The average monthly electricity consumption of a single vCPU with ~3 GB vRAM is 1.4 kWh, which translates to approximately $0.4
The datacenter of my customer is located in Central Europe, and they pay 0.33 USD for 1 kWh of electricity in a Tier 3 datacenter facility (UPS + cooling included in energy cost).
Here are my questions for the broader worldwide infrastructure community.
- Q1: How much do you pay for electricity in your data center or server facility?
- Q2: What are the statistics of your cluster (CPU, memory, # of VMs, # of vCPUs, # of vRAM)?
- Q3: How much power do your physical servers consume on average?
For vSphere Cluster statistics, you can use the PowerCLI script Get-ClusterDensity. I use it to compare virtual cluster metrics (# of VMs, # of vCPUs) with the actual power consumption of physical servers, which you can obtain from your hardware’s management tools.
If you want to dive deeper into my quick analysis, read on.
Detailed Analysis
CLUSTER01
Cluster has the following specifications
- 10 pServers (Admission control 5+5)
- pServer Specification:
- 2x Intel Xeon Gold 6226R @ 2.90 GHz
- 1,024 GB RAM (16x 64 GB DIMM)
- 632 PowerOn Virtual Machines
- 1526 PowerOn vCPUs
- with 4479 GB of PowerOn vRAM
- 1 vCPU has 3 GB vRAM
Here is power consumption of
- 292 W [ACTIVE]
- 203 W [STANDBY - NO VMs]
- 270 W [ACTIVE]
- 211 W [STANDBY - NO VMs]
- 442 W [ACTIVE]
- 287 W [STANDBY - NO VMs]
- 544 W [ACTIVE]
- 335 W [STANDBY - NO VMs]
- 207 W [STANDBY - NO VMs]
- 320 W [ACTIVE]
Average Electricity Consumption of Physical Server is 0.311 kWh (3111 W / 10 pServers) => 228 kWh/month (0.311 kWh x 24 hours x 30.5 days in month).
Electricity Cost of Physical Server is 75.24 USD/month (228 kWh/month x 0.33 USD)
Cluster's VM and vCPU statistics (averages)
- 63.2 Virtual Machines per Physical Server (632 VMs / 10 pServer)
- 1 VM average electricity consumption is 3.6 kWh/month (228 kWh/month / 63.2 VMs)
- 1 VM average electricity cost is 1.2 USD/month (3.6 x 0.33 USD)
- 152.6 vCPUs per Physical Server (1526 vCPUs / 10 pServers)
- 1 vCPU average electricity consumption is 1.5 kWh/month (228 kWh/month / 152.6 vCPUs)
- 1 vCPU average electricity cost is 0.5 USD/month (1.5 x 0.33 USD)
CLUSTER02
CLUSTER02 has the following specifications
- 24 pServers (Admission control 24+2)
- pServer Specification:
- 2x Intel Xeon Gold 6226R @ 2.90 GHz
- 768 GB RAM (12x 64 GB DIMM)
- 936 PowerOn Virtual Machines
- 3127 PowerOn vCPUs
- with 11661 GB of PowerOn vRAM
- 1 vCPU has 3.7 GB vRAM
Here is power consumption of
- 283 W [ACTIVE]
- 199 W [MAINTENANCE - NO VMs]
- 239 W [ACTIVE]
- 289 W [ACTIVE]
- 273 W [ACTIVE]
- 255 W [ACTIVE]
- 280 W [ACTIVE]
- 273 W [ACTIVE]
- 271 W [ACTIVE]
- 227 W [ACTIVE]
- 235 W [ACTIVE]
- 234 W [ACTIVE]
- 320 W [ACTIVE]
- 211 W [STANDBY - NO VMs]
- 283 W [ACTIVE]
- 286 W [ACTIVE]
- 298 W [ACTIVE]
- 291 W [ACTIVE]
- 255 W [ACTIVE]
- 294 W [ACTIVE]
- 302 W [ACTIVE]
- 273 W [ACTIVE]
- 263 W [ACTIVE]
- 215 W [STANDBY - NO VMs]
Average Electricity Consumption of Physical Server is 0.265 kWh (6349 W / 24 pServers) => 194 kWh/month (0.265 kWh x 24 hours x 30.5 days in month).
Electricity Cost of Physical Server is 64.02 USD/month (194 kWh/month x 0.33 USD)
Cluster's VM and vCPU statistics (averages)
- 63.2 Virtual Machines per Physical Server (632 VMs / 10 pServer)
- 1 VM average electricity consumption is 3.1 kWh/month (194 kWh/month / 63.2 VMs)
- 1 VM average electricity cost is 1 USD/month (3.1 x 0.33 USD)
- 152.6 vCPUs per Physical Server (1526 vCPUs / 10 pServers)
- 1 vCPU average electricity consumption is 1.27 kWh/month (194 kWh/month / 152.6 vCPUs)
- 1 vCPU average electricity cost is 0.41 USD/month (1.27 x 0.33 USD)
Conclusion
In my region (Central Europe), I pay 0.33 USD for electricity in Tier 3 datacenter (UPS + cooling included).
The average monthly electricity consumption for 1 vCPU is 1.4 kWh ((1.5 kWh + 1.27 kWh) / 2) , which translates to approximately 0.5 USD (1.4 kWh * 0.33 USD).
Please leave a comment with your answers to the questions at the beginning of this post, and don’t forget to share it, because sharing is caring.
You forgot to add the power usage for other equipment needed to run the cluster, switches, routers. In a cluster setup we are talking about at least two switches and redundant multipath routers.
ReplyDeleteFirst of all, thanks for sharing!
DeleteIt is good to know that my client's €0.28 per kWh is in the lower part of €0.25 to €0.55 range you have in Romania datacenters.
You are right. There is other infrastructure participating in power consumption. Typically LAN, but also SAN in older architectures. We typically use converged (LAN+SAN) infrastructures nowadays, don't we?
Anyway, I typically prepare a full TCO Model for the Infrastructure Architecture Design I do for my clients. We typically have vServer, vLAN, and vDisk types of products.
Networking and Storage costs outside of the pServer are typically calculated into other vProducts like vLAN and vDisk. vServer product cost is calculated just from pServer cost + virtualization licenses.
For your information, in this particular infrastructure where I did the analysis, they use Cisco UCS, therefore, pServers are in a chassis with two IOMs (I/O Modules). Each I/O module consumes around 300 W. Dozens of these chassis are connected to two UCS Fabric Interconnects (you can treat them as access switches). Each Cisco UCS Fabric Interconnect consumes 360 W. On top of these Fabric Interconnects are Nexus Switches acting as (L2/L3 border routers) for LAN and Fibre Channel Switches for FC Storage Arrays, because within Cisco UCS is FCoE. That's the reason why all these networking and storage components are not calculated for the vServer product, but vLAN and vDisk products.
Hope this makes sense for you.
Any other comments are appreciated.