Showing posts with label scripting. Show all posts
Showing posts with label scripting. Show all posts

Thursday, June 22, 2017

CLI for VMware Virtual Distributed Switch

A few weeks ago I have been asked by one of my customers if VMware Virtual Distributed Switch (aka VDS) supports Cisco like command line interface. The key idea behind was to integrate vSphere switch with open-source tool Network Tracking Database (NetDB) which they use for tracking MAC addresses within their network. I have been told by customer that NetDB can telnet/ssh to Cisco switches and do screen scraping so would not it be cool to have the most popular switch CLI commands for VDS? These commands are

  • show mac-address-table
  • show interface status
The official answer is NO, but wait a minute. Almost anything is possible with VMware API. So my solution is leveraging VMware's vSphere Perl SDK to pull information out of Distributed Virtual Switches. I have prepared PERL script vdscli.pl which currently supports two commands mentioned above. It goes through all VMware Distributed Switches on single vCenter.

Script along with shell wrappers are available on GITHUB here https://github.com/davidpasek/vdscli
See screenshots below to get an idea what script does.

The output of the command
vdscli.pl --server=vc01.home.uw.cz --username readonly --password readonly --cmd show-port-status
looks as depicted in screenshot below.


and output of the command
vdscli.pl --server=vc01.home.uw.cz --username readonly --password readonly --cmd show-mac-address-table

So now we have Perl scripts to get information from VMware Distributed Virtual Switch which is nice, however, we would like to have Interactive CLI to have the same user experience as we have on physical switches CLI, right? For Interactive CLI I have decided to use Python ishell (https://github.com/italorossi/ishell) to emulate Cisco like CLI. To start interactive VDSCLI shell you must have Python with iShell installed and then you can simply run script

./vdscli-ishell.py

which is just a wrapper around vdscli.pl The screenshot of VDSCLI shell is in the figure below

VDSCLI Interactive Shell
And the last step is to allow SSH or Telnet access to VDSCLI shell. It can be very easily done via standard Linux possibility to change a shell for the particular user. The VDSCLI over ssh is depicted on the screenshot below.

VDSCLI Interactive Shell over SSH
To operationalize all these scripts, I would highly encourage you to read my another blog post ...
"CLI for VMware Virtual Distributed Switch - implementation procedure".

Hope somebody else in VMware community will find it useful.

Wednesday, April 06, 2016

ESXi host vCPU/pCPU reporting via PowerCLI to LogInsight

Some time ago I had a discussion with one of my customers how to achieve vCPU/pCPU ratio 1:1 on their Tier 1 cluster. Unfortunately, there is not any out-of-the box vSphere policy to achieve it. You can try to use vSphere HA Cluster admission control with advanced settings to achieve such requirement but it is based on CPU reservations in MHz so it would be tricky settings anyway with some additional risks for example after physical server hardware replacement.

At the end of the day we agreed that the goal can be achieved by Monitoring and Capacity Planning process. One can probably leverage VMware vRealize Operations Manager (aka vROps) or similar monitoring platform but because my customer does not have vROps and I'm not vROps expert I realized there is very simple alternative.

Let's leverage PowerShell/PowerCLI to report vCPU/pCPU ratio of ESXi hosts. 

As you can see in the script below it is pretty easy task to prepare PowerCLI report however the question is how to visualize it and send alerts in case of exceeded threshold.

And that's another simple idea. Why not leverage vRealize LogInsight?

All my readers most probably know what VMware's LogInsight (LI) is but just in case - LI is highly available and scalable syslog server appliance which main business value is an excellent reporting capabilities from unstructured data. I don't want to describe LogInsight in this blog post but another interesting feature of LI besides syslog messages it also accepts JSON messages sent via API. For more details look here.

So the whole solution is conceptually pretty easy. Bellow is the high level process.

  1. PowerCLI : Go through each ESXi and calculates vCPU/pCPU ratio
  2. PowerCLI : Compose a message including vCPU/pCPU ratio together with additional context information like timestamp, cluster name, ESXi name, number of vCPU and pCPU
  3. PowerCLI : Send the message to LogInsight via REST API
  4. LogInsight : Prepare custom analytics and create Dashboard
  5. LogInsight: Create alert to send e-mail message or trigger web hook when threshold is exceeded    
The latest script version is at GITHUB. Below is complete PowerCLI script ...

 #################################  
 # vCenter Server configuration  
 #  
 $vcenter = “vc01.home.uw.cz“  
 $vcenteruser = “readonly“  
 $vcenterpw = “readonly“  
 $loginsight = "192.168.4.51"  
 #################################  
   
 $o = Add-PSSnapin VMware.VimAutomation.Core  
 $o = Set-PowerCLIConfiguration -InvalidCertificateAction Ignore -Confirm:$false  
   
 #################################  
 # Connect to vCenter Server  
 $vc = connect-viserver $vcenter -User $vcenteruser -Password $vcenterpw  
   
 #################################  
 # Send Message to LogInsight  
 function Send-LogInsightMessage ([string]$ip, [string]$message)  
 {  
  $uri = "http://" + $ip + ":9000/api/v1/messages/ingest/1"  
  $content_type = "application/json"  
  $body = '{"messages":[{"text":"'+ $message +' "}]}'  
  $r = Invoke-RestMethod -Uri $uri -ContentType $content_type -Method Post -Body $body  
 }  
   
 #################################  
 # Count vCPU/pCPU Ratio  
 foreach ($esx in (Get-VMHost | Sort-Object Name)) {  
  $pCPUs = $esx.NumCpu  
  $vCPUs = ($esx | get-vm | Measure-Object -Sum NumCPU).Sum  
  $CPU_ratio = $vCPUs / $pCPUs  
  $date = (Get-Date).ToUniversalTime()  
  $cluster_name = get-cluster -VMHost $esx  
   
  $message = "UTC date time: $date Cluster: $cluster_name ESX name: $esx.Name pCPUs: $pCPUs vCPUs: $vCPUs vCPU/pCPU ratio: $CPU_ratio"  
  Write-Output $message  
  Send-LogInsightMessage "192.168.4.51" $message  
 }  

 disconnect-viserver -Server $vc -Force -Confirm:$false  

The PowerCLI script running in my home lab generate messages depicted below ...

 PS C:\Users\Administrator\Documents\scripts> .\Cluster_hosts_vCPU_pCPU_report.ps1  
 UTC date time: 04/06/2016 12:49:32 Cluster: Cluster ESX name: esx01.home.uw.cz.Name pCPUs: 2 vCPUs: 18 vCPU/pCPU ratio: 9  
 UTC date time: 04/06/2016 12:49:33 Cluster: Cluster ESX name: esx02.home.uw.cz.Name pCPUs: 2 vCPUs: 12 vCPU/pCPU ratio: 6  

I use scheduled tasks to send these messages periodically to LogInsight. You can see LogInsight messages in screenshot below ...

LogInsight Interactive Analysis
It is very simple to create dashboard from the analytic ...

LogInsight vCPU/pCPU Dashboard

And the last task is to create alert in LogInsight when vCPU/pCPU ratio is higher then 1 or you can be informed little bit earlier so you can set an alert when ratio is higher then 0.8 ...


Pretty easy, right?
Hope this helps broader VMware community.

And as always, any comments and thoughts are very welcome.

Monday, February 09, 2015

How to change VM Network using linux command line?

Yesterday I read this Cormac's blog post and one of his reader (Philip Orleans) posted following comment ...
Just a personal favor, can you ask from the Vmware managers to enforce parity of functionality between management command line tools in Linux and Windows? It is a shame that the Linux tools are so far behind Power Shell.
Very well known PowerCLI scripting guru and VMware's Product Manager for CLIs Alan Renouf answered very quickly ...
Philip, thanks for the comment on Cormacs site, I am the Product Manager for CLIs at VMware and I can tell you we hear you loud and clear and we have a plans to bring the linux side of the house up to speed at some point in the future.

By the way, that's exactly what I really like on VMware's community and efficient communication with big corporation what VMware already is ...

Philip point out to one practical example where PowerCLI is the the only way how to automate VM network reassign to different network. I absolutely agree that PowerCLI is much simpler and feature rich scripting platform than linux alternatives. That's exactly what linux/*nix oriented vSphere administrators would like to see from VMware. However I don't agree it is not possible to achieve your goal.

Below is the question Philip raised ...
I am very happy that somebody is listening. A few weeks ago, here, I posted a question about how to change network for a VM using only Linux command line tools, and some idiot mocked me, told me to study, when in fact, there is no way, as far as I researched. This can be achieved only with Powershell. Unfortunately, one of my customers is so paranoid, that I am not allowed on premises with a Windows machine. If I did try, I would have to flee to Moscow.

In the history, the first scripting toolkit was targeted to linux administrators. It was Perl SDK nowadays known af vCLI. I used to use Perl SDK to develop several automation projects in the past. I even wrote something like VMware's "Site Recovery Manager" to achieve automated disaster recovery fail over, test, and fail back. To achieve DR testing in isolated network bubble you need to change VM networks, right? In that times I've developed perl function to do it. Based on Philip's question I've realized that it can be still valuable to some folks even the script was developed back in 2009. I spent just two hours with some quick re-factoring and below is single linux command which can be used to change virtual network on particular VM for particular VM Network Adapter.

/usr/lib/vmware-vcli/apps/vm/vmchnet.pl --server 10.10.4.70 --username administrator --vmname test-vm --vnic 2 --network my-test-network
Ok, it's kind of a joke :-) There is single command above but you need the script below to achieve it ...

 #!/usr/bin/perl -w  
 ###############################################################################  
 # Author: David Pasek  
 # Email: david.pasek[at]gmail.com  
 # Blog: http://blog.igics.com  
 #  
 # Created: 01/27/2009  
 # Updated: 02/09/2015  
 #  
 # Abstract:  
 # Reconfigure particular VM Network iAdapter to be in particular network label  
 # in VMware standard virtual Switch. Network label is also known as PortGroup.  
 #  
 # Script requires VMware's PERL SDK (aka vCLI)therefore it must be placed  
 # in apropriate directory tree location to work correctly.  
 # Optimal location is at /usr/lib/vmware-vcli/apps/vm  
 #  
 # Disclaimer: Use this script at your own risk. Author is not responsible  
 # for any impacts of using this script.  
 ###############################################################################  
 use strict;  
 use warnings;  
 use FindBin;  
 use lib "$FindBin::Bin/../";  
 use VMware::VIRuntime;  
 use XML::LibXML;  
 use AppUtil::VMUtil;  
 use AppUtil::XMLInputUtil;  
 use Data::Dumper;  
 $Util::script_version = "1.0";  
 my %opts = (  
   'vmname' => {  
    type => "=s",  
    help => "Name of virtual machine",  
    required => 1,  
   },  
   'vnic' => {  
    type => "=i",  
    help => "VM Network Adapter number - 1, 2, ...",  
    required => 1,  
   },  
   'network' => {  
    type => "=s",  
    help => "Name of new virtual network",  
    required => 1,  
   },  
 );  
 Opts::add_options(%opts);  
 Opts::parse();  
 Opts::validate(\&validate);  
 # connect to the server  
 Util::connect();  
 my $vmname = Opts::get_option('vmname');  
 my $vnic = Opts::get_option('vnic');  
 my $network = Opts::get_option('network');  
 &vm_change_net('vmname' => $vmname,  
         'vnic' => $vnic,  
         'network' => $network);  
 Util::disconnect();  
 exit;  
 sub vm_change_net {  
  my %params = @_;  
  my $vmname =$params{vmname};  
  my $vnic =$params{vnic};  
  my $network = $params{network};  
  my $vm_view;  
  $vm_view = Vim::find_entity_view(view_type => 'VirtualMachine',  
        filter => {'name' => $vmname });  
  if(!defined $vm_view) {  
   print "Cannot find VM: $vmname\n";  
   return(255);  
  }  
  my $devices = $vm_view->config->hardware->device;  
  foreach my $dev (@$devices) { # DEVICE  
   my $device_type = ref($dev);  
   my $device_name = $dev->deviceInfo->label;  
   my $device_key = $dev->key;  
   #print "Device type: $device_type\n";  
   #print "Device name: $device_name\n";  
   #print "Device device key: $device_key\n";  
   if ( $device_name eq "Network adapter $vnic") { # NETWORK ADAPTER  
    print "Device type: $device_type\n";  
    print "Device name: $device_name\n";  
    print "Device key: $device_key\n";  
    # Change network information  
    my $changed_device;  
    my $backing_info = VirtualEthernetCardNetworkBackingInfo->new( deviceName => $network );  
    if ($device_type eq "VirtualPCNet32") {  
     $changed_device = VirtualPCNet32->new(key => $device_key,  
                        backing => $backing_info);  
    }  
    if ($device_type eq "VirtualE1000") {  
     $changed_device = VirtualE1000->new(key => $device_key,  
                       backing => $backing_info);  
    }  
    if ($device_type eq "VirtualVmxnet3") {  
     $changed_device = VirtualVmxnet3->new(key => $device_key,  
                        backing => $backing_info);  
    }  
    my $config_spec_operation;  
    $config_spec_operation = VirtualDeviceConfigSpecOperation->new('edit');  
    my $device_spec =  
    VirtualDeviceConfigSpec->new(operation => $config_spec_operation,  
                   device => $changed_device);  
    my @device_config_specs = ();  
    push(@device_config_specs, $device_spec);  
    my $vmspec = VirtualMachineConfigSpec->new(deviceChange => \@device_config_specs);  
    # RECONFIGURE VM  
    eval {  
     print "Changing Network Adapter $vnic in virtual machine $vmname\n";  
     $vm_view->ReconfigVM( spec => $vmspec );  
     print "Success\n";  
    };  
    if ($@) {  
     print "Reconfiguration failed:\n";  
     print($@);  
    }  
   } # NETWORK ADAPTER - END  
  } # DEVICES - END  
  return;  
 }  
 sub validate {  
   my $valid = 1;  
  return $valid;  
 }  

Script above is limited to standard VMware virtual switch. If you are looking for similar script dealing with VMware distribute virtual switch look here. There is a link to script developed by Javier Viola.

As you can see PERL SDK is not easy for normal vSphere admins and when there are no precooked vCLI commands it is almost useless for someone who doesn't have some programming background. PowerCLI is absolutely another story and as VMware is moving out of Microsoft technologies I would expect some PowerCLI alternative in PERL or Python and based on Alan's comment I believe VMware is already working on some alternative. I'm really looking forward for linux alternative and I'm not alone.

By the way, nowadays there is another possibility to achieve the goal without MS Windows systems. vCenter Orchestrator (or vRealize Orchestrator) however it is another story.

Hope this helps to someone.
 


Wednesday, December 31, 2014

esxcli formatters

I have just read great blog post "Hidden esxcli Command Output Formats You Probably Don’t Know" where the author (Steve Jin) exposed undocumented esxcli options to choose different formatters of esxcli output. Following esxcli formatters are available:

  • xml
  • csv
  • keyvalue

Here is example of one particular esxcli command without formatter.
~ # esxcli system version get
   Product: VMware ESXi
   Version: 5.5.0
   Build: Releasebuild-1746018
   Update: 1
Here is example with csv formatter
~ # esxcli --formatter=csv system version get
Build,Product,Update,Version,
Releasebuild-1746018,VMware ESXi,1,5.5.0,

Here is example with keyvalue formatter
~ # esxcli --formatter=keyvalue system version get
VersionGet.Build.string=Releasebuild-1746018
VersionGet.Product.string=VMware ESXi
VersionGet.Update.string=1
VersionGet.Version.string=5.5.0

It is quite obvious how xml formatter output looks like, right? All these formatters can help with output parsing which can be very helpful for automation scripts. Even more formatters are available when debug option is used. Following esxcli debug formatters are available:

  • python
  • json
  • html
  • table
  • tree
  • simple


Here is example with json formatter
~ # esxcli --debug --formatter=json system version get
{"Build": "Releasebuild-1746018", "Product": "VMware ESXi", "Update": "1", "Version": "5.5.0"}

Conclusion

esxcli formatters might be very handy for those who do automation and need easier parsing of esxcli output. You probably know that esxcli can be also used from PowerCLI so formatters can significantly simplified the script code.   

Wednesday, February 05, 2014

Configure default settings on a VMware virtual distributed switch


Original blog post and full text is here. All credits go to http://kickingwaterbottles.wordpress.com

Here is the PowerCLI script that will set the ‘Teaming and Failover’ defaults on the vDS to work with etherchannel and two active uplinks.

connect-viserver vCenter
$vDSName = “”
$vds = Get-VDSwitch $vDSName
$spec = New-Object VMware.Vim.DVSConfigSpec
$spec.configVersion = $vds.ExtensionData.Config.ConfigVersion

$spec.defaultPortConfig = New-Object VMware.Vim.VMwareDVSPortSetting
$uplinkTeamingPolicy = New-Object VMware.Vim.VmwareUplinkPortTeamingPolicy

# Set load balancing policy to IP hash
$uplinkTeamingPolicy.policy = New-Object VMware.Vim.StringPolicy
$uplinkTeamingPolicy.policy.inherited = $false
$uplinkTeamingPolicy.policy.value = “loadbalance_ip”

# Configure uplinks. If an uplink is not specified, it is placed into the ‘Unused Uplinks’ section.
$uplinkTeamingPolicy.uplinkPortOrder = New-Object VMware.Vim.VMwareUplinkPortOrderPolicy
$uplinkTeamingPolicy.uplinkPortOrder.inherited = $false
$uplinkTeamingPolicy.uplinkPortOrder.activeUplinkPort = New-Object System.String[] (2) # (#) designates the number of uplinks you will be specifying.
$uplinkTeamingPolicy.uplinkPortOrder.activeUplinkPort[0] = “dvUplink1″
$uplinkTeamingPolicy.uplinkPortOrder.activeUplinkPort[1] = “dvUplink2″

# Set notify switches to true
$uplinkTeamingPolicy.notifySwitches = New-Object VMware.Vim.BoolPolicy
$uplinkTeamingPolicy.notifySwitches.inherited = $false
$uplinkTeamingPolicy.notifySwitches.value = $true

# Set to failback to true
$uplinkTeamingPolicy.rollingOrder = New-Object VMware.Vim.BoolPolicy
$uplinkTeamingPolicy.rollingOrder.inherited = $false
$uplinkTeamingPolicy.rollingOrder.value = $true

# Set network failover detection to “link status only”
$uplinkTeamingPolicy.failureCriteria = New-Object VMware.Vim.DVSFailureCriteria
$uplinkTeamingPolicy.failureCriteria.inherited = $false
$uplinkTeamingPolicy.failureCriteria.checkBeacon = New-Object VMware.Vim.BoolPolicy
$uplinkTeamingPolicy.failureCriteria.checkBeacon.inherited = $false
$uplinkTeamingPolicy.failureCriteria.checkBeacon.value = $false

$spec.DefaultPortConfig.UplinkTeamingPolicy = $uplinkTeamingPolicy
$vds.ExtensionData.ReconfigureDvs_Task($spec)
and here is simplified version


$vDSName = “XXX”  ## << dvSwitch name
$vds = Get-VDSwitch $vDSName
$spec = New-Object VMware.Vim.DVSConfigSpec
$spec.configVersion = $vds.ExtensionData.Config.ConfigVersion

$spec.defaultPortConfig = New-Object VMware.Vim.VMwareDVSPortSetting
$uplinkTeamingPolicy =  New-Object VMware.Vim.VmwareUplinkPortTeamingPolicy

# Set load balancing policy to IP hash
$uplinkTeamingPolicy.policy = New-Object VMware.Vim.StringPolicy
$uplinkTeamingPolicy.policy.inherited = $false
$uplinkTeamingPolicy.policy.value = “loadbalance_ip”   ## << Teaming Policy Type

$spec.DefaultPortConfig.UplinkTeamingPolicy = $uplinkTeamingPolicy
$vds.ExtensionData.ReconfigureDvs_Task($spec)
 

Wednesday, November 13, 2013

VMware ESXi vim-cmd Command: A Quick Tutorial

Very nice blog post on www.doublecloud.org ...
Command lines are very important for system administrors when it comes to automation. Although GUIs are more likely (not always as I’ve seen too many bad ones) to be more intuitive and easier to get started with, sooner or later administrators will use command lines more for better productivity. There are a few command line options in VMware ESXi, among which is the vim-cmd ... READ MORE.

Monday, October 28, 2013

VMware vSphere: Script to change Default PSP to Round Robin


Automated way how to set default PSP for particular SATP.
vCLI example:
esxcli --server myESXi --username user1 --password 'my_password' storage nmp satp set --default-psp=VMW_PSP_RR --satp=VMW_SATP_DEFAULT_AA

PowerCLI example:
$esxcli=Get-EsxCli -VMHost
$esxcli.storage.nmp.satp.set($null,"VMW_PSP_RR","VMW_ SATP_DEFAULT_AA")

Please note that for both examples ESX Server needs Reboot to take Effect.

Tuesday, August 20, 2013

Best Practices for Faster vSphere SDK Scripts

Reuben Stump published excellent blog post at http://www.virtuin.com/2012/11/best-practices-for-faster-vsphere-sdk.html about performance optimization of PERL SDK Scripts.

The main takeaway is to minimize the ManagedEntity's Property Set.

So instead of

my $vm_views = Vim::find_entity_views(view_type => "VirtualMachine") ||
  die "Failed to get VirtualMachines: $!";

you have to use

# Fetch all VirtualMachines from SDK, limiting the property set
my $vm_views = Vim::find_entity_views(view_type => "VirtualMachine",
          properties => ['name', 'runtime.host', 'datastore']) ||
  die "Failed to get VirtualMachines: $!";

This small improvement have significant impact on performance because it eliminates big data (SOAP/XML) generation and transfer between vCenter service and the SDK script.

It helped me improve performance of my script from 25 seconds to just 1 second. And the impact is even better for bigger vSphere environment. So my old version of script was almost useless and this simple improvement help me so much.

Thanks Reuben for sharing this information.
 

Monday, April 15, 2013

How to get Managed Object Reference ID ( aka MoRef ) from vSphere

If you've already scripted vSphere infrastructure you probably already know that everything has software representation also known as Managed Object. Each Managed Object has unique identifier referenced as Managed Object ID. Sometimes this Managed Object ID is needed.

In PowerCLI you can get it via following two lines
$VM = Get-VM -Name $VMName 
$VMMoref = $VM.ExtensionData.MoRef.Value
You can also use Perl script leveraging VMware vSphere Perl SDK to get Managed Object ID for particular virtual machine or datastore. If you need MOID for another entity it's pretty easy to slightly change the script below.

Script is developed and tested on vMA (VMware management Assistant) in directory /usr/lib/vmware-vcli/apps/general and script name is getmoid.pl

Here is usage example how to get MOID of datastore called FreeNAS-iSCSI-01:
./getmoid.pl --server --username --password --dsname FreeNAS-iSCSI-01

Manage Object ID: datastore-162

Here is usage example how to get MOID of virtual machine called VMA:
./getmoid.pl --server --username --password --vmname VMA
Manage Object ID: vm-122

Any feedback or comments are welcome.

Wednesday, December 05, 2012

Best Practices for Faster vSphere SDK Scripts

Source at http://www.virtuin.com/2012/11/best-practices-for-faster-vsphere-sdk.html 
The VMware vSphere API is one of the more powerful vendor SDKs available in the Virtualization Ecosystem.  As adoption of VMware vSphere has grown over the years, so has the size of Virtual Infrastructure environments.  In many larger enterprises, the increasing number of VirtualMachines and HostSystems is driving the architectural requirement to deploy multiple vCenter Servers.
In response, the necessity for automation tooling has grown just as quickly.  Automation to create daily reports, perform bulk operations, and aggregate data from large, distributed Virtual Infrastructure environments is a common requirement for managing the increasing virtual sprawl.
In a Virtual Infrastructure comprised of thousands of objects, even a simple script to list all VirtualMachines and their associated HostSystem and Datastores can result in very slow runtime execution.  Developing automation with the following, simple best practices can take orders of magnitude off your vSphere API tool's runtime.

 READ FULL ARTICLE

Friday, April 20, 2012

VMware Perl SDK - SSL Certificate verification

PROBLEM:

[root@MON-PROXY vm]# ./vminfo.pl --url https://10.10.4.70/sdk/vimService --username dpasek -vmname mon_proxy
Enter password:
Server version unavailable at 'https://10.10.4.70:443/sdk/vimService.wsdl' at /usr/lib/perl5/5.8.8/VMware/VICommon.pm line 545, line 1.

SOLUTION:

[root@MON-PROXY vm]# export PERL_LWP_SSL_VERIFY_HOSTNAME=0
[root@MON-PROXY vm]# ./vminfo.pl --url https://10.10.4.70/sdk/vimService --username dpasek -vmname mon_proxy

Thursday, May 14, 2009

How to shutdown windows from linux

If you have samba you can use "net rpc SHUTDOWN -C "some comment here" -f -I x.x.x.x -U user_name%password"