Leveraging VMware LogInsight for VM hardware inventory

There is no doubt that VMware LogInsight is a must for any properly managed vSphere environment. I'm explaining LogInsight benefits to all my customers. The main use case for LogInsight is troubleshooting but there are infinite number of other use cases where LogInsight can help.

During last LogInsight presentation to one of my customers I have got an interesting question if LogInsight can be used for change management. Change management is very broad term but all VM hardware changes should be in log files therefore there is potential to prepare specific interactive analytics to track VM hardware changes.

You can very simply find reconfiguration tasks for particular VMs logged by vCenter or ESXi hosts.

In ESXi logs reconfiguration log entries looks like

 2016-10-01T21:01:27.126Z esx01.home.uw.cz Hostd: info hostd[31C40B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/d22648f3-b9848863/test_cmdb/test_cmdb.vmx opID=7c3fe7f0-976b-4afa-a182-6d9f074a6446-45173-ngc-25-78-fc98 user=vpxuser:UW.CZ\cdave] Send config update invoked  

And in vCenter tasks it looks like

 2016-10-01 21:01:27.434 vc01.home.uw.cz vcenter-server: Reconfigured test_cmdb on esx01.home.uw.cz in home  

The problem is that even we know there were some reconfigurations, there are no VM reconfiguration details like CPU, RAM, disks and NICs. This is the current state in vSphere 6.0 U2 and it would be great if logging of such details would be improved in future vSphere version but this is what it is at the moment.

However, lot of VM hardware details should be in vmware.log which is located in home directory of each VM. The problem is that these logs are placed on VMFS filesystem and not forwarded to syslog so LogInsight cannot be used for log analysis.

Fortunately, there is the blog of the best VMware virtualization blogger William Lam and more specifically his blog post "A Hidden vSphere 5.1 Gem – Forwarding Virtual Machine Logs (vmware.log) to Syslog Part 1". William very nicely explains how to redirect VM logs also to syslog.

You have to change following two advanced settings for each Virtual Machine you would like to redirect VM hardware logs to syslog server.

vmx.log.destination=syslog-and-disk 

vmx.log.syslogID=[YOUR-VM-NAME-OR-OTHER-ID]

And when VM hardware logs are available in LogInsight you can prepare custom analytics to visualize VM hardware changes over time and put it on to dashboard for future use. You can see my dashboard on screenshot below.

VMware LogInsight dashboard - VM Hardware Inventory 

I have exported my VM Hardware Inventory dashboard as LogInsight content pack and shared it here on github so you can download and import it in to your LogInsight and test it.

Content Pack source code from GitHub is embedded below ...
[ Download Content Pack from GitHub ]

And as always, any comments are more then welcome.

How to clear all jobs on DELL Lifecycle Controller via iDRAC

When you have problem with DELL Lifecycle Controller jobs you can delete all jobs by single iDRAC command. This command

racadm -r ip address -u user name -p password jobqueue delete -i JID_CLEARALL_FORCE

deletes all of the jobs plus the orphaned pending and restarts the data manager service on the iDRAC. It will take about 90-120 secs before the iDRAC is able to process another job.

Occasionally the iDRAC may need to be reset after issuing the above command due to other issues outside the job queue processing. In such case you can issue commands in the following order:

• racadm -r ip address -u user name -p password jobqueue delete -i JID_CLEARALL_FORCE
• Wait 120 secs
• racadm –r ip address –u user name –p password racreset

How social media and community sharing help entrprise customers

I'm always happy when someone finds my blog article or shared document useful. Here is one example of recent email communication from one DELL customer who Googled my DELL OME (Open Manage Essentials is basic system management for DELL hardware inforastructure) document explaining network communication flows among DELL Open Manage components.

All personal information are anonymized so customer real name is changed to Mr.Customer. 

Hardware Infrastructure Monitoring Proxy

Every enterprise infrastructure product like server, blade system, storage array, fibre-channel or ethernet switch has some kind of CLI or API management. Lot of products support SNMP but it usually doesn't return everything what CLI/API offers. This project is set of connectors to different enterprise systems like DELL iDRAC and blade Chassis Management Controller, VMware vCenter and/or ESX, DELL Compellent. The framework is universal and other connectors to other systems can be simply developed.

Available connectors:
DELL CMC (racadm)
DELL DRAC (racadm)
General IPMI (ipmitools)
VMware vCLI (vcli)
Compellent Enterprise manager (odbc to MS-SQL)
Compellent Storage Center (CompCU.jar)
Brocade FC Switch (cli over telnet)

... other connectors and sensors can be simply developed. So if you have any need don't hesitate to contact me.

See video introduction at  https://www.youtube.com/watch?v=JRomfnfymlY
Project page: https://sourceforge.net/projects/monitoringproxy/ 
Author: David Pasek

Open Manage Essentials Network Connection (tcp/udp ports)

I was recently engaged to implement DELL Datacenter version of OME (Open Manage Essentials). DELL OME is quite easy and efficient tool for basic DELL hardware management. In other words it is free of charge element system management for DELL Servers, Network and also some Storage elements. It allows you to  do typical administrator tasks like

• Hardware Discovery and Inventory
• Monitor Hardware Status
• Send email Notification or trigger SNMP trap to another system management
• Inventory and System Reporting
• In-band (OMSA) or Out-of-band (DRAC) Server Firmware Management - upgrades and downgrades

It is important to note that DELL OME is not enterprise management system like Altiris, MS System Center, or so. For customers considering integration DELL hardware in to some enterprise management system it is very likely DELL has integration toolkit or management plugin for particular enterprise management system. But that's another story.

DELL OME  is straight forward to install in small environment but it is usually more complex to implement it in bigger enterprise environment where exist firewalls with strict polices. In such environments you have to tightly cooperate with network departments for creating firewall rules allowing communication among OME server and hardware elements.

Unfortunatelly OME User Guide doesn't describe details network connections. There are listed TCP/UDP ports but for firewall rules you need to know detail network flows and flow directions.

That's the reason I have created document "Open Manage Essentials Network Connection and useful information for creating firewall rules" and publish it on slide share here.

Direct link to the document:
http://www.slideshare.net/davidpasek/ome-network-connections-and-firewall-rules-v04

And as always ... any comments are highly appreciated.

OpenManage Integration for VMware vCenter 2.0

OpenManage Integration for VMware vCenter 2.0 is new generation of DELL vCenter Management Plugin targeted as plugin for vSphere 5.5 Web Client.

Looking forward to test it with vSphere 5.5 in my lab.

Configure Force10 S4810 for SNMP

Enable SNMP in Force10 S4810 switches is straight forward. Bellow is configuration sample.

conf

! Enable SNMP for read only access
snmp-server community public ro

! Enable SNMP traps and send it to SNMP receiver 192.168.12.70
snmp-server host 192.168.12.70 version 1
snmp-server enable traps

DELL OpenManage Essentials (OME)

OpenManage Essentials (OME) is a systems management console that provides simple, basic Dell hardware management and is available as a free download.

DELL OME can be downloaded at https://marketing.dell.com/dtc/ome-software?dgc=SM&cid=259733&lid=4682968

Patch 1.2.1 downloadable at
http://www.dell.com/support/drivers/us/en/555/DriverDetails?driverId=P1D4C

For more information look at DELL Tech Center.

DELL Open Manage Essentials 1.2 has been released

Dell OpenManage Essentials is a 'one to many' console used to monitor Dell Enterprise hardware. It can discover, inventory, and monitor the health of Dell Servers, Storage, and network devices. Essentials can also update the drivers and BIOS of your Dell PowerEdge Servers and allow you to run remote tasks. OME can increase system uptime, automate repetitive tasks, and prevent interruption in critical business operations.

It can be downloaded here.

Fixes & Enhancements
Fixes:

1. Multiple defect fixes and performance improvements

Enhancements:

1. Support for Discovery, Inventory and Map View for Dell PowerEdge VRTX devices. 
2. Addition of Microsoft Windows Server 2012 as a supported operating system for the management station.
3. Context sensitive Search functionality. 
4. Ability to configure OpenManage Essentials to send the warranty status of your devices through email at periodic intervals. 
5. Ability to configure OpenManage Essentials to generate a warranty scoreboard based on your preference and display a notification icon in the heading banner when the warranty scoreboard is available.
6. Enhanced support for Dell Compellent, Dell Force10 E-Series and C-Series, Dell PowerConnect 8100 series, Dell PowerVault FS7500, and PowerVault NX3500 devices. 
7. Support for installing OpenManage Essentials on the domain controller.
8. Device Group Permissions portal. 
9. Additional reports: Asset Acquisition Information, Asset Maintenance Information, Asset Support Information, and Licensing Information. 
10. Addition of a device group for Citrix XenServers and Dell PowerEdge C servers in the device tree. 
11. Availability of storage and controller information in the device inventory for the following client systems: Dell OptiPlex, Dell Latitude, and Dell Precision.
12. CLI support for discovery, inventory, status polling, and removal of devices from the device tree. 
13. Availability of sample command line remote tasks for uninstalling OpenManage Server Administrator and applying a server configuration on multiple managed nodes. 
14. Support for SUDO users in Linux for system updates and OMSA deploy tasks. 
15. Display of a notification icon in the heading banner to indicate the availability of a newer version of OpenManage Essentials. 
16. Support for enabling or disabling rebooting after system update for out-of band (iDRAC) system updates.
17. Support for re-running system update and OpenManage Server Administrator (OMSA) deployment tasks.
18. Support for Single Sign-On (SSO) for iDRAC and CMC devices. 
19. Ability to log on as a different user.

PXE Manager for vCenter

PXE Manager for vCenter enables ESXi host state (firmware) management and provisioning. Specifically, it allows:

• Automated provisioning of new ESXi hosts stateless and stateful (no ESX)
• ESXi host state (firmware) backup, restore, and archiving with retention
• ESXi builds repository management (stateless and statefull)
• ESXi Patch management
• Multi vCenter support
• Multi network support with agents (Linux CentOS virtual appliance will be available later)
• Wake on Lan
• Hosts memtest
• vCenter plugin
• Deploy directly to VMware Cloud Director
• Deploy to Cisco UCS blades

TBD - TEST & WRITE REVIEW

Increasing vCenter Server and ESX/ESXi logging levels

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004795

PuppetLabs | Razor: Next-Generation Provisioning

System administrators require the same agility and productivity from their hardware infrastructure that they get from the cloud. In response, Puppet Labs and EMC collaboratively developed Razor, a next-generation physical and virtual hardware provisioning solution. Razor provides you with unique capabilities for managing your hardware infrastructure, including:

• Auto-Discovered Real-Time Inventory Data
• Dynamic Image Selection
• Model-Based Provisioning
• Open APIs and Plug-in Architecture
• Metal-to-Cloud Application Lifecycle Management

Together, Razor and Puppet enable system administrators to automate every phase of the IT infrastructure lifecycle, from bare metal to fully deployed cloud applications.

More at ... https://puppetlabs.com/solutions/next-generation-provisioning/

DELL OMSA LiveCD / LiveDVD

http://linux.dell.com/files/openmanage-contributions/om65-firmware-live/