VCDX #200 The Ultimate Way to VirtualizeBlog of one VMware Infrastructure Designer: ESXi

Showing posts with label ESXi. Show all posts

Sunday, January 28, 2018

ESXi - optical diagnostic information from VMNIC

I have been asked by one customer how to get optical diagnostic information from NIC which is Intel X540 10GbE Controller. NIC was identified by ESXi host as vmnic6 and more info about VNMIC can show command

esxcli network nic get -n vmnic6

 esxcli network nic get -n vmnic6:  
  Advertised Auto Negotiation: true  
   Advertised Link Modes: 1000baseT/Full, 10000baseT/Full  
   Auto Negotiation: true  
   Cable Type: FIBRE  
   Current Message Level: 7  
   Driver Info:  
      Bus Info: 0000:03:00.0  
      Driver: ixgbe  
      Firmware Version: 0x800008fd, 18.0.16  
      Version: 4.4.1-iov  
   Link Detected: true  
   Link Status: Up  
   Name: vmnic6  
   PHYAddress: 0  
   Pause Autonegotiate: false  
   Pause RX: true  
   Pause TX: true  
   Supported Ports: FIBRE  
   Supports Auto Negotiation: true  
   Supports Pause: true  
   Supports Wakeon: false  
   Transceiver: external  
   Virtual Address: 00:50:56:5c:dd:9a  
   Wakeon: None

On ESXi host optics diagnostic is not implemented. I have found that ethtool in linux has the option -m which displays optic diagnostic information but ESXi ethtool do not support it. Monitoring Switch Port SFP/QSFP Hardware Information Using ethtool in Cumulus linux is described here.

During the research, I have found that linux -m option reads these diagnostic information from EEPROM. On ESXi host, we can dump EEPROM by following esxcli command.

 esxcli network nic eeprom dump -l 1048560 -o 0 -n vmnic6

Here is the EEPROM dump from Intel X520 NIC

 Offset Value                        
 ------ ------------------------------------------------  
 0000  61 05 ea ff 00 00 40 00 42 80 60 81 3f 02 55 02   
 0010  5b 02 61 02 69 02 71 02 74 02 77 02 87 02 82 80   
 0020  00 05 ff ff ff ff ff ff ff ff fa fa 2c 06 2d 03   
 0030  40 40 30 40 ff ff ff ff ff ff ff ff ff ff ff ff   
 0040  32 06 ff ff ff ff ff ff ff ff ff ff ff ff 23 06   
 0050  1d 06 51 40 46 00 ff ff f5 ff a0 05 00 80 a7 02   
 0060  20 01 00 40 5a 23 3b 40 21 01 00 40 0d 31 27 03   
 0070  4c 01 ff ff ff ff ff ff ff ff ff ff 02 80 ea 1a   
 0080  fe 01 00 70 06 06 00 7a c0 c0 62 70 02 02 63 70   
 0090  de de 64 70 02 02 66 70 11 11 67 70 90 90 68 70   
 00a0  2c 2c 69 70 78 78 6a 70 40 40 74 70 01 01 06 72   
 00b0  00 00 06 74 00 00 06 76 00 00 06 78 00 00 12 72   
 00c0  e2 e2 12 74 e2 e2 12 76 e2 e2 12 78 e2 e2 13 72   
 00d0  36 36 13 74 36 36 13 76 36 36 13 78 36 36 14 72   
 00e0  98 98 14 74 98 98 14 76 98 98 14 78 98 98 15 72   
 00f0  1c 1c 15 74 1c 1c 15 76 1c 1c 15 78 1c 1c 18 72   
 0100  f2 f2 18 74 f2 f2 18 76 f2 f2 18 78 f2 f2 1b 72   
 0110  0a 0a 1b 74 0a 0a 1b 76 0a 0a 1b 78 0a 0a 1c 72   
 0120  0c 0c 1c 74 0c 0c 1c 76 0c 0c 1c 78 0c 0c 2c 72   
 0130  02 02 2c 74 02 02 2c 76 02 02 2c 78 02 02 5a 72   
 0140  02 02 5a 74 02 02 5a 76 02 02 5a 78 02 02 5b 72   
 0150  02 02 5b 74 02 02 5b 76 02 02 5b 78 02 02 5c 72   
 0160  00 00 5c 74 00 00 5c 76 00 00 5c 78 00 00 5e 72   
 0170  03 03 5e 74 03 03 5e 76 03 03 5e 78 03 03 01 7a   
 0180  98 98 09 7a 22 22 0a 7a 1d 1d 0c 7a 02 02 0f 7a   
 0190  98 98 10 7a 19 19 40 7a 61 61 7f 7a 00 00 80 7a   
 01a0  00 00 84 7a 00 00 85 7a 00 00 86 7a 00 00 87 7a   
 01b0  60 60 88 7a 61 61 89 7a 61 61 8a 7a 61 61 8b 7a   
 01c0  61 61 8c 7a 61 61 8d 7a 61 61 8e 7a 11 11 8f 7a   
 01d0  00 00 90 7a 30 30 91 7a 00 00 92 7a ac ac 93 7a   
 01e0  aa aa 94 7a aa aa 95 7a aa aa 96 7a 00 00 97 7a   
 01f0  00 00 98 7a 11 11 a2 7a 00 00 a3 7a 40 40 a4 7a   
 0200  ad ad a5 7a 60 60 a6 7a a9 a9 a7 7a 5a 5a a8 7a   
 0210  95 95 a9 7a aa aa aa 7a 55 55 ab 7a 01 01 ac 7a   
 0220  f8 f8 ad 7a 03 03 ae 7a 00 00 af 7a 09 09 b0 7a   
 0230  09 09 b1 7a 00 00 b2 7a 00 00 b3 7a 00 00 b4 7a   
 0240  80 80 b5 7a 07 07 b6 7a 40 40 b7 7a 10 10 b8 7a   
 0250  04 04 b9 7a 01 01 bf 7a ff ff c0 7a a3 a3 c1 7a   
 0260  7e 7e 52 72 06 06 52 74 06 06 52 76 06 06 52 78   
 0270  06 06 53 72 1b 1b 53 74 1b 1b 53 76 1b 1b 53 78   
 0280  1b 1b 54 72 1e 1e 54 74 1e 1e 54 76 1e 1e 54 78   
 0290  1e 1e 55 72 53 53 55 74 53 53 55 76 53 53 55 78   
 02a0  53 53 56 72 31 31 56 74 31 31 56 76 31 31 56 78   
 02b0  31 31 57 72 1b 1b 57 74 1b 1b 57 76 1b 1b 57 78   
 02c0  1b 1b 51 72 0c 0c 51 74 0c 0c 51 76 0c 0c 51 78   
 02d0  0c 0c 00 70 07 07 90 00 ff 00 00 00 00 00 00 00   
 02e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 02f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0300  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0310  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0320  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0330  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0340  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0350  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0360  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0370  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0380  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0390  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 03a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 03b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 03c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 03d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 03e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 03f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0400  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0410  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0420  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0430  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0440  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0450  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0460  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0470  00 00 00 00 00 00 00 7a df df 00 00 00 00 15 00   
 0480  3b 3f ba 81 07 00 3f 00 14 00 00 00 07 4e 61 1f   
 0490  28 10 a6 10 02 00 f7 17 15 15 35 00 64 6e 01 00   
 04a0  24 6e 96 72 78 18 e2 ea 07 00 05 00 3f 06 28 15   
 04b0  00 00 04 14 00 00 05 00 3f 16 28 15 00 00 04 14   
 04c0  00 00 07 00 24 6e 96 72 78 18 06 01 82 05 00 00   
 04d0  01 00 07 00 24 6e 96 72 78 1a 06 01 82 05 00 00   
 04e0  01 00 02 00 40 0f 00 00 02 00 40 0f 00 00 0f 00   
 04f0  44 04 00 00 80 00 00 03 ff ff ff ff 00 03 ff ff   
 0500  ff ff 00 03 ff ff ff ff 00 03 ff ff ff ff 0f 00   
 0510  44 04 00 00 80 00 00 03 ff ff ff ff 00 03 ff ff   
 0520  ff ff 00 03 ff ff ff ff 00 03 ff ff ff ff 10 3d   
 0530  20 00 12 ff c6 c4 00 00 05 03 00 03 00 00 00 c8   
 0540  00 00 00 00 00 00 00 00 00 00 a2 02 00 02 82 15   
 0550  00 58 35 34 30 20 31 30 47 62 45 20 43 6f 6e 74   
 0560  72 6f 6c 6c 65 72 90 79 00 56 30 0a 46 46 56 31   
 0570  38 2e 30 2e 31 36 50 4e 06 47 31 34 38 34 33 4d   
 0580  4e 04 31 30 32 38 56 31 12 44 53 56 31 30 32 38   
 0590  56 50 44 52 2e 56 45 52 31 2e 30 56 33 06 44 54   
 05a0  49 4e 49 43 56 34 17 44 43 4d 31 30 30 31 30 30   
 05b0  38 37 44 35 32 31 30 31 30 30 38 37 44 35 56 35   
 05c0  04 4e 50 59 32 56 36 04 50 4d 54 31 56 37 0d 4e   
 05d0  4d 56 49 6e 74 65 6c 20 43 6f 72 70 52 56 03 39   
 05e0  00 00 78 00 ff ff ff ff ff ff ff ff ff ff ff ff   
 05f0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0600  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0610  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0620  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0630  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0640  ff ff ff ff ff ff ff ff ff ff ff ff ff ff 24 6e   
 0650  96 72 78 18 24 6e 96 72 78 1a 69 53 e0 05 01 00   
 0660  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0670  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0680  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0690  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 06a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 06b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 06c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 06d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 06e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 06f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0700  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0710  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0720  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0730  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0740  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0750  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0760  00 12 10 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0770  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0780  00 00 03 3d 00 00 00 00 00 00 00 00 00 00 00 00   
 0790  00 00 00 00 00 00 bc 0c 00 00 00 00 00 00 00 00   
 07a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 07b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 07c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 07d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 07e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 07f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0800  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0810  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0820  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0830  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0840  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0850  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0860  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0870  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0880  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0890  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 08a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 08b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 08c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 08d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 08e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 08f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0900  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0910  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0920  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0930  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0940  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0950  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0960  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0970  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0980  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0990  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 09a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 09b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 09c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 09d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 3d   
 09e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 09f0  00 00 bc 0c 00 00 00 00 00 00 00 00 00 00 00 00   
 0a00  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0a10  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0a20  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0a30  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0a40  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0a50  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0a60  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0a70  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0a80  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0a90  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0aa0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0ab0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0ac0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0ad0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0ae0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0af0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0b00  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0b10  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0b20  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0b30  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0b40  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0b50  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0b60  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0b70  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0b80  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0b90  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0ba0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0bb0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0bc0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0bd0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0be0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0bf0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0c00  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0c10  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0c20  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
 0c30  00 00 00 00 00 00 00 00 00 00 24 6e 96 72 78 19   
 0c40  24 6e 96 72 78 1b 03 00 24 6e 96 72 78 19 24 6e   
 0c50  96 72 78 1b 00 20 01 20 06 00 35 48 30 34 39 34   
 0c60  30 2d 35 30 56 4c 01 08 00 00 00 00 ff ff ff ff   
 0c70  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0c80  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0c90  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0ca0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0cb0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0cc0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0cd0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0ce0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0cf0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0d00  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0d10  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0d20  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0d30  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0d40  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0d50  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0d60  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0d70  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0d80  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0d90  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0da0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0db0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0dc0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0dd0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0de0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0df0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0e00  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0e10  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0e20  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0e30  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0e40  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0e50  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0e60  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0e70  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0e80  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0e90  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0ea0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0eb0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0ec0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0ed0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0ee0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0ef0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0f00  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0f10  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0f20  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0f30  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0f40  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0f50  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0f60  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0f70  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0f80  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0f90  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0fa0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0fb0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0fc0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0fd0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0fe0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
 0ff0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

Memory maps are described in specification SFF-8472 available on multiple places around the Internet.

Digital Diagnostic Monitoring Interface for SFP and SFP+ Optical Transceivers at https://www.finisar.com/sites/default/files/resources/an-2030_ddmi_for_sfp_rev_e2-20140404_updated.pdf
SFF-8472 Specification for Diagnostic Monitoring Interface for Optical Transceivers is available at http://www.10gtek.com/templates/wzten/pdf/SFF-8472-(Diagnostic%20Monitoring%20Interface).pdf

Two 256 bytes memory maps are described in SFF-8472 Specification but the problem is how to locate these maps in EEPROM dump which has, in this particular case (Intel X520) 4,080 bytes.

I have prepared simple Perl script to convert hex numbers into ASCII characters and look what text is available in the dump. Perl script is here

 #!/usr/bin/perl  
 print "EEPROM view\n";  
 my $filename = 'eeprom.dump';  
 open(my $fh, '<', $filename) or die "Could not open file '$filename' $!";  
 $line = 0;  
 while (my $row = <$fh>) {  
  $line++;  
  chomp $row;  
  if ($line>2) {  
   print "$row\n";  
   $data_str = substr($row, 7, 48);  
   @data = split(" ",$data_str);  
   print "    ";  
   foreach $b_str (@data) {  
    $b = hex $b_str;  
    $char = chr($b);  
    if ( ($b>=32) and ($b<=126) ) {  
     print $char;  
     print " ";  
    } else {  
     print "- ";  
    }  
   }   
   print "\n";  
  }   
 }

... and dump with ASCII translation is here ...

 EEPROM view  
 0000  61 05 ea ff 00 00 40 00 42 80 60 81 3f 02 55 02   
     a - - - - - @ - B - ` - ? - U -   
 0010  5b 02 61 02 69 02 71 02 74 02 77 02 87 02 82 80   
     [ - a - i - q - t - w - - - - -   
 0020  00 05 ff ff ff ff ff ff ff ff fa fa 2c 06 2d 03   
     - - - - - - - - - - - - , - - -   
 0030  40 40 30 40 ff ff ff ff ff ff ff ff ff ff ff ff   
     @ @ 0 @ - - - - - - - - - - - -   
 0040  32 06 ff ff ff ff ff ff ff ff ff ff ff ff 23 06   
     2 - - - - - - - - - - - - - # -   
 0050  1d 06 51 40 46 00 ff ff f5 ff a0 05 00 80 a7 02   
     - - Q @ F - - - - - - - - - - -   
 0060  20 01 00 40 5a 23 3b 40 21 01 00 40 0d 31 27 03   
       - - @ Z # ; @ ! - - @ - 1 ' -   
 0070  4c 01 ff ff ff ff ff ff ff ff ff ff 02 80 ea 1a   
     L - - - - - - - - - - - - - - -   
 0080  fe 01 00 70 06 06 00 7a c0 c0 62 70 02 02 63 70   
     - - - p - - - z - - b p - - c p   
 0090  de de 64 70 02 02 66 70 11 11 67 70 90 90 68 70   
     - - d p - - f p - - g p - - h p   
 00a0  2c 2c 69 70 78 78 6a 70 40 40 74 70 01 01 06 72   
     , , i p x x j p @ @ t p - - - r   
 00b0  00 00 06 74 00 00 06 76 00 00 06 78 00 00 12 72   
     - - - t - - - v - - - x - - - r   
 00c0  e2 e2 12 74 e2 e2 12 76 e2 e2 12 78 e2 e2 13 72   
     - - - t - - - v - - - x - - - r   
 00d0  36 36 13 74 36 36 13 76 36 36 13 78 36 36 14 72   
     6 6 - t 6 6 - v 6 6 - x 6 6 - r   
 00e0  98 98 14 74 98 98 14 76 98 98 14 78 98 98 15 72   
     - - - t - - - v - - - x - - - r   
 00f0  1c 1c 15 74 1c 1c 15 76 1c 1c 15 78 1c 1c 18 72   
     - - - t - - - v - - - x - - - r   
 0100  f2 f2 18 74 f2 f2 18 76 f2 f2 18 78 f2 f2 1b 72   
     - - - t - - - v - - - x - - - r   
 0110  0a 0a 1b 74 0a 0a 1b 76 0a 0a 1b 78 0a 0a 1c 72   
     - - - t - - - v - - - x - - - r   
 0120  0c 0c 1c 74 0c 0c 1c 76 0c 0c 1c 78 0c 0c 2c 72   
     - - - t - - - v - - - x - - , r   
 0130  02 02 2c 74 02 02 2c 76 02 02 2c 78 02 02 5a 72   
     - - , t - - , v - - , x - - Z r   
 0140  02 02 5a 74 02 02 5a 76 02 02 5a 78 02 02 5b 72   
     - - Z t - - Z v - - Z x - - [ r   
 0150  02 02 5b 74 02 02 5b 76 02 02 5b 78 02 02 5c 72   
     - - [ t - - [ v - - [ x - - \ r   
 0160  00 00 5c 74 00 00 5c 76 00 00 5c 78 00 00 5e 72   
     - - \ t - - \ v - - \ x - - ^ r   
 0170  03 03 5e 74 03 03 5e 76 03 03 5e 78 03 03 01 7a   
     - - ^ t - - ^ v - - ^ x - - - z   
 0180  98 98 09 7a 22 22 0a 7a 1d 1d 0c 7a 02 02 0f 7a   
     - - - z " " - z - - - z - - - z   
 0190  98 98 10 7a 19 19 40 7a 61 61 7f 7a 00 00 80 7a   
     - - - z - - @ z a a - z - - - z   
 01a0  00 00 84 7a 00 00 85 7a 00 00 86 7a 00 00 87 7a   
     - - - z - - - z - - - z - - - z   
 01b0  60 60 88 7a 61 61 89 7a 61 61 8a 7a 61 61 8b 7a   
     ` ` - z a a - z a a - z a a - z   
 01c0  61 61 8c 7a 61 61 8d 7a 61 61 8e 7a 11 11 8f 7a   
     a a - z a a - z a a - z - - - z   
 01d0  00 00 90 7a 30 30 91 7a 00 00 92 7a ac ac 93 7a   
     - - - z 0 0 - z - - - z - - - z   
 01e0  aa aa 94 7a aa aa 95 7a aa aa 96 7a 00 00 97 7a   
     - - - z - - - z - - - z - - - z   
 01f0  00 00 98 7a 11 11 a2 7a 00 00 a3 7a 40 40 a4 7a   
     - - - z - - - z - - - z @ @ - z   
 0200  ad ad a5 7a 60 60 a6 7a a9 a9 a7 7a 5a 5a a8 7a   
     - - - z ` ` - z - - - z Z Z - z   
 0210  95 95 a9 7a aa aa aa 7a 55 55 ab 7a 01 01 ac 7a   
     - - - z - - - z U U - z - - - z   
 0220  f8 f8 ad 7a 03 03 ae 7a 00 00 af 7a 09 09 b0 7a   
     - - - z - - - z - - - z - - - z   
 0230  09 09 b1 7a 00 00 b2 7a 00 00 b3 7a 00 00 b4 7a   
     - - - z - - - z - - - z - - - z   
 0240  80 80 b5 7a 07 07 b6 7a 40 40 b7 7a 10 10 b8 7a   
     - - - z - - - z @ @ - z - - - z   
 0250  04 04 b9 7a 01 01 bf 7a ff ff c0 7a a3 a3 c1 7a   
     - - - z - - - z - - - z - - - z   
 0260  7e 7e 52 72 06 06 52 74 06 06 52 76 06 06 52 78   
     ~ ~ R r - - R t - - R v - - R x   
 0270  06 06 53 72 1b 1b 53 74 1b 1b 53 76 1b 1b 53 78   
     - - S r - - S t - - S v - - S x   
 0280  1b 1b 54 72 1e 1e 54 74 1e 1e 54 76 1e 1e 54 78   
     - - T r - - T t - - T v - - T x   
 0290  1e 1e 55 72 53 53 55 74 53 53 55 76 53 53 55 78   
     - - U r S S U t S S U v S S U x   
 02a0  53 53 56 72 31 31 56 74 31 31 56 76 31 31 56 78   
     S S V r 1 1 V t 1 1 V v 1 1 V x   
 02b0  31 31 57 72 1b 1b 57 74 1b 1b 57 76 1b 1b 57 78   
     1 1 W r - - W t - - W v - - W x   
 02c0  1b 1b 51 72 0c 0c 51 74 0c 0c 51 76 0c 0c 51 78   
     - - Q r - - Q t - - Q v - - Q x   
 02d0  0c 0c 00 70 07 07 90 00 ff 00 00 00 00 00 00 00   
     - - - p - - - - - - - - - - - -   
 02e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 02f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0300  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0310  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0320  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0330  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0340  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0350  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0360  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0370  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0380  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0390  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 03a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 03b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 03c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 03d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 03e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 03f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0400  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0410  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0420  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0430  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0440  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0450  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0460  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0470  00 00 00 00 00 00 00 7a df df 00 00 00 00 15 00   
     - - - - - - - z - - - - - - - -   
 0480  3b 3f ba 81 07 00 3f 00 14 00 00 00 07 4e 61 1f   
     ; ? - - - - ? - - - - - - N a -   
 0490  28 10 a6 10 02 00 f7 17 15 15 35 00 64 6e 01 00   
     ( - - - - - - - - - 5 - d n - -   
 04a0  24 6e 96 72 78 18 e2 ea 07 00 05 00 3f 06 28 15   
     $ n - r x - - - - - - - ? - ( -   
 04b0  00 00 04 14 00 00 05 00 3f 16 28 15 00 00 04 14   
     - - - - - - - - ? - ( - - - - -   
 04c0  00 00 07 00 24 6e 96 72 78 18 06 01 82 05 00 00   
     - - - - $ n - r x - - - - - - -   
 04d0  01 00 07 00 24 6e 96 72 78 1a 06 01 82 05 00 00   
     - - - - $ n - r x - - - - - - -   
 04e0  01 00 02 00 40 0f 00 00 02 00 40 0f 00 00 0f 00   
     - - - - @ - - - - - @ - - - - -   
 04f0  44 04 00 00 80 00 00 03 ff ff ff ff 00 03 ff ff   
     D - - - - - - - - - - - - - - -   
 0500  ff ff 00 03 ff ff ff ff 00 03 ff ff ff ff 0f 00   
     - - - - - - - - - - - - - - - -   
 0510  44 04 00 00 80 00 00 03 ff ff ff ff 00 03 ff ff   
     D - - - - - - - - - - - - - - -   
 0520  ff ff 00 03 ff ff ff ff 00 03 ff ff ff ff 10 3d   
     - - - - - - - - - - - - - - - =   
 0530  20 00 12 ff c6 c4 00 00 05 03 00 03 00 00 00 c8   
       - - - - - - - - - - - - - - -   
 0540  00 00 00 00 00 00 00 00 00 00 a2 02 00 02 82 15   
     - - - - - - - - - - - - - - - -   
 0550  00 58 35 34 30 20 31 30 47 62 45 20 43 6f 6e 74   
     - X 5 4 0   1 0 G b E   C o n t   
 0560  72 6f 6c 6c 65 72 90 79 00 56 30 0a 46 46 56 31   
     r o l l e r - y - V 0 - F F V 1   
 0570  38 2e 30 2e 31 36 50 4e 06 47 31 34 38 34 33 4d   
     8 . 0 . 1 6 P N - G 1 4 8 4 3 M   
 0580  4e 04 31 30 32 38 56 31 12 44 53 56 31 30 32 38   
     N - 1 0 2 8 V 1 - D S V 1 0 2 8   
 0590  56 50 44 52 2e 56 45 52 31 2e 30 56 33 06 44 54   
     V P D R . V E R 1 . 0 V 3 - D T   
 05a0  49 4e 49 43 56 34 17 44 43 4d 31 30 30 31 30 30   
     I N I C V 4 - D C M 1 0 0 1 0 0   
 05b0  38 37 44 35 32 31 30 31 30 30 38 37 44 35 56 35   
     8 7 D 5 2 1 0 1 0 0 8 7 D 5 V 5   
 05c0  04 4e 50 59 32 56 36 04 50 4d 54 31 56 37 0d 4e   
     - N P Y 2 V 6 - P M T 1 V 7 - N   
 05d0  4d 56 49 6e 74 65 6c 20 43 6f 72 70 52 56 03 39   
     M V I n t e l   C o r p R V - 9   
 05e0  00 00 78 00 ff ff ff ff ff ff ff ff ff ff ff ff   
     - - x - - - - - - - - - - - - -   
 05f0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0600  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0610  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0620  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0630  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0640  ff ff ff ff ff ff ff ff ff ff ff ff ff ff 24 6e   
     - - - - - - - - - - - - - - $ n   
 0650  96 72 78 18 24 6e 96 72 78 1a 69 53 e0 05 01 00   
     - r x - $ n - r x - i S - - - -   
 0660  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0670  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0680  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0690  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 06a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 06b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 06c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 06d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 06e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 06f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0700  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0710  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0720  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0730  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0740  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0750  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0760  00 12 10 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0770  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0780  00 00 03 3d 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - = - - - - - - - - - - - -   
 0790  00 00 00 00 00 00 bc 0c 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 07a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 07b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 07c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 07d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 07e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 07f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0800  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0810  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0820  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0830  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0840  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0850  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0860  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0870  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0880  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0890  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 08a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 08b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 08c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 08d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 08e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 08f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0900  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0910  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0920  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0930  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0940  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0950  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0960  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0970  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0980  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0990  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 09a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 09b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 09c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 09d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 3d   
     - - - - - - - - - - - - - - - =   
 09e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 09f0  00 00 bc 0c 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0a00  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0a10  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0a20  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0a30  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0a40  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0a50  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0a60  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0a70  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0a80  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0a90  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0aa0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0ab0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0ac0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0ad0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0ae0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0af0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0b00  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0b10  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0b20  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0b30  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0b40  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0b50  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0b60  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0b70  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0b80  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0b90  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0ba0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0bb0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0bc0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0bd0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0be0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0bf0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0c00  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0c10  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0c20  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   
     - - - - - - - - - - - - - - - -   
 0c30  00 00 00 00 00 00 00 00 00 00 24 6e 96 72 78 19   
     - - - - - - - - - - $ n - r x -   
 0c40  24 6e 96 72 78 1b 03 00 24 6e 96 72 78 19 24 6e   
     $ n - r x - - - $ n - r x - $ n   
 0c50  96 72 78 1b 00 20 01 20 06 00 35 48 30 34 39 34   
     - r x - -   -   - - 5 H 0 4 9 4   
 0c60  30 2d 35 30 56 4c 01 08 00 00 00 00 ff ff ff ff   
     0 - 5 0 V L - - - - - - - - - -   
 0c70  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0c80  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0c90  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0ca0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0cb0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0cc0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0cd0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0ce0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0cf0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0d00  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0d10  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0d20  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0d30  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0d40  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0d50  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0d60  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0d70  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0d80  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0d90  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0da0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0db0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0dc0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0dd0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0de0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0df0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0e00  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0e10  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0e20  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0e30  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0e40  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0e50  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0e60  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0e70  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0e80  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0e90  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0ea0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0eb0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0ec0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0ed0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0ee0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0ef0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0f00  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0f10  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0f20  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0f30  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0f40  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0f50  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0f60  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0f70  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0f80  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0f90  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0fa0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0fb0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0fc0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0fd0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0fe0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -   
 0ff0  ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   
     - - - - - - - - - - - - - - - -

The most interesting part of the dump from text (ASCII) perspective is the part between 0550h and 05E0h.

  0550 00 58 35 34 30 20 31 30 47 62 45 20 43 6f 6e 74  
    - X 5 4 0  1 0 G b E  C o n t  
  0560 72 6f 6c 6c 65 72 90 79 00 56 30 0a 46 46 56 31  
    r o l l e r - y - V 0 - F F V 1    
  0570 38 2e 30 2e 31 36 50 4e 06 47 31 34 38 34 33 4d    
    8 . 0 . 1 6 P N - G 1 4 8 4 3 M    
  0580 4e 04 31 30 32 38 56 31 12 44 53 56 31 30 32 38    
    N - 1 0 2 8 V 1 - D S V 1 0 2 8    
  0590 56 50 44 52 2e 56 45 52 31 2e 30 56 33 06 44 54  
    V P D R . V E R 1 . 0 V 3 - D T  
  05a0 49 4e 49 43 56 34 17 44 43 4d 31 30 30 31 30 30  
    I N I C V 4 - D C M 1 0 0 1 0 0  
  05b0 38 37 44 35 32 31 30 31 30 30 38 37 44 35 56 35   
    8 7 D 5 2 1 0 1 0 0 8 7 D 5 V 5  
  05c0 04 4e 50 59 32 56 36 04 50 4d 54 31 56 37 0d 4e    
    - N P Y 2 V 6 - P M T 1 V 7 - N    
  05d0 4d 56 49 6e 74 65 6c 20 43 6f 72 70 52 56 03 39    
    M V I n t e l  C o r p R V - 9    
  05e0 00 00 78 00 ff ff ff ff ff ff ff ff ff ff ff ff  
    - - x - - - - - - - - - - - - -

There are adapter model and vendor name but it seems to me that it does not fit into the memory maps defined in the SFF-8472. Therefore I do not know how to find optic (SFP) diagnostic values in EEPROM memory dump.

If you have any idea if I am in the right track and how to continue, please let me know.

Monday, August 14, 2017

Remote text based console to ESXi over IPMI SOL

I have just bought another server into my home lab. I already have 6 Intel NUCs but a lot of RAM is needed for full VMware SDDC with all products like LogInsight, vROps, vRNI, vRA, vRO, ... but that's another story.

Anyway, I have decided to buy used Dell rack server (PowerEdge R810) with 256 GB RAM mainly because of the amount of RAM but also because of all Dell servers older than 9 Generation support IPMI which is very useful. The server can be remotely managed (power on, power off, etc.) over IPMI and it also supports SOL which stands for Serial-over-LAN for server consoles. IPMI SOL is an inexpensive alternative to the iDRAC enterprise virtual console.

You can read more about IPMI on links below

So, if you will follow instructions on links above, you will be able to use IMPI SOL to see and manage server during the boot process and change for example BIOS settings. I have tested it and it works like a charm. You see the booting progress, you can go to the BIOS and change anything how you want. Console redirection works and the keyboard can be used to control the server during POST. However, after the server POST phase and boot loading of ESXi, the ESXi console was not, unfortunately, redirected to SOL. I think it is because ESXi DCUI is not pure text based console. Instead, it is a graphics mode simulating text mode. A graphics mode consoles cannot be, for obvious reasons, transferred over IPMI SOL.

So there is another possibility. ESXi Direct Console (aka DCUI) can be redirected to a Serial Port. The setup procedure is nicely described in the documentation here. It is done via ESXi host advanced setting "VMkernel.Boot.tty2Port" to the value "com2". It is worth to mention that server console redirection and ESXi DCUI redirection cannot be done on the same time for obvious reasons. So I have unconfigured server console redirection and configured ESXi DCUI redirection. It worked great, but the keyboard was not working. It is pretty useless to see ESXi DCUI without the possibility to use it, right? To be honest, I do not know why my keyboard did not work over IPMI SOL.

So what is the conclusion? Unfortunately, I have hit another AHA effect ...

"Aha, IPMI SOL will not help me too much with remote access to ESXi DCUI console."

And as always, any feedback or tips and tricks are more than welcome as comments to this blog post.

Update: I have just found and bought very cheap iDRAC Enterprise Remote Access Card on Ebay, which supports remote virtual console and media. So, it is hardware workaround to my software problem :-)

iDrac6 on Ebay

Sunday, April 02, 2017

ESXi Host Power Management

I have just listened to Qasim Ali's VMworld session "INF8465 - Extreme Performance Series: Power Management's Impact on Performance" about ESXi Host Power Management (P-States, C-States, TurboMode and more) and here are his general recommendations

Configure BIOS to allow ESXi host the most flexibility in using power management features offered by the hardware
Select "OS Control mode", "Performace per Watt", or equivalent
Enable everything P-States, C-States and Turbo mode
To achieve the best performance per watt for most workloads, leave the power policy at default which is "Balanced"
For applications that require maximum performance, switch to "High Performance" from within ESXi host

Ali's VMworld session linked above is really worth to watch. I encourage you to watch it by yourself.

Wednesday, January 11, 2017

Using esxtop to identify storage performance issues for ESX / ESXi

ESXi performance are exposing to administrators through vSphere Clients. You can see real-time performance statistics which are collected in 5 minute intervals where each interval consists of fifteen 20 seconds samples. It is obvious that 20 second sample is pretty large for storage performance where we are working in mili or even micro second scale.

20 seconds contains 20,000 milliseconds

Let's be clear here, we will never have full visibility but smaller monitoring sample will give as better clue what is really happening inside the system. It is similar to microscope device.

The smallest monitoring samples can be achieved by ESXi utility ESXTOP. The default esxtop delay between monitoring points (sample) is 5 seconds. However, it can be lowered up to 2 seconds by parameter -d 2

For real analytics the esxtop data must be exprted to external file. In esxtop terminology it is batch mode and it is achieved by parameter -b

Another important factor is what statistics (metrics) we are going to collect. The best is to collect all statistics because during performance analytics you have to correlate multiple values against each other. It is achieved by parameter -a

And last parameter is -n which defines how many iterations you want to perform in batch mode. So in example below we will have 30 iterations with delay between each other 2 seconds. So we will do total monitoring for 60 seconds.

esxtop -b -a -d 2 -n 30 > esxtop-data.csv

For all esxtop parameters see screenshot below.

 [root@esx11:~] esxtop -h  
 usage: esxtop [-h] [-v] [-b] [-l] [-s] [-a] [-c config file] [-R vm-support-dir-path]   
         [-d delay] [-n iterations]  
        [-export-entity entity-file] [-import-entity entity-file]   
        -h prints this help menu.  
        -v prints version.  
        -b enables batch mode.  
        -l locks the esxtop objects to those available in the first snapshot.  
        -s enables secure mode.  
        -a show all statistics.  
        -c sets the esxtop configuration file, which by default is .esxtop60rc  
        -R enables replay mode.  
        -d sets the delay between updates in seconds.  
        -n runs esxtop for only n iterations. Use "-n infinity" to run esxtop forever.  
        -----Experimental Features-------------  
        -export-entity writes the entity ids into a file, which can be modified  
         to select interesting entities.  
        -import-entity reads the file of selected entities. If this opion   
         is used, esxtop only shows the data for the selected entities.

It is important to know, that esxtop will give you significantly more statistics you can see in vSphere Client level. That's another important benefit of esxtop. But each benefit has also some drawbacks or impact. The impact is, that single esxtop output line can have several thousands statistic counters. For example ESXi 6.0 host with just 2 running VMs in my home lab has 27,314 counters. My customer's product ESXi host has over 330,000 counters! So the output file can be pretty large in case you run it for 24 hours. Count on it.

In the file are very interesting counters. Following counters for physical disk devices are the most interesting

### Reponse times
Average Guest MilliSec/Command
Average Kernel MilliSec/Command
Average Queue MilliSec/Command
Average Queue MilliSec/Read
Average Driver MilliSec/Command
Average Driver MilliSec/Write
### Queue
Adapter Q Depth
### IOPS
Reads/sec
Writes/sec
Commands/sec
### MB/s
MBytes Read/sec
MBytes Written/sec"
### Split commands
Split Commands/sec
### SCSI Reservations
Reserves/sec
Failed Reserves/sec
Conflicts/sec
### Failures
Failed Commands/sec
Failed Reads/sec
Failed Writes/sec
Failed Bytes Read/sec
Failed Bytes Written/sec
Aborts/sec
Resets/sec

Some of above counters are not available in vSphere Client but the big benefit is that esxtop will give you data in 2 second interval which is much better granularity.

I hear your questions - So what now? How to analyze esxtop output file?
Well, you can replay it back in esxtop or you can use any of following tools

VisualEsxtop
perfmon
excel
esxplot

To be honest, none of tools above fulfilled my requirements therefore I'm writing my own Python script for esxtop output analysis.

The Python script is available on github here.

Tuesday, December 20, 2016

ESXi sched-stats command

Frank Denneman has shared on twitter very interesting ESXi command to show CPU scheduling statistics and information.

@FrankDenneman tweet

There are not so much information about this command so one have to rely on command help ...

[root@esx01:~] sched-stats -h
Usage:
-c : use vsi-cache instead of live kernel
-t : specify the output type from the following list
: vcpu-state-times
: vcpu-run-times
: vcpu-state-counts
: vcpu-run-states
: vcpu-alloc
: vcpu-migration-stats
: vcpu-load
: vcpu-relations
: vcpu-comminfo
: ncpus
: cpu
: pcpu-stats
: pcpu-load
: overhead-histo
: sys-service-stats
: run-state-histo
: wait-state-histo
: coSched-stats
: groups
: worldlet-state-times
: worldlet-state-counts
: worldlet-mig-state
: worldlet-load
: worldlet-relations
: worldlet-comminfo
: power-pstates
: power-cstates
: numa-clients
: numa-migration
: numa-cnode
: numa-pnode
: numa-global
-f : ignore version check
-w : only show stats of the specified world
-p : only show stats of the specified pcpu
-m : only show stats of the specified module
-r : reset scheduler statistics
-s : 1 to enable advanced cpu sched stats gathering, 0 to disable.
-l , : comma separated list of ids to restrict the report to;
(not supported by all reports)
-k : check the correctness of scheduling stats
-v : verbose
-h : print friendly help message

Note:
Sched-stats reads the stats data from vmkernel for each vcpu one
by one via the VSI interface. Since the scheduling stats may
continue to change during the VSI calls, what's reported by
sched-stats is not a consistent snapshot of the kernel stats.
But the inconsistency is expected to be small.

Sunday, July 24, 2016

ESXi PSOD and HeartbeatPanicTimeout

A Purple Screen of Death (PSOD) is a diagnostic screen with white type on a purple background that is displayed when the VMkernel of an ESX/ESXi host experiences a critical error, becomes inoperative and terminates any virtual machines that are running. For more info look here.

Nobody is happy to see PSOD in ESXi host but it is important to say that it is just another safety mechanism how to protect your server workloads because PSOD is intentionally initiated by ESXi's vmkernel in situations when something really bad happens in low level. It is usually related to hardware, firmware or driver issue. You can find further information in VMware KB article - Interpreting an ESX/ESXi host purple diagnostic screen (1004250).

The main purpose of this blog post is to explain the timing of PSOD for just single type of error message - "Lost heartbeat". If there is no heartbeat in some time interval PSOD looks like screenshot below.

no heartbeat

There is no doubt that something serious has to happened in ESXi vmkernel, however regardless what exactly happened following two vSphere advanced settings are used to control heartbeat time interval in which heartbeat must be received otherwise PSOD is executed.

ESXi - Misc.HeartbeatPanicTimeout
VPXD (aka vCenter) - vpxd.das.heartbeatPanicMaxTimeout

Let's start with ESXi advanced setting Misc.HeartbeatPanicTimeout. It defines interval in seconds after which vmkernel goes to panic if no heartbeat is received. Please, don't mixed this "Panic Heartbeat" with "HA network heartbeat". These two heartbeats are very different. "HA network heartbeat" is heart beating mechanism between HA cluster members (master<-><->sleaves) over ethernet network but "Panic Heartbeat" is heartbeat inside single ESXi host between vmkernel and COS software components. You can see "Panic Heartbeat" settings by issuing following esxcli command

esxcli system settings advanced list | grep -A10 /Misc/HeartbeatPanicTimeout

 [root@esx01:~] esxcli system settings advanced list | grep -A10 /Misc/HeartbeatPanicTimeout  
   Path: /Misc/HeartbeatPanicTimeout  
   Type: integer  
   Int Value: 14  
   Default Int Value: 14  
   Min Value: 1  
   Max Value: 86400  
   String Value:  
   Default String Value:  
   Valid Characters:  
   Description: Interval in seconds after which to panic if no heartbeats received

I have tested that Misc.HeartbeatPanicTimeout has different values in different situations. Default value is always 14 seconds but

if you have single standalone ESXi host not connected to HA Cluster effective value is 900 seconds
if you have ESXi host as a member of vSphere HA Cluster then the value is 14 seconds

So now we know that the value in ESXi host with enabled HA is 14 seconds (panicTimeoutMS = 14000) and it usually works without any problem. However, if you will, from whatever reasons, decide to change this value it is worth to know that in HA enabled ESXi host is in HA code hardcoded cap of 60 seconds on this value. It is a cap so it does not change the value if it is already less than 60. However, if you use for example the value 900 it will be caped to 60 seconds anyway. I did a test in vSphere6/ESXi6 and it works exactly like that and I assume it works in the same way in vSphere5/ESXi5.

Side note: It was very different in vSphere4/ESXi4 because HA cluster was rewritten in vSphere 5 from the scratch but it is already a history and I hope nobody use vSphere4 anymore.

Behavior justification:
Behavior described in paragraph above makes perfect sense if you ask me. If you have standalone ESXi host and you are experiencing some hardware issue it is better to wait 900 seconds (15 minutes) before ESXi goes to PSOD state because virtual machines running on top of this ESXi host cannot be automatically restarted in other ESXi hosts anyway. And guess what, if ESXi host have some significant hardware failure, it has most probably negative impact on virtual machines running on top of this particular ESXi host, right? Unfortunately, if you have just a single ESXi host vSphere cannot do anything for you.

On the other hand, if affected ESXi host is a member of vSphere HA cluster then it is better to wait only 14 seconds (by default) or maximally 60 seconds and put ESXi host into PSOD quicker because HA cluster will restart affected virtual machines automatically and helps to mitigate the risk of unavailable virtual machines and with that application services running inside these virtual machines.

So that's the explanation how ESXi setting /Misc/ HeartbeatPanicTimeout behaves. Now we can look what vpxd.das.heartbeatPanicMaxTimeout setting is. My understanding is that vpxd.das.heartbeatPanicMaxTimeout is vCenter (VPXD) global configuration for ESXi advanced setting Misc.HeartbeatPanicTimeout. But don't forget that HA cluster is capping Misc.HeartbeatPanicTimeout value on ESXi hosts as described above.

You can read further details about vpxd.das.heartbeatPanicMaxTimeout in VMware KB 2033250 but I think that following description is little bit misleading.

"This option impacts how long it takes for a host impacted by a PSOD to release file locks and hence allow HA to restart virtual machines that were running on it. If not specified, 60s is used. HA sets the host Misc.HeartbeatPanicTimeout advanced option to the value of this HA option. The HA option is in seconds."

My understanding is that description should be reworded to something like ...

"This option is in seconds and impacts how long it takes for ESXi host experiencing some critical issue to go into a PSOD. Setting vpxd.das.heartbeatPanicMaxTimeout is a global setting used for vCenter managed ESXi advanced option Misc.HeartbeatPanicTimeout however Misc.HeartbeatPanicTimeout is adjusted automatically in certain situations.
In standalone ESXi host 900s is used. In vSphere HA Cluster ESXi host it is automatically changed to 14s and capped to maximum of 60s. This setting have indirect impact on time when file locks are released and hence allow HA cluster to restart virtual machines that were running on affected ESXi host."

Potential side effects and impacts

ESXi HA Cluster restart of virtual machines - if your Misc.HeartbeatPanicTimeout is set to 60 seconds than HA cluster will most probably try to restart VMs on another ESXi hosts because network heartbeat (also 14 seconds) will not be received. However because it is not in PSOD the file lock still exist and VM restart will be unsuccessful.
ESXi Host Profiles - if you use the same host profile for HA protected and also non-protected ESXi hosts then it can report difference of Misc.HeartbeatPanicTimeout against compliance.

Blog posts in blogosphere covering "no heartbeat" issues:

Monday, June 27, 2016

ESXi boot mode - UEFI or BIOS

Legacy BIOS bootstrapping along with a master boot record (MBR) is uses with x86 compatible systems for ages. The concept of MBRs was publicly introduced in 1983 with PC DOS 2.0. It is unbelievable that we are still using the same concept after more then 30 years.

However, there must be some limitations in 30 years old technology, isn't it?

BIOS limitations (such as 16-bit processor mode, 1 MB addressable space and PC AT hardware) had become too restrictive for the larger server platforms. The effort to address these concerns began in 1998 and was initially called Intel Boot Initiative later renamed to EFI. In July 2005, Intel ceased its development of the EFI specification at version 1.10, and contributed it to the Unified EFI Forum, which has evolved the specification as the Unified Extensible Firmware Interface (UEFI).

What is EFI (or UEFI) firmware?

UEFI replaces the the old Basic Input/Output System (BIOS). UEFI can be used on

physical server booting ESXi hypervisor or
Virtual Machine running on top of ESXi hypervisor.

This blog post is about ESXi boot mode however just for completeness I would like to mention that VMware Virtual Machine with at least hardware version 7 supports UEFI as well. For further information about VM UEFI look at [1].

Originally called Extensible Firmware Interface (EFI), the more recent specification is known as Unified Extensible Firmware Interface (UEFI), and the two names are used interchangeably.

EFI (Extensible Firmware Interface) is a specification for a new generation of system firmware. An implementation of EFI, stored in ROM or Flash RAM, provides the first instructions used by the CPU to initialize hardware and pass control to an operating system or bootloader. It is intended as an extensible successor to the PC BIOS, which has been extended and enhanced in a relatively unstructured way since its introduction. The EFI specification is portable, and implementations may be capable of running on platforms other than PCs.

For more information, see the Wikipedia page for Unified Extensible Firmware Interface.

General UEFI Advantages

UEFI firmware provides several technical advantages over a traditional BIOS system:

Ability to boot from large disks (over 2 TB) with a GUID Partition Table (GPT)
CPU-independent architecture
CPU-independent drivers
Flexible pre-OS environment, including network capability
Modular design
Since UEFI is platform independent, it may be able to enhance the boot time and speed of the computer. This is especially the case when large hard drives are in use.
UEFI can perform better while initializing the hardware devices.
UEFI can work alongside BIOS. It can sit on top of BIOS and work independently.
It supports MBR and GPT partition types.

Note: Modern systems are only emulating the legacy BIOS. They are EFI native.

UEFI on ESXi

vSphere 5.0 and above supports booting ESXi hosts from the Unified Extensible Firmware Interface (UEFI). With UEFI, you can boot systems from hard drives, CD-ROM drives, USB media, or network.

UEFI benefits

ESXi can boot from a disk larger than 2 TB provided that the system firmware and the firmware on any add-in card that you are using supports it.

UEFI drawbacks

Provisioning with VMware Auto Deploy requires the legacy BIOS firmware and is not available with UEFI BIOS configurations. I hope that this limitation will be lifted soon.

Notes: Changing the host boot type between legacy BIOS and UEFI is not supported after you install ESXi 6.0. Changing the boot type from legacy BIOS to UEFI after you install ESXi 6.0 might cause the host to fail to boot.

Conclusion

UEFI is meant to completely replace BIOS in the future and bring in many new features and enhancements that can’t be implemented through BIOS. BIOS can be used in servers that do not require large storage for boot. To be honest, even you can, it is not very common to use boot disks greater then 2 TB for ESXi hosts therefore you may be using BIOS at the moment, but I would recommend shifting to UEFI, as it is the future while BIOS will fade away slowly.

ESXi hypervisor supports both boot modes therefore if you have modern server hardware and don't use VMware Auto Deploy then UEFI should be your preferred choice.

References:
[1] VMware : Using EFI/UEFI firmware in a VMware Virtual Machine
[2] VMware : Best practices for installing ESXi 5.0 (VMware KB 2005099)
[3] VMware : Best practices to install or upgrade to VMware ESXi 6.0 (VMware KB 2109712)
[4] Usman Khurshid : [MTE Explains] Differences Between UEFI and BIOS
[5] Wikipedia : Unified Extensible Firmware Interface

Wednesday, May 25, 2016

ESXi : How to mask storage device causing some issues

I have heard about the issue with ESXi 6 Update 2 and HP 3PAR storage where VVOLs are enabled. I have been told that the issue is caused by issuing unsupported SCSI command to PE LUN (256). PE stands for Protocol Endpoint and it is VVOL technical LUN for data path between ESXi and remote storage system.

Observed symptoms:

ESX 6 Update 2 – issues (ESXi disconnects from vCenter, console is very slow)
Hosts may take a long time to reconnect to vCenter after reboot or hosts may enter a "Not Responding" state in vCenter Server
Storage-related tasks such as HBA rescan may take a very long time to complete
I have been told that ESX 6 Update 1 doesn't experience such issues (there are entries are in log file but no other symptoms occur)

Below is a snippet from a log file ..

 2016-05-18T11:31:27.319Z cpu1:242967)WARNING: NMP: nmpDeviceAttemptFailover:603: Retry world failover device "naa.2ff70002ac0150c3" - issuing command 0x43a657470fc0  
 2016-05-18T11:31:27.320Z cpu31:33602)WARNING: NMP: nmpCompleteRetryForPath:352: Retry cmd 0x28 (0x43a657470fc0) to dev "naa.2ff70002ac0150c3" failed on path "vmhba0:C0:T2:L256" H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x25 0x0.  
 2016-05-18T11:31:27.320Z cpu31:33602)WARNING: NMP: nmp_PathDetermineFailure:2973: Cmd (0x28) PDL error (0x5/0x25/0x0) - path vmhba0:C0:T2:L256 device naa.2ff70002ac0150c3 - triggering path failover  
 2016-05-18T11:31:27.320Z cpu31:33602)WARNING: NMP: nmpCompleteRetryForPath:382: Logical device "naa.2ff70002ac0150c3": awaiting fast path state update before retrying failed command again.

Possible workarounds

ESXi hostd restart helps therefore SSH to ESXi hosts was enabled for quick resolution in case of problem
LUN masking of LUN 256

UPDATE 2016-09-30: There is most probably another workaround. Changing the Disk.MaxLUN parameter on ESXi Hosts as described in VMware KB 1998

Final solution

Application of HP 3PAR firmware patch (unfortunately patch is not available for current firmware thus firmware upgrade has to be planned and executed)
Investigation of root cause why ESXi 6 Update 2 is more sensitive then ESXi 6 Update 1

Immediate steps

Application of workarounds mentioned above

HOME LAB EXERCISE
I have tested in my home lab how to mask particular LUN on ESXi host just to be sure I know how to do it.

Below is quick solution for impatient readers.

Let's sat we have following device with following path.

Device: naa.6589cfc000000bf5e731ffc99ec35186
Path: vmhba36:C0:T0:L1

LUN Masking
esxcli storage core claimrule add -P MASK_PATH -r 500 -t location -A vmhba36 -C 0 -T 0 -L 1
esxcli storage core claimrule load
esxcli storage core claiming reclaim -d naa.6589cfc000000bf5e731ffc99ec35186

LUN Unmasking
esxcli storage core claimrule remove --rule 500
esxcli storage core claimrule load
esxcli storage core claiming unclaim --type=path --path=vmhba36:C0:T0:L1
esxcli storage core claimrule run

... continue reading for details.

LUN MASKING DETAILS
Exact LUN masking procedure is documented in vSphere 6 Documentation here. It is also documented in these KB articles 1009449 and 1014953.

List storage devices

 [root@esx02:~] esxcli storage core device list  
 naa.6589cfc000000bf5e731ffc99ec35186  
   Display Name: FreeNAS iSCSI Disk (naa.6589cfc000000bf5e731ffc99ec35186)  
   Has Settable Display Name: true  
   Size: 10240  
   Device Type: Direct-Access  
   Multipath Plugin: NMP  
   Devfs Path: /vmfs/devices/disks/naa.6589cfc000000bf5e731ffc99ec35186  
   Vendor: FreeNAS  
   Model: iSCSI Disk  
   Revision: 0123  
   SCSI Level: 6  
   Is Pseudo: false  
   Status: degraded  
   Is RDM Capable: true  
   Is Local: false  
   Is Removable: false  
   Is SSD: true  
   Is VVOL PE: false  
   Is Offline: false  
   Is Perennially Reserved: false  
   Queue Full Sample Size: 0  
   Queue Full Threshold: 0  
   Thin Provisioning Status: yes  
   Attached Filters:  
   VAAI Status: supported  
   Other UIDs: vml.010001000030303530353661386131633830300000695343534920  
   Is Shared Clusterwide: true  
   Is Local SAS Device: false  
   Is SAS: false  
   Is USB: false  
   Is Boot USB Device: false  
   Is Boot Device: false  
   Device Max Queue Depth: 128  
   No of outstanding IOs with competing worlds: 32  
   Drive Type: unknown  
   RAID Level: unknown  
   Number of Physical Drives: unknown  
   Protection Enabled: false  
   PI Activated: false  
   PI Type: 0  
   PI Protection Mask: NO PROTECTION  
   Supported Guard Types: NO GUARD SUPPORT  
   DIX Enabled: false  
   DIX Guard Type: NO GUARD SUPPORT  
   Emulated DIX/DIF Enabled: false
  
 naa.6589cfc000000ac12355fe604028bf21  
   Display Name: FreeNAS iSCSI Disk (naa.6589cfc000000ac12355fe604028bf21)  
   Has Settable Display Name: true  
   Size: 10240  
   Device Type: Direct-Access  
   Multipath Plugin: NMP  
   Devfs Path: /vmfs/devices/disks/naa.6589cfc000000ac12355fe604028bf21  
   Vendor: FreeNAS  
   Model: iSCSI Disk  
   Revision: 0123  
   SCSI Level: 6  
   Is Pseudo: false  
   Status: degraded  
   Is RDM Capable: true  
   Is Local: false  
   Is Removable: false  
   Is SSD: true  
   Is VVOL PE: false  
   Is Offline: false  
   Is Perennially Reserved: false  
   Queue Full Sample Size: 0  
   Queue Full Threshold: 0  
   Thin Provisioning Status: yes  
   Attached Filters:  
   VAAI Status: supported  
   Other UIDs: vml.010002000030303530353661386131633830310000695343534920  
   Is Shared Clusterwide: true  
   Is Local SAS Device: false  
   Is SAS: false  
   Is USB: false  
   Is Boot USB Device: false  
   Is Boot Device: false  
   Device Max Queue Depth: 128  
   No of outstanding IOs with competing worlds: 32  
   Drive Type: unknown  
   RAID Level: unknown  
   Number of Physical Drives: unknown  
   Protection Enabled: false  
   PI Activated: false  
   PI Type: 0  
   PI Protection Mask: NO PROTECTION  
   Supported Guard Types: NO GUARD SUPPORT  
   DIX Enabled: false  
   DIX Guard Type: NO GUARD SUPPORT  
   Emulated DIX/DIF Enabled: false

So we have two device with following NAA IDs

naa.6589cfc000000bf5e731ffc99ec35186
naa.6589cfc000000ac12355fe604028bf21

Now let's list paths of both of my iSCSI devices

[root@esx02:~] esxcli storage nmp path list
iqn.1998-01.com.vmware:esx02-096fde38-00023d000001,iqn.2005-10.org.freenas.ctl:test,t,257-naa.6589cfc000000bf5e731ffc99ec35186
   Runtime Name: vmhba36:C0:T0:L1
   Device: naa.6589cfc000000bf5e731ffc99ec35186
   Device Display Name: FreeNAS iSCSI Disk (naa.6589cfc000000bf5e731ffc99ec35186)
   Group State: active
   Array Priority: 0
   Storage Array Type Path Config: {TPG_id=1,TPG_state=AO,RTP_id=3,RTP_health=UP}
   Path Selection Policy Path Config: {current path; rank: 0}

iqn.1998-01.com.vmware:esx02-096fde38-00023d000001,iqn.2005-10.org.freenas.ctl:test,t,257-naa.6589cfc000000ac12355fe604028bf21
   Runtime Name: vmhba36:C0:T0:L2
   Device: naa.6589cfc000000ac12355fe604028bf21
   Device Display Name: FreeNAS iSCSI Disk (naa.6589cfc000000ac12355fe604028bf21)
   Group State: active
   Array Priority: 0
   Storage Array Type Path Config: {TPG_id=1,TPG_state=AO,RTP_id=3,RTP_health=UP}
   Path Selection Policy Path Config: {current path; rank: 0}

Let's mask iSCSI devices exposed as a LUN 1.
So our path we want to mask is vmhba36:C0:T0:L1 and device UID is naa.6589cfc000000bf5e731ffc99ec35186

So let's create masking rule of path above. In this particular case we have just a single path because it is local device. In real environment we have usually multiple paths and all paths should be masked.

 esxcli storage core claimrule add -P MASK_PATH -r 500 -t location -A vmhba36 -C 0 -T 0 -L 1
 esxcli storage core claimrule load

We can list our claim rules to see the result

 [root@esx02:~] esxcli storage core claimrule list  
 Rule Class  Rule Class  Type    Plugin   Matches                  XCOPY Use Array Reported Values XCOPY Use Multiple Segments XCOPY Max Transfer Size  
 ---------- ----- ------- --------- --------- ---------------------------------------- ------------------------------- --------------------------- -----------------------  
 MP       0 runtime transport NMP    transport=usb                            false            false            0  
 MP       1 runtime transport NMP    transport=sata                           false            false            0  
 MP       2 runtime transport NMP    transport=ide                            false            false            0  
 MP       3 runtime transport NMP    transport=block                           false            false            0  
 MP       4 runtime transport NMP    transport=unknown                          false            false            0  
 MP      101 runtime vendor   MASK_PATH vendor=DELL model=Universal Xport                  false            false            0  
 MP      101 file   vendor   MASK_PATH vendor=DELL model=Universal Xport                  false            false            0  
 MP      500 runtime location  MASK_PATH adapter=vmhba36 channel=0 target=0 lun=1              false            false            0  
 MP      500 file   location  MASK_PATH adapter=vmhba36 channel=0 target=0 lun=1              false            false            0  
 MP     65535 runtime vendor   NMP    vendor=* model=*                          false            false            0

We can see that new claim rule (500) is in configuration file (/etc/vmware/esx.com) and also loaded in runtime.

However, to really mask our particular device without ESXi host reboot we have to reclaim device

 [root@esx02:~] esxcli storage core claiming reclaim -d naa.6589cfc000000bf5e731ffc99ec35186

The particular device disappear from ESXi host immediately. ESXi host reboot is not needed.
So we are done. Particular device is not visible to ESXi host anymore.

Note: I was unsuccessful when I was testing LUN masking with local device. Therefore I assume that LUN masking works only with remote disks (iSCSI, Fibre Channel).

LUN UNMASKING
Just in case you would like to unmask device and use it again here is the procedure.

Let's start with removing claimrules for our previously masked path.

 [root@esx02:~] esxcli storage core claimrule remove --rule 500  
 [root@esx02:~] esxcli storage core claimrule list  
 Rule Class  Rule Class  Type    Plugin   Matches                  XCOPY Use Array Reported Values XCOPY Use Multiple Segments XCOPY Max Transfer Size  
 ---------- ----- ------- --------- --------- ---------------------------------------- ------------------------------- --------------------------- -----------------------  
 MP       0 runtime transport NMP    transport=usb                            false            false            0  
 MP       1 runtime transport NMP    transport=sata                           false            false            0  
 MP       2 runtime transport NMP    transport=ide                            false            false            0  
 MP       3 runtime transport NMP    transport=block                           false            false            0  
 MP       4 runtime transport NMP    transport=unknown                          false            false            0  
 MP      101 runtime vendor   MASK_PATH vendor=DELL model=Universal Xport                  false            false            0  
 MP      101 file   vendor   MASK_PATH vendor=DELL model=Universal Xport                  false            false            0  
 MP      500 runtime location  MASK_PATH adapter=vmhba36 channel=0 target=0 lun=1              false            false            0  
 MP     65535 runtime vendor   NMP    vendor=* model=*                          false            false            0  
 [root@esx02:~]

You can see that rule is removed from file configuration but it is still running. We have to re-load claimrules from file to runtime.

 [root@esx02:~] esxcli storage core claimrule load  
 [root@esx02:~] esxcli storage core claimrule list  
 Rule Class  Rule Class  Type    Plugin   Matches              XCOPY Use Array Reported Values XCOPY Use Multiple Segments XCOPY Max Transfer Size  
 ---------- ----- ------- --------- --------- --------------------------------- ------------------------------- --------------------------- -----------------------  
 MP       0 runtime transport NMP    transport=usb                        false            false            0  
 MP       1 runtime transport NMP    transport=sata                        false            false            0  
 MP       2 runtime transport NMP    transport=ide                        false            false            0  
 MP       3 runtime transport NMP    transport=block                       false            false            0  
 MP       4 runtime transport NMP    transport=unknown                      false            false            0  
 MP      101 runtime vendor   MASK_PATH vendor=DELL model=Universal Xport              false            false            0  
 MP      101 file   vendor   MASK_PATH vendor=DELL model=Universal Xport              false            false            0  
 MP     65535 runtime vendor   NMP    vendor=* model=*                       false            false            0  
 [root@esx02:~]

Here we go. Now there is no rule with id 500.

But the device is still not visible and we cannot execute command

esxcli storage core claiming reclaim -d naa.6589cfc000000bf5e731ffc99ec35186

because such device is not visible to ESXi host. We mask it, right? So it is exactly how it should behave.

ESXi host would probably help but can we do it without ESXi host reboot?
The answer is yes we can.
We have to unclaim the path to our device and re-run claim rules.

 esxcli storage core claiming unclaim --type=path --path=vmhba36:C0:T0:L1  
 esxcli storage core claimrule run

and now we can see both paths to iSCSI LUNs again.

 [root@esx02:~] esxcli storage nmp path list  
 iqn.1998-01.com.vmware:esx02-096fde38-00023d000001,iqn.2005-10.org.freenas.ctl:test,t,257-naa.6589cfc000000bf5e731ffc99ec35186  
   Runtime Name: vmhba36:C0:T0:L1  
   Device: naa.6589cfc000000bf5e731ffc99ec35186  
   Device Display Name: FreeNAS iSCSI Disk (naa.6589cfc000000bf5e731ffc99ec35186)  
   Group State: active  
   Array Priority: 0  
   Storage Array Type Path Config: {TPG_id=1,TPG_state=AO,RTP_id=3,RTP_health=UP}  
   Path Selection Policy Path Config: {current path; rank: 0}  
 iqn.1998-01.com.vmware:esx02-096fde38-00023d000001,iqn.2005-10.org.freenas.ctl:test,t,257-naa.6589cfc000000ac12355fe604028bf21  
   Runtime Name: vmhba36:C0:T0:L2  
   Device: naa.6589cfc000000ac12355fe604028bf21  
   Device Display Name: FreeNAS iSCSI Disk (naa.6589cfc000000ac12355fe604028bf21)  
   Group State: active  
   Array Priority: 0  
   Storage Array Type Path Config: {TPG_id=1,TPG_state=AO,RTP_id=3,RTP_health=UP}  
   Path Selection Policy Path Config: {current path; rank: 0}

Hope this helps to other vmware users having a need for LUN masking / unmasking.

Thursday, May 19, 2016

ESXi 6 - manual partitioning for multiple VMFS filesystems on single disk device

I have to test SDRS initial placement exact behavior (blog post here) therefore I need multiple VMFS datastores to form an Datastore Cluster with SDRS. Unfortunately, I'm constraint with storage resources in my home lab therefore I would like to use one local 220GB SSD to simulate multiple VMFS datastores.

Warning: This is not recommended practice for productional systems. It is recommended to have single partition (LUN) per single device.

I did not find GUI way how to create multiple partitions per single disks therefore I used CLI.

You cannot use old good fdisk for VMFS partitions. Instead PartedUtil has to be used because of GPT. PartedUtil is included in ESXi so you can login to ESXi over SSH and use partedUtil.

Note: It is worth to mention that you can have maximally 16 partitions in GUID Partition Table (GPT) in ESXi 6. This is what I have tested. When I tried to create 17 partitions the partedUtil failed with message - "Too many partitions (17)".

In my lab I have Intel NUC with local SSD identified as "t10.ATA_____INTEL_SSDMCEAW240A4_____________________CVDA4426003E240M____".

To get partition table use following command

 partedUtil getptbl "t10.ATA_____INTEL_SSDMCEAW240A4_____________________CVDA4426003E240M____"

If disk is empty you should see follwoing output:

msdos
29185 255 63 468862128

I would like to have 5 x 10Gb LUNs.

10Gb is 20971519 in sectors. Sector is 512Bytes

First 2048 should be skipped to keep space for GPT and to be aligned with 512B sectors. Below is my partition plan in format [Partition number, Start sector, End Sector]

P1 2048-20973567
P2 20973568-41945087
P3 41945088-62916607
P4 62916608-83888127
P5 83888128-104859647

Following command is used to create these 5 partitions.

 partedUtil setptbl "t10.ATA_____INTEL_SSDMCEAW240A4_____________________CVDA4426003E240M____" gpt "1 2048 20973567 AA31E02A400F11DB9590000C2911D1B8 0" "2 20973568 41945087 AA31E02A400F11DB9590000C2911D1B8 0" "3 41945088 62916607 AA31E02A400F11DB9590000C2911D1B8 0" "4 62916608 83888127 AA31E02A400F11DB9590000C2911D1B8 0" "5 83888128 104859647 AA31E02A400F11DB9590000C2911D1B8 0"

Now you can list partitions again and you should see following output:

gpt
29185 255 63 468862128
1 2048 20973567 AA31E02A400F11DB9590000C2911D1B8 vmfs 0
2 20973568 41945087 AA31E02A400F11DB9590000C2911D1B8 vmfs 0
3 41945088 62916607 AA31E02A400F11DB9590000C2911D1B8 vmfs 0
4 62916608 83888127 AA31E02A400F11DB9590000C2911D1B8 vmfs 0
5 83888128 104859647 AA31E02A400F11DB9590000C2911D1B8 vmfs 0

So we have 5 partitions and the last step is to format it to VMFS5 file system. We have to leverage vmkfstools which is also included in ESXi 6 system.

Let's start with first partition and we will use datastore name Datastore1.

 vmkfstools -C vmfs5 -S Datastore1 t10.ATA_____INTEL_SSDMCEAW240A4_____________________CVDA4426003E240M____:1

The same procedure has to be repeated for each partition.

Here is Perl script which generates commands for partedUtil and vmkfstools based on several variables.

 #!/usr/bin/perl  
 #  
 my $disk_device = "t10.ATA_____INTEL_SSDMCEAW240A4_____________________CVDA4426003E240M____";  
 my $starting_sector = 2048;  
 my $partition_size_in_sectors = 4000000;  
 my $datastore_prefix = "Datastore";  
 my $num_of_partitions = 16; # Maximum is 16 partitions  
 my $partitions;  
 my $start = $starting_sector;  
 my $end;  
 my $i;  
 # Generate command for GPT partitions  
 for ($i=1; $i<$num_of_partitions; $i=$i+1) {  
     $end = $start + $partition_size_in_sectors;  
     $partitions .= "\"$i $start $end AA31E02A400F11DB9590000C2911D1B8 0\" ";  
     $start = $end + 1;  
 }  
 print "partedUtil setptbl \"$disk_device\" gpt $partitions\n";  
 # Generate commands to format partitions with VMFS5 file system  
 #  
 for ($i=1; $i<$num_of_partitions; $i=$i+1) {  
     print "vmkfstools -C vmfs5 -S $datastore_prefix$i $disk_device:$i\n";  
 }

If you want to unmount 16 datastores it is very handy to use following command line:

 esxcli storage filesystem unmount -l Datastore1

Hope this helps at least one other vSphere dude in his lab exercises.

VCDX #200 The Ultimate Way to Virtualize
Blog of one VMware Infrastructure Designer

Pages

Sunday, January 28, 2018

ESXi - optical diagnostic information from VMNIC

Monday, August 14, 2017

Remote text based console to ESXi over IPMI SOL

Sunday, April 02, 2017

ESXi Host Power Management

Wednesday, January 11, 2017

Using esxtop to identify storage performance issues for ESX / ESXi

Tuesday, December 20, 2016

ESXi sched-stats command

Sunday, July 24, 2016

ESXi PSOD and HeartbeatPanicTimeout

Monday, June 27, 2016

ESXi boot mode - UEFI or BIOS

Wednesday, May 25, 2016

ESXi : How to mask storage device causing some issues

Thursday, May 19, 2016

ESXi 6 - manual partitioning for multiple VMFS filesystems on single disk device

Pages

Sunday, January 28, 2018

Monday, August 14, 2017

Sunday, April 02, 2017

Wednesday, January 11, 2017

Tuesday, December 20, 2016

Sunday, July 24, 2016

Monday, June 27, 2016

Wednesday, May 25, 2016

Thursday, May 19, 2016

Subscribe To