Tuesday, September 17, 2013

DELL Force10 configuration for VMware VXLAN transportation

Right now I work on vSphere Design where network virtualization is leveraged to simplify network management and provide segmentation of multiple tenants. Therefore I was tested VXLANs in my lab. I have equipment listed bellow:
  • 1x DELL Blade Chassis  M1000e
  • 2x DELL Force10 IOA (IO Aggregators - blade chassis network modules)
  • 2x DELL Force10 S4810 as top of the rack switches
  • 1x DELL Force10 S60 acting as physical router (L3 switch)
  • 1x DELL EqualLogic storage PS-4110 (iSCSI storage module inside Blade Chassis)

Here are VXLAN physical switch requirements 
  • Minimum MTU size requirement is 1600 however we will use maximum Jumbo Frames across physical network
  • IGMP snooping should be enabled on L2 switches, to which VXLAN participating hosts are attached.
  • IGMP Querier enabled on router or L3 switch with connectivity to the multicast enabled networks.
  • Multicast routing (PIM-SM) must be enabled on routers.

Force10 switches are by default configured to allow Jumbo Frames. However physical interfaces, vlan interfaces and port-channels has to be configured explicitly. 
Force10 S-series switches interfaces MTU can be set up to 12000. In CISCO Nexus environments max MTU is 9216.

Force10 IOA (I/O Aggregator) is by default set to MTU 12000 so it is already prepared for VXLAN and nothing has to be configured.

Let's assume we use VLAN 14 for VXLAN transport.

Router config (Force10 S60)
config
igmp snooping enable
ip multicast-routing
interface vlan 14
  ip pim sparse-mode
  mtu 12000
  tagged gigabitethernet 0/46-47
  exit
! For all interfaces in VLAN 14 we have to set MTU at least 1600
interface range gigabitethernet 0/46 - 47
  mtu 12000
  end
Switch config (Force10 S4810)
! IGMP snooping must be enabled
conf
ip igmp snooping enable
interface vlan 14
  mtu 12000
  exit
interface range tengigabitethernet 0/46 , tengigabitethernet 0/48 - 51 , fortyGigE 0/56 , fortyGigE 0/60
  mtu 12000
  exit
interface range port-channel 1 , port-channel 128
  mtu 12000
  exit
end
IO Aggregator (Force10 IOA)
Force10 IOA default configuration has maximum MTU  already in default factory settings so it is VXLAN ready and no changes are required.

Here are Force10 IOA default values:
mtu 12000
ip mtu 11982
igmp snooping enabled
Check out these excellent blog articles for more details on VXLAN theory and implementation:

VXLAN requirements
http://www.yellow-bricks.com/2012/10/04/vxlan-requirements/

VXLAN on UCS and vSphere: from L3 to Nexus 1000V
http://vmtrooper.com/vxlan-on-ucs-and-vsphere-from-l3-to-nexus-1000v/

http://www.force10networks.com/CSPortal20/TechTips/0008_mtu-settings.aspx
Adjusting MTU and Configuring Jumbo Frame Settings

UPDATE 2015-02-02:
I have multicast router enabled on my VLAN 14 (see configuration of Force10 S60) therefore it works like IGMP querier. However if you have a need to have VXLAN overlay over the network without multicast router you should configure IGMP Querier on particular VLAN otherwise multicast traffic will be flooded into whole broadcast network (VLAN). IGMP querier can be configured by following command:
ip igmp snooping querier 



No comments: