How to enable iSCSI and Jumbo Frames on VMware ESX/ESXi

I am redoing my lab at home (more on that in coming weeks) and wanted to test out jumbo frames support in vmware esxi with my iSCSI NAS device. This article by vmware provides information related to setting up a NAS (Network Attached Storage) and an iSCSI environment with Jumbo Frames.

Note: Jumbo Frames on software iSCSI for ESX 3.x is an experimental feature and is not a fully supported configuration. Jumbo Frames are fully supported in ESX/ESXi 4.x and ESXi 5.0.

Ensure that you read this important information about Jumbo Frames before working with them:

  • ESX/ESXi supports a maximum MTU size of 9000. Note: Some switch configurations for Jumbo Frames need to have an MTU set more than 9000. For more information, see Cisco Nexus 5000 Series NX-OS Software Configuration Guide.
  • Any packet larger than 1500 MTU is a Jumbo Frame. ESX/ESXi supports frames up to 9Kb (9000 Bytes).
  • Jumbo Frames are limited to data networking only (virtual machines and the vMotion network).
  • It is possible to configure Jumbo Frames for an iSCSI network. It is not a fully supported configuration in ESX 3.5, but it is supported in ESX/ESXi 4.x and ESXi 5.0.
  • You can enable Jumbo Frames for each vSwitch or VMkernel interface through the command line interface on your ESX host.
  • To allow ESX host to send larger frames out onto the physical network, the network must support Jumbo Frames end to end.
  • Ensure that your NIC or LOM supports Jumbo Frames.
  • For experimental support of Jumbo Frames in ESX 3.5, these NICs are supported:
    • Intel (82546, 82571)
    • Broadcom (5708, 5706, 5709, 57710, 57711)
    • Netxen (NXB-10GXxR, NXB-10GCX4)
    • Neterion (Xframe, Xframe II, Xframe E)
  • For ESX/ESXi 4.x and ESXi 5.0, contact your NIC hardware vendor regarding support for Jumbo Frames.
  • You cannot use Jumbo Frames on a Broadcom card that is configured as a hardware initiator performing iSCSI offload functions. You can either use Jumbo Frames or iSCSI Offload and you cannot use both together with the Broadcom adapters.

Creating a Jumbo Frames-enabled vSwitch

To create a Jumbo Frames-enabled vSwitch:

  1. Log in directly to the ESX host console.
  2. To set the MTU size for the vSwitch:
    1. Run this command for ESX 3.5 and ESX/ESXi 4.x:esxcfg-vswitch -m MTU vSwitch#

       

    2. Run this command for ESXi 5.0:esxcli network vswitch standard set -m MTU -v vSwitch#

      Note: This command sets the MTU for all uplinks on that vSwitch. Set the MTU size to the largest MTU size among all the virtual network adapters connected to the vSwitch.

  3. To display a list of vSwitches on the host, and to check that the configuration of the vSwitch is correct:
    • Run this command for ESX 3.5 and ESX/ESXi 4.x:esxcfg-vswitch -l

       

    • Run this command for ESXi 5.0:esxcli network vswitch standard list

Creating a Jumbo Frames-enabled VMkernel interface

To create a Jumbo Frames-enabled VMkernel interface:

  1. Log in directly to the ESX host console.
  2. To create a VMkernel connection with Jumbo Frames support, run this command to create the port group on an existing switch:esxcfg-vswitch -A vmkernel_port_group_name vSwitch#

    Note:If you plan to have a vSwitch which just contains the iSCSI port group you can specify the MTU for the vSwitch to be 9000 and not specify the MTU when creating the vmkernel port in the next step.

    esxcfg-vswitch -m 9000 vSwitch#

     

  3. To create a VMkernel connection with Jumbo Frame support:
    • Run this command for ESX 3.5 and ESX/ESXi 4.x:esxcfg-vmknic -a -i ip_address -n netmask -m MTU portgroup_name

      Note:If the vmnic port is already created, use the command for ESX/ESXi 4.1 only:

      esxcfg-vmknic -m 9000 portgroup_name

       

    • Run this command for ESXi 5.0:esxcli network ip interface set -m 9000 -i vmk_interface

       

  4. To display a list of VMkernel interfaces, and to check that the configuration of the Jumbo Frame‐enabled interface is correct:
    • Run this command for ESX 3.5 and ESX/ESXi 4.x:esxcfg-vmknic -l

       

    • Run this command for ESXi 5.0:esxcli network ip interface list