Jumbo Frames/MTU en Hyper-V Server (Windows Server 2008/8 Core)

Enabling Jumbo Frame support in Hyper-V Server 2008 R2 (or Windows Server Core) has proven to be a bit of an adventure.  It really just involves setting the MTU size, but it has to be done in the OS (to affect the TCP/IP stack) as well as the network cards’ driver.  Since Core versions of Windows do not have a network control, setting the MTU on the cards proves to be a bit of a trick.  This is what I had to do to enable Jumbo Frames on several iSCSI nics, and since it differs for Intel vs Broadcom adapters, there are two procedures.

I should point out that this does not address configuring the network switch that these nics are attached to.  That is a whole ‘nother can of worms, but suffice it to say that the switch must not only support Jumbo Frames but have that support enabled, along with a whole host of other settings.

Enable Jumbo Frames on the OS

The first thing you need to do is make sure that your server will allow jumbo frames.  You do this by setting the MTU on your adapters to 9000.  The easiest way to do this is by running a netsh command on each adapter you want to use Jumbo Frames.

Get a list of interface names by running “netsh int show int”

In this case I have already re-named the Interfaces that I intend to use for iSCSI.  You might just see a whole list of “Local Area Connection” interfaces.  You can use ipconfig or netsh to further identify which ones you want to use.

Now for each interface you want jumbo frames enabled, run this command:

netsh int ipv4 set subint “” mtu=9000 store=persistent

Now you have to configure Jumbo Frames in the driver for each interface.

Enable Jumbo Frames on Intel cards

The Intel driver stores it’s “Jumbo Frame” settings in the registry.  Thankfully, Hyper-V Server (and Windows Core) comes with Regedit, so you can just launch that from command line (regedit.exe) and browse to the following key:

Here you will see all the network interfaces listed by GUID.  I have found that the easiest way to determine which GUID is which adapter is by finding the IP address and being able to correlate it to the right Interface name.

At this point you should start making a list to help keep things straight.  Copy the GUID into notepad and list the IP address next to it and do this for each card you want to configure.  So for this server, my list looks like this:

Now we have to navigate to a new registry key to configure the driver.  Go here:

Here you will again see a list of all network interfaces, only this time they are under 4 digit identifiers.  From here, search for the GUID that you copied to your list and you should find it as the “NetCfgInstanceId” key of one of the adapters.  Once found, it’s not a bad idea to update your list to keep track of what’s what.  Mine looks like this now:

Scroll up to find the “*JumboPacket” key and double click it to change the default value of 1514 to 9014.  Note the extra 14 bytes here represents packet headers that normally are not counted in MTU size.

Repeat this for each Intel adapter you need to configure, and then reboot the server for the setting to take effect.

Enable Jumbo Frames on Broadcom cards

First make sure you have the latest Broadcom drivers.  Make sure you get the 2008 R2 x64 set.

If you haven’t already, then download and install the driver and then reboot the host.  Note: Make sure you migrate any existing guest servers off the host before you install the drivers.  The temporary outage of the card due to the update seems to make a failover cluster angry.

Now get the Broadcom Management Application suite.  Again, get the x64 set from the same page.

Install the management app.  I opt’d not to install the BASP component (see screenshot below) since we do not want failover or teaming in this scenario.  It’ll likely warn you that you need the dotNet Framework 2.0 and you should be able to ignore this because the installer just does not recognize the “Core” framework, but the application still runs.  To make sure you do in fact have the framework installed, run “oclist | findstr /i netfx” and look for a line stating that NetFx is installed.  For example, “Installed:NetFx2-ServerCore”.  If not, you can install it by running “start /w ocsetup NetFx2-ServerCore” or instead you can install dotNet 3.0 and 3.5 by running “start /w ocsetup NetFx3-ServerCore”.

From C:Program FilesBroadcomBACS run “BACSCLi” to run in interactive mode.  It will show you a list of all network adapter drivers installed.  You only care about the “NDIS” adapters so enter “list ndis” and you’ll see something like this:

If you did the Intel configuration you’ll notice the four digit number in square braces of the Name field matches the ControlClass registry key.

Use some combination of “ipconfig /all” in another window or CtxAdmTools’ Visual Core Configurator 2008 or the four digit registry code to identify the adapter that you want to configure.  In this example we want Connection #6.  Select it by using “select 6” or whatever number is in the “C” column that matches your adapter.  Now validate that you have selected the correct adapter by reviewing some of its details.  Run “info” to see it’s MAC/IP, etc.

Notice the MTU setting is set to 1500 by default.  Now run “cfg advanced” to list its advanced properties.

Run “cfg advanced “Jumbo MTU”=9000” to set Jumbo frames to 9000 bytes.  Note that you do not have to account for the 14 bytes of header data here.  It’ll take a few seconds to apply the change but you should not need to reboot (yay!).  You can now run “cfg advanced” and “info” to list the settings and ensure that the MTU is in fact set to 9000.

You should also enable Flow Control for Transmit (Tx) and Receive (Rx).  With the correct adapter already selected, run “cfg advanced “Flow Control”=”Rx & Tx enabled””.

Once that is complete you can enter “q” to exit BACScli or start over using “list ndis” and select another interface to configure.  You can also use this utility to select non-Broadcom adapters to display some of their info like MTU size.

Testing Jumbo Frames

To test if Jumbo Frames are working you can ping another host target that also supports Jumbo Frames.  The easiest way that I have found to do this was to just change the IP of your test NIC and your test target NIC to something that no other adapter has.  This is because there is no way to tell windows specifically what NIC to send traffic over, so setting the NIC’s to their own network ip space is the only way to ensure that the ping traverses a particular adapter.

For example, I changed the source test nic to 172.16.1.4 and the target to 172.16.1.8 and no other adapters on either host saw set in the 172.16.1.* range.

First try a normal “ping 172.16.1.8” and it should work fine.  Then use “ping -f -l 6000 172.16.1.8” to test jumbo frames and it should also work, only this time you’ll see it sending 6000 bytes instead of 32.

So that about covers it.  I had to do this for each of the 32 iSCSI nics spread across the 8 host servers, but it works!  You should be aware that if you do a driver update or if you share a NIC with a virtual network (as a Hyper-V Host) your settings may be lost and you’ll have to go through this again.

Obtained from this link http://mrshannon.wordpress.com/2010/01/13/jumbo-frames-on-hyper-v-server/#comment-13