Increasing MTU on Pi 4

[smishra12345]

I tried increasing the MTU on Pi 4 from the default 1500 but it would not let me do so.

With a Gigabit Ethernet, higher MTU values would be very helpful.

Can we recompile the kernel/driver and increase the MTU ?

[epoch1970]

I don’t recall MTU as a Kconfig parameter so recompiling the kernel is possibly not an option.
In any case defaulting to jumbo seems a questionable choice since mtu 9000 (?) is not a standard and regular Ethernet devices will not be able to handle the large payload and will drop it...
Changing the mtu per interface with the “ip” command seems a better idea.

This being said if the Ethernet hardware cannot handle large frames, the point is moot. Do you know the max frame size supported by the Ethernet adapter on pi4? (I don’t.)

[trejan]

Can we recompile the kernel/driver and increase the MTU ?

The Broadcom GENET driver doesn't have any support for setting MTU so it'll involve more than just recompiling it assuming the hardware even supports > 1500. The init_umac function does set a maximum frame length but I've no idea what happens if you increase that number and whether you need to mess about the ring buffers or DMA configuration as well.

[jdb]

In theory the hardware supports up to 16k packet sizes but increasing the MTU will, as you say, need tweaks to the packet buffer configuration. My hunch is that the buffers need to end up 4k page-sized for compatibility with the rest of Linux which means jumbo frames span more than 1 tx or rx buffer. The driver already supports generic segmentation offload so I assume it can handle multi-buffer tx.

[Muny]

Has anyone had any success with increasing the MTU on their Pi 4?
I'm trying to get larger packets working so I can get better performance out of my GigE vision camera.

[akrantz01]

Has there been any progress on this? I am having the same issue when trying to set up BATMAN-adv.

[ehem]

Well, does Raspian ship with the "ifupdown" package? Have you tried `ifdown eth0`, then in /etc/network/interfaces, under "iface eth0" adding "mtu 9000" and then doing `ifup eth0`?

If your network uses DHCP, then DHCP option 26 can be used to specify the frame size of the network (I can confirm this works with at least some DHCP client programs). A bigger issue is many switches have a fixed limit of 1500 bytes. You will need a noticeably higher-end switch for 9K packets. Mine is at the bottom end of this category and sending a jumbo frame after a long gap of no jumbo frames for a while causes it to temporarily drop all packets for a bit (OS bug on a switch, wonderful).

In theory the hardware supports up to 16k packet sizes but increasing the MTU will, as you say, need tweaks to the packet buffer configuration. My hunch is that the buffers need to end up 4k page-sized for compatibility with the rest of Linux which means jumbo frames span more than 1 tx or rx buffer. The driver already supports generic segmentation offload so I assume it can handle multi-buffer tx.

I was wanting to claim the standards for gigabit ethernet required all gigabit computer NICs to support jumbo frames, yet a quick search suggests this may not be the case. I'm wondering if perhaps the hardware of gigabit ethernet NICs is required to support jumbo frames, but many drivers do not...

[trejan]

Has there been any progress on this? I am having the same issue when trying to set up BATMAN-adv.

No change. The bcmgenet driver in the latest kernel doesn't have any support for setting the MTU.

[trejan]

I was wanting to claim the standards for gigabit ethernet required all gigabit computer NICs to support jumbo frames, yet a quick search suggests this may not be the case. I'm wondering if perhaps the hardware of gigabit ethernet NICs is required to support jumbo frames, but many drivers do not...

There is no requirement to support jumbo frames. I've seen controllers that only supported the stock 1500 MTU and some that only supported "baby jumbo" where it'll take 1518-1522. Even the controllers that do support jumbo frames don't have any guarantee as to how big it is with some as low as 2000.

[wang_Bob]

Have you sloved the problem？
Now,I'm trying to connect a gigabit camera, but the frame was badly missing.

[trejan]

Have you sloved the problem？
Now,I'm trying to connect a gigabit camera, but the frame was badly missing.

No. The bcmgenet driver still doesn't have any support for changing MTU.

[thomaage]

Has anyone had any success with increasing the MTU on their Pi 4?
I'm trying to get larger packets working so I can get better performance out of my GigE vision camera.

Did you find a solution to this? I'm trying to solve the same problem. Would a USB3 to Ethernet converter do the trick?

[waryishe]

If your willing to get your hands dirty and compile a kernel of your own, you can get jumbo frames working. In the file include/linux/if_vlan.h change the lines

Code: Select all

#define VLAN_ETH_DATA_LEN	1500
#define VLAN_ETH_FRAME_LEN	1518

to

Code: Select all

#define VLAN_ETH_DATA_LEN	9000
#define VLAN_ETH_FRAME_LEN	9018

In include/uapi/linux/if_ether.h change

Code: Select all

#define ETH_DATA_LEN	1500
#define ETH_FRAME_LEN	1514

to

Code: Select all

#define ETH_DATA_LEN	9000
#define ETH_FRAME_LEN	9014

and finally change drivers/net/ethernet/broadcom/genet/bcmgenet.c

Code: Select all

#define RX_BUF_LENGTH		2048

to

Code: Select all

#define RX_BUF_LENGTH		10240

compile and you will have fully functional jumbo frames. I used 9000 as the limit because that works with all the gear on my network. Beware that you are changeing the default mtu for EVERYTHING. Some other network drivers will not like that. But if all you are using is the on-board ethernet, this should work. Iperf gives me 990Mbit after the changes. Very close to line speed.

[SMAHH]

Thank you very much for the information.
It seems to be that entries help to fix the issue partly.
I have done some more tests after I have compiled a new kernel with those entries.

I have tried to transfer image data form a GigE camera via UDP protocol. It works only up to 2000 bytes packet size even MTU is set to 9000 on pi.

I had a look with wireshark at the switch port pi connected to and on the PI. I can see the packet gets damaged on PI when I send the packet > 2000bytes.

Please have a look into the attached files.

[waryishe]

I'm not sure why it is having trouble after 2000 byte size? Are you sure its going over the ethernet and not wifi? Wifi has a mtu around 2k. Try running this command on the pi

Code: Select all

 ping -M do -s 8972 IPADDRESS

Replace IPADDRESS with a computer on your network. If you get normal ping responses then it's not the network drivers issue. Also are you running in 32 bit or 64 bit mode? I only checked it in 64 bit, but i didn't think that would make a difference.

[dickon]

Ethernet has an MTU of 1500 bytes. Anything more is considered 'jumbo frames', and may or may not be supported. 9000B is common, but I'm not at all surprised to see a ~2000B limit. Gives you 48B to play with for metadata.

[JohnDuffy]

Thanks for posting the information on how to change the MTU setting in the kernel.

I've made the changes, compiled a new kernel and very happily have an MTU of 9000.

I was wondering where the 10240 value comes from for the RX_BUF_LENGTH? Is this the maximum that can be set for the driver? And I was wondering whether, for maximum performance, this value (or a multiple) should be reflected in any kernel TCP parameters, such as 'tcp_rmem'? Or are they unrelated?

Kind regards

[iandk]

Any plans to support this out of the box?

[DougieLawson]

Any plans to support this out of the box?

Is packet fragmentation a big problem for you? My network works with MTU 1500 on every device.
If you support it out of the box you can't set big packets as default because that will break some folk's networks.

[geerlingguy]

Just wanted to follow up and say thanks for posting this—I did this while testing the CM4 with a 4-interface Intel PCIe NIC (plus the internal interface), and went from getting a maximum of 3.4 Gbps (with overclock) across all interfaces, to getting 4.15 Gbps! (details are in this GitHub issue for those interested).

[graysky]

Was wondering if anyone else tried testing different MTU values for throughput on a RPi4. Based on the post above by waryishe, I made this patch to enable them in the 5.4.y series of kernel.

Upon booting into 5.4.75 with the patch applied, I ran some tests using rsync to copy 10 x 330 MB unique files from my x86_64 box to the RPi4 using different MTU values on each end, 1500, 4000, and 9000. With 10 files per test, I could get some stats around the time it took. What I found was that a value of 1500 gave the fastest, 4000 was slower (about 12%), and 9000 was slower yet (about 34%).

Here is the script I used to do generate the unique files and the timing of the rsync transfer if others want to play with it.

Conclusion: mtu values of 4000 and 9000 are inferior to 1500 for copying files of 330 MB.

Stats on the time which is in seconds:
For 1500:

Code: Select all

# NumSamples = 10; Min = 4.73; Max = 5.33
# Mean = 4.911205; Variance = 0.040310; SD = 0.200774; Median 4.818305

For 4000:

Code: Select all

# NumSamples = 10; Min = 5.31; Max = 5.95
# Mean = 5.481886; Variance = 0.039424; SD = 0.198554; Median 5.415320

For 9000:

Code: Select all

# NumSamples = 10; Min = 6.57; Max = 6.58
# Mean = 6.577341; Variance = 0.000012; SD = 0.003519; Median 6.577062

[kolsi]

I applied the patch to kernel 5.9. Now MTU is 9000 bytes but I still cannot transfer packets larger than 2000 bytes. We have a camera with high frame rate and higher MTU is required for efficient result. Is there a solution to use higher packet size?

[Shorlak]

I recompiled the kernel with the changes that were mentioned above and copied the new kernel to the rPi.
It boots fine, but my network interface (eth0) is gone.
Doesn't matter what I connect to the LAN port, ifconfig only displays the loopback interface.
2 LEDS on the port are on, but I got no internet..

How can I solve this issue?