Reports so far.
Been hitting the ethernet with a stick for the last few days, that stick being iperf3. I have been testing the onboard, and two different USB->GigE dongles.
The main conclusion we have come to - if your switch does NOT have flow control turned on, you will suffer a large drop in performance. This is due to the Pi being unable to service incoming data at gig speeds, and therefor a lot of retries are required for dropped frames.
Now, desktop switches have flow control on permanently, and managed switches appear to have it on by default. Big data centre switches sometimes have it turned off because there is a slight improvement in performance with it off, if all devices on the network can cope with the speed. But, if you are seeing really low speeds when downloading from the net, you really need to check your network devices to ensure flow control is turned on.
Now, that being said, we have seen some really good performance from a USB dongle with the r8153 chipset/driver. This consistently performances well above the speed of the onboard device even with flow control off. Investigating further, this simply appears to be a REALLY GOOD chip. It does a lot of offloading on to HW (for example, it appears it coalesces TCP packets from the standard 1400 size in to much larger lumps, massively reducing the load on the USB link and the CPU processing required), and that really seems to make a massive difference to overall performance. We are investigating whether we can get the Microchip devices to work in a similar way. I suspect that the amount of buffer memory on the chip is the real limiting case here. What is interesting is that this dongle, with flow control off, goes from 0 retries to over 10k when testing with iPerf over 10s. This is about 5 times that of the onboard device, but its performance is still very good....so its doing some really clever stuff which we haven't yet figured out.
Another dongle tried with an ax88179 (ASIX chipset) showed performance BELOW that off the onboard device in the same tests.
All devices tested showed degradation in performance when flow control is off.
Note, I am unable to test internet performance, since we have a max of 100Mbits/s in the office, and both the 3B and 3B+ hit that limit in our tests (well, I get mid 90's for both)
Finally, we are talking to Microchip to see what can be done to improve performance. It's their chip and their driver. However, we are still investigating.
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.