Where is the Bottleneck?

[gjs]

With a RPi set up as an HTTP or FTP server, where would the bottleneck be when downloading from it e.g. The ethernet port, the USB bus, the storage device (USB flash, USB SSD) etc.

With everything set up optimally what maximum download rate would be expected?

Thanks.

[RaTTuS]

the NIC / USB all share the same interface so it's that that will be the bottleneck.

but your looking at max 100Mbps [really]

YMMV

[jamesh]

All data has to pass down a single USB 2 bus, so 480Mbits/s max. Take away half that for USB overhead, 280Mbits/s. Now that need to be shared between the HD/USB stick and the ethernet. So 140MBits/s. That's probably an absolute max you are going to get (about 15MB/s)

Now you probably have some more overheads in there (on my desktop I get about 12MB/s transfer on a USB device).

With a max. 100mbit/s ethernet port, 12.5MB/s. With wireless N, half that, but with more overheads.

With all that said, without actual testing its difficult to figure out all the overheads, so a wild guestimate would be about 5MB/s.

The of course, you need to factor in the speed of the software serving up the data.

[gordon@drogon.net]

gjs said:


With a RPi set up as an HTTP or FTP server, where would the bottleneck be when downloading from it e.g. The ethernet port, the USB bus, the storage device (USB flash, USB SSD) etc.

With everything set up optimally what maximum download rate would be expected?

Thanks.



Probably not in the RPi.

If the SD card can be read at over 10MBytes/sec then it can be streamed to the Ethernet port at the same speed. (Assuming the internal bus that connects peripherals together can go faster than 100Mb/sec - I'd hope it can, although I don't know the actual architecture)

A quick check suggests that an SD cards with an x66 or greater rating, or a Class 10 card can work at 10MBytes/sec.

However there's more to it than that - the application that serves the data for example - a big MySQL type database, apache with a lot of cgi behind it, etc.

I'd have to suggest that if you are concerned about its throughput then perhaps the RPi isn't the right tool for your application...

G

[Jim Manley]

Since Class 10 SD cards don"t work with the R-Pi (nor are they expected to without a currently-unplanned hardware mod, due to a voltage incompatibility issue), and even some recent lower-class cards have problems, including some Micro SD cards in adapters, the best you"re going to do is under 6 MB/s, not including bus overhead and application/service delays, as others have noted. A USB flash drive will do somewhat better, but, you"ll still max out below 15 MB/s, plus overhead. However, a USB hard drive will easily do 40 ~ 100+ MB/s sustained and they have nice-sized RAM buffers to help keep data going whenever there"s time on the bus, plus, you can get TBs of capacity for well under $100.

[jojopi]

Jim Manley said:


Since Class 10 SD cards don"t work with the R-Pi (nor are they expected to without a currently-unplanned hardware mod, due to a voltage incompatibility issue)


Do have references for either of these new parenthetic claims?  Because there is only one signalling voltage for SD cards, and two for SDHC, and they appear to have no direct relation to the class system[1].  And the last official statement I saw was that the SD incompatibilities were being looked at, not that there were no plans to try to fix them[2].

[1] http://www.sdcard.org/download.....100518.pdf

[2] http://www.raspberrypi.org/archives/837

[Nr90]

There have been multiple places where the issue with the class 10 cards was explained.
One of the latest is the webinar Eben gave earlier this week.

[Jim Manley]

jojopi said:


Do have references for either of these new parenthetic claims?  Because there is only one signalling voltage for SD cards, and two for SDHC, and they appear to have no direct relation to the class system[1].  And the last official statement I saw was that the SD incompatibilities were being looked at, not that there were no plans to try to fix them.


SDHC (including Micro SD) and SDXC are based on 1.8 volt technology due to narrower traces deposited during the fabrication process, vs. the wider traces associated with the 3.3 volt technology used in original SD cards. 1.8 volt technology introduction made possible the debut of higher capacities starting at 8 GBs and, given the higher capacities and narrower traces, higher speeds are possible owing to lower current and associated lower heat generated in the traces. This was a necessary development as SD card capacities increased along with processor, bus, and other I/O interface speeds (e.g., 1 GB/s Ethernet). So, there is a coincident overlapping correlation between higher capacities and higher speed/Class SD card interfaces.

The SD card interface on the R-Pi can"t generate the commands to shift the interface between 3.3 and 1.8 volt mode without a hardware upgrade, per Eben"s Q&A answer toward the end of the April 4th webinar/Ebenar.

Recall that price is _the_ predominant design factor in the R-Pi, hence USB 2.0 not 3.0, 100 Mb/s Ethernet instead of 1 Gb/s, 256 MB RAM rather than 512 MB, 700 MHz ARM11v6 vs. 1+ GHz v7 Cortex multi-core, etc.

[jojopi]

Jim Manley said:


SDHC (including Micro SD) and SDXC are based on 1.8 volt technology due to narrower traces deposited during the fabrication process, vs. the wider traces associated with the 3.3 volt technology used in original SD cards.


But by my reading of the abridged phy spec, they are still required to support 3.3V operation at the original bus speeds?


The SD card interface on the R-Pi can"t generate the commands to shift the interface between 3.3 and 1.8 volt mode without a hardware upgrade, per Eben"s Q&A answer toward the end of the April 4th webinar/Ebenar.


What he appears to say is that interface is 3.3V only.  In that case it should not be attempting to switch to the higher modes.

If the cards or the Pi are violating the spec then this would seem to be a much bigger issue than Class 10 flakiness.  As you say it potentially applies to any cards over 2GB (non-SDSC) and will only become more common with ongoing process shrinkage.

[dom]

My understanding:
On BCM2835 each GPIO bank is fed from a supply voltage which determines the I/O level supported. I believe there are 3 banks and the sdcard bank runs off a fixed 3.3V.
All sdcards start of at 3.3V I/O and only will switch to lower voltage/higher speed after negotiation with the host.
The initial firmware blob and kernel are loaded by GPU which never accepts the lower voltage negotiation. So far we"ve not seen sdcard acccess fail at this stage for sdhc.
The kernel shipped with Debian image incorrectly accepts the negotiation to a lower voltage it cannot support. It crashes soon after.
The latest kernel on GitHub handles the negotiation correctly, and remains at 3.3V (and lower speed).
We are aware of one additional cause of failure (possibly too small a timeout when erasing multiple sectors) and now have a card that shows this behaviour and we are investigating.
If we get through the firmware boot, then I"m not aware of any reason why a card will not be usable in theory, albeit not necessarily at its highest speed. However the driver may need tweaking as we come across cards with new foibles.
If there exist cards that the GPU bootloader cannot read, then we"re out of luck as that code is in rom. As this code is quite basic (no negotiation of enhanced features) hopefully there will be no problems.

[Vindicator]

In my area a Raspi server would be bottle necked at my ISP, I am paying for 10Mbps and I get 8.5 on a good day so the Raspi probably would not be the problem LOL.

[arm2]

JamesH said:


All data has to pass down a single USB 2 bus, so 480Mbits/s max. Take away half that for USB overhead, 280Mbits/s. Now that need to be shared between the HD/USB stick and the ethernet. So 140MBits/s.


Yes if you are say reading data from the Ethernet and then writing it to a USB device but if say you are just loading into RAM there would not be the halving! Yes?

[arm2]

dom said:


My understanding:
On BCM2835 each GPIO bank is fed from a supply voltage which determines the I/O level supported. I believe there are 3 banks and the sdcard bank runs off a fixed 3.3V


Is the choice of bank fixed in the chip or in the PCB wiring?

i.e. could reworked PCB fix this problem?

[dom]

Yes, this is a PCB issue rather than a chip issue.

Some extra logic to switch the GPIO bank supply voltage (using a GPIO pin) is possible in theory.

[rew]

I've found hints (in the datasheet?) that there are just two IO banks.

That would surely mean that there are other devices on that IO bank that the SD card is connected to. I'm not sure that you'll be able to just drop the voltage to 1.8V on the whole bank without messing up a whole lot of other stuff.

However, we don't have the datasheet, so I can only guess that there IS other stuff on that bank and that it can or cannot be changed to 1.8V without problems....

[sge]

All data has to pass down a single USB 2 bus, so 480Mbits/s max. Take away half that for USB overhead, 280Mbits/s. Now that need to be shared between the HD/USB stick and the ethernet. So 140MBits/s. That's probably an absolute max you are going to get (about 15MB/s)

Now you probably have some more overheads in there (on my desktop I get about 12MB/s transfer on a USB device).

Sorry for bringing back this post, but it's probably better to keep discussion to as few posts as possible.

Could somebody elaborate/link on the exact reasons the USB interface won't reach anywhere near its theoretical maximum? If it's true that we only ever can get to half the theoretical maximum it's for all intents and purposes 240Mbits/s interface... D:

[gsh]

You should be able to reach quite close to theoretical maximum for simple applications (i.e. just sending big blocks of data over bulk interface to a USB 2.0 device) I would expect it to hit 480 (90%) MBits throughput... But then the data has to be handled on the ARM... And that takes processing power which is also needed to send data...

Class 10 SDCards should not be a problem, they have to support 3.3V but only up to 50MHz, we cannot reach the higher speeds because this requires us to change down the voltage)

SDCard read speed seems to be able to reach 25MByte/s doing block reads so I don't think it's likely to be a problem there...

Bottleneck is almost definitely the CPU creating / copying / moving the data around... So it really depends upon what the server is doing...

Gordon

[Heater]

Simple arithmetic.

480Mbits per second is about 48MBytes per second.
Our processor is running at about 700M instructions per second. So we have about 14 instructions available to handle each byte!
Would you really expect that to be sufficient to keep up?

[ech0]

Sorry to bump an old thread but I'm both amazed and curious to how you can get such great speeds.
I've got a 500/50 mbit connection, cat 6 ethernet cables and I reach speeds around 50 MB/s on my desktop but on my pi I barely reach download speeds around 2.2~ MB/s. I'm using a pretty much "blank" raspbian image. Is there a way to investigate this further or some kind of optimization that needs to be done? Keep in mind, I'm writing to an external harddrive (2TB USB 3.0, however pi only got 2.0). I'm curious to know what the bottleneck may be as well, if I need to overclock something even more than what I have. Currently I'm running the CPU on 900 MHz, core at 450 MHz and RAM on 450 as well, 4 overvolt.
Also, when downloading around the 2.2 MB/s the CPU is using around 80-95%, RAM using 65-75MB.

[jamesh]

Easy test - drop the overclock to normal, then see if your download speed changes. If so its CPU related. Which is almost certainly the case - see gsh's post earlier.

[ech0]

I dropped the overclocking to the standard values and I get the same speed.
I also did some additional testing, and it seems that if I download a file online to the SD card I get about 7-8 MB/s, and if I download the very same file to the external harddrive it's about 1.6-1.8 MB/s. There must be some limitations to the USB port because the drive itself is normally working fine, if I connect it to any other machine I get good and normal 2.0 and 3.0 speeds. So I kind of doubt it's a CPU issue and more USB related, perhaps?

[JoeDaStudd]

How is the hdd powered? I'd try it via a powered hub and see if that helps.
It's also worth testing the speed from SD to hdd.

[ech0]

The hdd is powered with the power source I got when buying the drive, its output is 12V and 1.5A.

Copying a file using Rsync shows about avg: 3.06 MB/s writing from SD card to the external drive.
Writing from external drive to SD card is about avg: 6.95 MB/s

[KeithSloan]

In my area a Raspi server would be bottle necked at my ISP, I am paying for 10Mbps and I get 8.5 on a good day so the Raspi probably would not be the problem LOL.

Is that 8.5 download or upload speed?

Here in the UK my router syncs with exchange at 7 Mbps and on speed tests I get 6 Mbps, but that is downloading from the internet, upload is only 448 Kps. A Pi installed at my home would be uploading to serve people on the internet so only able to do 448 Kps at max..

[ghans]

Is the harddisk NTFS-formatted ? NTFS-3G sucks on embedded
systems.


ghans