Benchmarking a raspberrypi compared to my own PC [obsolete and locked]

[KarlSplatz]

I don't have a raspberrypi and I'm wondering if it's what I'm looking for. I'd like it to be able to do some number crunching in a couple of minutes running some software that does that on my PC in the same time. The code runs in a single thread on my PC (Windows 7.0) but also in a Virtualbox running Ubuntu.

So the main thing is I'd just like be able to run something as a benchmark in the virtualbox that has also been run on the latest raspberry pi and published. Can anyone point to a benchmark review where I can access the benchmark software?

All help greatly received.

Karl

[DougieLawson]

You're probably wasting your time. The Raspberry is about as powerful as your cell phone was five years ago. It's also a completely different architecture to your PC, so you'd be comparing apples vs oranges.

The easiest way if you insist on benchmarking (a process that's exceedingly good at demonstrating nothing other than the speed a benchmark runs) spend £35 and get a RPi to play with.

[ejolson]

I'd like it to be able to do some number crunching in a couple of minutes running some software that does that on my PC in the same time.

The almost definitive source of Raspberry Pi benchmark results were performed by Roy Longbottom and may be found at http://www.roylongbottom.org.uk/Raspber ... hmarks.htm. I've written a number of parallel CPU benchmarks to test the Intel/MIT Cilk extensions to the C programming language on the Raspberry Pi2B. A simple benchmark for finding prime numbers shows that a single ARMv7 core has about the same compute performance as a 650Mhz Pentium III CPU from 1999. A slight modification of that same prime finder to list the first 664579 primes up to 10000000 gives the following results:

Code: Select all

CPU                     32-bit  64-bit  Compiler
-------------------------------------------------------
i7-6700K 4.0GHz          0.600   1.820  gcc-5.2.1 -O3
i7-4850HQ 2.3GHz                 1.850  LLVM-7.0.2 -O3
i7-3770k 4GHz            0.975   2.146  gcc-5.2 -O3
i5-3570k 3.4GHz                  2.497  gcc-4.8.4 -O3
Xeon E5-2620v3 2.4GHz    1.135   2.545  gcc-4.7.2 -O3
Xeon E5-2650v2 2.6GHz    1.155   2.592  gcc-4.4.7 -O3
AMD A6-5400K 3.6GHz      2.023   2.095  gcc-4.7.2 -O3
ARMv8 NanoPi T3 1.4Ghz   2.876   3.088  gcc-5.4.0 -O3
ARMv8 Pi 3B+ 1.4Ghz      3.332  14.691  gcc-7.3.0 -O3 \
                                        unknown optimization
Opteron 6212 2.6GHz      3.407   3.421  gcc-5.1.0 -O3
Phenom II X4 3.4GHz      3.473   3.479  gcc-4.7.2 -O3
ARMv8 Pi 3B 1200MHz      3.611  51.734  gcc-4.9.2 -O3 \
                                        -mcpu=cortex-a53
Pentium 4 3.4Ghz         3.759   5.181  gcc-5.2.1 -O3
AthlonII X2 255 3.1GHz   3.828   3.836  gcc-4.7.2 -O3
Athlon64 X2 5400+ 2.8GHz 4.601   7.893  gcc-4.6.3 -O3
Pentium 4D CPU 2.80GHz   4.612   6.271  gcc-4.7.2 -O3
ARMv7 Pi 2B 900MHz       7.187  74.790  gcc-5.2 -O3 \
                                        -mcpu=cortex-a7
Athlon 1400Mhz           9.726  12.807  gcc-4.7.2 -O3
Pentium III 866MHz      14.999  20.169  gcc-4.7.2 -O3
ARMv8 Pi 3B 1200MHz     17.670          gcc-4.9.2 -O3
Pentium III 650MHz      19.891  26.735  gcc-4.7.2 -O3
AMD-K6 3D 350MHz        26.726  45.469  gcc-4.7.2 -O3
ARMv7 Pi 2B 900MHz      27.741  74.987  gcc-4.6.3 -O3
ARMv6 Pi B+ 700MHz      74.027          gcc-4.6.3 -O3
i586 Pentium 75MHz     155.804 303.710  gcc-2.7.2.3 -O3
i486 DX/2 66MHz        282.180 919.130  gcc-2.6.3 -O3

Updated to distinguish timings using 32-bit integers from 64-bit integers. The effects of running the program under a 32-bit kernel versus a 64-bit kernel are not carefully tabulated. If in doubt, please refer to the posts below to determine the specific conditions under which each timing was made.

[solar3000]

You're probably wasting your time. The Raspberry is about as powerful as your cell phone was five years ago. It's also a completely different architecture to your PC, so you'd be comparing apples vs oranges.

The easiest way if you insist on benchmarking (a process that's exceedingly good at demonstrating nothing other than the speed a benchmark runs) spend £35 and get a RPi to play with.

Liar. He's comparing humming birds to pterodactyls.

[KarlSplatz]

Thanks for the replies so far.

Let me explain a little more. I teach numerical simulation and I want to be able to let my students play with a simulation model to understand the system being modelled AND the process of simulation. The raspberry pi appears to be excellent because I can configure it myself and then bring it to the class room (which is variable and I don't have access to the PCs to install the software - it's cumbersome for various reasons). The fact that the RPi works from a memory card makes this even better as I could encourage the students to purchase the Rpi and I distribute the software.

I do realise that you have to be careful with benchmarks but a simple numerical test would be a good start. If we're talking an order of magnitude difference in run time then I need to look for a better solution.

I tried the ejolson's benchmark and my PC runs from just under 0.009s to 0.025. I guess a longer problem would be more stable. Still that's 5-10 times faster than the numbers quoted.

I was unsure of the RPi2 might be faster than is quoted above as it's difficult to tell but this helps. I need something faster.

Thanks.
Karl

[jahboater]

On my Pi2 (at 900Mhz) the "benchmark" takes from 0.123 sec to 0.180 sec
Using the standard (on Jessie) gcc 4.9.2

[DougieLawson]

Give us some self contained ready to compile code and some of us can try running it on our RPis.

[ejolson]

I tried the ejolson's benchmark and my PC runs from just under 0.009s to 0.025. I guess a longer problem would be more stable.

You are right that the original problem size is not large enough for meaningful timings. If you change the program to find the primes up to 10000000 as

Code: Select all

#include <stdio.h>
#include <math.h>
#include <stdint.h>

#define N 10000000

typedef uint32_t Integer;
//typedef uint64_t Integer;
static Integer prime[1000000];
static int count=0;

static int isprime(Integer n){
    int sqrtn=sqrt((double)n)+0.5;
    int k;
    for(k=0;k<count;k++){
        if(prime[k]>sqrtn) return 1;
        if(n%prime[k]==0) return 0;
    }
    return 1;
}

int main(){
    Integer n;
    for(n=2;n<=N;n++){
        if(isprime(n)) prime[count++]=n;
    }
    printf("Found a total of %d primes (%d-bit)\n",
        count,(int)sizeof(n)*8);
    return 0;
}

and rerun with

Code: Select all

$ gcc -O3 prime.c -lm
$ time ./a.out; time ./a.out; time ./a.out

and report the best real elapsed time among the three runs, I would be happy to have another data point for the table I posted above.

At our university it is possible to boot a CD on any of the classroom computers. A few years ago I set up an NFS server which contained a read-only Debian-live image and then passed out CDs with Linux kernels that had been configured to automatically mount root from that NFS server. It was not necessary to install any software on the Windows PCs and it worked amazingly well.

Last year IT support installed Linux virtual machines on Windows in our lab and then offered to install the same virtual machine on each student's personal laptop as needed. All the individual images were difficult to update and maintain. I think the Raspberry Pi2B could make a good platform for teaching numerical methods; however, lots of little computers might also be difficult to maintain.

To make maintenance easier, a thin-client solution called PiNET is sometimes employed, but that negates the advantage of students using the same Pi computers at home. At the same time, it seems entirely reasonable for university students to setup and maintain their own Linux computers. The Pi also includes a free copy of Mathematica, if that makes any difference. Assuming your students write their code using C or Fortran and the problems are not research sized, I would not expect CPU speed to be a significant problem.

Note: Code for prime.c has been updated to explicitly specify the size of the integers used for the computation.

[leiptrstormr]

https://github.com/phoronix-test-suite/ ... est-suite/

[jahboater]

For 900MHz Pi2, built with gcc 4.9.2:-

pi@pi2 ~ $ time ./a.out; time ./a.out; time ./a.out
Found a total of 664579 primes!

real 0m27.697s
user 0m27.690s
sys 0m0.000s
Found a total of 664579 primes!

real 0m27.721s
user 0m27.690s
sys 0m0.020s
Found a total of 664579 primes!

real 0m27.722s
user 0m27.700s
sys 0m0.010s
pi@pi2 ~/ge $

For 700MHz B+, built with gcc 4.6.3

pi@pi ~ $ time ./a.out; time ./a.out; time ./a.out
Found a total of 664579 primes!

real 1m14.038s
user 1m13.440s
sys 0m0.230s
Found a total of 664579 primes!

real 1m14.039s
user 1m13.530s
sys 0m0.150s
Found a total of 664579 primes!

real 1m14.027s
user 1m13.490s
sys 0m0.170s
pi@pi ~/ge $

For 4GHz Intel Core i7-3770k, built with gcc 5.2:-

~ * time ./a.out; time ./a.out; time ./a.out
Found a total of 664579 primes!

real 0m2.183s
user 0m2.180s
sys 0m0.000s
Found a total of 664579 primes!

real 0m2.159s
user 0m2.156s
sys 0m0.000s
Found a total of 664579 primes!

real 0m2.146s
user 0m2.144s
sys 0m0.000s

[ejolson]

Updated the table above with data from jahboater and information about compiler version!

[ghans]

Forum user RoyLongbottom maintains a website with a great amount of synthetic benchmarks (and their results) :

http://www.roylongbottom.org.uk/Raspber ... hmarks.htm

ghans

[jahboater]

For the 900 MHz Pi2, I tried adding the usual compiler options to specify the exact CPU type and the results were dramatically improved (down from 27.7 to 7.2 secs) putting it closer the 2.8GHz Pentium 4.

pi@pi2 ~ $ gcc -O3 -mcpu=cortex-a7 -mfpu=neon-vfpv4 prime.c -lm
pi@pi2 ~ $ time ./a.out; time ./a.out; time ./a.out
Found a total of 664579 primes!

real 0m7.253s
user 0m7.240s
sys 0m0.010s
Found a total of 664579 primes!

real 0m7.227s
user 0m7.220s
sys 0m0.000s
Found a total of 664579 primes!

real 0m7.187s
user 0m7.170s
sys 0m0.010s
pi@pi2 ~ $

[KarlSplatz]

I tried the longer calculation for primes.

I got error messages when I tried to compile with the new compile command but reverted to the old one in the example link given above. I also tried the -mcpu option but that is "deprecated" with my version of gcc so switched to -march=native and -mtune=native. Neither of these options made much difference.

I'm getting a run time of 2.5s plus or minus 0.03s I would say. 3 successive runs are more alike but if I wait a minute or two the average for three successive runs goes up or down by the stated uncertainty.

I'm compiling using gcc (Ubuntu 4.8.4-2ubuntu1~14.04) 4.8.4 which is run on a "Vagrant" controlled version of Ubuntu 14.04.3 LTS (GNU/Linux 3.13.0-68-generic x86_64) on Virtualbox in Windows 7 (64bit). My processor is an Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz.

Karl

[KarlSplatz]

Hi ejolson,
I take your point about setting up live CDs. I've discussed this with our IT guys but they're not happy from a security point of view.

The problem isn't them so much though. I teach the course off campus and I'm going into computer labs where I have limited access or call on IT guys time to set everything up properly. The big attraction the PI (apart from low cost) is the ability to control and manage a small number of boxes (I also like their size as I'd have to carry them with me).

We also have "Distance Learning" students who have to be able to set up the software themselves. The memory card image approach for the PI would be ideal.

I did play with live bootable USB sticks a wee while back (2013 or early 2014). The software I am trying to run is not mine and I struggled with the installation. However, a newer version is available so I may explore that.

Karl

[ejolson]

For the 900 MHz Pi2, I tried adding the usual compiler options to specify the exact CPU type and the results were dramatically improved (down from 27.7 to 7.2 secs) putting it closer the 2.8GHz Pentium 4.

Good call. Thanks for using the newer compiler and the good optimization flags. It seems the results are in, at least concerning this particular algorithm for finding prime numbers. The fact that the Pi 2B is about 3 times slower than the fastest PCs currently available is quite remarkable considering it uses less than 1/100 the power and costs 1/100 the amount.

As with any such comparisons, it is useful to reflect on the nature of the code being tested. The memory footprint to store 664579 primes is about 2.5 MB, however, less than 4K of these are ever needed for the inner prime-testing loop. Therefore, the inner loop should execute from the level 1 cache of most modern processors. The calculation itself relies almost solely on integer arithmetic, which is one of the Pi 2Bs relative strengths. I would expect greater differences for a floating point calculation, because the FPU in the particular ARMv7 chip under consideration is a single-issue non-vector type.

[blc]

That fact the Pi 2B is about 3 times slower than the fastest PCs currently available is quite remarkable considering it uses less than 1/100 the power and costs 1/100 the amount.

Your numbers make interesting reading and I appreciate the effort but: I'm sorry, I can't let this slide:

...and costs 1/100 the amount.

I get the point you're making, but it doesn't cost $3,500 (~£2300) to build a PC with an i7-3770k. It wouldn't even cost that much to build a PC with an i7-6700K.

Interesting numbers nonetheless, and it certainly makes the case for all the optimisation work going on.

[jamesh]

I think the real improvement in there is use of NEON. Can make a huge difference.

[jahboater]

And the Rpi2 is the size of a credit card, compared to the PC which is a medium sized tower!

It goes fractionally faster on the Pi2 with -mthumb perhaps because more instructions fit in the I-cache. Using gcc 5.2 on the Pi2 did not help.

That benchmark is very synthetic. I have just timed a real C compilation which took 23sec on the Core i7 PC, but took 5 min 53 sec on the Pi2 - about 15 times slower. The PC was about 45 times more expensive by the way (and drinks power). Love the Pi!

[jamesh]

And the Rpi2 is the size of a credit card, compared to the PC which is a medium sized tower!

It goes fractionally faster on the Pi2 with -mthumb perhaps because more instructions fit in the I-cache. Using gcc 5.2 on the Pi2 did not help.

That benchmark is very synthetic. I have just timed a real C compilation which took 23sec on the Core i7 PC, but took 5 min 53 sec on the Pi2 - about 15 times slower. The PC was about 45 times more expensive by the way (and drinks power). Love the Pi!

I suspect big compiles will also be limited by SD card random access speed. Would be interesting to see performance differences with 1) Going for USB HW, 2) Using all the cores (make -j 4) if not already.

[experix]

I did a bunch of comparisons using various PCs, a Pi B+ and a Pi 2, all running http://experix.sourceforge.net (a scripting language for device control and data acquisition and analysis) and doing these tests:

generating ramp and sine functions (also compared vs. octave on a Linux PC)

generating a binary control function for stepper motor

rapid-fire graph drawing (also compared octave/ftlk and octave/gnuplot on Linux PC and MatLab on Windows PC); experix does it much faster than octave or MatLab)

a multi-threaded data acquisition (simulated) and display task with file storage operations

These are discussed in the book/performance file which you can get from the file download section of my sourceforge page (look for the most recent version) or if that's not convenient I'll send you that file.

[jahboater]

The biggest build I ever did on the Pi2 was gcc 5.2 itself which took:-

-j1 48 hours
-j2 23 hours
-j3 17 hours
-j4 15 hours

(takes about 2 hours or so on my 16GB core i7 with -j4).

-j2 or more required additional swap space. I think there are a few very large source files in gcc. Especially with j3 or j4, the ACT light was solid much of the time and yes it would no doubt benefit from a faster disk. On the core i7 I do the entire build in /tmp which is in memory.

[jamesh]

The biggest build I ever did on the Pi2 was gcc 5.2 itself which took:-

-j1 48 hours
-j2 23 hours
-j3 17 hours
-j4 15 hours

(takes about 2 hours or so on my 16GB core i7 with -j4).

-j2 or more required additional swap space. I think there are a few very large source files in gcc. Especially with j3 or j4, the ACT light was solid much of the time and yes it would no doubt benefit from a faster disk. On the core i7 I do the entire build in /tmp which is in memory.

The fact those times steps are not linear indicates it's storage access that is the limiting factor.

[ejolson]

The fact those times steps are not linear indicates it's storage access that is the limiting factor.

To do the math

Code: Select all

Linear     Actual     Deviation from Linear
 48          48           0%
 24          23          -4% (superlinear)
 14.3        17          18%
 12          15          25%

The superlinear speedup when moving from 1 to 2 cores is interesting. Do you notice this with the i7 builds as well? Even though there are only 4 cores, I wonder if the build time would continue to decrease with -j5 or -j6.

[jahboater]

The superlinear speedup when moving from 1 to 2 cores is interesting. Do you notice this with the i7 builds as well? Even though there are only 4 cores, I wonder if the build time would continue to decrease with -j5 or -j6.

No, with 16GB of memory it is never worth building with less than -j4 on the i7. As for the Pi2, next time I do this (when gcc 5.3 arrives) I'll probably just put the swapfile on a separate fast SSD. Otherwise its hammering the poor little sd card too much. I tried -j >4 on the i7 but it didn't seem to help. Kudos to the Pi that it can even be done!