energyi
Posts: 184
Joined: Tue Mar 24, 2015 9:39 pm

Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Wed Jun 26, 2019 10:57 pm

Thought if anyone was interested I would post my Octane Benchmark score for the Raspberry Pi 4 1 GB RAM straight out of the box, no logging into anything, just going immediately to http://chromium.github.io/octane/.

Score for the Raspberry Pi 4 1 GB RAM was 7652.

This benchmark is outdated, but I still like to compare SBCs with it. Lots of variables, YMMV. For comparison, my other testing showed:
Jetson Nano was 7671
Rpi 3B+ was 2802.
energyi

User avatar
HawaiianPi
Posts: 6685
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Thu Jun 27, 2019 1:27 am

Raspberry Pi 3B+ in Flirc Gen2 case
Raspbian Stretch with Desktop
Octane Score: 3198

HP Chromebook x2 (Intel Core M3-7Y30)
Chrome OS 74.0.3729.159
Octane Score: 23442

Lenovo Legion Y520 budget gaming laptop (i7-7700HQ)
Windows 10 Insider Preview build (slow ring)
Octane Score: 40013

Still waiting for my Pi4 ... :(
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups, and where is that annoying music coming from?

binaryhermit
Posts: 57
Joined: Sun Apr 13, 2014 1:26 am
Location: Lockport, Illinois
Contact: Website

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Thu Jun 27, 2019 1:42 am

And apparently there's a major issue in the Chromium build in Buster that requires that they disable GPU to work around it. Apparently the octane score will likely go up fairly significantly once they reenable GPU.

User avatar
HawaiianPi
Posts: 6685
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Thu Jun 27, 2019 2:19 am

binaryhermit wrote:
Thu Jun 27, 2019 1:42 am
... Apparently the octane score will likely go up fairly significantly once they reenable GPU.
Doubtful.

Octane is a Javascript benchmark that doesn't do much in terms of graphics. But out of curiosity I set my gaming laptop to switch to the Nvidia GPU when running Chrome, and I got a score of ... (drum roll please) ... 40077

Which is nearly identical to the score I posted above using the Intel integrated graphics.
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups, and where is that annoying music coming from?

energyi
Posts: 184
Joined: Tue Mar 24, 2015 9:39 pm

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Thu Jun 27, 2019 8:11 pm

Outdated Octane Benchmark Score on Raspberry Pi 4 4 GB RAM is 8454.

Just trying to show rpi scores. Yes, have lots of other scores from other OS and boxes, but hoping this is helpful to the Rpi community to compare and contrast old benchmarks to new 4 versions.
energyi

binaryhermit
Posts: 57
Joined: Sun Apr 13, 2014 1:26 am
Location: Lockport, Illinois
Contact: Website

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Fri Jun 28, 2019 4:47 am

GPU might actually matter.

On my n3350-based Chromebook on M75, the octane score is 10903.
Running the version of chromium in Debian Stable, M73 I think, via Crostini (which on most chromebooks doesn't support GPU acceleration) the score is 8885.

There is a cpu penalty, although the benchmarks I've done seem to suggest it's much smaller than that, perhaps so much so that the cpu overhead from ChromeOS with the vm layer might be less than the overhead of running windows on the same hardware.

Anyway, we'll see,

The best way to check might be to have someone run it from the menu and then run it from the command line to see if the score is significantly different.

User avatar
HawaiianPi
Posts: 6685
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Jun 29, 2019 1:57 am

energyi wrote:
Thu Jun 27, 2019 8:11 pm
Just trying to show rpi scores. Yes, have lots of other scores from other OS and boxes, but hoping this is helpful to the Rpi community to compare and contrast old benchmarks to new 4 versions.
Since the Pi4 is being touted as having true desktop PC performance now, comparisons to other systems are both inevitable and important. By the time you add the PSU, case, SD card and other accessories needed to turn a $35-$55 Pi4 into a Desktop PC, you may approach the pricing of a budget Chromebook, so I imagine a lot of people would like to know how they compare.

The Pi4 score posted above is significantly lower than my Chromebook, but mine is a mid-tier model. However, it is over 2.6X faster than my Pi 3B+ score, so that's pretty cool.

Still a week before my Pi4 computers ship (estimated ship date, July 5) ... :(
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups, and where is that annoying music coming from?

binaryhermit
Posts: 57
Joined: Sun Apr 13, 2014 1:26 am
Location: Lockport, Illinois
Contact: Website

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Jun 29, 2019 5:37 am

8k or so octane score is very usable.

And as far as I can tell, one fast core is gonna beat more not as fast cores in octane. Or at the very least, 2 fast cores beats 4 not so fast cores. Look at the n3060 getting 8.8k-ish vs the n3160 getting 8k-ish. Apparently the 3060's turbo boost is higher than the 3160's because they have the same thermal and power budget, but the n3160 has 4 cores vs the n3060's 2

EDITED to add, the octane score on something like a pi4 might vary somewhat depending on how badly you abuse it thermally, if it's in the official case in a hot room, it wouldn't surprise me if it was going in and out of throttling during an octane test.

energyi
Posts: 184
Joined: Tue Mar 24, 2015 9:39 pm

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Jun 29, 2019 9:31 pm

Saw the official case and thought that the pi4 is going to need a fan because my temps were already higher than I like without a case so didn't purchase. The outdated octane scores I posted didn't include any case in a relatively cool room.
energyi

danormsby
Posts: 1
Joined: Thu Apr 07, 2016 11:02 am

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Jun 29, 2019 9:39 pm

Just tried that on my 2018 iPad Pro. 41794

Cheers,
Dan.

pierro78
Posts: 29
Joined: Fri Nov 29, 2019 5:27 am

10800 on RPi OS 32bit. What about RPi OS 64bit ??

Wed Feb 03, 2021 5:02 pm

about 10800 on RPi4 4GB, chromium on RPi OS 32bit, overclocked to 2.05Ghz (flirc case, vertical & without the "doors" for better air cooling) ...
... did somebody try with RPi OS 64bit ????
thanks !!

Heater
Posts: 18758
Joined: Tue Jul 17, 2012 3:02 pm

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Wed Feb 03, 2021 5:52 pm

Just for fun and reference:

My MacBook Pro M1 with 16GB RAM scores 63463.
Memory in C++ is a leaky abstraction .

ejolson
Posts: 8631
Joined: Tue Mar 18, 2014 11:47 am

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Wed Feb 03, 2021 7:02 pm

Heater wrote:
Wed Feb 03, 2021 5:52 pm
Just for fun and reference:

My MacBook Pro M1 with 16GB RAM scores 63463.
Would you mind running my Pi Pie Chart program on the M1 both single core and OpenMP versions to see how it does?

viewtopic.php?f=63&t=227177

I'm interested in how the M1 compares with the Amazon Graviton 2 as well as the Pi.

I assume you know how the mandatory anti-malware component of MacOS on the M1 sends hashes of each program you run unencrypted through the Internet.

https://sneak.berlin/20201112/your-computer-isnt-yours/

The project manager of the FidoBasic port to the Pico told me this feature makes it easy to count how often each individual developer recompiles the code on which they're working. Unfortunately, when I asked whether that invaded any privacy laws, the reply sounded only like barking. I guess one month is all most New Year's resolutions last anyway.

In spite of the barking, I finally understood that the development was being done off-planet and the local laws on site implied collecting any unencrypted information automatically provided by the operating system was fine. Presumably those who compile more frequently do less thinking and therefore produce more bugs per line of code; however, the project manager ended the call before I was able to confirm my conjecture one way or the other.

Heater
Posts: 18758
Joined: Tue Jul 17, 2012 3:02 pm

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Wed Feb 03, 2021 9:54 pm

ejolson wrote:
Wed Feb 03, 2021 7:02 pm
Would you mind running my Pi Pie Chart program on the M1 both single core and OpenMP versions to see how it does?
I'd Love to. If you have a link to a GitHub repo I can clone it from. Or whatever else. With instructions to build and run it. I'm your man.
ejolson wrote:
Wed Feb 03, 2021 7:02 pm
I assume you know how the mandatory anti-malware component of MacOS on the M1 sends hashes of each program you run unencrypted through the Internet.
Yeah. I have my reservations.

Thing is my partner wanted to blow some cash on toys before we had to pay tax on it at the year end. I did not want yet another big box PC. I have never seen an Intel laptop I like. My MS Surface Pro suffered a battery explosion. So, what the heck, let's try out the new fangled ARM based Mac.

Never used a Mac before. It's a whole new world to me.
ejolson wrote:
Wed Feb 03, 2021 7:02 pm
Presumably those who compile more frequently do less thinking and therefore produce more bugs per line of code;
That is an interesting assertion.

Back in the day (1976 in my case) we wrote code in ALGOL. Which we punched into Holarith cards. We gave the card decks to the priests of the mainframe to run over night. If they deemed it worthy. We got the results back in the morning. Often with only syntax errors. It was a long and tedious process so we spent a lot of time poring over our code, checking the syntax was correct never mind the actual logic, before submitting our jobs.

Today, we throw code into an IDE. It immediately does a partial compilation on it and points out syntactic and other problems with red underlining. Within seconds or milliseconds. We fix it immediately. That is a lot of bookkeeping "busy work" a programmer does not need to do anymore.

Hopefully that gives the programmer more time to think about. the actual logic of his problem, rather than the minutia of syntactic details.

Never mind an IDE. Often I use just plain Vim. Just now I do all my programming in Rust. I have the Rust compiler watching every change I make and recompiling. Second by second.

Do I produce more bugs per line of code?

I cannot imagine that is possible given Rust's anal fussing about types, object life times, race conditions and so on.

But the jury is still out on that.
Last edited by Heater on Wed Feb 03, 2021 10:53 pm, edited 1 time in total.
Memory in C++ is a leaky abstraction .

User avatar
jahboater
Posts: 7552
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Wed Feb 03, 2021 10:50 pm

ejolson wrote:
Wed Feb 03, 2021 7:02 pm
Presumably those who compile more frequently do less thinking and therefore produce more bugs per line of code;
If its compiled and tested, then it is easier to locate any freshly introduced bugs in the small change - and so less time is spent debugging.

Heater
Posts: 18758
Joined: Tue Jul 17, 2012 3:02 pm

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Wed Feb 03, 2021 11:05 pm

I don't know who does the most thinking. I like to avoid it where I can.

That is why we have an accountant to do all the boring tax code work, who in turn has a bookkeeper to do all the boring double entry bookkeeping, who in turn uses some computer program to do all that.

That is why I like a compiler that checks all my work as much as possible.
Memory in C++ is a leaky abstraction .

ejolson
Posts: 8631
Joined: Tue Mar 18, 2014 11:47 am

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Feb 06, 2021 12:59 am

Heater wrote:
Wed Feb 03, 2021 9:54 pm
ejolson wrote:
Wed Feb 03, 2021 7:02 pm
Would you mind running my Pi Pie Chart program on the M1 both single core and OpenMP versions to see how it does?
I'd Love to. If you have a link to a GitHub repo I can clone it from.
The up-to-date version is just the tarball linked in the first post of that thread or here

http://fractal.math.unr.edu/~ejolson/pi ... urrent.tgz

Simply fetch it with something like

Code: Select all

wget http://fractal.math.unr.edu/~ejolson/pi/pichart/pichart-current.tgz
The instructions have not changed much. Unpack the archive, check the Makefile to see if there are any special optimization flags you want to add and then type make. After this there should be two programs: pichart-serial and pichart-openmp.

Run them as

Code: Select all

$ ./pichart-serial -t "M1"
$ ./pichart-openmp -t "M1"
In addition to the text output, there should be two files pichart-serial.svg and pichart-openmp.svg in the working directory. I think it is necessary to convert the svg files to png if you want to upload them as an attachment to a post. This can be done with Gimp, for example.

Since the Pi has only 4 cores it would be nice to also make an OpenMP run on the M1 restricted to 4 cores. In Linux this would be done with

Code: Select all

$ taskset -c 0-3 ./pichart-openmp -t "M1 (4 cores)"
However, the tool to set the affinity mask is likely different in MacOS. If you can figure this out, a four-core run would make a nice point of comparison.

Technologov
Posts: 33
Joined: Wed Jan 20, 2021 4:49 am

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Feb 06, 2021 1:05 am

Rpi1: 103
Rpi2 1392
Rpi3 2360
Rpi4 8630


Using recent Chromium 84 on Raspbian OS.

ejolson
Posts: 8631
Joined: Tue Mar 18, 2014 11:47 am

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Feb 06, 2021 2:20 am

Technologov wrote:
Sat Feb 06, 2021 1:05 am
Rpi1: 103
Rpi2 1392
Rpi3 2360
Rpi4 8630


Using recent Chromium 84 on Raspbian OS.
Is that the original Pi 2 or the updated one? Similarly is that a 3B or 3B+ reported?

Technologov
Posts: 33
Joined: Wed Jan 20, 2021 4:49 am

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Feb 06, 2021 2:22 am

original Pi2 (v1.1) and original Pi3 (not 3+). Model B of each. No case, no cooling.

Heater
Posts: 18758
Joined: Tue Jul 17, 2012 3:02 pm

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Feb 06, 2021 6:06 am

ejolson wrote:
Sat Feb 06, 2021 12:59 am
The up-to-date version is just the tarball linked in the first post of that thread or here
Oddly enough I found all that last night. Which led me down a deep habit hole...

First problem being there is no wget on MacOS out of the box.
So one has to install it using homebrew.
Which does not work on ARM based Macs yet.
So one needs to install the Rosetta x86 emulator to run brew.
Which is easy or impossible to install depending which stack overflow solutions you try...

Of course I could have just used curl but that is not the point.

Second problem being the the compilation fails on M1. Turns out Apple dropped support for OpenMP some time ago and the -fopenmp option is rejected. I gave up trying to add support for OpenMP so I only have a single threaded result for now.

Third problem being that I have never used a Mac prior to getting this M1. So doing almost anything becomes a research project. Like getting a screen shot of the resulting .svg.

Anyway, here we go:
Screenshot 2021-02-06 at 7.46.23.png
Screenshot 2021-02-06 at 7.46.23.png (91 KiB) Viewed 1467 times
Seems I get a final score ratio of 41.

Code: Select all

pichart -- Raspberry Pi Performance Serial version 35

Prime Sieve          P=14630843 Workers=1 Sec=1.18669 Mops=787.339
Merge Sort           N=16777216 Workers=2 Sec=1.52591 Mops=263.878
Fourier Transform    N=4194304 Workers=1 Sec=0.328173 Mflops=1405.89
Lorenz 96            N=32768 K=16384 Workers=2 Sec=0.199056 Mflops=16182.5

My Computer has Raspberry Pi ratio=41.4031
Making pie charts...done.
Memory in C++ is a leaky abstraction .

Heater
Posts: 18758
Joined: Tue Jul 17, 2012 3:02 pm

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Feb 06, 2021 8:10 am

Turns out that today HomeBrew 3.0 was released, with support for the ARM based Mac M1's. Which means I could get GCC v10.2.0 installed on my MacBook. With support for OpenMP.

So here is the multi-processor piechart for M1:
Screenshot 2021-02-06 at 10.03.13.png
Screenshot 2021-02-06 at 10.03.13.png (86.59 KiB) Viewed 1444 times
I now get an M1/PI ratio of 172.

Code: Select all

$ pichart-35 ./pichart-openmp -t "MacBook Pro M1"
pichart -- Raspberry Pi Performance OPENMP version 35

Prime Sieve          P=14630843 Workers=16 Sec=0.192798 Mops=4846.16
Merge Sort           N=16777216 Workers=16 Sec=0.266217 Mops=1512.5
Fourier Transform    N=4194304 Workers=16 Sec=0.0869724 Mflops=5304.83
Lorenz 96            N=32768 K=16384 Workers=16 Sec=0.0896742 Mflops=35921.4

The MacBook Pro M1 has Raspberry Pi ratio=171.661
Making pie charts...done.
That using all the cores available. At least when running "Prime Sieve""top" shows nearly 800% CPU usage. For the others more like 200%, which sounds like they don't parallelise well.

I have no idea how to do any thread affinity on a Mac yet...
Memory in C++ is a leaky abstraction .

Technologov
Posts: 33
Joined: Wed Jan 20, 2021 4:49 am

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Feb 06, 2021 1:49 pm

Are there any good TensorFlow benchmarks?
Good meaning that they don't change every week, so I can use the same benchmarks for years across a large variety of hardware, and have consistent results?

Heater
Posts: 18758
Joined: Tue Jul 17, 2012 3:02 pm

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Feb 06, 2021 1:54 pm

No idea. You could always do what we do. Make our own benchmarks. Take something you have written using tensor flow and define it as your benchmark. Invite others to run it for comparisons.
Memory in C++ is a leaky abstraction .

ejolson
Posts: 8631
Joined: Tue Mar 18, 2014 11:47 am

Re: Outdated Octane Benchmark Score on Raspberry Pi 4 1 GB RAM

Sat Feb 06, 2021 4:35 pm

Heater wrote:
Sat Feb 06, 2021 8:10 am
Turns out that today HomeBrew 3.0 was released, with support for the ARM based Mac M1's. Which means I could get GCC v10.2.0 installed on my MacBook. With support for OpenMP.

So here is the multi-processor piechart for M1:

Screenshot 2021-02-06 at 10.03.13.png

I now get an M1/PI ratio of 172.

Code: Select all

$ pichart-35 ./pichart-openmp -t "MacBook Pro M1"
pichart -- Raspberry Pi Performance OPENMP version 35

Prime Sieve          P=14630843 Workers=16 Sec=0.192798 Mops=4846.16
Merge Sort           N=16777216 Workers=16 Sec=0.266217 Mops=1512.5
Fourier Transform    N=4194304 Workers=16 Sec=0.0869724 Mflops=5304.83
Lorenz 96            N=32768 K=16384 Workers=16 Sec=0.0896742 Mflops=35921.4

The MacBook Pro M1 has Raspberry Pi ratio=171.661
Making pie charts...done.
That using all the cores available. At least when running "Prime Sieve""top" shows nearly 800% CPU usage. For the others more like 200%, which sounds like they don't parallelise well.

I have no idea how to do any thread affinity on a Mac yet...
It's worth pointing out that each parallel task tries using different numbers of workers starting with k=2n where n is the number hardware threads, then trying n, then n/2 and so forth until only one is left. This serves two purposes:
  • It automatically tunes the test to optimise tradeoffs in scheduling latency, memory bandwidth, symmetric multi-threading and additional parallelization.
    • It gives the CPU a chance to cool off so underengineered cooling has less effect on later parts of the run.
    In the case of the M1, it would appear the optimal values occurred with 16 workers in each of the different tests. That uniformity doesn't occur with many systems.

    Since the parallel results were made using the GNU toolchain, it would be nice to run the serial tests with gcc. My suspicion is single-thread floating-point performance on the M1 with the Apple compiler is likely to be much faster than gcc.

    Return to “General discussion”