Is it possible to add a new forum Parallel programming

[richrarobi]

What I thought I said was that Binary, Octal and Hexadecimal are not as (initially) important as they once were (I used punched paper tape) - As I remember we spent a lot of time learning these basics (Not BASIC) first, before we reached apparently more productive subjects like (yuk) CESIL and (better) Fortran. (I taught myself Coral 66)
Yes, they are important, at the engineering level. Even today Python error messages are baffling to many ( and I hate(d) core dumps!)

True, I don't know what is in the curriculum, but it is a matter of opinion what would be more useful as background knowledge at the starter levels and after. (Dare I say it, but teachers probably won't get enough support anyway?)

My point was, and still is, that higher theory (ideas) would be understood, even if not taught to a great depth. Some would maybe be encouraged to investigate further. Half the battle in knowledge is knowing what the question is. I know It isn't easy to decide what is best. I am NOT arguing about something I cannot change.

RichR

[DougieLawson]

You can't make sense of a SQL database unless you know what the bits and bytes mean. There's no avoiding that. That's the whole raison d'être for the Raspberry Pi and the Raspberry Pi Foundation's educational charity objectives.

Making sense (or trying to) of SQL database pages in post-mortem dumps is something I used to do for my day job at IBM.

[DavidS]

On the original topic:
I see no reason not to increase the awareness of multiprocessing.

There are many applications where it is the best way around, and cache coherency is not as arcane as some in this thread are attempting to make it sound. Just make sure to flush the correct cache lines at the correct time, make the correct pages write through/read through, make sure that all semaphores are non cachable, and your have 99% of cache issues covered (there are a few rare exceptions).

Most applications that would have the most benefit from parallel processing do not require continuous access to the same subset of the data set being worked with. At least that has been my experience.

And true parallel processing has been an issue on the RPi ever since we got the complete datasheet for the VideoCore IV, as that is a multi core processor.

On personal computing, there have been multi processor desktop personal computers ever since the early 1980's, that were able to run all processors at the same time. So all these issues had to be learned by kids in my generation, we did not always know the 'correct' terms, though we knew what we had to watch out for. And if you have to teach these basic concepts to a 13 year old then that kid has no interest in programming, or they would have learned it on there own years earlier.

[Heater]

DavidS,
I agree, the possibility of parallel processing should not be hidden. It should even be presented as "normal".

Just make sure to flush the correct cache lines at the correct time, make the correct pages write through/read through, make sure that all semaphores are non cachable,

This is all low level implementation specific detail that one should not have to worry about. And rather orthogonal to the concepts of parallel processing.

I'm more thinking of languages that make parallel code a natural thing. Like the xc language for the multi-threaded, multi-cored, XMOS processors where one can write things like:

Code: Select all

par {
    f();
    g();
}

Here functions f and g get run in parallel. Clean, simple.

[ejolson]

We have these ideas about what young minds can or cannot do and we teach them accordingly.

Luckily one in every million or billion young minds is way ahead of the game. Perhaps through genius or excessive curiosity or weird obsession.

Along these lines the web page for the world record computation of Pi reads

y-cruncher - A Multi-Threaded Pi-Program
From a high-school project that went a little too far...

which implies that parallel computation is within reach of at least some high school students.

While multi-CPU computers have been available for home use since the mid 1990's, the fact that multi-core computers are now the norm, I believe, makes parallel processing much more central to the teaching of computer literacy. This additional processing power also makes the Raspberry Pi 2B a more effective desktop computer as well as opening the door to more interesting autonomous robotics and other physical computing projects.

I'm more thinking of languages that make parallel code a natural thing. Like the xc language for the multi-threaded, multi-cored, XMOS processors

In Intel/MIT Cilk, which is now working on the Raspberry Pi 2B, the equivalent code is

Code: Select all

    cilk_spawn f();
    g();
    cilk_sync;

This also abstracts the details of thread synchronization and does so by means of the surprisingly efficient scheduling technique called work stealing.

[ejolson]

...I have tried to parallelize my own Fast Fourier Transform with OpenMP. So I know first hand it is very hard to get performance to scale linearly with the number of cores available.
...

I thought you might be interested in seeing an FFT coded with MIT/Intel Cilk. Using a simple parallel recursive implementation I obtain a 3.4 fold increase in performance using 4 cores on the Raspberry Pi 2B. More information is at

viewtopic.php?p=736504#p736504

[Heater]

ejolson,

You are right. I am very interested. I have not looked at Cilk before. I do love that recursive FFT algorithm, it's so elegant. I found that on my 4 core PC the recursive algorithm was quite a bit slower than the normal approach for small sample sizes, but as the sample size went up to a million or whatever it started to be a winner. Something to do with the way it uses the caches I guessed.

My FFT was an weird thing. Integer maths only and only for a sample size of 1024. The C version was a prototype for creating a assembler version to run on the Parallax Inc. Propeller MCU. I used the standard 3 nested loop and bit reversal approach.

The Propeller has 8 cores so I had to try out OpenMP to see what it did. I forget the speed up now, the OpenMP in their version of GCC was a bit buggy at the time. I should try it again. I do remember it gave me headache trying to figure out how to parallelize it with OMP.

The code is here if you are curious: https://github.com/ZiCog/fftbench

[ejolson]

My FFT was an weird thing. Integer maths only and only for a sample size of 1024. The C version was a prototype for creating a assembler version to run on the Parallax Inc. Propeller MCU. I used the standard 3 nested loop and bit reversal approach.

I've looked at the code. It see that it was used for benchmarking various hardware. Do you have a list of the timings? It would be interesting to see how the Pi 2B does in comparison.

[dmc1954]

Sorry, this post has nothing to do with FFTs.

When teaching parallel programming it is sometimes helpful for the student to visualize the work being done on a multi-core processor like RPI2's armv7. A lot of people know that running in a separate window the linux command top and then type "1" to list of load on the different cores, but visually it's pretty bland.



There is another linux utility written by Nigel Griffiths called nmon and type "c" to toggle the the CPU utilization display. Now the work load on the each core is presented as a colorful dynamic bar graph.

Main nmon display with help information



Nmon CPU utilization display showing four active cores


Nmon CPU utilization display showing two active cores


The U is the percentage of time that the core is spent in user mode
The s is the percentage of time that the core is spent in system mode

The nmon is not currently installed by default with NOOBS or raspbian, but can be added later using:

sudo apt-get install nmon

More information about nmon is available at http://nmon.sourceforge.net/pmwiki.php

Enjoy!

Ok, back to the FFT discussion.

[richrarobi]

wiki.erights.org/wiki/Walnut/Distributed_Computing

wiki.erights.org/wiki/Walnut/Distribute ... _Computing

As in the document above, and in many modern languages, Haskell, scala, etc and in some add on libraries for concurrency, threading models are changing.
Forget multi-cores, use those cores for smaller, distributed tasks across your rack of pi's. Use promises, when-catch, etc. Much more fun.
RichR

[ejolson]

http://wiki.erights.org/wiki/Walnut/Dis ... _Computing

Forget multi-cores, use those cores for smaller, distributed tasks across your rack of pi's. Use promises, when-catch, etc.

Have you got this E programming language to work on the Pi? How does it perform? I couldn't find any benchmark results on the website.

While this post is clearly about parallel and concurrent programming, the E programming language is pretty far from C/C++ and its derivatives. If you start a new thread in "Other Languages" could you please link to it from here?

Also, while we're on the topic of other programming languages for parallel and concurrent processing, I wonder if anyone here has experience with Julia http://julialang.org/. Google search indicates that Julia should run on the Raspberry Pi.

[richrarobi]

Because E is available as a jvm language it is easy to get it running on pi http://erights.org/ - I tried it briefly (in java only form - an eclipse plugin is available) , but dropped back to jruby as I thought it would be easier to do pi things with that. Also the more popular languages like jruby have better clients for tools like couchdb and rabbitmq. It is worth reading the chapter and trying the language if you are interested in language development.
More notes on places to look here:
viewtopic.php?f=63&t=101480&p=771066#p771066

On Julialang, It looks promising, but (my opinion) it isn't quite there yet. You might try groovy or go(lang)? I did try to install Julia, but the clvm install threw me. At that point I went back to ruby and fighting the problems I had with jruby.
RichR

[richrarobi]

viewtopic.php?f=31&t=114106

RichR

[dmc1954]

Check this out.

https://github.com/CODE-at-TACC/summer- ... raph500.md

The Texas Advanced Computing Center is teaching a two week class to high school students about the Raspbery Pi 2 and an introduction to parallel computing.

I wonder where they will place in the Graph500 list.

[ejolson]

Check this out.

https://github.com/CODE-at-TACC/summer- ... raph500.md

That writeup could be used as a lesson plan for a similar camp at another school. Are any other materials available? Maybe a new thread with an expanded summary in the "Teaching and Learning Resources" forum topic would allow teachers to make use of these materials. Otherwise, the link will remain buried in this rather long and meandering thread about parallel processing.

By the way, I noticed that the graphics showing screen-shots of nmon running have mysteriously disappeared from your post viewtopic.php?p=740546#p740546. Is there any way to fix this?

[dmc1954]

The high school students attending the "CODE-at-TACC" summer class were able to place 64 in the July 2015 Graph500 list announced at ISC 2015 in Frankfurt, Germany. see http://www.graph500.org/results_jul_2015.

If you were a student in this class then Congratulations!

Ejolson, to answer your question the complete course material is at https://github.com/CODE-at-TACC/summer-2015/.