scheduler ignores niceness

[olsen]

I observed a strange behavior on Raspbian: sometimes the niceness of processes gets ignored.
Raspbian is up to date and I'm using the latest firmware (rpi-update).

Scenario:
I start two cpu-greedy processes (cat /dev/zero > /dev/null). I give one of these processes a niceness of 19, the other one gets the default niceness of 0.
What I (sometimes!) observe is that both processes get the same amount of cpu:

Code: Select all

top - 21:35:41 up 1 day,  7:05,  8 users,  load average: 2,40, 2,50, 2,04
Tasks: 118 total,   3 running, 109 sleeping,   6 stopped,   0 zombie
%Cpu(s):  6,4 us, 92,7 sy,  0,9 ni,  0,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
KiB Mem:    497544 total,   474864 used,    22680 free,    13848 buffers
KiB Swap:   102396 total,      108 used,   102288 free,   158676 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND                                                           
22104 pi        39  19  3748  524  424 R  45,7  0,1   0:13.51 cat /dev/zero                                                     
22103 pi        20   0  3748  524  424 R  45,4  0,1   0:14.58 cat /dev/zero                                                    

As you can see, processes 22103 and 22104 are my greedy processes. Process 22104 has a nice value of 19, process 22103 has a nice value of 0.
But: both processes get around 45% cpu.
This should not happen on a single core processor!

As I can see from more complex experiments, in such a situation the niced process really gets too much cpu, it is not a failure in top.

Surprisingly, this behavior shows only sometimes. I couldn't figure out the circumstances yet. I have seen it on Raspbmc as well.

Has anyone else seen this behavior?

[olsen]

Ok, the cause for the described behavior seems to be the 'miracle patch'. This patch seems to be enabled by default in Raspbian.
Its purpose is to increase responsiveness of the system. If you run a (cpu bound) video player and a simple compile job, both get ~50% cpu. If the compile job forks 50 child processes, without the 'miracle patch' your video player would only get ~2% cpu. The miracle patch treats the compile job as a single job, no matter how many child processes it forks, so the video player still gets ~50% cpu while the compile processes only get ~1% cpu.

This is a neat idea; anyway this patch totally sucks because it effectively breaks the semantics of nice.
When I start a compile job with a niceness of 19, I want this job to only get otherwise unused cpu cycles. This means when I start a compile job (niceness 19) and a video player, the video player should get 100% cpu if necessary, regardless of the number of forked compile jobs.
This is the normal behavior. The miracle patch breaks this behavior; in the described scenario the video player only gets 50% instead of 100% cpu.

So, anyone who tries to do some interactive work on the pi while running a compile job (e.g. nice -n19 make); put the following line into /etc/sysctl.d/local.conf:

Code: Select all

kernel.sched_autogroup_enabled = 0

reboot, try again and enjoy the power of your pi!

(just calling 'sudo sysctl kernel.sched_autogroup_enabled=0' changed the value but had no effect on my system)

See

• http://www.phoronix.com/scan.php?page=a ... ideo&num=1
• http://serverfault.com/questions/456362 ... -processes
• http://choon.net/forum/read.php?21,89570

[saintdev]

Heh, Con (Kolivas) predicted something like this would happen.

He attempted something very similar with BFS, before the patch was even proposed to mainline. He gave up on it because it caused too many other problems, like the one you have found. Also nice 19 isn't strictly idle time only, it still has a very small percentage of time allocated. For true idle, you need to set the process SCHED_IDLE. I think this was implemented in mainline (there was at least a patch for it at one point).

Code: Select all

schedtool -D make -j2

[olsen]

Didn't know schedtool before; I'll give it a try. Thanks.

[AutoStatic]

Why not use chrt?

[saintdev]

Why not use chrt?

I wasn't aware of it, tbh.

[rew]

Also nice 19 isn't strictly idle time only, it still has a very small percentage of time allocated.

When I started using Unix in the latter half of the 1980ies, nice 19 was a special case that would disable the behaviour that lower-priority processes would still get some CPU time. So IMHO, a nice-19 process should have its "SCHED_IDLE" turned on automatically. (or the kernel should behave as if....)