SWAP on Raspberry Pi / SD = AUTOCRASH

[globiboulga666]

I wonder who had the brilliant idea to enable a SWAP system by default on Raspbian for the Raspberry Pi, which runs on SD cards.

This is a perfect way to ensure that the device will get completely stuck for hours as soon as something is trying to use a little bit too much of RAM, instead of just interrupting it (or letting it handle std::bad_alloc if it's correctly written)

Now and again, I have to do some savage power interrupt while the SD card operation is 100% (and may trash some files) because Raspbian is delivered with this autocrash function enabled

Then I will type systemctl disable dphys-swapfile && reboot

This already was a problem long time ago and we are almost 2020, it may be time to get rid of that "enabled by default autocrash" some day soon or late don't you think ?

[topguy]

Stuck for hours is not the same as a crash if it recovers in the end.

[trejan]

What autocrash? I've never had it actually crash. It just goes slow because it is busy swapping. If your Pi crashes immediately when using swap then there is something wrong with your system and it isn't because swap is enabled.

The system going slowly because it is swapping is still preferable IMO to random processes getting hit by the OOM killer or your software not running at all. Another factor for having swap is that the Raspbian images are designed to be backwards compatible all the way back to the first Pi with only 256MB RAM.

[bjtheone]

I would gently suggest that as shipped Raspbian is setup to deliver the best response to the largest volume of folks. Said "best response" may not be suitable for your particular application nor is it suitable for corner cases and high swapping loads. Given that there is not vast hue and cry about it tends to support this theory.

We have been using Pi's for years to run vision processing for robotics and have never encountered this issue. We are moving from 3s to 4s this season to up the frame rate we get, but were not having freezing issues.

There are a number of ways you can address this: change the load, change the amount of RAM (buy a bigger Pi), change the swap device (SSD), or change swap (either the swappiness factor, the type of swap, or just disable swap).

For example, trying to use a 3B+, with 1 GB of ram as a general purpose computer and medium browsing activity is not a particularly wonderful experience. Moving from a SD to a SSD for the file system and more importantly adding 1 GB of swap on said SSD makes it much much better. Still not perfect, but usable. Moving from a 3b+ to a 4GB 4B solves the rest of the issue. With 4GB it basically stops swapping and the SSD makes it more responsive.

[globiboulga666]

Thanks to all of you for your answers

As I was working remotely on some Pi with VNC I was pretty grumpy against that sorry !
Building some OpenCV based App on a Pi3 B+ (1GB RAM) until the RAM usage increased near to maximum and the remote access (be it SSH or VNC) wasn't responding anymore.

After having a look at "kernel.log" file it seems that it wasn't only stuck for several hours (topguy, when you have work to do, it is the same as crashed ) but actually it really crashed :

Code: Select all

Nov  7 14:18:02 Pi3-Gate-Picrobe kernel: [953180.541203] INFO: task kworker/3:1:6368 blocked for more than 120 seconds.
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541212]       Tainted: G         C        4.19.75-v7+ #1270
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541215] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541219] kworker/3:1     D    0  6368      2 0x00000000
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541238] Workqueue: events_freezable mmc_rescan
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541258] [<80831314>] (__schedule) from [<80831984>] (schedule+0x50/0xa8)
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541266] [<80831984>] (schedule) from [<80686c38>] (__mmc_claim_host+0x120/0x228)
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541276] [<80686c38>] (__mmc_claim_host) from [<80686d78>] (mmc_get_card+0x38/0x3c)
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541284] [<80686d78>] (mmc_get_card) from [<8068fde4>] (mmc_sd_detect+0x24/0x7c)
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541292] [<8068fde4>] (mmc_sd_detect) from [<806893a0>] (mmc_rescan+0x1cc/0x39c)
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541303] [<806893a0>] (mmc_rescan) from [<8013bf64>] (process_one_work+0x170/0x458)
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541312] [<8013bf64>] (process_one_work) from [<8013c2a8>] (worker_thread+0x5c/0x5a4)
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541321] [<8013c2a8>] (worker_thread) from [<80142594>] (kthread+0x138/0x168)
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541331] [<80142594>] (kthread) from [<801010ac>] (ret_from_fork+0x14/0x28)
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541335] Exception stack(0x86653fb0 to 0x86653ff8)
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541340] 3fa0:                                     00000000 00000000 00000000 00000000
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541345] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Nov  7 14:18:06 Pi3-Gate-Picrobe kernel: [953180.541350] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421710] INFO: task kworker/3:1:6368 blocked for more than 120 seconds.
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421724]       Tainted: G         C        4.19.75-v7+ #1270
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421727] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421736] kworker/3:1     D    0  6368      2 0x00000000
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421765] Workqueue: events_freezable mmc_rescan
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421795] [<80831314>] (__schedule) from [<80831984>] (schedule+0x50/0xa8)
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421807] [<80831984>] (schedule) from [<80686c38>] (__mmc_claim_host+0x120/0x228)
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421815] [<80686c38>] (__mmc_claim_host) from [<80686d78>] (mmc_get_card+0x38/0x3c)
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421824] [<80686d78>] (mmc_get_card) from [<8068fde4>] (mmc_sd_detect+0x24/0x7c)
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421834] [<8068fde4>] (mmc_sd_detect) from [<806893a0>] (mmc_rescan+0x1cc/0x39c)
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421846] [<806893a0>] (mmc_rescan) from [<8013bf64>] (process_one_work+0x170/0x458)
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421858] [<8013bf64>] (process_one_work) from [<8013c2a8>] (worker_thread+0x5c/0x5a4)
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421870] [<8013c2a8>] (worker_thread) from [<80142594>] (kthread+0x138/0x168)
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421882] [<80142594>] (kthread) from [<801010ac>] (ret_from_fork+0x14/0x28)
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421886] Exception stack(0x86653fb0 to 0x86653ff8)
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421894] 3fa0:                                     00000000 00000000 00000000 00000000
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421900] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Nov  7 14:20:05 Pi3-Gate-Picrobe kernel: [953303.421905] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
Nov  7 14:22:07 Pi3-Gate-Picrobe kernel: [953426.301928] INFO: task kworker/3:1:6368 blocked for more than 120 seconds.
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.301939]       Tainted: G         C        4.19.75-v7+ #1270
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.301942] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.301946] kworker/3:1     D    0  6368      2 0x00000000
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.301969] Workqueue: events_freezable mmc_rescan
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.301990] [<80831314>] (__schedule) from [<80831984>] (schedule+0x50/0xa8)
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302000] [<80831984>] (schedule) from [<80686c38>] (__mmc_claim_host+0x120/0x228)
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302008] [<80686c38>] (__mmc_claim_host) from [<80686d78>] (mmc_get_card+0x38/0x3c)
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302015] [<80686d78>] (mmc_get_card) from [<8068fde4>] (mmc_sd_detect+0x24/0x7c)
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302022] [<8068fde4>] (mmc_sd_detect) from [<806893a0>] (mmc_rescan+0x1cc/0x39c)
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302032] [<806893a0>] (mmc_rescan) from [<8013bf64>] (process_one_work+0x170/0x458)
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302041] [<8013bf64>] (process_one_work) from [<8013c2a8>] (worker_thread+0x5c/0x5a4)
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302050] [<8013c2a8>] (worker_thread) from [<80142594>] (kthread+0x138/0x168)
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302059] [<80142594>] (kthread) from [<801010ac>] (ret_from_fork+0x14/0x28)
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302063] Exception stack(0x86653fb0 to 0x86653ff8)
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302068] 3fa0:                                     00000000 00000000 00000000 00000000
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302074] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Nov  7 14:22:08 Pi3-Gate-Picrobe kernel: [953426.302079] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182334] INFO: task kworker/3:1:6368 blocked for more than 120 seconds.
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182346]       Tainted: G         C        4.19.75-v7+ #1270
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182349] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182355] kworker/3:1     D    0  6368      2 0x00000000
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182377] Workqueue: events_freezable mmc_rescan
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182401] [<80831314>] (__schedule) from [<80831984>] (schedule+0x50/0xa8)
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182409] [<80831984>] (schedule) from [<80686c38>] (__mmc_claim_host+0x120/0x228)
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182419] [<80686c38>] (__mmc_claim_host) from [<80686d78>] (mmc_get_card+0x38/0x3c)
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182429] [<80686d78>] (mmc_get_card) from [<8068fde4>] (mmc_sd_detect+0x24/0x7c)
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182437] [<8068fde4>] (mmc_sd_detect) from [<806893a0>] (mmc_rescan+0x1cc/0x39c)
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182450] [<806893a0>] (mmc_rescan) from [<8013bf64>] (process_one_work+0x170/0x458)
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182460] [<8013bf64>] (process_one_work) from [<8013c2a8>] (worker_thread+0x5c/0x5a4)
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182469] [<8013c2a8>] (worker_thread) from [<80142594>] (kthread+0x138/0x168)
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182480] [<80142594>] (kthread) from [<801010ac>] (ret_from_fork+0x14/0x28)
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182487] Exception stack(0x86653fb0 to 0x86653ff8)
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182492] 3fa0:                                     00000000 00000000 00000000 00000000
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182498] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Nov  7 14:24:10 Pi3-Gate-Picrobe kernel: [953549.182502] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062641] INFO: task kworker/3:1:6368 blocked for more than 120 seconds.
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062652]       Tainted: G         C        4.19.75-v7+ #1270
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062655] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062659] kworker/3:1     D    0  6368      2 0x00000000
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062681] Workqueue: events_freezable mmc_rescan
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062702] [<80831314>] (__schedule) from [<80831984>] (schedule+0x50/0xa8)
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062712] [<80831984>] (schedule) from [<80686c38>] (__mmc_claim_host+0x120/0x228)
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062720] [<80686c38>] (__mmc_claim_host) from [<80686d78>] (mmc_get_card+0x38/0x3c)
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062727] [<80686d78>] (mmc_get_card) from [<8068fde4>] (mmc_sd_detect+0x24/0x7c)
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062735] [<8068fde4>] (mmc_sd_detect) from [<806893a0>] (mmc_rescan+0x1cc/0x39c)
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062746] [<806893a0>] (mmc_rescan) from [<8013bf64>] (process_one_work+0x170/0x458)
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062756] [<8013bf64>] (process_one_work) from [<8013c2a8>] (worker_thread+0x5c/0x5a4)
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062764] [<8013c2a8>] (worker_thread) from [<80142594>] (kthread+0x138/0x168)
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062774] [<80142594>] (kthread) from [<801010ac>] (ret_from_fork+0x14/0x28)
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062778] Exception stack(0x86653fb0 to 0x86653ff8)
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062784] 3fa0:                                     00000000 00000000 00000000 00000000
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062789] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Nov  7 14:26:13 Pi3-Gate-Picrobe kernel: [953672.062794] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
Nov  7 14:28:16 Pi3-Gate-Picrobe kernel: [953794.942733] INFO: task kworker/3:1:6368 blocked for more than 120 seconds.
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942743]       Tainted: G         C        4.19.75-v7+ #1270
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942746] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942749] kworker/3:1     D    0  6368      2 0x00000000
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942770] Workqueue: events_freezable mmc_rescan
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942790] [<80831314>] (__schedule) from [<80831984>] (schedule+0x50/0xa8)
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942800] [<80831984>] (schedule) from [<80686c38>] (__mmc_claim_host+0x120/0x228)
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942808] [<80686c38>] (__mmc_claim_host) from [<80686d78>] (mmc_get_card+0x38/0x3c)
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942815] [<80686d78>] (mmc_get_card) from [<8068fde4>] (mmc_sd_detect+0x24/0x7c)
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942823] [<8068fde4>] (mmc_sd_detect) from [<806893a0>] (mmc_rescan+0x1cc/0x39c)
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942832] [<806893a0>] (mmc_rescan) from [<8013bf64>] (process_one_work+0x170/0x458)
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942842] [<8013bf64>] (process_one_work) from [<8013c2a8>] (worker_thread+0x5c/0x5a4)
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942850] [<8013c2a8>] (worker_thread) from [<80142594>] (kthread+0x138/0x168)
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942860] [<80142594>] (kthread) from [<801010ac>] (ret_from_fork+0x14/0x28)
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942864] Exception stack(0x86653fb0 to 0x86653ff8)
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942869] 3fa0:                                     00000000 00000000 00000000 00000000
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942876] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Nov  7 14:28:17 Pi3-Gate-Picrobe kernel: [953794.942881] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
Nov  7 14:30:19 Pi3-Gate-Picrobe kernel: [953917.823142] INFO: task kworker/3:1:6368 blocked for more than 120 seconds.
Nov  7 14:30:20 Pi3-Gate-Picrobe kernel: [953917.823155]       Tainted: G         C        4.19.75-v7+ #1270
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823158] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823162] kworker/3:1     D    0  6368      2 0x00000000
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823187] Workqueue: events_freezable mmc_rescan
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823210] [<80831314>] (__schedule) from [<80831984>] (schedule+0x50/0xa8)
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823221] [<80831984>] (schedule) from [<80686c38>] (__mmc_claim_host+0x120/0x228)
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823231] [<80686c38>] (__mmc_claim_host) from [<80686d78>] (mmc_get_card+0x38/0x3c)
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823239] [<80686d78>] (mmc_get_card) from [<8068fde4>] (mmc_sd_detect+0x24/0x7c)
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823249] [<8068fde4>] (mmc_sd_detect) from [<806893a0>] (mmc_rescan+0x1cc/0x39c)
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823263] [<806893a0>] (mmc_rescan) from [<8013bf64>] (process_one_work+0x170/0x458)
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823275] [<8013bf64>] (process_one_work) from [<8013c2a8>] (worker_thread+0x5c/0x5a4)
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823284] [<8013c2a8>] (worker_thread) from [<80142594>] (kthread+0x138/0x168)
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823296] [<80142594>] (kthread) from [<801010ac>] (ret_from_fork+0x14/0x28)
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823311] Exception stack(0x86653fb0 to 0x86653ff8)
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823317] 3fa0:                                     00000000 00000000 00000000 00000000
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823322] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Nov  7 14:30:21 Pi3-Gate-Picrobe kernel: [953917.823327] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703349] INFO: task kworker/3:1:6368 blocked for more than 120 seconds.
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703358]       Tainted: G         C        4.19.75-v7+ #1270
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703362] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703366] kworker/3:1     D    0  6368      2 0x00000000
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703385] Workqueue: events_freezable mmc_rescan
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703408] [<80831314>] (__schedule) from [<80831984>] (schedule+0x50/0xa8)
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703417] [<80831984>] (schedule) from [<80686c38>] (__mmc_claim_host+0x120/0x228)
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703426] [<80686c38>] (__mmc_claim_host) from [<80686d78>] (mmc_get_card+0x38/0x3c)
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703433] [<80686d78>] (mmc_get_card) from [<8068fde4>] (mmc_sd_detect+0x24/0x7c)
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703442] [<8068fde4>] (mmc_sd_detect) from [<806893a0>] (mmc_rescan+0x1cc/0x39c)
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703453] [<806893a0>] (mmc_rescan) from [<8013bf64>] (process_one_work+0x170/0x458)
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703462] [<8013bf64>] (process_one_work) from [<8013c2a8>] (worker_thread+0x5c/0x5a4)
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703471] [<8013c2a8>] (worker_thread) from [<80142594>] (kthread+0x138/0x168)
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703480] [<80142594>] (kthread) from [<801010ac>] (ret_from_fork+0x14/0x28)
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703483] Exception stack(0x86653fb0 to 0x86653ff8)
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703488] 3fa0:                                     00000000 00000000 00000000 00000000
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703495] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Nov  7 14:32:22 Pi3-Gate-Picrobe kernel: [954040.703499] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
Nov  7 14:34:25 Pi3-Gate-Picrobe kernel: [954163.583809] INFO: task kworker/3:1:6368 blocked for more than 120 seconds.
Nov  7 14:34:26 Pi3-Gate-Picrobe kernel: [954163.583818]       Tainted: G         C        4.19.75-v7+ #1270
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583822] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583827] kworker/3:1     D    0  6368      2 0x00000000
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583853] Workqueue: events_freezable mmc_rescan
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583876] [<80831314>] (__schedule) from [<80831984>] (schedule+0x50/0xa8)
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583887] [<80831984>] (schedule) from [<80686c38>] (__mmc_claim_host+0x120/0x228)
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583896] [<80686c38>] (__mmc_claim_host) from [<80686d78>] (mmc_get_card+0x38/0x3c)
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583904] [<80686d78>] (mmc_get_card) from [<8068fde4>] (mmc_sd_detect+0x24/0x7c)
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583913] [<8068fde4>] (mmc_sd_detect) from [<806893a0>] (mmc_rescan+0x1cc/0x39c)
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583926] [<806893a0>] (mmc_rescan) from [<8013bf64>] (process_one_work+0x170/0x458)
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583935] [<8013bf64>] (process_one_work) from [<8013c2a8>] (worker_thread+0x5c/0x5a4)
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583944] [<8013c2a8>] (worker_thread) from [<80142594>] (kthread+0x138/0x168)
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583953] [<80142594>] (kthread) from [<801010ac>] (ret_from_fork+0x14/0x28)
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583957] Exception stack(0x86653fb0 to 0x86653ff8)
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583963] 3fa0:                                     00000000 00000000 00000000 00000000
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583970] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Nov  7 14:34:27 Pi3-Gate-Picrobe kernel: [954163.583975] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
Nov  7 14:21:50 Pi3-Gate-Picrobe kernel: [    0.000000] Booting Linux on physical CPU 0x0
Nov  7 14:21:50 Pi3-Gate-Picrobe kernel: [    0.000000] Linux version 4.19.75-v7+ (dom@buildbot) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611)) #1270 SMP Tue Sep 24 18:45:11 BST 2019
[...]

This is not the first time that it happens but I don't know if this happenned into logs at each time.

[trejan]

Posting useful information is much more productive than an angry rant :p

It looks to be the SD card getting stuck and refusing to respond before the timeout. Are you using a Samsung EVO+ card? There are several issue reports on the bugtracker and that seems to be a common factor in several issues along with high memory usage causing swapping and heavy I/O to the SD card at the same time.

You can try the workaround mentioned on https://github.com/raspberrypi/linux/is ... -456787673 which adjusts how the kernel deals with swap.

If you have a different card then I'd suggest cloning your install onto that and trying that as well. It may be the card itself is failing somehow.