Has Raspberry Pi OS some specificities to care for SD cards?

[rin67630]

I just wondered: Has Raspberry Pi OS implemented some specificities to care for SD cards wearing reduction?
Or is it just a plain Debian writing on the SD cards as if they were SSDs?

[dickon]

The latter. It's why I netboot everything important.

[thagrol]

The latter. It's why I netboot everything important.

That's also why I use tmpfs for /tmp, put /home on a different physical device to the OS, and do the same with swap if I need any.

[redvli]

I just wondered: Has Raspberry Pi OS implemented some specificities to care for SD cards wearing reduction?
Or is it just a plain Debian writing on the SD cards as if they were SSDs?

plain Debian normally has the option to do installation, then you can choose which filesystem to use. If you use Ext4, there is not much you can do to prevent wearing out. However, there is F2fs for example, designed for flash-storage. Or Btrfs, which can be configured and use in such a way that it is also flash-friendly. And don't forget to enable fstrim.timer.
I use all my Pi's Raspberry Pi OS with Btrfs as rootfs and use a tool called snapper to lock readonly snaphots. New data is then written to new locations until the snapshots get deleted.Can be done automatically based on quotas for various sorts of data. I let a backup tool do it at the time the snapshots are cloned to my NAS.

[dickon]

All of which are -- or should be, on reasonably reputable cards -- utterly irrelevant. The cards themselves should do wear-levelling to fix this problem, but they're all eventually prone to failure, and it's simply better to avoid the issue in the first place.

[redvli]

All of which are -- or should be, on reasonably reputable cards -- utterly irrelevant. The cards themselves should do wear-levelling to fix this problem, but they're all eventually prone to failure, and it's simply better to avoid the issue in the first place.

Right, in the end it will all fail. If you buy a new card, buy an 'expensive' one with 'guaranteed perfect wear-leveling'. That would be then the advice. But not all is perfect with existing stuff. I also thought some time ago, from now on I do everything via netboot/nfsroot. But some use-cases are just wifi/wireless (or not network-cabled at least) or have no persistent IP network connection at all. But maybe people should read articles like https://en.wikipedia.org/wiki/Write_amplification to see if it is all irrelevant.

[jahboater]

The latter. It's why I netboot everything important.

That's also why I use tmpfs for /tmp, put /home on a different physical device to the OS, and do the same with swap if I need any.

I use tmpfs for /var/log too, though its quite small.

Code: Select all

tmpfs /tmp tmpfs defaults,noatime 0 0
tmpfs /var/log tmpfs defaults,noatime,size=16m 0 0

You may reduce writes by extending the commit interval. By default the dirty blocks in the disk cache are flushed very 5 seconds - very conservative.
Add this to the / ext4 line in /etc/fstab:

Code: Select all

,noatime,commit=600

to change it to every 10 minutes (say).
This reduces writes and at the same time improves performance.

noatime is probably the default now, it avoids a disk write for every disk read.

[Daniel Gessel]

You may reduce writes by extending the commit interval. By default the dirty blocks in the disk cache are flushed very 5 seconds - very conservative.
Add this to the / ext4 line in /etc/fstab:

Code: Select all

,noatime,commit=600

to change it to every 10 minutes (say).
This reduces writes and at the same time improves performance.

noatime is probably the default now, it avoids a disk write for every disk read.

I don't think noatime is default, at least not on some Linux distros (relatime shows up alot). But it looks like it's added in /etc/fstab for root on the latest Raspberry Pi OS builds.

I also add it to /etc/udisks2/mount_options.conf

Code: Select all

[defaults]
defaults=noatime

I think I verified this adds noatime to removable drives when they are mounted in /media/<user>. I'll add the commit=600 in to the mix. Thanks!

[thagrol]

noatime is probably the default now, it avoids a disk write for every disk read.

Depends what you mean by "default".

IIRC it's not in the "defaults" mount option but at least on RPiOS it is in the options list in fstab for the root partition.

So fresh out of the box RPiOS will have the noatime mount option set on the root partition. Other OS and user modified fstabs? I have no idea.

[dickon]

relatime is the default these days, so you don't gain as much as you'd expect from noatime.

[Daniel Gessel]

relatime is the default these days, so you don't gain as much as you'd expect from noatime.

That sounds right from what I've read. Somewhere I saw a post where the person had built a custom kernel to make noatime the default (or even remove atime support, I don't remember exactly).

I also don't like atime. It seems unreliable to me, easily a victim of indexing, backup or even a misplaced recursive grep. Since I don't want SW to depend on it, it's just a performance sink (however small) and, on principle, turn it off.

I'm not administering systems - they're all personal use - so I can take such liberties...

[dickon]

It has its uses, mostly for shells and MUAs to detect when the mailbox has been written to, but not yet read from, so they can flag 'You have new mail'. Now, as most people no longer have mailspools on the machine they're using, it's a bit legacy niche, but I'm one of those who does, and I use it.

[Daniel Gessel]

I realize some SW uses it, but for the reasons I mentioned, I wouldn't use that software (at least not on purpose). Thankfully I'm not in charge of the kernel - I'd have ripped it out a long time ago.

[edit: btw, I'm making fun of my own opinionated self here - the aspersions I cast are inward only]

[jahboater]

It has its uses, mostly for shells and MUAs to detect when the mailbox has been written to, but not yet read from, so they can flag 'You have new mail'. Now, as most people no longer have mailspools on the machine they're using, it's a bit legacy niche, but I'm one of those who does, and I use it.

Perhaps inotify would do the job, perhaps more efficiently?

[butchkemper]

I just wondered: Has Raspberry Pi OS implemented some specificities to care for SD cards wearing reduction?
Or is it just a plain Debian writing on the SD cards as if they were SSDs?

Take a look at the "log2ram" program which is available on github.

Butch

[Daniel Gessel]

Take a look at the "log2ram" program which is available on github.

Is that better than mounting ramdisks in fstab?

[jahboater]

Take a look at the "log2ram" program which is available on github.

Is that better than mounting ramdisks in fstab?

I took a quick look, and no, I think:

Code: Select all

tmpfs /var/log tmpfs defaults,noatime,size=16m 0 0

in /etc/fstab is simpler, does not require a service to be running, and does not require two pieces of software to be installed.

It looks like, log2ram mounts some kind of memory disk at /var/log just like the above, but I don't know if its tmpfs or as good as tmpfs.

[dickon]

It has its uses, mostly for shells and MUAs to detect when the mailbox has been written to, but not yet read from, so they can flag 'You have new mail'. Now, as most people no longer have mailspools on the machine they're using, it's a bit legacy niche, but I'm one of those who does, and I use it.

Perhaps inotify would do the job, perhaps more efficiently?

Quite probably. But I'm not going to maintain my own forks of mutt and bash just to be able to remove atime support from my filesystems...

[Gavinmc42]

For 24/7 Pi's I tend to use PiCore Linux.
But I log to ram drives and write only once a day to uSD.
I might lose a day max of data but it reboots when power comes back without issue.

A1 uSD seem to last longer than most.
Found the way to kill uSD's on Pi3's was do long overnight compiles with big swap file.
Pi4's with 4GB and more are better at those compiles

SSD hold more and are a bit more tolerant.

There are Linux tricks others will tell you about, might learn them one day if I need to use Linux

[lop]

maybe F2FS is soon available (with rpi-imager it is only a little step to support it)?

[Daniel Gessel]

Quite probably. But I'm not going to maintain my own forks of mutt and bash just to be able to remove atime support from my filesystems...

Definitely makes sense. My interest is more "academic" than practical - I think I'll put a warning about that in my signature, so people get it when my priorities seem weird.

[thagrol]

with rpi-imager it is only a little step to support it

Almost certainly not a "liitle step" and likely to break or complicate things for those of us not using flash drives for the OS.

[Daniel Gessel]

To us users, RPi Imager could let us choose from images with a variety of filesystems for root. And if somebody else has to build and test all those images, it's easy enough for us...

I didn't read all of kernel docs yet (https://docs.kernel.org/filesystems/f2fs.html) but the archlinux wiki has some warnings (https://wiki.archlinux.org/title/F2FS) about reliability.

By the time we know what really works, we'll be on to storage on holographic crystals with problems caused by quantum fluctuations (you have to keep your drives in a box and never look inside).

[thagrol]

To us users, RPi Imager could let us choose from images with a variety of filesystems for root. And if somebody else has to build and test all those images, it's easy enough for us...

I hope that was intended at least half in jest.

• You're talking about doubling the number of images.
• More choice will likely confuse many users.
• Someone has to do the testing. If that's RP's engineers that has to be paid for.
• It's also an increase in support burden, which like it or not means an increased burden on the volunteers offering help on here.
• Getting things reliable will take time and effort by beta testers too - it's much more than just "will it install and boot".
• I've not checked the kernel config but it'll likely mean having f2fs as a builtin rather than as a module (it might be now, I dunno) which means a bigger kernel for everyone, a different kernel configuration for an f2fs image, or an initrd for an f2fs image.
• ...

Is the benefit enough to justify all of that and more? I have no idea. Given the low rate of failure of SD cards I've had using ext4 (two in ten years) I'm not going to spend the time and effort to find out.

[ejolson]

Quite probably. But I'm not going to maintain my own forks of mutt and bash just to be able to remove atime support from my filesystems...

Definitely makes sense. My interest is more "academic" than practical - I think I'll put a warning about that in my signature, so people get it when my priorities seem weird.

Will inotify trigger on a change to the atime metadata field if the filesystem is mounted noatime?

I don't think inotify could replace atime in mutt as that would require mutt to be continuously running. My experience with the dog developer is that mutts get tired quickly when running, so that does not seem practical.

Unsurprisingly, Fido's preferred setting is lazytime. In that case, I'd suggest adding strictatime as well, or the laziness may get out of hand.

https://unix.stackexchange.com/question ... ime-writes