RonR
Posts: 3017
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Convert BOOT Partition to 2048 Byte Clusters

Mon Dec 05, 2022 11:14 pm

Due to a bug in the Raspberry Pi 4/400 vl805 USB3 controller, a 'flush' option was added to the /etc/fstab /boot entry to prevent file corruption problems when the BOOT partition uses 512 byte clusters in its FAT filesystem.

Adding a 'flush' option to the /etc/fstab /boot entry imposes a significant performance penalty on writes to the boot partition.

Newer Raspberry Pi OS images use 2048 byte clusters, eliminating the need to add a 'flush' option to the /etc/fstab /boot entry.

The attached script will check the BOOT partition cluster size and offer to convert it to 2048 byte clusters, if needed, in order to remove the 'flush' option from the /etc/fstab /boot entry.

In all cases, the 'flush' option of the /etc/fstab /boot entry will be added or removed as appropriate.

As always, make a backup before performing any kind of conversion process.
Attachments
cvt-boot-2k.zip
(1.11 KiB) Downloaded 19 times

timg236
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1210
Joined: Thu Jun 21, 2018 4:30 pm

Re: Convert BOOT Partition to 2048 Byte Clusters

Tue Dec 06, 2022 9:53 am

N.B. This is not necessary if you have the latest Linux Kernel from APT.

See https://github.com/raspberrypi/linux/pull/5173

RonR
Posts: 3017
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Convert BOOT Partition to 2048 Byte Clusters

Tue Dec 06, 2022 6:12 pm

timg236 wrote:
Tue Dec 06, 2022 9:53 am
N.B. This is not necessary if you have the latest Linux Kernel from APT.

See https://github.com/raspberrypi/linux/pull/5173

According to P33M the following day, 512 byte clusters are still vulnerable to the file corruption issue.

See https://github.com/raspberrypi/linux/is ... 1246277232
Older images will still be vulnerable as they use a cluster size of 512B.

Also see 'add_remove_flush_option ()' function in raspberrypi-sys-mods package (20221019 release).

This leaves 3 possible solutions for older systems with 512 byte cluster boot filesystems:

1. Add a 'flush' option to the /etc/fstab /boot entry, imposing a significant performance penalty on writes to the boot partition.

2. Rebuild your older system using a newer image that has 2k clusters.

3. Convert your existing 512 byte cluster boot filesystem to 2k clusters.
Last edited by RonR on Tue Dec 06, 2022 7:35 pm, edited 2 times in total.

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2774
Joined: Thu Jul 11, 2013 2:37 pm

Re: Convert BOOT Partition to 2048 Byte Clusters

Wed Dec 07, 2022 9:07 am

Both statements are correct. If you have an older OS image without the updated kernel, the bug exists. It's not a firmware fix. The title of this thread should be "update your kernel to the latest in apt, or use a newer OS release".
Rockets are loud.
https://astro-pi.org

RonR
Posts: 3017
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Convert BOOT Partition to 2048 Byte Clusters

Wed Dec 07, 2022 6:48 pm

jdb wrote:
Wed Dec 07, 2022 9:07 am
Both statements are correct. If you have an older OS image without the updated kernel, the bug exists. It's not a firmware fix. The title of this thread should be "update your kernel to the latest in apt, or use a newer OS release".

You statement is very confusing.

I just tested a system that had been installed with an older OS release image (before 20220809) which uses 512 byte FAT clusters, updated to the latest kernel in apt. Boot partition file corruption is still reproducible. Updating your kernel to the latest in apt does not eliminate the bug. As you stated, this is not a firmware fix.

Updating using apt also installs the raspberrypi-sys-mods package (20221019 release) which adds a 'flush' option to the /etc/fstab /boot entry if 512 byte FAT clusters are being used. While this eliminates the bug, it also imposes a significant performance penalty on writes to the boot partition.

Yes, using a newer OS release image (20220809 or after) also eliminates the bug, but this is due to newer OS release images using 2K FAT clusters, as stated by P33M:
Newer images use a FAT32 cluster size of 2K which obviates the need to use the -o flush mount option, and avoids the VL805 bug. Older images will still be vulnerable as they use a cluster size of 512B.

I stand by everything I've stated in this topic. If you used an older OS release image (before 20220809) and you don't want the possibility of corrupted files in your boot partition, you have 3 options:

1. Add a 'flush' option to the /etc/fstab /boot entry, which imposes a significant performance penalty on writes to the boot partition.
[The raspberrypi-sys-mods package (20221019 release) from apt does this for you.]

2. Rebuild your older system using a newer OS release image that has 2K clusters and no 'flush' option in the /etc/fstab /boot entry.

3. Convert your existing 512 byte cluster boot filesystem to 2K clusters, which also removes the 'flush' option from the /etc/fstab /boot entry.

redvli
Posts: 431
Joined: Thu Sep 03, 2020 8:09 am

Re: Convert BOOT Partition to 2048 Byte Clusters

Thu Dec 08, 2022 9:16 am

Thanks RonR for the script, although I need a certain label and ID for the boot FAT, I was curious how you use the -s option (and not -S). I see some potential problems for my combinations of hardware and OSses so I reformatted the boot partition with option -s 4 . I have a 3.5inch drive that has physical 4096 sized sectors and for that, bcache with its defaults is known to corrupt data on the block level, for example.

But why 2048 and not the more common 4096 (matching the CPU pagesize) ?

I use multiple boot loaders from the same 1stFAT/ESP and seen some strange failures (I have workarounds).

RonR
Posts: 3017
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Convert BOOT Partition to 2048 Byte Clusters

Thu Dec 08, 2022 9:34 am

redvli wrote:
Thu Dec 08, 2022 9:16 am
Thanks RonR for the script, although I need a certain label and ID for the boot FAT, I was curious how you use the -s option (and not -S). I see some potential problems for my combinations of hardware and OSses so I reformatted the boot partition with option -s 4 . I have a 3.5inch drive that has physical 4096 sized sectors and for that, bcache with its defaults is known to corrupt data on the block level, for example.

But why 2048 and not the more common 4096 (matching the CPU pagesize) ?

I use multiple boot loaders from the same 1stFAT/ESP and seen some strange failures (I have workarounds).

You're confusing Logical Sector Size with FAT Filesystem Sectors Per Cluster.

bls
Posts: 2803
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA

Re: Convert BOOT Partition to 2048 Byte Clusters

Thu Dec 08, 2022 4:57 pm

RonR, which command shows whether my SD Card or IMG has 512 or 2048 Byte Clusters? Thx

EDIT: NM, found it. fsck.vfat
Pi tools:
Quickly and easily build customized exactly as-you-want SSDs/SD Cards: https://github.com/gitbls/sdm
Easily run and manage your network's DHCP/DNS servers on a Pi: https://github.com/gitbls/ndm
Easy and secure IPSEC/IKEV2 VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

redvli
Posts: 431
Joined: Thu Sep 03, 2020 8:09 am

Re: Convert BOOT Partition to 2048 Byte Clusters

Thu Dec 08, 2022 5:58 pm

RonR wrote:
Thu Dec 08, 2022 9:34 am
redvli wrote:
Thu Dec 08, 2022 9:16 am
Thanks RonR for the script, although I need a certain label and ID for the boot FAT, I was curious how you use the -s option (and not -S). I see some potential problems for my combinations of hardware and OSses so I reformatted the boot partition with option -s 4 . I have a 3.5inch drive that has physical 4096 sized sectors and for that, bcache with its defaults is known to corrupt data on the block level, for example.

But why 2048 and not the more common 4096 (matching the CPU pagesize) ?

I use multiple boot loaders from the same 1stFAT/ESP and seen some strange failures (I have workarounds).

You're confusing Logical Sector Size with FAT Filesystem Sectors Per Cluster.
Yes sorry, I wrote the wrong thing, it is the logical sector size that is 4096, physical is anyhow 4096 for most modern larger drives. For BtrFS, the blocksize it is working with is 4096, the most used pagesize of a CPU, at least for Arm and Intel AFAIK.
I just wonder why the RPi engineers haven't chosen 8 512-byte sized sectors per cluster. If I would boot RasPiOS from this 4096/4096 (physical/logical) HDD, sectors per cluster should be 1 I think, leading to 4096-sized clusters for the FAT boot filesystem.

cleverca22
Posts: 6972
Joined: Sat Aug 18, 2012 2:33 pm

Re: Convert BOOT Partition to 2048 Byte Clusters

Thu Dec 08, 2022 6:07 pm

redvli wrote:
Thu Dec 08, 2022 5:58 pm
If I would boot RasPiOS from this 4096/4096 (physical/logical)
the pi4 firmware lacks support for 4096 byte sector drives

it will just give an error if you try to boot from one

redvli
Posts: 431
Joined: Thu Sep 03, 2020 8:09 am

Re: Convert BOOT Partition to 2048 Byte Clusters

Thu Dec 08, 2022 7:01 pm

cleverca22 wrote:
Thu Dec 08, 2022 6:07 pm
redvli wrote:
Thu Dec 08, 2022 5:58 pm
If I would boot RasPiOS from this 4096/4096 (physical/logical)
the pi4 firmware lacks support for 4096 byte sector drives

it will just give an error if you try to boot from one
Hmm, thanks for this info. That helps a lot with planning what to do with a combination of a 1T SSD and that 8T HDD, I currently have them connected to an older Atom IPMI/server board (SATA and 12V/5V cables).

cleverca22
Posts: 6972
Joined: Sat Aug 18, 2012 2:33 pm

Re: Convert BOOT Partition to 2048 Byte Clusters

Thu Dec 08, 2022 7:54 pm

you have basically 3 options to work around that
  1. put /boot on an SD card, and ignore the usb-boot support, use the old hack of /boot on SD and / on usb
  2. maybe use a 2nd 512 usb drive as /boot, and same thing
  3. turn on 512 emulation mode in the usb drive, one guy i helped found a "windows xp compatability" tool that could switch the drive into 512 mode

RonR
Posts: 3017
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Convert BOOT Partition to 2048 Byte Clusters

Thu Dec 08, 2022 8:14 pm

cleverca22 wrote:
Thu Dec 08, 2022 7:54 pm
you have basically 3 options to work around that
  1. put /boot on an SD card, and ignore the usb-boot support, use the old hack of /boot on SD and / on usb
  2. maybe use a 2nd 512 usb drive as /boot, and same thing
  3. turn on 512 emulation mode in the usb drive, one guy i helped found a "windows xp compatability" tool that could switch the drive into 512 mode

If I recall correctly (and I may not - it's been a while since I encountered this), some USB-to-SATA adapters present drives with 4096 byte physical sectors as 512/4096 (Logical/Physical Sector Size) and work fine on Pi's.

This discussion is unrelated to FAT filesystem cluster sizes and the conversion thereof, and should be moved to a new topic.

cleverca22
Posts: 6972
Joined: Sat Aug 18, 2012 2:33 pm

Re: Convert BOOT Partition to 2048 Byte Clusters

Fri Dec 09, 2022 3:12 am

RonR wrote:
Thu Dec 08, 2022 8:14 pm
If I recall correctly (and I may not - it's been a while since I encountered this), some USB-to-SATA adapters present drives with 4096 byte physical sectors as 512/4096 (Logical/Physical Sector Size) and work fine on Pi's.
one of my nasty usb-sata adapters, converts a 512 drive, to a 4096/4096 interface!!

the MBR table is then pointing to all the wrong places, and the drive looks like it doesnt even have anything on it!!

Return to “Advanced users”