DonJuane
Posts: 24
Joined: Tue Dec 15, 2020 1:15 am

Can large drive eXFat support be added to Pi ?

Thu Oct 14, 2021 2:02 am

I have put this project on hold over a year hoping that the Pi distribution would be fixed to support large eXFat volumes but after re-purposing my old Pi 4B and starting all over fresh today, I still see there is not large drive eXFat support. And while the disk activity lamp lights up just like it does when I plug it in Android OTG adapter, or on Linux Mint desktop, and while I can access it watch videos off it using any other Linux distribution including Android, on the Pi 4B it is as if nothing is plugged in.

The ongoing suggestion for all the problems that I have seen brought up here for the failed large volume support on eXFat drives says to re-format them EXT4, which is of course is an options but since I happen to need the drive on Windows as well to make edits to the contents, this means I'll be out a near couple of hundred bucks having to buy another drive and have to manage duplicates.

Any hope of contacting the exfuse developer to see if they will be willing to give support for Raspberrian distribution? THANKS!!!

(I tried searching and can't find who the owner of exfuse package is?????)

***** YES! These have been added but that fixes nothing:

sudo apt-get install exfat-fuse
sudo apt-get install exfat-utils

Code: Select all

Using username "pi".
Linux raspberrypi 5.10.63-v7l+ #1457 SMP Tue Sep 28 11:26:14 BST 2021 armv7l
..........
Last login: Thu Oct 14 01:42:53 2021 from 192.168.16.174
...........
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.10.63-v7l+ #1457 SMP Tue Sep 28 11:26:14 BST 2021 armv7l GNU/Linux

pi@raspberrypi:~ $ sudo fdisk -l
Disk /dev/ram0: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram1: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram2: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram3: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram4: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram5: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram6: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram7: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram8: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram9: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram10: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram11: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram12: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram13: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram14: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/ram15: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk /dev/mmcblk0: 29.8 GiB, 32010928128 bytes, 62521344 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xe6e9a232

Device         Boot  Start      End  Sectors  Size Id Type
/dev/mmcblk0p1        8192   532479   524288  256M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      532480 62521343 61988864 29.6G 83 Linux
pi@raspberrypi:~ $ df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/root       30476156 2054700  27150008   8% /
devtmpfs         1800564       0   1800564   0% /dev
tmpfs            1965428       0   1965428   0% /dev/shm
tmpfs            1965428    8696   1956732   1% /run
tmpfs               5120       4      5116   1% /run/lock
tmpfs            1965428       0   1965428   0% /sys/fs/cgroup
tmpfs            1965428       0   1965428   0% /tmp
/dev/mmcblk0p1    258095   49232    208863  20% /boot
folder2ram       1965428    2384   1963044   1% /var/log
folder2ram       1965428       0   1965428   0% /var/tmp
folder2ram       1965428     272   1965156   1% /var/lib/openmediavault/rrd
folder2ram       1965428     752   1964676   1% /var/spool
folder2ram       1965428   12776   1952652   1% /var/lib/rrdcached
folder2ram       1965428       4   1965424   1% /var/lib/monit
folder2ram       1965428    1288   1964140   1% /var/cache/samba
tmpfs             393084       0    393084   0% /run/user/1000

pi@raspberrypi:~ $ lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk0     179:0    0 29.8G  0 disk
├─mmcblk0p1 179:1    0  256M  0 part /boot
└─mmcblk0p2 179:2    0 29.6G  0 part /

pi@raspberrypi:~ $ sudo lsblk -f
NAME FSTYPE LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINT
mmcblk0
│
├─mmcblk0p1
│    vfat   boot  5DE4-665C                               204M    19% /boot
└─mmcblk0p2
     ext4   rootfs
                  7295bbc3-bbc2-4267-9fa0-099e10ef5bf0   25.9G     7% /

pi@raspberrypi:~ $ sudo blkid
/dev/mmcblk0p1: LABEL_FATBOOT="boot" LABEL="boot" UUID="5DE4-665C" TYPE="vfat" PARTUUID="e6e9a646-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="7295bbc3-bbc2-4267-9fa0-099e10ef5bf0" TYPE="ext4" PARTUUID="e6e9a646-02"
/dev/mmcblk0: PTUUID="e6e9a646" PTTYPE="dos"

pi@raspberrypi:~ $ lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk0     179:0    0 29.8G  0 disk
├─mmcblk0p1 179:1    0  256M  0 part /boot
└─mmcblk0p2 179:2    0 29.6G  0 part /
pi@raspberrypi:~ $

LTolledo
Posts: 6010
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Can large drive eXFat support be added to Pi ?

Thu Oct 14, 2021 9:16 am

why the need for exFAT?

use the file system natively supported by RPiOS / Linux
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

User avatar
jojopi
Posts: 3542
Joined: Tue Oct 11, 2011 8:38 pm

Re: Can large drive eXFat support be added to Pi ?

Thu Oct 14, 2021 11:39 am

You do not appear to have any /dev/sdX devices. "It is as if nothing is plugged in," indeed.

The fact that the drive is large may be a contributing factor, but the fact that is formatted exFAT is not. Linux is not yet talking to the drive well enough to know how it is partitioned or formatted.

Low-level drive issues can be caused by insufficient power to the drive or Pi, marginal-quality cables, or other deviations from the USB specifications.

Have you seen the thread about USB Mass Storage Device versus USB Attached SCSI? viewtopic.php?t=245931

User avatar
thagrol
Posts: 5839
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Can large drive eXFat support be added to Pi ?

Thu Oct 14, 2021 11:58 am

How large are we talking? It's possible you're hitting the 32 bit limit in the kernel, the exFATfuse driver, or in exFAT itself.
I'm a volunteer. Take me for granted or abuse my support and I will walk away

All advice given is based on my experience. it worked for me, it may not work for you.
Need help? https://github.com/thagrol/Guides

DonJuane
Posts: 24
Joined: Tue Dec 15, 2020 1:15 am

Re: Can large drive eXFat support be added to Pi ?

Fri Oct 15, 2021 3:34 pm

5Tb - the drive is a portable "shock proof" Western Digital 2.5 external USB 3.0 laptop drive. I can plug the same drive in an Amazon fire stick OTG adapter, use "Stick Mount" and watch videos off the drive at very rapid access speeds. Yet when I plug it in the Pi4b 4g it doesn't recognize it. I've also run chkdsk x: /F on Windows and no errors are found, drive comes right up and can read and write fine on Windows.

The same happened with another Pi4b 4g with this same drive not showing up there either on an older version of Raspbian, so it can't be the PI hardware.

**** also note that I've copied files to/from this same drive using Linux Mint desktop, latest release.

trejan
Posts: 3735
Joined: Tue Jul 02, 2019 2:28 pm

Re: Can large drive eXFat support be added to Pi ?

Fri Oct 15, 2021 3:43 pm

Plug it in and run the "dmesg" command. Show us the last page or two.

User avatar
thagrol
Posts: 5839
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Can large drive eXFat support be added to Pi ?

Fri Oct 15, 2021 4:22 pm

DonJuane wrote:
Fri Oct 15, 2021 3:34 pm
5Tb
Might be a partition table issue. AIUI for a drive that size you need to be using GPT not MSDOS for the partition table. RPiOS supports both. I'd expect it to have been set to GPT at the factory though.

Or a power issue. If the drive needs more than 1.2A (6W), you have a lot of other power hungry devices connected, you have a low current rating on your PSU (less than 3A), or a poor quality power cable you're in for trouble.
I'm a volunteer. Take me for granted or abuse my support and I will walk away

All advice given is based on my experience. it worked for me, it may not work for you.
Need help? https://github.com/thagrol/Guides

DonJuane
Posts: 24
Joined: Tue Dec 15, 2020 1:15 am

Re: Can large drive eXFat support be added to Pi ?

Fri Oct 15, 2021 4:41 pm

Yes, I have tried powered USB 3.0 hub but will check the partition table today, thank you.

LTolledo
Posts: 6010
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Can large drive eXFat support be added to Pi ?

Fri Oct 15, 2021 9:23 pm

need interoperability with w$ system..... why not use NTFS instead....
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

DonJuane
Posts: 24
Joined: Tue Dec 15, 2020 1:15 am

Re: Can large drive eXFat support be added to Pi ?

Fri Oct 15, 2021 11:00 pm

Because past experience using a NTFS driver on the OpenWRT platform while setting up a Samba server and using their NTFS driver was even worse than using eXFat drive on OpenWRT Samba server and I figured it might be the same on Raspbian. The OpenWRT NTFS driver I think will write at only about 8M a second while their eXFat driver can max out around 25M.

User avatar
thagrol
Posts: 5839
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Can large drive eXFat support be added to Pi ?

Sat Oct 16, 2021 1:09 am

You do realise that the underlying file system is completely irrelevant to samba clients? They don't even know what it is.

Unless you're planning on physically connecting the drive to a Windows box use ext4.

And NFS rather than samba/CIFS/SMB for Linux clients.
I'm a volunteer. Take me for granted or abuse my support and I will walk away

All advice given is based on my experience. it worked for me, it may not work for you.
Need help? https://github.com/thagrol/Guides

DonJuane
Posts: 24
Joined: Tue Dec 15, 2020 1:15 am

Re: Can large drive eXFat support be added to Pi ?

Sat Oct 16, 2021 4:04 am

I was referencing the copy speed when you have a series of fast drives formatted differently and you place them on OpenWrt the performance with OpenWRT is bad and with Raspbian it is very bad. A NTFS drive is written at a speed of around 8GB on a wired GB Ethernet link and a eXFat is written at a rate of 25M and a EXT4 drive on the same server is written at 80M-100M, I would suspect that it is a NTFS driver that is at fault.

I've got my Ubuntu laptop booted up now and copying at an alarming rate off the eXFat that won't even be recognized by the PI4B 4G, which I point again to drivers. The Pi just doesn't take Windows drives seriously and yes I would not battle over the file system if I didn't need to use the drive on Windows.

Something hokey with the driver. Wish someone would help me find the developer or support person for eXFat. These Linux vs. Windows battles are now a half a century old or more. Some Linux distributions support them but Raspbian doesn't do it very well if at all.

Still I'd like to if possible to remain on topic and try to resolve the eXFat driver issue. Slow is better than nothing at all, which I am now dealing with, nothing at all.

trejan
Posts: 3735
Joined: Tue Jul 02, 2019 2:28 pm

Re: Can large drive eXFat support be added to Pi ?

Sat Oct 16, 2021 4:27 am

DonJuane wrote:
Sat Oct 16, 2021 4:04 am
Still I'd like to if possible to remain on topic and try to resolve the eXFat driver issue.
I previously asked you for the output of dmesg after plugging in the drive but you've not done so.

User avatar
thagrol
Posts: 5839
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Can large drive eXFat support be added to Pi ?

Sat Oct 16, 2021 10:24 am

DonJuane wrote:
Sat Oct 16, 2021 4:04 am
Something hokey with the driver. Wish someone would help me find the developer or support person for eXFat.
I suspect both RPiOS and Ubuntu (assuming same kernel etc.) are using the same base code for the exFAT driver as both are derived from Debian. There are also many, many differenece between a Pi and an x86 laptop.

A though occurs: which USB port are you connecting the drive to?

I suggest the following actions:
  1. Try the drive with Ubuntu on Pi.
  2. Try the drive on the Pi formatted to ext4.
  3. Open an issue on the RPiOS github when doing so include the mkae and model number of the HDD.
    That's probably the best way to reach the devs.
I suspect the problem is not in the exFAT driver but lower in the stack.
These Linux vs. Windows battles are now a half a century old or more.
No. Neither Linux nor Microsoft Windows existed in 1971.
I'm a volunteer. Take me for granted or abuse my support and I will walk away

All advice given is based on my experience. it worked for me, it may not work for you.
Need help? https://github.com/thagrol/Guides

DonJuane
Posts: 24
Joined: Tue Dec 15, 2020 1:15 am

Re: Can large drive eXFat support be added to Pi ?

Sat Oct 16, 2021 3:12 pm

Will check these things, thank you everyone. I think I understand that being at a lower level must mean that this is a hardware or architecture issue and beyond the power of the driver, which does make sense.

(I actually am intuitive enough to know where this is going so I am now 14 hours into an exFat to EXT4 copy, dropping $150 for another drive realizing I'll have to manage 2 copies for my own needs). Thanks Ubuntu, as Windows or Raspbian won't pull me out of this mess.

Regarding half a decade, I left it open with the world "near". "Jest" was the idea and thus inferring there's no need to keep hashing it out and "punishing" people who like to sample from all platforms. For example if anyone is a Windows user or at least knows the basics, if they haven't used WinSCP to edit files on a Linux server, they are missing a super time saver. DOS is an interesting OS as well. They all have their merits and quirks and I try to call on each of them all when needed and when practical. In fact I still use a port of an old mainframe editor from the 70's called Kedit (port of old IBM mainframe CMS editor Xedit) which allows inserting characters or spaces in the entire selected column, as well as provides an old macro language I sometimes use to quickly make replacements and changes as well as providing all the text standard editing stuff. I am sure someone will say "you could also do that with grep" LOL.. Then there's Context editor in Windows where I recently popped in an Android SD card, deleted my non-music related files and changed the drive letter pointer for about 40 playlists to the new SD drive path name and did this to all 40 files in under 30 seconds. So thus a point is to keep any tool box full and stay open to new and old ideas and to keep in the mix as well a good bit of "simply giving up" as I have done with the current disk copy underway. :-) (P.S. I know ..... "I can do that with Spotify and I don't need play list, as they can read my mind and know what's best for me" ... just what I get from always being behind the times :-) )

dustnbone
Posts: 521
Joined: Tue Nov 05, 2019 2:49 am

Re: Can large drive eXFat support be added to Pi ?

Sat Oct 16, 2021 11:40 pm

This looks more like a USB power problem than a filesystem one. Plug the drive into a powered hub or use a self powered enclosure.

DonJuane
Posts: 24
Joined: Tue Dec 15, 2020 1:15 am

Re: Can large drive eXFat support be added to Pi ?

Sun Oct 17, 2021 3:39 pm

That didn't seem to help. (actually symptoms were the same with it externally powered)

I am currently using Ubuntu to copy everything to a large EXT4 drive, I will format the portable drive back to EXTFAT using another utility rather than windows and then copy all the files back and see if that works.

User avatar
thagrol
Posts: 5839
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Can large drive eXFat support be added to Pi ?

Sun Oct 17, 2021 4:40 pm

DonJuane wrote:
Sun Oct 17, 2021 3:39 pm
I am currently using Ubuntu to copy everything to a large EXT4 drive, I will format the portable drive back to EXTFAT using another utility rather than windows and then copy all the files back and see if that works.
Have you checked dmesg and lsusb with the drive connected to the Pi yet? If the issue is not power but is lower in the driver stack than the file system driver you may be wasting your time.
I'm a volunteer. Take me for granted or abuse my support and I will walk away

All advice given is based on my experience. it worked for me, it may not work for you.
Need help? https://github.com/thagrol/Guides

Return to “Raspberry Pi OS”