RDPUser
Posts: 215
Joined: Tue Jan 30, 2018 12:18 pm

Samba on Raspi copy large files gives error 0x8007003B solution so far: don't use ExFAT or FAT32

Fri Mar 02, 2018 9:30 pm

I've setup Samba, copying small files with a few MBs works fine. Reading large files from Rapi also works fine. However copying large files to Raspi doesn't work. Windows always tells "calculating" and then gives error 0x8007003B on Windows 10
On Windows 7 it looks like the server went away, thats why it is failing.
Freespace is 2 GB on an 8 GB USB dongle with ExFat. Copying files with 1 GB gives this error. Reading files with 1 GB works and gives speed of about 3 MB/s.

Any ideas whats wrong?

Code: Select all

pi@raspberrypi:~ $ cat /etc/samba/smb.conf
[global]
workgroup = WORKGROUP
server string = RaspberryServer
netbios name = raspberrypi
dns proxy = no
passdb backend = tdbsam
### Authentication
security = user
map to guest = never
guest account = nobody
encrypt passwords = true

[freigabe]
path = /backup
browseable = no
read only = no
create mask = 0775
directory mask = 0775
writeable = yes
mounted with

Code: Select all

sudo mount /dev/sdc /backup
Last edited by RDPUser on Sat Feb 23, 2019 3:53 pm, edited 2 times in total.

RDPUser
Posts: 215
Joined: Tue Jan 30, 2018 12:18 pm

Re: Samba on Raspi Pi 1 Model B 512 MB Copy large files gives error 0x8007003B

Sat Mar 03, 2018 9:48 pm

Problem seems to be with ExFat
I dug into with iostat and have seen samba is writing/allocating the whole 1 GB on the ExFAT USB dongle. Because this is rather slow, a timeout occurs https://www.samba.org/samba/docs/curren ... onf.5.html
strict allocate (S)

This is a boolean that controls the handling of disk space allocation in the server. When this is set to yes the server will change from UNIX behaviour of not committing real disk storage blocks when a file is extended to the Windows behaviour of actually forcing the disk system to allocate real storage blocks when a file is created or extended to be a given size. In UNIX terminology this means that Samba will stop creating sparse files.

This option is really designed for file systems that support fast allocation of large numbers of blocks such as extent-based file systems. On file systems that don't support extents (most notably ext3) this can make Samba slower. When you work with large files over >100MB on file systems without extents you may even run into problems with clients running into timeouts.

When you have an extent based filesystem it's likely that we can make use of unwritten extents which allows Samba to allocate even large amounts of space very fast and you will not see any timeout problems caused by strict allocate. With strict allocate in use you will also get much better out of quota messages in case you use quotas. Another advantage of activating this setting is that it will help to reduce file fragmentation.

To give you an idea on which filesystems this setting might currently be a good option for you: XFS, ext4, btrfs, ocfs2 on Linux and JFS2 on AIX support unwritten extents. On Filesystems that do not support it, preallocation is probably an expensive operation where you will see reduced performance and risk to let clients run into timeouts when creating large files. Examples are ext3, ZFS, HFS+ and most others, so be aware if you activate this setting on those filesystems.

Default: strict allocate = no
So the maximum filesize before this happens depends on the write speed of the usb stick and other IO load on the drive. Windows handles this better. When I put the share on the ExFAT Dongle, it also allocates the whole filesize but without writing it completely. That is also very bad concering wearleveling because it writes each file twice.

Searched a bit and found this one https://github.com/relan/exfat/issues/45

Tested everything on a RPI 3, same problem.

RDPUser
Posts: 215
Joined: Tue Jan 30, 2018 12:18 pm

Re: Samba on Raspi copy large files gives error 0x8007003B solution don't use ExFAT

Sat Mar 03, 2018 10:43 pm

Exfat-nofuse has the same problem https://github.com/dorimanx/exfat-nofuse just in case someone wants to try, it won't be useful.
We should tell samba not to allocate the whole filesize. Any ideas how to do?

RDPUser
Posts: 215
Joined: Tue Jan 30, 2018 12:18 pm

Re: Samba on Raspi copy large files gives error 0x8007003B solution don't use ExFAT

Wed Mar 07, 2018 9:46 pm

Same problem with "normal" FAT.

blindox
Posts: 4
Joined: Sat Feb 21, 2015 7:12 pm

Re: Samba on Raspi copy large files gives error 0x8007003B solution don't use ExFAT

Sun Dec 02, 2018 3:16 pm

Any solution for this problem ?

andrum99
Posts: 2033
Joined: Fri Jul 20, 2012 2:41 pm

Re: Samba on Raspi copy large files gives error 0x8007003B solution don't use ExFAT

Sun Dec 02, 2018 4:08 pm

Per this thread, don't set strict_allocate, since it is apparently meant to default to no. Or force to no with strict_allocate=no in your smb.conf. This is a share level parameter so to be sure, set it at share level. If that doesn't work then something else is going on.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 32828
Joined: Sat Jul 30, 2011 7:41 pm

Re: Samba on Raspi copy large files gives error 0x8007003B solution don't use ExFAT

Sun Dec 02, 2018 8:58 pm

blindox wrote:
Sun Dec 02, 2018 3:16 pm
Any solution for this problem ?
I thought that this was fixed some months ago. Is your Raspian up to date?
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

2_of_8
Posts: 1
Joined: Thu Jan 03, 2019 7:59 pm

Re: Samba on Raspi copy large files gives error 0x8007003B solution don't use ExFAT

Thu Jan 03, 2019 8:16 pm

Pi 2 B (1GB) running Raspbian
4.14.79-v7+
USB2.0 powered HDD mounted using exfat-fuse
Wi-fi USB connection (TP-Link WN823N) on Pi, ethernet out of Windows 10

Copying fails to start if the file is > 2 GB . Sits at 0% complete for about a minute, followed by the error 0x8007003B

Happy to provide more info if needed!

LTolledo
Posts: 6657
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Samba on Raspi copy large files gives error 0x8007003B solution don't use ExFAT

Sun Jan 06, 2019 5:49 am

for your reference, below is a test that I made for copying large files over the network

Samba share hosted by RPi1B... this time as the host to receive the large files

A. Samba Server: Raspberry Pi (c) 2011.12 with 512MB (lets call it RPi1B) , with
A1. Raspbian Stretch with Desktop on 8GB Transcend SD card (updated and upgraded as of July 23, 2018)
A2. Ethernet connection to Ethernet Switch HUB -B
A3. Samba server and client installed, running fine
A4. 3 share folder in Ext4 format (inside 8G BSD Card)
A5. NTFS formatted 128GB Cruzerfit USB flash memory plugged into vacant USB port - shared in samba
A6. of the 512MB on board memory, 192MB is allocated for video because of Kodi
A7. RPi1B connected to 8" HDMI monitor, 2.4GHz wireless keyboard and mouse
A8. RPi1B is autoboot to Kodi (used mainly as Kodi Media Player)
A9. PSU is 5v 3.0A, shared with 8" Monitor
A10. I2S DAC HifiBerry clone connected to 26-pin GPIO
A11. previously copied 4GB and 8GB file still in 128GB USB flash.
A12. Kodi running in the background (it is autoboot to Kodi as described in A8)

B. NAS (EXT4 format) connected to Ethernet Switch HUB- A

C. Ethernet Switch HUB -B connected to Ethernet Switch Hub -A

D. Ethernet Switch Hub-A connected to WiFi Router

E. Client: Raspberry Pi 3 Model B+ (lets call it RPi3B+) , with
E1. Raspbian Stretch with Desktop on 32GB Sandisk Cruzerfit USB Flash memory (native USB boot, Stretch is updated and upgraded as of July 22, 2018)
E2. Network Connection via WiFi to WiFi router described in D.
E3. Samba server and client installed, running fine
E4. 1 share folder in Ext4 format (inside 32GB USB Flash Memory)
E5. of the 1GB on board memory, 192MB is allocated for video because of Kodi
E6. RPi3B+ connected to HDMI monitor, 2.4GHz wireless keyboard and mouse
E7. PSU is 5v 2.5A
E8. I2S DAC HifiBerry clone connected to 40-pin GPIO

What I did.
1. Using the RPi3B+ Stretch desktop file manager, connected to the samba share of RPi1B (as indicated in A5)
3. Using the file manager of RPi3B+ selected a 4GB video file ( a single file! not a directory of smaller files) that was saved in desktop (from previous test above)
4. Copied the 4GB video file to the USB share folder in RPi1B
5. Copy completed in under 30 minutes... no errors...

4GB Route: RPi3B+ Desktop ---> WiFi Router ---> HUB-A ---> HUB-B ---> RPi1B ---> USB Flash Memory

6. Again using the RPi3B+ Stretch desktop manager connected to the samba share of NAS (as described in B)
7. Selected an 8GB file from the NAS samba share and directly copied it to the USB samba share of RPi1B
8. Copy completed in about 65 minutes... no errors

8GB Route: RPi3B+ ---* WiFi Router ---* HUB-A ---* NAS ---> HUB-A ---> HUB-B ---> RPi1B ---> USB Flash Memory

Copying 4GB file from RPi3B+ desktop and an 8GB from the NAS samba share to the USB Flash memory shared over samba on RPi1B server did not produce any error or freezes.

The RPi1B was also accessed via VNC from RPi3B+ during the test...
if 8GB is still "small" I can make a bigger file transfer test....
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

RDPUser
Posts: 215
Joined: Tue Jan 30, 2018 12:18 pm

Re: Samba on Raspi copy large files gives error 0x8007003B solution don't use ExFAT

Sat Feb 23, 2019 3:37 pm

A4. 3 share folder in Ext4 format (inside 8G BSD Card)
A5. NTFS formatted 128GB Cruzerfit USB flash memory plugged into vacant USB port - shared in samba
Ext4 and NTFS work. This problem only affects ExFat and Fat32 Filesystems.

Problem still exists with FAT32 filesystem. Just tested. Before I did sudo apt update && sudo apt upgrade

Update: Problem still exits for ExFat, too
Last edited by RDPUser on Fri Aug 02, 2019 12:22 am, edited 1 time in total.

RDPUser
Posts: 215
Joined: Tue Jan 30, 2018 12:18 pm

Re: Samba on Raspi copy large files gives error 0x8007003B solution so far: don't use ExFAT or FAT32

Fri Aug 02, 2019 12:21 am

Update: As the ntfs-3g driver has very poor performance, I suggest to use ext4 as filesystem. You can read the files within windows with Linux-Reader (Freeware) https://www.diskinternals.com/linux-reader/
If you need "live acces" there is a solution from paragon, 10 days free, after that, still free, but then slower speed https://www.paragon-software.com/home/linuxfs-windows/#
Did some tests with x86 raspbian on a Core i3, about 7 years old now and a samsung SSD 850 pro 512 GB. With NTFS dd with bs=32M got 123 MB/s, one CPU core was fully utilized. With same bs direct to the disk it was 538 MB/s and with ext4 on the disk 536 MB/s.

LTolledo
Posts: 6657
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Samba on Raspi copy large files gives error 0x8007003B solution so far: don't use ExFAT or FAT32

Fri Aug 02, 2019 9:45 am

my upgraded NAS servers are now using EXT4 (main {Rock64-1G} and mirror {RPi2B} )....

so mirroring time is less than before (not timed though, but just seemed faster than before) ;)
sometimes mirroring a single file with more than 8GB size....and no errors.
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

basj
Posts: 21
Joined: Tue Mar 03, 2015 1:37 pm

Re: Samba on Raspi copy large files gives error 0x8007003B solution so far: don't use ExFAT or FAT32

Mon Dec 07, 2020 3:51 pm

Does anyone have a solution for this?

When uploading a 2.2 GB file to the RPi from Windows with Samba, there are 30 seconds at the beginning where nothing happens ... and only then, the copy begins (quite quickly).

Only with exFAT. The problem doesn't appear with ext4 or NTFS.

Return to “Troubleshooting”