F2fs, Ext4, FAT USB Benchmark

[RoyLongbottom]

Via Linux - Hopefully RPi soon.

I have run a benchmark, via USB 2 on a Linux based PC, using a high speed SanDisk Extreme USB 3.0 drive, with write/read ratings of 110/190 MB/s, formatted as FAT, Ext4 and F2fs. The main source code is the same as Raspberry Pi DriveSpeed, where this and the execution file can be found in:

http://www.roylongbottom.org.uk/Raspber ... hmarks.zip

The OS is Ubuntu 13.10 with Linux 3.12.0. I downloaded the ISO for Ubuntu 14.04 but 13.10 was installed?? I downloaded f2fs-tools from:

http://www.ubuntuupdates.org/package/co ... f2fs-tools

GParted Formatted a 16 GB partition as F2fs but it appeared as an unknown file system. It was mounted as shown in the following. Even then it was not visible in Ubuntu, but the directory path could be accessed by the benchmark (using sudo).

https://wiki.archlinux.org/index.php/F2fs

The benchmark measures writing and reading speeds and execution times of writing and reading large and small files and random access. Results (fairly constant on three runs) for FAT, Ext4 and F2fs are below. Default small file sizes are 4, 8 and 16 KB. An extra test using 1, 2 and 4 KB was also run.

All have similar speeds on large files. F2fs is slow on random access, particularly on writing. FAT is slowest on writing small files and F2fs fastest but is seems that the real advantage over Ext4 becomes more apparent as the file sizes are reduced.

Next, I have to find how to run, using the F2fs partition, on the Raspberry Pi ARM based system [HELP!].

Code: Select all

               16 MB Files MBytes/Second
  MB    Write1   Write2   Write3    Read1    Read2    Read3

  FAT    28.32    28.59    28.40    39.07    39.78    39.05
  Ext4   30.31    30.44    30.32    38.94    38.72    38.25
  F2fs   29.64    30.01    29.58    39.56    39.66    39.40


                  Random Milliseconds
 From MB     4        8       16        4        8       16

  FAT    0.549    0.502    0.502     0.94     0.72     0.82
  Ext4   0.505    0.501    0.500     0.86     0.73     0.65
  F2fs   0.602    0.624    0.624     1.76     1.74     1.76


                2oo Files milliseconds per file
             Write                      Read               
 File KB     4        8       16        4        8       16

  FAT     1.91     1.76     2.01     0.65     0.72     0.77
  Ext4    0.75     0.63     0.93     0.42     0.53     0.77
  F2fs    0.53     0.61     0.89     0.39     0.52     0.77

 File KB     1        2        4        1        2        4

  Ext4    0.98     0.81     0.67     0.41     0.52     0.41
  Ext4    1.32     0.84     0.67     0.48     0.51     0.40
  F2fs    0.51     0.59     0.51     0.39     0.51     0.40

[RoyLongbottom]

I downloaded and installed 2013.12.20 Rasbian with Linux 3.10. This provides support at least for reading and writing F2fs partitions. Initially, the existing USB drive partition was not visible using the df command but, as the drive had another partition, the Filesystem path could be assumed and mounted (me - sudo mount -t f2fs /dev/sdc1 /mnt). The benchmark was run on this and on an Ext4 partition, then later as FAT. Results are below.

Speeds are a lot slower than those for PC Linux but with similar relationships, where Ext4 and F2fs are fastest on particular types of access.

Code: Select all

               Raspberry Pi Results
  
               16 MB Files MBytes/Second
  MB    Write1   Write2   Write3    Read1    Read2    Read3

  FAT    14.02    14.18    14.18    25.35    25.63    25.54
  Ext4   24.84    24.99    25.07    24.64    25.49    25.11
  F2fs   22.02    21.97    22.28    24.93    25.43    25.77


                  Random Milliseconds
             Read                       Write
 From MB     4        8       16        4        8       16

  FAT    0.884    0.902    0.884     0.84     1.12     0.71
  Ext4   0.861    0.900    0.894     1.03     0.90     0.93
  F2fs   0.851    0.903    0.896     2.10     3.22     2.25


                2oo Files milliseconds per file
             Write                      Read               
 File KB     4        8       16        4        8       16

  FAT     3.93     4.18     4.63     0.92     1.05     1.40
  Ext4    2.17     1.76     2.05     0.99     1.13     1.40
  F2fs    1.56     1.51     1.87     0.92     1.05     1.36

[RoyLongbottom]

I have determined why the above benchmark results show that random writing with F2fs formatting is slower than using Ext4. To check this out, I increased random reading and writing times to four seconds per test, instead of one, in order to measure system loading using VMSTAT. Then, besides the original 1 KB block sizes, I reran the benchmark with 4 KB accesses. Results for the random access tests are below.

Response times for writing at 4 KB to F2fs were much faster than at 1 KB and, now, a little faster than with Ext4. All other response times were slightly longer at 4 KB. The number of accesses in 4 seconds were calculated and are also shown below.

Code: Select all

                   BENCHMARK RESULTS

             Read                       Write
 From MB     4        8       16        4        8       16

              1 KB Blocks Random Milliseconds

  Ext4   0.828    0.845    0.859     0.90     0.98     1.00
  F2fs   0.841    0.869    0.888     2.22     2.27     2.34

              1 KB Blocks Accesses in 4 Seconds

  Ext4    4831     4734     4657     4444     4082     4000
  F2fs    4756     4603     4505     1802     1762     1709
  
  Ext4    Total Reads      14221     Total Writes     12526
  F2fs                     13864                       5273
 

              4 KB Blocks Random Milliseconds

  Ext4   0.928    0.940    0.950     1.01     1.00     1.01
  F2fs   0.926    0.943    0.946     0.92     0.93     0.93

              4 KB Blocks Accesses in 4 Seconds

  Ext4    4310     4255     4211     3960     4000     3960
  F2fs    4320     4242     4228     4348     4301     4301

  Ext4    Total Reads      12776     Total Writes     11921
  F2fs                     12790                      12950

Next we have the VMSTAT results for total KB read and written and extrapolated CPU seconds for each 12 second set of tests, the latter showing that performance is not CPU limited. Using 1 KB requests, Ext4 appeared to handle these perfectly but, on writing to F2fs, 4 KB was read and 4 KB written, for every 1 KB that needed to be written. At 4 KB there were no addittional reading.

Note - For this benchmark, Direct I/O is used for random access (saying “don’t cache the data”).

Code: Select all

               VMSTAT Results 12 seconds each read and write

                        CP secs CP secs    secs    Read           Write
          KB in  KB out    user  system     WIO     IOs   KB/IO     IOs   KB/IO

                                 1 KB Blocks
 Ext4
 Read      14042      68     0.2     6.3     5.2   14221     1.0
 Write         0   11621     1.3     6.7     2.9                   12526     0.9

 F2fs
 Read      13772      48     0.3     6.4     5.2   13864     1.0
 Write     21314   21362     0.2     6.2     5.7             4.0    5273     4.1


                                  4 KB Blocks
 Ext4
 Read      51300     112     0.3     5.5     6.2   12776     4.0
 Write         0   47976     1.6     7.0     3.4                   11921     4.0

 F2fs
 Read      51300      60     0.3     5.6     6.0   12790     4.0
 Write         0   52048     0.3     8.9     2.9                   12950     4.0

[RoyLongbottom]

The last quote was that Direct I/O is used in the benchmark, to try to ensure that drive speed is measured and not that for cached data. However, enabling the cache identifies how the outstanding performance is achieved on random writing.

Below are results so far for random reading and writing. The first two are as earlier showing that F2fs writing speed is slow using small block sizes. Next, enabling caching produces ridiculously fast speeds using small files. For further details of the following, see:

http://www.roylongbottom.org.uk/Raspber ... #anchor21a

Next we have larger files where F2fs is much faster on random writing with wide variations for Ext4 and both with similar but consistent reading response times at all file sizes.

Finally, with 40 second tests, Ext4 speeds are not as variable and F2fs writing performance is still outstanding. Reading from 256 MB is faster than from larger files as a higher proportion of data is cached. VMSTAT was run at the same time, showing that F2fs was writing to the drive at near maximum USB speed, with 90%+ CPU utilisation organising data.

Apparently, the reason for this superior performance is that F2fs is based on a Log-structured File System, where all modifications are written sequentially in a log-like structure that also contains indexing information. See:

https://www.kernel.org/doc/Documentatio ... s/f2fs.txt

Next, I am looking into copying small files, where some inexplicable benchmark results have been reported.

Code: Select all

              Random milliseconds on  a SanDisk  Extreme USB 3.0 drive

                         Read                       Write
              From MB      4        8       16        4        8       16

 Direct I/O 1 KB Ext4  0.839    0.822    0.845      0.9      0.8      0.8
                 F2fs  0.851    0.903    0.896      2.1      3.2      2.3

 Direct I/O 4 KB Ext4  0.928    0.940    0.950      1.0      1.0      1.0
                 F2fs  0.926    0.943    0.946      0.9      0.9      0.9

 Cached 4-16 MB  Ext4  0.024    0.034    0.114     0.03     0.03     0.04
                 F2fs  0.025    0.021    0.191     0.01     0.01     0.01

              From MB    256      512     1024      256      512     1024

 Cached 4 secs   Ext4  1.168    1.137    1.117     0.32     1.07     0.56
                 F2fs  1.212    1.160    1.149     0.13     0.12     0.14

 Cached 40 secs  Ext4  0.099    0.617    0.967     0.80     1.35     1.26  
                 F2fs  0.107    0.636    0.997     0.14     0.17     0.18

[RoyLongbottom]

Performance investigation of USB drives formatted with F2fs, compared with Ext4, were prompted by reports in XBMC Community Forum, that copying files with the former was up to nine times faster than with the same drive formatted as Ext4. The particular page is not now directly available but might still be found via Googling for “OpenELEC Testbuilds for RaspberryPi Part 2” page 199 or searching the site. As copying files involves passing data via the RAM based cache, a modified version of my DriveSpeed benchmark was produced to write and read 1000 files of increasing sizes, between 4 KB and 1 MB, with caching enabled. Detailed results are in the following, showing that, at most file sizes, F2fs speeds are mainly slightly faster - range 0.99 to 1.7 times on writing.

http://www.roylongbottom.org.uk/Raspber ... #anchor21b

Next stage involved producing a series of directories, with average file sizes between 6 KB and 500 KB, occupying >100 MB (similar to sizes quoted in XBMC Forum). Results, via above link, show that, still using the SanDisk Extreme USB 3.0 drive, F2fs is a little faster at the larger file sizes, but the position is reversed at reducing file sizes. Most significant is at 6 KB, where Ext4 is 70% faster, with the du command reporting 178 MB, compared with 269 MB with F2fs.

I installed XBMC Media Center, on a Windows based PC, to produce a Thumbnails directory from photographs, included in the mix, in case there was something special about them. The directory comprised 4370 JPG files at around 34 KB average size, occupying 161 MB with Ext4 and 178 MB under F2fs, the former being slightly faster. These directories were also copied, using two other USB sticks, via the Raspberry Pi and a Linux based PC (plus limited tests with FAT formatting). Linux was faster on all, and the other drives were slower than the Extreme, but there were no significant variations between Ext4 and F2fs formatting. Results are again shown via above link.

The last phase is to try using XBMC on the RPi. I had not realised that this involved installing OpenElec (Open Embedded Linux Entertainment Center) that pre-configures XBMC. The current XBMC version is 3.2.4 and Linux version 3.10.20. So far, DriveSpeed benchmarks and file copying tests have been run using Ext4 formatting, a surprise being that speeds were much slower than via Raspbian (down to near half speed). These are still being analysed. So far, unlike the XMBC Forum reports, I have not been able to mount the F2fs directory (might need an earlier version of XMBC?). I am appealing for HELP in different places. Can anyone here oblige?

[RoyLongbottom]

I have managed to install versions of OpenElec/XBMC that can handle F2fs files. Source download details, test results and analysis are in:

http://www.roylongbottom.org.uk/Raspber ... hmarks.htm

The software has quite high background activity, particularly with an RSS Feeds option that scrolls news headlines on the screen, where CPU utilisation is around 95%. I ran some of my processor benchmarks to see if the activity could affect main foreground applications. Example results are below. These show that the overheads can produce degraded performance, compared with Raspbian and possibly quite severely with RSS enabled.

Code: Select all

                  No RSS News Feeds                RSS News Feeds Scrolling

                  Dhrystone Vax MIPS, Linpack MFLOPS, higher is faster

                    MIPS  MFLOPS                       MIPS  MFLOPS

   Raspbian          822    40.8
   OE 3.2.4          835    42.8                        824    33.6
   OE Later          833    41.1                        812    37.4

                          MP-MFLOPS 1 nad 2 Threads, higher is faster

                    2 Ops/Word     32 Ops/Word         2 Ops/Word     32 Ops/Word
   KBytes           12.8    12.8   12800   12800       12.8    12.8   12800   12800
   Threads             1       2       1       2          1       2       1       2

   Raspbian           50      52     166     169
   OE 3.2.4           42      41     129     130         22      32      74     102
   OE Later           43      43     128     130         21      29      70      95

The next step was to run my DriveSpeed benchmark, to compare OpenElec Ext4/F2fs and Raspbian/OpenElec F2fs speeds. On most tests, there was no clear winner on these comparisons, the exception being the test that measures cached speed, where Raspbian was much more effective. Then further tests were run using 1000 files of varying sizes, with caching enabled. Example results are below, showing Raspbian faster but similar on F2fs/Ext4 ratios. Best ratios are with RSS at the slow crawl. Clearly, any benchmarks should indicate whether RSS is enabled and one wonders whether other XBMC facilities can make even greater differences.

Code: Select all

   1000 Files With Caching, Write + Read Milliseconds/File, Ratio >1 F2fs is faster

  
  File KB                     4      8     16     32     64    128    256    512   1024
  
  Raspbian        Ext4     0.57   0.59   0.81   1.32   2.58   6.06  13.64  39.40  78.58
                  F2fs     0.58   0.48   0.57   0.98   2.19   6.41  20.49  39.97  78.98
                  Ratio    0.99   1.22   1.41   1.34   1.18   0.95   0.67   0.99   0.99

  Later OpenElec  Ext4     0.56   0.71   1.02   2.14   4.33  10.51  26.96  50.54  97.93
                  F2fs     0.44   0.57   0.79   1.82   3.99  18.54  29.86  57.05 117.20
                  Ratio    1.26   1.25   1.30   1.17   1.08   0.57   0.90   0.89   0.84
  
  With Scrolling RSS News Feeds
  
  Later OpenElec  Ext4     1.24   1.52   2.25   5.02  10.33  21.68  36.05  69.84 128.63
                  F2fs     0.89   1.00   1.64   3.08   6.47  23.12  Crashed
                  Ratio    1.38   1.51   1.37   1.63   1.60   0.94 

Next are some real file copying tests with F2fs and Ext4 and consideration of data volumes. Here, OpenElec Linux du command can indicate much too high a value, invalidating MB/second speeds based on this.

[jamesh]

Hmm. I do think that disabling lots of the 'cruft' in XBMC would make it a lot faster on the Raspi. I really must see what can and cannot be disabled.

[RoyLongbottom]

The last phase was to generate data via OpenElec/XBMC and run file copying using Raspbian and OpenElec. Detailed results are in:

http://www.roylongbottom.org.uk/Raspber ... tm#anchor7

Copying via Raspbian was at least 10% faster with Ext4 and 50% using F2fs, with somewhat less proportions in CPU time used. None of the tests showed significant differences between F2fs and Ext4 speeds.

With the RSS scrolling news option enabled, copying time under OpenElec took at least 20% longer. The directories were copied to a slow, old USB stick but there was, again, little difference in F2fs and Ext4 speeds.

Using Raspbian and, based on using the du command, calculations showed that average F2fs file sizes, used for all copying tests, were 4 KB greater than those in the Ext4 format. OpenElec du wrongly reported file space approximately twice that indicated by Raspbian. To investigate further, the tests were repeated, concurrently using performance monitors to measure drive data volumes, memory occupancy and CPU utilisation - vmstat on Linux and Raspbian, top and iostat on OpenElec (via PuTTy SSH client terminals on a PC).

The monitors indicated that Linux, Raspbian and OpenElec were seeing the same volume of data transferred. This means that calculating drive MB/second based OpenElec du commands is invalid.

The OpenElec du results reflected cache occupancy for all systems (but why twice the data read?). The doubling was not applicable for an earlier test with 23,000 small files but was when the number of files were reduced to less than 6000.

In order to identify cache space actually claimed by copying, one of my stress test programs was used to reduce initial cache size. Under Raspbian, after a 450 MB demand there was 367 MB of free memory, minimum cache size and some space swapped out. Using OpenElec, cache size could not be reduced below 110 MB with only 220 MB free RAM. Also, the stress program could not allocate more than 200 MB and there was no sign of swapping (system settings?).

I have not been able to demonstrate that copying F2fs based files is significantly faster than with Ext4 formatting.