User avatar
HawaiianPi
Posts: 7640
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: PI3 microSD and wear leveling

Wed Nov 15, 2017 10:40 am

doublehp wrote:
Tue Nov 14, 2017 5:31 pm
32-400GB uSD cards are a bit more expensive than legacy SATA SSD of equivalent storage, but not that much.
Yes, but the problem is a large micro SD card is still a low-end flash memory device, while an SSD is a high performance flash memory device. An SSD is far superior to an SD card in every way except physical size. At the current pricing it actually makes sense to consider running a Pi from an SSD, rather than a large SD card.

If size is a concern, you can pay a little more for mSATA or M.2 form factor drives which fit in the footprint of a Pi3.
Image
Click picture for my review of the X850 mSATA expansion board.
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups, and where is that annoying music coming from?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 32227
Joined: Sat Jul 30, 2011 7:41 pm

Re: PI3 microSD and wear leveling

Wed Nov 15, 2017 11:34 am

doublehp wrote:
Tue Nov 14, 2017 4:43 pm
Ideal for Android, they mean, ideal for the external SD storage for phones. And even with this use, I had a 64GB card die in one year.

What they don't like is to boot on uSD, and store /var/log in there. ext3 is pretty much designed to kill classic SD cards. It's pretty much urgent Raspbian moves to F2FS.

Up to now, I was buying Sandisk Ultra cards (red-grey, usually). Now moving to High Endurance (installed it last week), for a try. For eMMC: https://www.amazon.fr/gp/product/B01DNV ... V087&psc=1 (installed it last month).

But I am also now trying an other alternative, which will be much cheaper. SD or MMC cards add from 8 to 40€ to the cost of an rPi. So, for only 4€ more than the usual cost of an rPi, I have found this Orange pi. The keyword we are looking for are the PLUS versions; this is what means "MMC soldered onboard":
https://www.amazon.fr/Orange-Support-Lu ... %2Bpc&th=1
It's said to be software compatible with rpi, and should support identical raspbian configuration (I expect IO ports to have different adresses, like for example, the I2C settings). Some recent versions with newer CPU are said to be harder to boot from MMC. The procedure to migrate raspbian to MMC on this H3 should work as following the tutorial to the letter; but a friend has a more recent H5, and he claims that there is no working tuto for now. Note that there should exist "zero plus" versions (not 100% sure).

Should receive my orange pi in two weeks.
Some software MIGHT work on both Raspberry Pi and Orange, but not any Pi specific stuff (camera, anything that uses the VC4). Note also, Orange Pi support is basically rubbish in comparison to Pi.

eMMC will last longer, but when its does fail its soldered down and cannot be replaced.

If you set up your Pi to avoid logging to the SD card, you will greatly increase the SD cards life. If you are constantly writing things like video, then SD card life time will be an issue, whatever you do, and whatever platform.
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

Averell
Posts: 3
Joined: Sun Jun 17, 2018 2:55 pm

Re: PI3 microSD and wear leveling

Sat Jan 26, 2019 1:48 pm

Hello,
this topic contains highly interesting information. One thing was missing : real testing. Since the Sandisk High Endurance card was mentioned, I decided to buy one and test.
The test consists in writing series of :
10000 files of 17 bytes
1000 files of 17 KB
100 files of 17 MB
10 files of 170 MB
The total is around 3.5GB and is equivalent to writing 1MB every day for 10 years.
Then everything was read again and compared.
The card had only 8GB of free space (to evaluate the pseudo-leveling on free space, explained somewhere above.

First SD card tested had errors from the beginning, despite being from a known brand. Perhaps one day I will try again, but presently the test was discontinued.
Then I tested with the card arrived with the RPI with the Noobs distri. It ran without errors for 5000 cycles, which is the equivalent of writing 5GB of data every day for 10 years.
Then the errors appeared, and after 100 more cycles, the card died.
The conclusion of this test is that the wear leveling system is not bad. It does not use only the free space (the card would have died at 3000 cycles at most), but the general result is :
  • 17.5 TB written,
  • equivalent of 1093 writes on the full card.
  • equivalent of writing 5GB every day for 10 years
Since a good SSD allows 3000 writes, the result is not bad : who is writing 5GB every day on a RPI ?

Simultaneously, I was testing on a second Raspeberry Pi a Sandisk 32GB High Endurance card (the white card). The test lasted for 6 months, and then it was enough and I stopped it, to be able to use the card again. Result is :
  • 70TB written
  • 222 000 000 files
  • equivalent of 2187 writes of the full card
  • equivalent of writing 10GB in 30 000 files, every day for 20 years.
The conclusion is we have a real wear leveling on this card.
I could continue the test to the maximum possible of 28000 cycles (Erase count 3000), we cannot expect more. But presently i thought that the test was even better than expected, and I prefer to continue running my Raspberry Pi normally on this card.

User avatar
bstrobl
Posts: 103
Joined: Wed Jun 04, 2014 8:31 pm
Location: Germany

Re: PI3 microSD and wear leveling

Mon Jan 28, 2019 11:52 am

Thanks for testing this out, looks like we have a real winner with the High Endurance card. :)

kozlynx
Posts: 5
Joined: Fri Feb 08, 2019 7:59 am

Re: PI3 microSD and wear leveling

Fri Feb 08, 2019 8:21 am

What do you think of SanDisk Industrial cards? They are expensive, but reading datasheet here https://eu.mouser.com/datasheet/2/669/S ... 285144.pdf it seems they're very similar to High Endurance cards, but listed read/write speeds are faster. Not sure though if Raspberry Pi can make use of them.

bdg2
Posts: 10
Joined: Thu May 24, 2012 10:27 pm

Re: PI3 microSD and wear leveling

Thu Jul 04, 2019 5:55 pm

How can a microSD card possibly have wear levelling?

Surely in an SSD with wear leveling the mapping table needs to be in RAM and there needs to be a bank of capacitors so that when the power goes off the processor in the SSD still has time to save the RAM to flash. If the table was just in flash and not held in RAM during operation then nothing would be achieved since it would just mean the flash that holds the table would wear out quickly and the life of the device would still be short.

User avatar
rpdom
Posts: 21534
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: PI3 microSD and wear leveling

Thu Jul 04, 2019 6:17 pm

bdg2 wrote:
Thu Jul 04, 2019 5:55 pm
How can a microSD card possibly have wear levelling?

Surely in an SSD with wear leveling the mapping table needs to be in RAM and there needs to be a bank of capacitors so that when the power goes off the processor in the SSD still has time to save the RAM to flash. If the table was just in flash and not held in RAM during operation then nothing would be achieved since it would just mean the flash that holds the table would wear out quickly and the life of the device would still be short.
I think you need to read up on what Wear Levelling means. It has nothing to do with saving data when power is lost.
Unreadable squiggle

doublehp
Posts: 77
Joined: Wed May 02, 2012 1:11 am

Re: PI3 microSD and wear leveling

Thu Jul 04, 2019 11:30 pm

bdg2 wrote:
Thu Jul 04, 2019 5:55 pm
How can a microSD card possibly have wear levelling?

Surely in an SSD with wear leveling the mapping table needs to be in RAM and there needs to be a bank of capacitors so that when the power goes off the processor in the SSD still has time to save the RAM to flash. If the table was just in flash and not held in RAM during operation then nothing would be achieved since it would just mean the flash that holds the table would wear out quickly and the life of the device would still be short.
It's offtopic; but, I will try a short answer.

I have read about WL in 2008, so my knowledges are outdated, and things are now much more powerfull than then.

But, already in that time, there were at least 5 completely different technologies; and 3 of them were already dedicated to SD cards (and 2 were possible only in larger SSDs).

One proof: go on Sandisk website, and, either read their web pages, or start a chat with the tech support. Since at least 2013, all their cards are certified to include WL.

How ? Let's make things simple: chips need to store a table to convert logical sectors into physical NAND gates. This allocation table is a huge mess, and needs to be stored. It's critical, but, much smaller than the data sectors; so, this table can be stored in a more reliable kind of gates (using larger transistors, less susceptible to aging). Considering modern technology, SD cards physically have plenty room for some RAM and a small power capacitor; so, it can be possible to store frequent modifications in RAM, and commit changes to long term storage only at unplug time. Also, keep in mind that SRAM can be certified 10 years (but are susceptible to cosmic rays, much more than other kings of memory).

There also are several different kinds of algorythm to handle that conversion table; the linear ones (which look like FAT32) are crappy but easy to implement. More complex ones are subbecjt to expensive patents, and involve algorythms similar to MD5 (a small seed and a medium algorythm are used to generate a long list of non consecutive numbers; but in the case of WL, unlike MD5 and GSM requirements, you need to generate numbers only once; the algo is similar to the onces used in GSM or Rolling Codes, except each output number exists only once in the output sequence). The aim is to have a bijection between the logical sectors, and the physical NANDs, where two consecutive logical sectors are NOT stored on adjascent NANDs. Because corruption is often located on a specific place of the chip, spreading a logical block all over the chip increases the chance to rebuild the user data (using CRC) when corruption of one block is detected.

Underneath this, good manufacturers may add a second translation layer to remap dead NANDs.

You also need a third mapping layer so that consecutive writes (in time) of the same logical block will not burn always the same NANDs.

SDs usually have 20 to 40% hidden gates; while SSDs have from 60 to 300%. Cheap SSDs may have less than 5%; or even down to 0% (some even sell 4GB chips under a 32GB packaging).

Even if SDs have less hidden blocks, and weaker processing power, they share large lines of concepts with SSD. But to be able to perform WL with a weaker computing unit, they have to work harder on the math side. IE, SDs are less sensible to free space ratio than SSDs.

User avatar
HawaiianPi
Posts: 7640
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: PI3 microSD and wear leveling

Fri Jul 05, 2019 1:14 am

doublehp wrote:
Thu Jul 04, 2019 11:30 pm
Even if SDs have less hidden blocks, and weaker processing power, they share large lines of concepts with SSD.
Wear levelling is not a requirement for micro SD cards (I don't believe it's even mentioned in the SD Associan's specs). It is an entirely optional feature that may or may not be implemented by the card manufacturer.

Yes, SanDisk white papers do state their cards all have it. I would not blindly trust that all other brands are the same.
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups, and where is that annoying music coming from?

Return to “Advanced users”