Read/Write cycles of a SD card

[SearchMyAim]

Hello,
I just wondered if anyone has some thoughts about the read/write cycles of a sd card. If you use it as webserver you will reach 1million writes kinda fast I guess (a couple of months is what I consider as kinda fast).

Greetings,
Search

[malakai]

It really depends on a few things the architecture of the card and the size 1 million writes on an 8 gig is 8 million gigs that's a poor example. SD Cards generally have wear leveling http://en.wikipedia.org/wiki/Wear_leveling there are so many factors one company did an end to end write on cards and they got 3 months out of the last card. This was filling the card completely over and over non stop and only writing to the next block.

The life of SD Cards is limited but it could be a year or more on a Pi setup just depends on the setup and the logic it uses to update the card itself. So much of what the PI is doing is actually read not write which gets people confused consider the logs if you have 8 gigs of free space on your Pi the time it would take for you to fill that much data with just logs is quite long.

[Lob0426]

Probably the most destructive thing would be overuse of a Swap (vitual memory). This would cause constant read/writes to the SD card. Wear leveling will hide it for a while.

A webserver is mostly reads. Even if you use it as a blog, it will take quite a while to kill your SD card. I have a webserver up on a RasPi. It only host a web page not a blog. It has been running for a couple of months straight with no problems. The only time it seems to act up is when I try to use rpi-update. The Card is a Transcend 8GB. Others have been complaining about these cards. I guess if you were to setup too many logfiles you may see a problem in a relatively short time. But it would probably be more than a couple of months.

One cure for this would be to keep your system files on the SD card and send everything else (log files, swap and such) to a USB stick, which are cheaper, or a USB HDD which does not have the write cycle limitations. Another is to use an over-sized card, this "wear levels" over a larger card. A 8GB card instead of a 4GB card.

Remember the reads are not limited just the writes! Most new flash memory is no less than 100,000 writes. Most of it is 1 million writes now.

[hojnikb]

Probably the most destructive thing would be overuse of a Swap (vitual memory). This would cause constant read/writes to the SD card. Wear leveling will hide it for a while.

A webserver is mostly reads. Even if you use it as a blog, it will take quite a while to kill your SD card. I have a webserver up on a RasPi. It only host a web page not a blog. It has been running for a couple of months straight with no problems. The only time it seems to act up is when I try to use rpi-update. The Card is a Transcend 8GB. Others have been complaining about these cards. I guess if you were to setup too many logfiles you may see a problem in a relatively short time. But it would probably be more than a couple of months.

One cure for this would be to keep your system files on the SD card and send everything else (log files, swap and such) to a USB stick, which are cheaper, or a USB HDD which does not have the write cycle limitations. Another is to use an over-sized card, this "wear levels" over a larger card. A 8GB card instead of a 4GB card.

Remember the reads are not limited just the writes! Most new flash memory is no less than 100,000 writes. Most of it is 1 million writes now.

Thats is actually untrue. Most flash cards today are in the range of 10k-1k rewrites (depends on the type of nand used).Remember with the new generation of flash, rewrites actually decresses not increases.
But still wearout is nothing to worry about. In worts case scenario you end up with a card that cannot be written to.Data will not go away once card reaches it maximum writes..

[SearchMyAim]

Ok, thank you guys for the quick and accurate answer.

Greetings, Search

[Narf03]

Is there a way to know an SD is about to fail soon caused of wear leveling?
Thanks.

[Lob0426]

Probably the most destructive thing would be overuse of a Swap (vitual memory). This would cause constant read/writes to the SD card. Wear leveling will hide it for a while.

A webserver is mostly reads. Even if you use it as a blog, it will take quite a while to kill your SD card. I have a webserver up on a RasPi. It only host a web page not a blog. It has been running for a couple of months straight with no problems. The only time it seems to act up is when I try to use rpi-update. The Card is a Transcend 8GB. Others have been complaining about these cards. I guess if you were to setup too many logfiles you may see a problem in a relatively short time. But it would probably be more than a couple of months.

One cure for this would be to keep your system files on the SD card and send everything else (log files, swap and such) to a USB stick, which are cheaper, or a USB HDD which does not have the write cycle limitations. Another is to use an over-sized card, this "wear levels" over a larger card. A 8GB card instead of a 4GB card.

Remember the reads are not limited just the writes! Most new flash memory is no less than 100,000 writes. Most of it is 1 million writes now.

Thats is actually untrue. Most flash cards today are in the range of 10k-1k rewrites (depends on the type of nand used).Remember with the new generation of flash, rewrites actually decresses not increases.
But still wearout is nothing to worry about. In worts case scenario you end up with a card that cannot be written to.Data will not go away once card reaches it maximum writes..

You are correct:
Sorry, those figures were for SLC NOR flash used in SSD's. Removable media is normally NAND flash though there are several types as you stated. 10K writes are closer to the mark than 1k writes. The range is more like 5k to 10k for branded SD cards. These figures are for flash memory without wear leveling of any type. The figures with wear leveling go up. If you wrote the whole card full every time you would only get 1k to 10k writes. That would effectively cancel wear leveling out. This would be an atypical use of a RasPi SD card.

Wear leveling spreads the writes to different memory cells to extend the life of flash memory. A full card is going to "wear out" quicker. You have a 2 GB card and most RasPi images do fill a 2GB almost full. You will wear out the card quicker than if you were using a 4GB card. Wear leveling will not have as much space to spread the writes (this depends on the wear leveling system used). Overall most people are never going to wear out a SD card on a RasPi. Others may have to replace them occasionally.

This issue was discussed in these forums clear back in 2011.

I am not a proponent of placing a virtual memory onto a flash device. If I did it would be a nice cheap USB memory stick, so I could replace it as often as needed. I have used Windows ReadyBoost for a couple of years now. I have yet to have one of these USB flash devices go toes up. Of course ReadyBoost is not virtual memory but a cache. So writes to it are much less than in a virtual memory use. But these USB flash drives are still going strong after two years of constant use. I would expect that most SD cards will last at least a couple of years in average use.

I have an 8GB card in my first RasPi. It has a swap partition of 512MB. It has been used continuously since June. It is still operating just fine. I would recommend using a swap file rather than a swap partition.

[sjwright]

I can tell you it didn't take long for a 4gb Kingston SD card to start corrupting data with my project. I run a website hosted on a Raspberry Pi using Wordpress, and it was a matter of a couple of weeks when I noticed some odd things with it, then a few days later I went to do a routine update and it never came back on after a restart command.

I use an SD card to hold the boot instruction and the config, and run the OS from a Corsair Voyager GT USB flash drive, its a hell of a lot quicker than running from an SD card, and more reliable. I've posted some of my observations on my website (http://www.dingleberrypi.com).

I need to get round to writing a tutorial on how to use a Flash USB drive instead of an SD card, its very simple though.

[Aydan]

To alleviate this to a degree you should run fstrim every so often on a flash drive. No matter if USB or SD-card.
This way you free up blocks for wear leveling.

Rgeards
Aydan

[hojnikb]

To alleviate this to a degree you should run fstrim every so often on a flash drive. No matter if USB or SD-card.
This way you free up blocks for wear leveling.

Rgeards
Aydan

sd cards dont support trim as far as i know. So its kinda pointless..

[hojnikb]

I can tell you it didn't take long for a 4gb Kingston SD card to start corrupting data with my project. I run a website hosted on a Raspberry Pi using Wordpress, and it was a matter of a couple of weeks when I noticed some odd things with it, then a few days later I went to do a routine update and it never came back on after a restart command.

I use an SD card to hold the boot instruction and the config, and run the OS from a Corsair Voyager GT USB flash drive, its a hell of a lot quicker than running from an SD card, and more reliable. I've posted some of my observations on my website (http://www.dingleberrypi.com).

I need to get round to writing a tutorial on how to use a Flash USB drive instead of an SD card, its very simple though.

Currupting is not actually an issue of wearout of the card, but actually its a common issue with raspberry, which for some reason currupts data if its core clock is increased. It can also happen on a no OC pi, if the card reader doesnt like the card.

[Aydan]

To alleviate this to a degree you should run fstrim every so often on a flash drive. No matter if USB or SD-card.
This way you free up blocks for wear leveling.

Rgeards
Aydan

sd cards dont support trim as far as i know. So its kinda pointless..

Nope, it's not.
See here
The SD card block will be erased which tells the controller it can put the block into the wear leveling pool.

Regards
Aydan

[Heater]

hojnikb,

sd cards dont support trim as far as i know. So its kinda pointless..

I would not be sure about that. Om my Pi I can do the following:

Code: Select all

pi@raspberrypi ~ $ cp 6D-xuPIn_9c.flv temp
pi@raspberrypi ~ $ sudo fstrim -v /
/: 0 bytes were trimmed
pi@raspberrypi ~ $ rm temp 
pi@raspberrypi ~ $ sync
pi@raspberrypi ~ $ sudo fstrim -v /
/: 118149120 bytes were trimmed
pi@raspberrypi ~ $ 

So fstrim thinks it is doing something. A believe SD cards have an ERASE command which is used instead to to the same thing as an SSD TRIM command.

[hojnikb]

Well that is something new to me

Never thought sd cards support something like that
Will give it a try once i get to my Pi

[taking3271]

I tried to run a freeswitch server from a nextech USB on a pogoplug and it started to corrupt after 3 months.

I would like to run this on the Raspberry, but I think it would be safer to use a USB hard drive. It might be possible to configure a ram disk for the logs and use crontab to write the blocks, but this would be risky without a battery backup. Even the SSD manufactures are not recommending using the drives in servers (at least not the low end products) .

[SporkmasterD]

Or you could go with an industrial-rated SD card, like Rockwell Automation's (per RA KB article 31720):
Rockwell Automation Non-Volatile memory cards are rated for >2,000,000 write\erase cycles.

[honda4life]

I'm trying (1500 cycles atm)
https://xively.com/feeds/127941/

[Penguin]

I'm trying (1500 cycles atm)
https://xively.com/feeds/127941/

A very interesting test. I'm very curious about this (card life cycle) as I've just bought a Pi I'll be using for doing a lot of writes (or at least a lot in my view, which might be wrong).

For each "run" it will download 2000 files (and create 1000 directories, although I could get away with not doing this, I guess), perform several SQL queries (UPDATE/INSERT, to two tables) per download. After that, there will be file processing, but this will just be several UPDATE/INSERT to an SQL database (again probably just two tables).

I would be doing this approximately once per fortnight. I'm not sure how long I would get out of a card.

I've thought about two possibilities (to help prolong the card's life):

1. Have MySQL running on a NAS and also write the files there.
2. Connect a spare 2.5" SATA drive via USB and use that for storing the database and files.

[honda4life]

Think that isn't an issue.
40.000 and counting... very good card I suppose

[permaband]

The main problem is unclean dismounts/shutdowns corrupting the fs and maybe causing damage to the cards themselves too. I've been through 3 cards since I got my rev 1 pi way back when. All caused by catastrophic root fs corruption, all caused by power outtages, lock ups leading to power outages etc. None caused by the sdcard itself wearing out. FWIW I only buy sandisk class 4's now. They are cheap and have been the most robust for me thus far.

[electronicman]

I now have 5 Transcend class 10, 8 GB cards, and one of them has already failed. Unfortunately I do not remember which one is was, so I have no idea of how many times it has been written too, but it cannot be very many.

[ppisa]

There are many applications where it is possible use temporary overlay over root filesystem and left root read-only for (allmost) whole operation.

In the fact, I would not believe in read-write SDcard root-fs without automatic second device boot replacement for serious/industrial application when standard consumer grade cards are used.

We have been contracted to enhance reliability of such application build on Raspbian image and we propose to use Aufs to build overlay in RAM/tmpfs to ensure reliable operation. Raspbian and generally any other distribution can be used with our scripts without noticing that SD card root is in the fact read-only.
The separate partition is allocated to keep persistent data of target application. That partition is cleaned/formated in the case of that partition FS corruption.

Kernel with Aufs patches included is available there

https://github.com/ppisa/linux-rpi

The QEMU ARMv6/ARM1176 versatile patches are included as well to allow Raspbian image testing on PC.

The scripts to setup overlay based on Aufs and replace init are part of another repository

https://github.com/ppisa/rpi-utils
- init-overlay contains key parts of Raspbian filesytem which has been modified/added

* sbin/init-overlay - contains standard init replacement which setup overlay
and then executes original init
* sbin/overlayctl - provides command to install, uninstall, enable and disable
overlay feature for next boot
* etc/init.d/mountpersistent.sh - rc script to check and mount persistent data partition
- WIP material for now

The preliminary experimental setup has been tested by company using RPi in industrial application. They randomly cycled RPi power and root file-system has survived 3 weeks. They have problems with SDcards corruptions and wearing in the field before. Some probably caused by Flash device not bearing well power lost during write - even that ext4 is journaled, there seems to be working no transaction barriers on standard grade SDcards and file-system cannot reconstruct state for next boot.

The other cards seems to really experience wear/tear problems.

We provide all above mentioned enhancements in open-source form same as all future enhancements of this project. We would be glad if Aufs or OverlayFS is included in official Raspbian kernels. Aufs is/was standard part of Debian kernels. Our scripts are prepared for OverlayFS and UnionFS as well.

I hope we find time to prepare better documentation in future but I hope that code can be already for some use and experimenting for others so I am providing this notice and public offer to use our work. Some basic little dated documentation is there

https://github.com/ppisa/rpi-utils/blob ... verlay.txt

overlayctl can be used for most of management tasks now.

Pavel Pisa

[ppisa]

I forgot to note another observation. The original Raspbian image (2014-01-07-wheezy-raspbian.img) which we have used has very strange setup because kernel command line leads to mounting root read-write directly by kernel because /boot/cmdline.txt is missing ro option. Then /etc/init.d/checkroot.sh remounts ro and attempt to check and correct root filesystem. But that is late, kernel already tried to reply journal and in more severe problem could make thing worse because it does not use more complete mechanisms to recover found in e2fsck.

Same risky behavior is in mounting /boot read-write by default. The FAT32 requires marking as dirty after mount and mark as clean after correct unmount. But coincidence of power failure with these operations can lead to Flash content damage - sector or may be even larger eraseblock. Again there is no reason to mount /boot RW for most times. But when kernel or boot parameters are updated it requires remount /boot RW.

[jan_paulussen]

I have had about a dozen different Raspi's running a LAMP server to control heatings, lights and a solar collector (solarair.livotel.com) from a distance. They al used a SD card, and the SQL was reading/writing about every minute or more... Al of them failed about after 6 to 12 months... I think it means it is really unusable for real applications. I am still running a test with a USB-SSD, of wich I could not resize to the real size, but hopefully the internal controller can still optimze.... And I am trying with other SBC's with eMMc on board to see if that's any better... I really wish the New Years resolution for the Raspberries organisation is to do something about this major problem... It would make the difference between the Raspberry being just a toy, or a real usable stable small computer...

[Heater]

Nothing needs to be done. Except by you.

Make your root file system read-only. Then it is not going to get corrupted.

Put your data on some other media on a USB port.

I strongly suspect that if you just simply move everything to on board FLASH storage it will suffer just as badly.