Kernel panic (me too!) *SOLVED*

[MrGibbage]

My raspberry pi digital picture frame has been running non stop on my kitchen counter since Feb. Today it shut down when my house lost power. When it rebooted after the power was restored, I got some scary messages (all copied from the screen onto a piece of paper, and then retyped here, so I may have transcribed some things wrong):

ext4_check_descriptors: block bitmap for group 0 not in group
group descriptors corrupted!
List of all partitiions
mmcblk0
mmcblk0p1
mmcblk0p2
No filesystem could mount root, tried...
Kernel panic-not syncing: VFS: unable to mount root from unknown-block(179,2)
Entering kbd (current=0xda82ac80, pid 1) due to Keyboard Entry

kbd>_

I don't have a keyboard hooked up to it, but even if I did, I wouldn't know what to type. I know a little about linux (when it works), but I suck at troubleshooting it. Can someone please help? In the spirit of Christmas?

[DougieLawson]

Kernel panic-not syncing: VFS: unable to mount root from unknown-block(179,2)

179 == /dev/mmcblk0
2 == /dev/mmcblk0p2

That's your ext4 root filesystem.

Pull the card, create a dummy F:\forcefsck file on it using Windows (I'm assuming Windows will mount your card as drive F:).
Put it back in the RPi and it will fsck your filesystems on the way up.

You can add some strange arcane incantations (or cast some runes) or in other words add forcefsck on the end of F:\cmdline.txt to do the same thing, but that's more complex to my way of thinking.

[MrGibbage]

I made an empty file named forcefsck and out it on the card (right next to bootcode.bin and start.elf). There is no file extension on the forcefsck file (double checked with directory opus).

When booting, I don't see anything different. It doesn't seem to be running fsck at all. All of the messages look exactly the same as before.

Did I goon something up?

[DougieLawson]

It can be fixed.

But we're going to have to aim for single user mode.

Take a copy of f:/cmdline.txt as f:/cmdline2.txt (belts and braces mode).
add "single" to the end of your cmdline.txt file.

Boot the Pi (keyboard needed).
When it boots you'll need to use your root password to login.

If you've never set a root password you'll need to add "init=/bin/sh" instead of single.

That gets you to a blank screen with a single # and the cursor

NOTE: It's a root shell so you can completely destroy the whole filesystem. (We're not planning on going there.)

fsck -aw /dev/mmcblk0p1 # this will check the FAT /boot partition
fsck -pf /dev/mmcblk0p2 # this will check the ext4 partition
sync
sync
halt

Pull the card and reverse the changes to F:\cmdline.txt
Reboot and it should be OK.

[MrGibbage]

Wow, that didn't make a difference either. I edited the cmdline.txt file to read

Code: Select all

dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait
single

I used notepad++, not the regular windows notepad

After booting, it just goes to that kbd> prompt. And even stranger, I did have my keyboard plugged in and I couldn't type anything there. Am I supposed to be able to type something there?

Bottom line is, I didn't get to a point where I could log in and go farther.

[jojopi]

If you've never set a root password you'll need to add "init=/bin/sh" instead of single.

That is not true; if no root password is set, single user mode will not prompt for one.

The init scripts do not look for forcefsck in the /boot partition, so only the "more complex" cmdline.txt method is possible.

Neither "forcefsck", "single", nor "init=/bin/sh" will solve this problem, though. Since the kernel is "unable to mount root" filesystem, there is no /sbin/e2fsck, no /sbin/sulogin, and no /bin/sh.

The only way to run fsck without access to another Linux computer will be to boot into the "safe mode" emergency initramfs by jumpering GPIO3 to ground: http://www.raspberrypi.org/phpBB3/viewt ... 13#p129413. Hit enter to get a prompt, remove the jumper, and run:

Code: Select all

e2fsck.static -fy /dev/mmcblk0p2
reboot

I am not confident that fsck will be able to repair the filesystem to a bootable state, however. Unfortunately, "block bitmap not in group" suggests some very serious corruption.

[MrGibbage]

I really hope that's not the case. I'll give the jumper a try tomorrow. I am hoping that maybe someone else will have some other ideas, maybe??? I really hate to lose those thousands of pictures on the card

[jdb]

I really hope that's not the case. I'll give the jumper a try tomorrow. I am hoping that maybe someone else will have some other ideas, maybe??? I really hate to lose those thousands of pictures on the card

Do not use the SD card to attempt to recover any images on the card.

If the filesystem is damaged, then attempting to repair it may result in data loss due to the uncertain behaviour of most filesystem repair methods.

First step is to plug the card into a PC, and READ the entire card using a disk imaging software to a file. This then will be a bomb-proof backup (although trapped in a broken ext4 shell) from which you can work from.

If you have no Linux machine available, then you can use a second SD card to write a clean OS to, boot from that and then use a USB-SD adapter to access the faulty drive from the Pi.

[MrGibbage]

Disk image has been made (using Win32 Disk Imager)

When I plugged the SD card into my Ubuntu machine, I got this error message:


Please don't give up on me folks! I just *KNOW* we can figure it out. I really appreciate all the help so far!

[jojopi]

That error is to be expected. Like the Pi, Ubuntu cannot mount the ext4 filesystem without repair.

Make a backup image of partition 2 of the SD card to a local file (I think this is more convenient than making a backup of the entire card, and no less useful):

Code: Select all

sudo dd bs=1M if=/dev/sde2 >BACKUP.ORIGINAL.IMG

Then make a copy of the backup image, and try to repair the copy (rather than the original, or the card itself):

Code: Select all

cp BACKUP.ORIGINAL.IMG BACKUP.COPY.IMG
e2fsck -fy BACKUP.COPY.IMG

If this is successful, you should be able to mount the repaired image, and access the recovered data:

Code: Select all

sudo mount -roloop BACKUP.COPY.IMG /mnt
ls -l /mnt

Otherwise, post any errors.

[MrGibbage]

Thank you very much! I am at work now, but I will try this when I get home.

In the meantime, should I buy a new SD card? If so, are there any recommendations for a really good, stable card? I have seen the various compatibility lists out there, and my current card is on the list. But I was wondering if there is a crowd favorite that I should look at.

[MrGibbage]

I don't think it worked

Code: Select all

xbmc@xbmc-desktop:~$ sudo dd bs=1M if=/dev/sde2 >backup.origial.img
[sudo] password for xbmc:
15323+0 records in
15323+0 records out
16067330048 bytes (16 GB) copied, 766.193 s, 21.0 MB/s
xbmc@xbmc-desktop:~$ ls -l
total 15858080
-rw-rw-r-- 1 xbmc xbmc 16067330048 Dec 20 15:06 backup.origial.img
xbmc@xbmc-desktop:~$ cp backup.origial.img backup.copy.img
xbmc@xbmc-desktop:~$ e2fsck -fy backup.copy.img
e2fsck 1.42.5 (29-Jul-2012)
ext2fs_check_desc: Corrupt group descriptor: bad block for block bitmap
e2fsck: Group descriptors look bad... trying backup blocks...
backup.copy.img: recovering journal
e2fsck: unable to set superblock flags on backup.copy.img
 
 
backup.copy.img: ***** FILE SYSTEM WAS MODIFIED *****
 
backup.copy.img: ********** WARNING: Filesystem still has errors **********
 
xbmc@xbmc-desktop:~$ sudo mount -roloop backup.copy.img /mnt
[sudo] password for xbmc:
xbmc@xbmc-desktop:~$ cd /mnt
xbmc@xbmc-desktop:/mnt$ ls
bin/        etc/        media/      root/       srv/        var/
boot/       home/       mnt/        run/        sys/        
.dbus/      lib/        opt/        sbin/       tmp/        
dev/        lost+found/ proc/       selinux/    usr/        

[jojopi]

That looks somewhat promising, actually. I suggest you unmount the image and run e2fsck at least another couple of times:

Code: Select all

cd
sudo umount /mnt
e2fsck -fy backup.copy.img
e2fsck -fy backup.copy.img

The first run may have exited prematurely because the block bitmap fix was so fundamental that it confused later tests. In that case, subsequent runs should fix more, until e2fsck is happy with the filesystem.

Or it might always stop at the same point, recovering the journal:

Code: Select all

backup.copy.img: recovering journal
e2fsck: unable to set superblock flags on backup.copy.img

In that case, remove the journal feature from the filesystem superblock, effectively downgrading from ext4 to ext2, and keep trying:

Code: Select all

tune2fs -O ^has_journal backup.copy.img
e2fsck -fy backup.copy.img

[MrGibbage]

Success! 1000 internet points to everyone! I have a new SD card on order. I am going to redo the setup and start anew! Thanks for all the help, everyone.

[MrGibbage]

Thanks again for the help with this, but I have one more question. As I reported, I was able to save all of the data. That was huge! What I would like to do next is to see how much of the configuration I can recover. Just to make it easier to set up the next time. I remember I built a few scripts to get everything working perfectly. If I can't recover the setup, that's fine--I am sure I can work it out again. But I would rather spend my Christmas holiday playing with my daughter than rebuilding this.

So, based on the screenshots and console messages pasted above, how would I get back into the linux stuff on the sd card. I don't need it to be bootable--I just want to read some of the scripts in /etc and /home.

Thanks!

[MrGibbage]

Can I get an assist on my one last question, please? You guys have been so helpful this far!

[mahjongg]

as this question is essentially answered, I would put a new question in a new thread so that it attracts new viewers.

[Lectere]

Thanks JoJoPi,

Your posts really helped me!

[chiwalfrm]

I could really use some help. I tried following jojopi's instructions, but the e2fs command failed:

Code: Select all

vmware-u004 ~ # e2fsck -fy BACKUP.COPY.IMG 
e2fsck 1.42.9 (4-Feb-2014)
ext2fs_open2: Bad magic number in super-block
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open BACKUP.COPY.IMG

The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>

I didn't know where the alternate superblock was, so I tried both examples:

Code: Select all

vmware-u004 ~ # e2fsck -b 8193 -fy BACKUP.COPY.IMG 
e2fsck 1.42.9 (4-Feb-2014)
e2fsck: Attempt to read block from filesystem resulted in short read while trying to open BACKUP.COPY.IMG
Could this be a zero-length partition?
vmware-u004 ~ # e2fsck -b 32768 -fy BACKUP.COPY.IMG 
e2fsck 1.42.9 (4-Feb-2014)
e2fsck: Attempt to read block from filesystem resulted in short read while trying to open BACKUP.COPY.IMG
Could this be a zero-length partition?

My situation is the same as OP's. Power failure at home and now the Pi won't boot with the "can't mount root" error. Thank you very much for any assistance!

[hdtodd]

THANKS, JoJoPi ... 6 years later and it helped me recover a damaged SD partition!