Ethernet dies after about 12 hours, nothing connected to USB

[ausserirdischegesund]

I've just got 2 Raspis, and everything works fine for a while. I had no luck letting them run for longer times (like overnight) because Ethernet dies after a few hours (I had it running for close to 24 once, sometimes only 6 or so. 12 hours seems to be the average, so quite a bit longer than problems that show after a few minutes, but short enough to make it useless as a headless server.

I've tried both having nothing connected, as well as various peripherials on a powered hub. It does not seem to make much difference.

I've tried 2 different power supplies, and even though I've got another, "good" one on order, I doubt this is the root cause of the problem, because even using a 2.5 A specified powered USB hub as a power source does not help. I have not been able to check voltages yet, this is the next thing I will try.

The log gives me nice messages, as the Raspi continues to run, only with dead network (and USB stack?). This always looks similar, just one example:

Code: Select all

Nov  2 21:07:16 pi kernel: [44813.532421] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000114
Nov  2 21:07:21 pi kernel: [44818.532491] smsc95xx 1-1.1:1.0: eth0: Failed to write register index 0x00000114
Nov  2 21:07:26 pi kernel: [44823.532560] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000114
Nov  2 21:07:31 pi kernel: [44828.532625] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000118
Nov  2 21:07:36 pi kernel: [44833.532694] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000114
Nov  2 21:07:41 pi kernel: [44838.532758] smsc95xx 1-1.1:1.0: eth0: Failed to write register index 0x00000114
Nov  2 21:07:46 pi kernel: [44843.532818] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000114
Nov  2 21:07:51 pi kernel: [44848.532890] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000118
Nov  2 21:07:57 pi kernel: [44854.632970] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000114
Nov  2 21:08:02 pi kernel: [44859.633043] smsc95xx 1-1.1:1.0: eth0: Failed to write register index 0x00000114
Nov  2 21:08:02 pi kernel: [44859.922962] ------------[ cut here ]------------
Nov  2 21:08:02 pi kernel: [44859.923024] WARNING: at net/sched/sch_generic.c:255 dev_watchdog+0x2a0/0x2c0()
Nov  2 21:08:02 pi kernel: [44859.923042] NETDEV WATCHDOG: eth0 (smsc95xx): transmit queue 0 timed out
Nov  2 21:08:02 pi kernel: [44859.923054] Modules linked in: evdev uinput btusb rfcomm bnep bluetooth snd_bcm2835 snd_pcm snd_seq snd_timer snd_seq_device snd snd_page_alloc
Nov  2 21:08:02 pi kernel: [44859.923159] [<c0013e1c>] (unwind_backtrace+0x0/0xf0) from [<c0028b24>] (warn_slowpath_common+0x4c/0x64)
Nov  2 21:08:02 pi kernel: [44859.923194] [<c0028b24>] (warn_slowpath_common+0x4c/0x64) from [<c0028bd0>] (warn_slowpath_fmt+0x30/0x40)
Nov  2 21:08:02 pi kernel: [44859.923226] [<c0028bd0>] (warn_slowpath_fmt+0x30/0x40) from [<c02eb38c>] (dev_watchdog+0x2a0/0x2c0)
Nov  2 21:08:02 pi kernel: [44859.923274] [<c02eb38c>] (dev_watchdog+0x2a0/0x2c0) from [<c0033fe8>] (run_timer_softirq+0x114/0x2a4)
Nov  2 21:08:02 pi kernel: [44859.923307] [<c0033fe8>] (run_timer_softirq+0x114/0x2a4) from [<c002e62c>] (__do_softirq+0x90/0x128)
Nov  2 21:08:02 pi kernel: [44859.923337] [<c002e62c>] (__do_softirq+0x90/0x128) from [<c002eaec>] (irq_exit+0x90/0x98)
Nov  2 21:08:02 pi kernel: [44859.923375] [<c002eaec>] (irq_exit+0x90/0x98) from [<c000e880>] (handle_IRQ+0x34/0x84)
Nov  2 21:08:02 pi kernel: [44859.923411] [<c000e880>] (handle_IRQ+0x34/0x84) from [<c037c974>] (__irq_svc+0x34/0xc8)
Nov  2 21:08:02 pi kernel: [44859.923443] [<c037c974>] (__irq_svc+0x34/0xc8) from [<c000e990>] (default_idle+0x24/0x28)
Nov  2 21:08:02 pi kernel: [44859.923473] [<c000e990>] (default_idle+0x24/0x28) from [<c000eb78>] (cpu_idle+0x9c/0xc4)
Nov  2 21:08:02 pi kernel: [44859.923513] [<c000eb78>] (cpu_idle+0x9c/0xc4) from [<c04c1718>] (start_kernel+0x280/0x2c8)
Nov  2 21:08:02 pi kernel: [44859.923532] ---[ end trace 14762c6030d9df86 ]---
Nov  2 21:08:07 pi kernel: [44864.633104] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000114
Nov  2 21:08:12 pi kernel: [44869.633172] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000118
Nov  2 21:08:17 pi kernel: [44874.633245] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000114
Nov  2 21:08:22 pi kernel: [44879.633312] smsc95xx 1-1.1:1.0: eth0: Failed to write register index 0x00000114
Nov  2 21:08:27 pi kernel: [44884.633365] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000114
Nov  2 21:08:32 pi kernel: [44889.633431] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000118
Nov  2 21:08:37 pi kernel: [44895.173506] smsc95xx 1-1.1:1.0: eth0: Failed to write register index 0x00000014
Nov  2 21:08:37 pi kernel: [44895.173539] smsc95xx 1-1.1:1.0: eth0: Failed to write HW_CFG_LRST_ bit in HW_CFG register, ret = -1

This goes on for hours.

Any ideas except for trying another power supply (which I doubt is the problem, as it should give me enough juice, and I've got nothing plugged in into the Raspi's USB)?

I am using a current Raspbian, nothing overclocked, stock kernel, no X, just a few idle gettys on the console. And each of my two Raspis seems to act that way (so it is not only one defective one). Did I get two broken ones? Is this the general "Elephant in the Room" USB flakiness? Something else? Is there hope to get my Raspi running continuously as a headless server?

TIA
Ralph

[jamesh]

Something like this has already been mentioned a few times in the forum - try a search form the front page of the website on some of the error messages.

[ausserirdischegesund]

Thanks, jamesh! I've read through at least 10 or 20 of these threads, but found nothing to help me, as most solutions that were discribed as working were "I unplugged USB device X, and it worked", which does not help me as I've got nothing plugged in anyway. Other threads were quite inconclusive. I also tried the kernel commandline to slow down USB, but that did not work either (in fact my Pi would not even boot all the way with that kernel commandline).

Of course it still might be a power problem ... can't test that before next week, though.

[pluggy]

I run all 3 of my Pi's headless on a long term basis. I have problems with the ones with USB wifi adapters sometimes (usually when my router glitches and loses the internet connection), but the wired one is rock solid. Uptime is presently reporting close on 5 days.

PS, I don't own a powered USB hub.

[gsh]

https://github.com/raspberrypi/linux/issues/151

[ausserirdischegesund]

Thanks pluggy&gsh, that sounds good (both that it works for some people and that there is a formal bugreport being worked on). I will certainly investigate this further as described in the bugreport, when it happens next time.

Strangely enough since the posting my pi Ethernet has not crashed. It might be because I've used yet another power supply (USB outlet of my other computer), or it is something else.

[SR-71]

Same problem here: I have an unattended system with one only device attached to the USB and of course the ethernet.
Nokia power supply as provided.
The system freezes every 6-24 hours.

uname -a
Linux raspberrypi 3.2.27+ #160 PREEMPT Mon Sep 17 23:18:42 BST 2012 armv6l GNU/Linux

[SR-71]

I solved the problem using suggestions found in

https://github.com/raspberrypi/linux/issues/151

It seems that a rpi-update cured definitively that problem.

[ewig]

https://github.com/raspberrypi/linux/issues/151

It seems that a rpi-update cured definitively that problem.

How can I tell if I have this fix on my system? I'm seeing network fails after about 12 hours uptime with identical messages.

Code: Select all

$ uname -a
Linux raspberrypi 3.2.27+ #250 PREEMPT Thu Oct 18 19:03:02 BST 2012 armv6l GNU/Linux

and

Code: Select all

Jan 11 14:58:10 raspberrypi kernel: [ 2088.329951] smsc95xx 1-1.1:1.0: eth0: Failed to write register index 0x00000114
Jan 11 14:58:15 raspberrypi kernel: [ 2093.330015] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000114
Jan 11 14:58:20 raspberrypi kernel: [ 2098.330087] smsc95xx 1-1.1:1.0: eth0: Failed to read register index 0x00000118
Jan 11 14:58:25 raspberrypi kernel: [ 2103.900156] smsc95xx 1-1.1:1.0: eth0: Failed to write register index 0x00000014
Jan 11 14:58:25 raspberrypi kernel: [ 2103.900189] smsc95xx 1-1.1:1.0: eth0: Failed to write HW_CFG_LRST_ bit in HW_CFG register, ret = -110
Jan 11 14:58:31 raspberrypi kernel: [ 2109.900248] smsc95xx 1-1.1:1.0: eth0: Failed to write register index 0x00000014
Jan 11 14:58:31 raspberrypi kernel: [ 2109.900282] smsc95xx 1-1.1:1.0: eth0: Failed to write HW_CFG_LRST_ bit in HW_CFG register, ret = -110
Jan 11 14:58:37 raspberrypi kernel: [ 2115.900318] smsc95xx 1-1.1:1.0: eth0: Failed to write register index 0x00000014
Jan 11 14:58:37 raspberrypi kernel: [ 2115.900352] smsc95xx 1-1.1:1.0: eth0: Failed to write HW_CFG_LRST_ bit in HW_CFG register, ret = -110
Jan 11 14:58:43 raspberrypi kernel: [ 2121.900411] smsc95xx 1-1.1:1.0: eth0: Failed to write register index 0x00000014

If this problem is supposedly fixed how can I tell if my system has the patch?

Thanks
ewig

[bgirardot]

It does not look like you have the development firmware installed from your uname -a output.

If you want the development firmware and kernel to see if it fixes your problem go to this page:

https://github.com/Hexxeh/rpi-update

and install the rpi-update tool and then run it.

I never had to do the 'preparation' steps, just the install and update.

There is no easy way to downgrade from the development stuff, so you might want to back up your SD card first so you can easily go back to it if the development code causes you any issues, not likely, but certainly possible.

[ewig]

It does not look like you have the development firmware installed from your uname -a output.

If you want the development firmware and kernel to see if it fixes your problem go to this page:

https://github.com/Hexxeh/rpi-update

and install the rpi-update tool and then run it.

I never had to do the 'preparation' steps, just the install and update.

There is no easy way to downgrade from the development stuff, so you might want to back up your SD card first so you can easily go back to it if the development code causes you any issues, not likely, but certainly possible.

Cool, thank you.
ewig

[ewig]

...
There is no easy way to downgrade from the development stuff, so you might want to back up your SD card first so you can easily go back to it if the development code causes you any issues, not likely, but certainly possible.

Cool, thank you.
ewig

My Pi has 48 hours uptime since applying this update. Could be the fix is working for me.

ewig

[JasonNaughton]

Hi ewig,

Can you do me a favour and post the following:

uname -a

after you did the firmware update? I noticed this morning the same log entry in my syslog. Now my pi is still operational but I was wondering if I was heading into issues to come. I'm using the same kernel version that you were using. My uname -a contains:

Linux XXXXXX 3.2.27+ #250 PREEMPT Thu Oct 18 19:03:02 BST 2012 armv6l GNU/Linux

Yet someone said you weren't using the development firmware from your kernel release. So I'm now curious what you have now after the update. Yet when I cat /etc/issue I have:

# cat /etc/issue
Debian GNU/Linux 7.0 \n \l

I've installed rasbian wheezy and have done a number of updates over the past few weeks.

Cheers

Jason

[5jwhw4g5]

I am experiencing this on a pi4 with:
- Ethernet plugged in (wifi disconnected)
- NVMe SSD in an external enclosure via a powered USB 3 hub
- Docker containers running on the OS (Raspberry Pi OS)
- Latest updates + no tinkering with kernel or e.g. rpi-eeprom-update
- No notable activity during the time of the error (other times in the day are high activity)

After 12-60 hours of uptime, the system completely stops communicating on ethernet. If I unplug / re-plug the ethernet cable, those actions register in DMESG, but nothing happens - the connection registers on both ends as "connected", but no activity lights and no IP is requested from the router). I'm dead in the water until I reboot. This is the DMESG error when it pukes:

Aug 17 02:05:11 raspberrypi kernel: [41098.091081] ------------[ cut here ]------------
Aug 17 02:05:11 raspberrypi kernel: [41098.091105] WARNING: CPU: 1 PID: 0 at net/sched/sch_generic.c:448 dev_watchdog+0x310/0x314
Aug 17 02:05:11 raspberrypi kernel: [41098.091112] NETDEV WATCHDOG: eth0 (bcmgenet): transmit queue 0 timed out
Aug 17 02:05:11 raspberrypi kernel: [41098.091117] Modules linked in: xt_nat xt_tcpudp veth xt_conntrack xt_MASQUERADE nf_conntrack_netlink nft_chain_nat nf_nat br_netfilter bridge xt_connmark nf_conntrack nf_defrag_ipv4 xt_mark nft_counter xt_comment xt_addrtype nft_compat nf_tables nfnetlink wireguard ip6_udp_tunnel udp_tunnel overlay cmac bnep hci_uart btbcm bluetooth ecdh_generic ecc 8021q garp stp llc sg vc4 cec drm_kms_helper brcmfmac brcmutil bcm2835_codec(C) sha256_generic v3d libsha256 gpu_sched drm cfg80211 rpivid_mem bcm2835_v4l2(C) bcm2835_isp(C) drm_panel_orientation_quirks rfkill bcm2835_mmal_vchiq(C) v4l2_mem2mem videobuf2_vmalloc videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 videobuf2_common snd_soc_core raspberrypi_hwmon snd_compress snd_bcm2835(C) snd_pcm_dmaengine videodev mc snd_pcm snd_timer vc_sm_cma(C) snd syscopyarea sysfillrect sysimgblt fb_sys_fops uio_pdrv_genirq uio ip_tables x_tables ipv6 nf_defrag_ipv6
Aug 17 02:05:11 raspberrypi kernel: [41098.091256] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G C 5.4.51-v7l+ #1333
Aug 17 02:05:11 raspberrypi kernel: [41098.091259] Hardware name: BCM2711
Aug 17 02:05:11 raspberrypi kernel: [41098.091262] Backtrace:
Aug 17 02:05:11 raspberrypi kernel: [41098.091273] [<c020d46c>] (dump_backtrace) from [<c020d768>] (show_stack+0x20/0x24)
Aug 17 02:05:11 raspberrypi kernel: [41098.091278] r6:ef912000 r5:00000000 r4:c129c8f8 r3:e3bab435
Aug 17 02:05:11 raspberrypi kernel: [41098.091287] [<c020d748>] (show_stack) from [<c0a39a44>] (dump_stack+0xe0/0x124)
Aug 17 02:05:11 raspberrypi kernel: [41098.091295] [<c0a39964>] (dump_stack) from [<c0221c70>] (__warn+0xec/0x104)
Aug 17 02:05:11 raspberrypi kernel: [41098.091301] r8:000001c0 r7:00000009 r6:c0e375c8 r5:00000000 r4:ef913d3c r3:e3bab435
Aug 17 02:05:11 raspberrypi kernel: [41098.091307] [<c0221b84>] (__warn) from [<c0221d0c>] (warn_slowpath_fmt+0x84/0xc0)
Aug 17 02:05:11 raspberrypi kernel: [41098.091312] r9:c0e375c8 r8:000001c0 r7:c09476ec r6:00000009 r5:c0e375e0 r4:c1204f88
Aug 17 02:05:11 raspberrypi kernel: [41098.091318] [<c0221c8c>] (warn_slowpath_fmt) from [<c09476ec>] (dev_watchdog+0x310/0x314)
Aug 17 02:05:11 raspberrypi kernel: [41098.091323] r9:003e40b8 r8:ef23c000 r7:00000001 r6:c1203d00 r5:ef23c2a8 r4:00000000
Aug 17 02:05:11 raspberrypi kernel: [41098.091332] [<c09473dc>] (dev_watchdog) from [<c02a05dc>] (call_timer_fn+0x40/0x180)
Aug 17 02:05:11 raspberrypi kernel: [41098.091336] r8:c09473dc r7:00000100 r6:ef912000 r5:ef23c2a8 r4:eff20440
Aug 17 02:05:11 raspberrypi kernel: [41098.091343] [<c02a059c>] (call_timer_fn) from [<c02a172c>] (run_timer_softirq+0x288/0x654)
Aug 17 02:05:11 raspberrypi kernel: [41098.091348] r9:00000000 r8:ef23c2a8 r7:ef912000 r6:003e40b8 r5:ef913e18 r4:eff20440
Aug 17 02:05:11 raspberrypi kernel: [41098.091354] [<c02a14a4>] (run_timer_softirq) from [<c020249c>] (__do_softirq+0x1a4/0x418)
Aug 17 02:05:11 raspberrypi kernel: [41098.091359] r10:00000004 r9:00000082 r8:ef848800 r7:00000100 r6:ef912000 r5:00000001
Aug 17 02:05:11 raspberrypi kernel: [41098.091362] r4:c1203084
Aug 17 02:05:11 raspberrypi kernel: [41098.091369] [<c02022f8>] (__do_softirq) from [<c0227d74>] (irq_exit+0x100/0x110)
Aug 17 02:05:11 raspberrypi kernel: [41098.091374] r10:00000000 r9:ef912000 r8:ef848800 r7:00000001 r6:00000000 r5:00000000
Aug 17 02:05:11 raspberrypi kernel: [41098.091377] r4:c10a82e4
Aug 17 02:05:11 raspberrypi kernel: [41098.091385] [<c0227c74>] (irq_exit) from [<c028304c>] (__handle_domain_irq+0x70/0xc4)
Aug 17 02:05:11 raspberrypi kernel: [41098.091391] [<c0282fdc>] (__handle_domain_irq) from [<c02022b8>] (gic_handle_irq+0x4c/0x88)
Aug 17 02:05:11 raspberrypi kernel: [41098.091396] r8:f0815000 r7:f0814000 r6:ef913f38 r5:f081400c r4:c1205a14 r3:ef913f38
Aug 17 02:05:11 raspberrypi kernel: [41098.091401] [<c020226c>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c)
Aug 17 02:05:11 raspberrypi kernel: [41098.091405] Exception stack(0xef913f38 to 0xef913f80)
Aug 17 02:05:11 raspberrypi kernel: [41098.091409] 3f20: c0209b94 00000000
Aug 17 02:05:11 raspberrypi kernel: [41098.091414] 3f40: 60000093 c021c160 c1204fb4 ef912000 c1204ffc 00000002 c12a32ed 410fd083
Aug 17 02:05:11 raspberrypi kernel: [41098.091419] 3f60: 00000000 ef913f94 c12053ac ef913f88 00000000 c0209b98 60000013 ffffffff
Aug 17 02:05:11 raspberrypi kernel: [41098.091424] r8:c12a32ed r7:ef913f6c r6:ffffffff r5:60000013 r4:c0209b98 r3:e3bab435
Aug 17 02:05:11 raspberrypi kernel: [41098.091433] [<c0209b64>] (arch_cpu_idle) from [<c0a5aa6c>] (default_idle_call+0x34/0x48)
Aug 17 02:05:11 raspberrypi kernel: [41098.091440] [<c0a5aa38>] (default_idle_call) from [<c0255b20>] (do_idle+0xec/0x170)
Aug 17 02:05:11 raspberrypi kernel: [41098.091445] [<c0255a34>] (do_idle) from [<c0255e80>] (cpu_startup_entry+0x28/0x2c)
Aug 17 02:05:11 raspberrypi kernel: [41098.091450] r8:00007000 r7:c12b5a50 r6:30c0387d r5:00000001 r4:0000008a r3:e3bab435
Aug 17 02:05:11 raspberrypi kernel: [41098.091457] [<c0255e58>] (cpu_startup_entry) from [<c0210b10>] (secondary_start_kernel+0x138/0x144)
Aug 17 02:05:11 raspberrypi kernel: [41098.091463] [<c02109d8>] (secondary_start_kernel) from [<002027ac>] (0x2027ac)
Aug 17 02:05:11 raspberrypi kernel: [41098.091467] r5:00000000 r4:2f86dac0
Aug 17 02:05:11 raspberrypi kernel: [41098.091471] ---[ end trace 9a458b64afee265e ]---

Has anyone gotten to the bottom of this?

[bgirardot]

I have no help for you, but thank you for searching and finding an old topic to solve your issue.

I love posts to old threads, it makes these forums feel like a useful resource.

Anyway, good luck, I hope someone can help with this.