Raspberry pi freezes completely if rebooted while mainboard is kept powered

General Discussion
1

I have been fighting against this issue for some days now, and can’t seem to find solutions elsewhere.
I have a very simple setup:

  • pi 3 powered by a raspberry pi power adapter
  • printer mainboard
  • usb cable connecting the two

I mentioned printer mainboard and not a specific model because this issue occurs with both a BTT skr mini e3 v3 and a Fysetc Cheetah 2.0. The pi has been tested with both 32bit and 64bit raspberry pi os lite

If I boot the pi with the mainboard off, and turn it on after boot the system works as expected. If I then reboot the pi while the mainboard is kept on it gets completely unresponsive, no ssh, no ping, no web ui, no display updates on the printer.

After trying lots of things like

  • swapping usb cable
  • swapping pi with another pi 3
  • swapping mainboard
  • swapping os
  • swapping microsd

I cannot get it to survive a restart apart from 1 in 5-ish times. System logs are clean except for some journal files that get corrupted when unplugging the pi since nothing else seems to work. I attached an hdmi screen and ran sudo dmesg -w on the console at boot, and it seems that the freezes are caused by a segmentation fault or similar not-your-usual problem.

A fresh os install doesn’t brick the system, as well as when the klipper service is disabled. It seems that something in the rpi-mainboard link is corrupted during reboots or left in a bad state which causes faults on next boot

More details have been logged on the klipper help thread on the voron discord (Discord)

The only solution found to date is to use a raspberry pi 4, which behaves as expected. Not sure what difference makes the pi 3 crash

youtube video link https://youtu.be/zHFonmDN_Dc

a similar issue is described in these posts: Rpi crashes when klipper is started with MCU connected and Pi crashes/doesn't boot with MCU connected - #3 by mykepredko

I believe on the klipper logs you can see where the pi freezes by looking at the corrupted lines that appear in the file, the mainboard connection seems to be established briefly and then the file gets corrupted

image
image906×381 18.9 KB

image
image1026×388 46.6 KB

image
image756×422 15 KB

klippy.log (2.8 MB)

2

Have you tried by disabling the 5V line of the USB?

3

just tried, doesn’t work

4

Without any error message or indication of what’s going wrong, this will be almost impossible to diagnose. See Pi crashed 3 times mid-print, almost same place - #15 by Sineos for some pointers on how to get some logging.

5

The issue is that I went over the logs and they don’t get updated following the crash. The only way to get something is to film the screen as I did.

  • tried checking dmesg and journalctl
  • enabled remote logging to my NAS
  • ran sudo dmesg -w >> dmesg.txt to print any log to a file from an auto login terminal

all options above don’t end up working or don’t show anything unusual. If you check the video there’s plenty of information flowing on the screen, but I cannot get it saved anywhere. SSH will freeze if I manage to log in quickly enough to start printing logs

Will try to enable persistent storage to journald to see if it manages to catch something more, but since it seems to be a kernel level issue I don’t have high hopes

6

You might be suffering from Kernel 6.6.16 freeze with rpi2 rev B when using usb device · Issue #6100 · raspberrypi/linux · GitHub
Try adding the line

dtoverlay=dwc2

to the end of the /boot/firmware/config.txt then reboot.

7

it works! Could you explain in simple terms what that option does to fix the issue? I assume and hope this won’t affect actual printing?

8

Well, my understanding is far from comprehensive:

  • It seems to be a RPi Kernel bug.
  • Switching to a different “USB driver” (from dwc_otg to dwc2) apparently solves the issue, as only the dwc_otg driver seems to be affected.
9

That’s more than enough, thank you for your help :slight_smile:

10

The fix you proposed also fixed an issue with my other printer, in which the pi becomes unresponsive on firmware restarts. I feel a bug has been recently-ish introduced in the offending module since I wasn’t having these issues until a couple of months ago

1 Like