Fill out above information andin all cases attach yourklippy.logfile (use zip to compress it, if too big). Pasting yourprinter.cfgis not needed Be sure to check our “Knowledge Base” Category first. Most relevant items, e.g. error messages, are covered there
Describe your issue:
Hi all! I’ve been fighting a problem with my pi and printers. I am trying to use a single RPi 3B+ and have, ultimately, 4 printers (all Ender 3 V3 SE) running off a single pi. Right now, I have only 2 connected for testing. I find that the pi doesn’t have a problem running this number of printers at once. I am finding that I am getting constant MCU timeouts. I have been doing basic troubleshooting (looking for short-circuits, good quality power supplies, quality USB cables, proper usb contact, using different ports, no extra usb devices. etc.). Power cycling the printers brings communication back up.
I have found a few things that seem to change how often the printers crash.
I had some cheap amazon usb a to c cables. These crashed a lot. I then got higher quality USB cables (shielded, thicker). The crashing was not nearly as much. But still are crashing.
When the power cables to the pi laid close to the usb cables, the printers crashed more often. I removed other power cables away from the usb cables. Crashing became less frequent
I wrapped aluminum foil around the usb cables. This didn’t really seem to do anything. I read this would be good for high frequency filtering.
I lowered the baud rate to 115200. This helped a lot, but still didn’t solve the problem.
It didn’t seem like I had any problems when I had only one printer connected. So, I am wondering if the pi itself or the printers are messing with each other.
So, now I’m at the point that I can’t really see an effective way forward to prevent the crashing. I’m considering the following, but really don’t know if it’ll make a difference or if I’m even looking in the right place for the root problem
thicker, better USB cables. I have no idea how to find a better usb cable than the 30-dollar USB cables I bought. They claim to be shielded and protect against interface.
dedicating a single pi zero 2 w to each printer. Not ideal at all since I am limited on power.
I’m trying to find a way to check if there is over the air interface or if it’s coming from the pi/printer. I don’t really know a good way to test for this without buying expensive tools.
lowering the baud rate from 115200 to 57600. I’m afraid that the print quality will start to stuffer if communication is too slow. It sucks to start big prints and find out in the middle of the print that lowering baud did affect the total quality of the printer. Anyone have opinions on this?
Is there a way to filter or ground each printer or USB port on the pi to prevent any interface and noise from one to another? Is this even something that could work that I should consider?
Ideally, I would connect 4 printers to one pi. I have a touchscreen on the pi and use klippyscreen (or whatever it was called). It would suck to lose this interface because having to buy a screen per pie just for that interface, ignoring the power consumption, would be costly and a waste.
That’s a great question! I just verified today that it does happen with just a single pointer connect and powered on. Which is strange because at the beginning of this year, I had a single printer connected and, at least, idle for literal weeks without ever experiencing this. The only thing I can think of is I did update the pi and related software once. And of course, I noticed the problem immediately after adding a second printer. Now a single printer is having problems. The only other thing I can think of is I moved the printers downstairs into my basement. But it’s a big, open and unimpeded by anything large electrical products other than lights and a dehumidifier.
To be on the safe side, I’m also currently testing all the printers one at a time, like you recommended. It takes about a day per printer to see if they’ll crash when individually hooked up. I’ll report back. Thanks for the advice so far!
I kept having MCU communication errors between an Ender 3 V2 with board v4.2.2 and a RPI3 B+. I did troubleshooting using information from the post referenced by Sineos above, but could not resolve the issue.
I finally solved the problem by not using the USB connection and making a serial connection between the RPI pins and the cable to the display unit. The header of the Klipper Ender 3 V2 printer configuration file has instructions on how to modify settings for the firmware bin file and gives the pinout for the display unit cable. This post was also helpful: [Reddit - Dive into anything]
This is what I am thinking as well. The only downfall is I’d like to have 4 printers on one pi 3B+. However, I’ve been testing a single printer at a time. One of the two printers (Printer 1) I am testing currently will crash even when by itself. The other printer (Printer 2) seems to be stable. I’ve had it one for more than 2 days now, and it hasn’t crashed. Granted, that isn’t much time, but it’s already 3 times longer without crashing.
Would you happen to know how many printers you can set up using direct serial? I believe this is called UART, right? I’ll also start reading your reference you sent
Thank you again for the added help! <3
I don’t know how to set up multiple printers on serial communication. It is probably primarily a RPI issue. This is the first time I have used a RPI, so I’m still going through the learning curve.
@EddyMI3D That’s super fair and a good point. I had to create a bunch of scripts to account for power lose and the pi crashing already. You’re probably right, if I’m really gonna stick with more than one printer on a pi, I should stick to just two. Or maybe I need to go the route of Pi zero 2 w for each printer. I’m not even worried about the money. It’s the amperage I have to rewire in my panel. (my goal is a printer farm with upwards of 20 printers).
@GB61 Got you! Yeah, I was looking it up. 3B+ only has a single set of TX RX pins. Pi 4 has two sets. but Eddy still brings up a good point on a single point of failure for more than one printer. I’d have to make more scripts to keep track of each printer’s most recent coordinates or line in the gcode. Another thought that wouldn’t be hard, build a script that runs on my main server to recover and start where the print ended. I think it would be a bit messy, but might work.
@Sineos So, I found that one of the printers is causing itself and others to crash. Even when this printer is the only come connected, it’ll crash. The other printer I tested with doesn’t crash when by itself. I guess it could be the USB port, or the printer itself producing more noise than the other printers. I’m going to try using another USB hub and see if the problem printer causes others to crash still. I don’t know if I have a good way or knowledge to begin troubleshooting the printer itself as to why it’s crashing when even on its own. If you’ve any ideas, I’d be glad to test.
I wanted to report back with my findings so far. I do need to solve this problem so I can at least move forward with my farm.
When I plug printer 1 in by itself, it does not crash.
When I plug printer 2 in by itself, it does crash.
When I plug printer 1 into the top usb port of the right usb stack, it does not crash.
when I plug printer 1 into the top of the left usb stack and printer 2 into the top of the right usb stack, printer 2 still crashed, but printer 1 did not.
I then plugged a third printer into the bottom usb port on the left stack while still leaving printer 1 and 2 plugged in. They all would crash.
I took a usb a to c cable and cut the power line within the cable, leaving the ground and data cables. almost double the time, but all printers crashed still.
I tried disabling the power to the usb hub via software, but this seems to stop traffic as well.
I tried using minicom to see what traffic was crossing the usb ports to the printers. I can’t figure the software out. I am seeing traffic, but it’s gibberish. I tried setting baud rate of 250000 (default my printers are set up at) and some recommended settings, but still nothing but gibberish.
I found that by disabling power on the pi via software, reenabling software, and then hitting both the “restart” and “restart frameware” buttons, the printers would come back online. Before, I had to power cycle the printers and restart “frameware” to restore communication.
I just noticed that without turning off and on the power to the usb ports via software, I can’t get the printers to respond using the “restart” and “restart frameware” buttons (we knew this already). But I pushed an emergency shutdown from mainsail to printer 3. I then hit “restart frameware” and “restart”. Printer 3 came back up. I did “restart frameware” to printer 1 and 2, without hitting “restart”, and they also came back out.
I’m going to continue to try and figure out a way to inspect the actual usb traffic. The Klipper logs show a lot, but I’m hoping to watch it live. I’d also like to see what a working printer and a non-responding printer’s traffic looks like live. Maybe there will be clues there.
At this point, it’s not a matter of should I, it’s a matter of I will. I must understand what isn’t working and fix it, for the sake of mankind! I’m sure I’m the only one that cares, but I’m going to keep researching and reporting back here. Maybe this will help someone else
Of course you can spend your time on whatever you want, but I doubt it will get you anywhere. The communication protocol between the Klipper host process and the MCU is not human readable, and you would have to have such deep knowledge to analyze anything there that you might as well start developing cool things for Klipper.
Also, given the error message and the nature of the error, it is very unlikely that you will be able to derive the root cause from this initiative.
@EddyMI3D you’re definitely right. I lost sight of that it could be localized to just this printer. At the very least, if printer 1 and 3 show to be stable on separate usb stacks, that’s already a win. Then we know for sure printer 2 has something going on with the hardware side. Thanks for reorienting me <3
@Sineos this is exactly what I needed to hear. I needed a gauge of how close to possible my troubleshooting could get. Since the traffic is not human travel, then I’m not going to continue to try, unless somehow these talk in assembly language. I’m going to continue looking at other solutions. A single Pi per printer, or maybe a Linux server with pci-e usb expansion boards running on a hyperv with dozens of VM’s. I’d have a single point of failure, but my plan is to have multiple locations. I could build failover into each location. If I’m going to be looking at a monstrous blade server per location, I’d definitely have the horsepower to run these vms. Of course then internet outage and power outage is a problem. But I can tell with some risk. If power goes out, then it is what it is.
For the sake of documentation, I’ll update my post as things happen. It’d be nice for the internet to have my future failures cemented online lol
@Sineos this is good to know! At least I won’t have to research that. Since I’m looking for stability, I’ll probably avoid vms then haha. I’ll just give pi zero 2 W’s a try and slowly ramp up. You’re very persuasive
Ok. I’m not the only one having random crashes like this.
Has anyone looked into increasing the serial timeout? I’ve seen references to the setting but can’t find it. I’ve read the default is something like 5 seconds and I wonder if increasing it would give the connection more time before crashing out.
@KLB I mean I’m totally game for trying this, but I’m not sure about this setting as well. I’ll have to check this out the next time I’m free. But good thought on this!
I noticed something. When I used minicom and screen, anytime I would try to use these tools to watch the traffic (know we know it’s not human reable) the printers crash. When I use minicom, I could recover from the crash by using restart frameware. When using screen, I had to power cycle the printer. So, whatever they are doing seems to be different and affects the printers differently. Just food for thought.
I have a Bigtree Tech touchscreen attached to my Pi by USB and HDMI and I was thinking about disconnecting it to see but candidly there’s no rhyme or reason to when it crashes.
I’ve done multiple hour long prints in a row without issue and the next print crashes the MCU.
I’ve wasted a huge number of hours and filament on this and I’m ready to walk away from Klipper.
It is under no circumstances recommended messing with Klipper’s time-out values or other timing relevant parameters.
A stable, uninterrupted and performant connection between the host and the MCUs is a must
When this can no longer be guaranteed, then Klipper will error out hard
When this can no longer be guaranteed, then you have an issue in your hardware setup and you should fix it.
Thousands of machines are able to run stable under such and a lot more complex setups. What makes you think that with your machines, it is a good idea to mess with internal settings?
In the worst case, you will get errors that are completely unrelated and impossible to diagnose.
On the other hand, it is your printers, so do what you want with them but do not come asking for support afterward!
<3