I’m super grateful for the additional info on noise / transients, thank you! There might be a DSO in my future (sigh).
what you mean when you say you put “ferrite beads on all the signal and power cables
Signal cables (USB-2)
While troubleshooting this, I bought 4 new 10’ USB-2 cables from Amazon, 1 for each FDM printer. The spec is that they’re “shielded”, presumably Faraday shielded, though I didn’t cut one open to check. But they’re hightly rated with thousands of reviews.
However, they did not have ferrite beads. So I bought a bag of ferrite beads, and installed 2 on each cable (1 on printer side, 1 on Sonic Pad side).
Power Cables
Sonic Pad has wall-mount external power supply. It shipped with a ferrite bead already on the DC power cable, on the device side.
Printers have internal power supply connected to external 110V AC cable, which connects to a UPS. I put ferrite beads on the printer external AC cables (downstream from the UPS). Did not do anything internal with the printers for this issue.
rerouting the cables” as that seems extreme.
Ordinarily, yes. But I have cabling for 4 FDM printers, 2 SLA printers, wash & cure, laser, ventilation, air filtration, lighting, 8 cameras, environmental sensors. I’ve not been careful about cabling best practices, and it’s been fine until now. Plus, while troubleshooting this issue, I did power down everything except the FDM printers and the Sonic Pad. But since I started having these intermittent halts when I moved to Klipper, I did some no-brainer improvements: distancing AC cables from USB cables; moving USB cables away from power supplies and fans; etc.
Are you using a single Sonic Pad for control or multiple units (I’m asking because people have reported USB problems with the Sonic Pad that results in shut downs - apparently one of the two ports works better than the other).
This is super interesting and fits my experience exactly. Yes, I’m using a single SP to control multiple printers. SP has 4 USB ports. My troubleshooting process included testing my SP with, respectively, 1, 2, 3, and 4 printers running concurrently.
When running multiple printers concurrently, any printer on any port might halt. Swapped printers and cables on different ports to confirm this.
When running one printer on any port (1 - 4), that printer might halt. Swapped printers and cables on different ports to confirm this.
Weirdly, printers on Ports 3 and 4 (back of SP) seem to halt sooner when running singly. Also, when running >1 printers concurrently, one of the printers seems to halt more quickly, if I have a printer running on port 3 or Port 4, although the printer which actually halts might on be on Port 1 or Port 2. I have also noticed that SP sometimes has trouble recognizing USB drive on Port 3.
The more concurrent printers I added, the faster a halt occurred on one of the printers.
This led me to swap SP power supply, which did not resolve the problem.
Finally, when testing a non-Klipper printer connected by USB-2 to a different computer (not the SP) running a simple job-monitoring app, I noticed intermittent USB disconnects. This is what pointed me toward EMI as the root cause.
So I do think that a contributing factor to my issues, though not the root cause, is that USB ports 3 and 4 on my SP are flaky. Creality is sending me “new USB ports,” whatever that means (probably just connectors).
What are the power supplies that are used in each of the printers?
The whole system (above) is powered by a mediocre $200 Amazon Basics UPS with AVR and EMI filtering. I do have a much nicer high-capacity enterprise-grade rack-mounted UPS and power conditioner which I use for other stuff, and I was going to test the 3D printers on that power if needed. But it looks like I don’t have to go there.
The printers themselves have the usual internal $25 Chinese power supplies, which probably suck.
Personally, I would have looked at power line interference first and started with a surge protection power bar as having multiple printers on the same AC circuit (with questionable power supplies) could result in some pretty nasty transients on the AC line that could have caused your problems. You might still want to look at getting some kind of AC surge protection for the long term.
Totally agree. See reply immediately above.
One other bit of relevant data is that I’m in a city apartment building. There might be a 20-year-old refrigerator above and/or below emitting who-knows-what EMI.
Anyway, thanks for the help and suggestions. So far so good, after adding the ferrites and making no-brainer USB cable reroutes.
Please keep me posted if you hear anything more about USB issues with Sonic Pad.