MCU shutdown: Missed scheduling of next digital out event

Could you start this as a new thread in “General Discussion” with all the requested information including your klippy.log?

For anybody to help you, we have to know what your system is and exactly what is happening (from the klippy.log) - I’m asking that you start a new thread so that what is discussed for your system doesn’t confuse people trying to understand what happened in this one.

1 Like

Hey Engineer!

Not yet. The new PSU’s didn’t resolve the issue, nor better heatsinking on the EBB36, so I’m now still working on replacing my main CANBus cable and how to split it between the mainboard and the PSU. I currently have both a USB and the CANBus cable coming off the EBB36, because if the CANBus cable doesn’t resolve it then I want to try USB and see if it’s really CANBus or if the boards are just not working right.

I would also pay attention to which mcu is crashing, as I’ve had the error with both the main ‘mcu’ and with the EBB36 ‘ebb36’.

*Also, the “Undervoltage” reports still show up on my klippy.logs, which now just has me confused unless it’s some kind of startup behavior with the 5160T drivers and the mainboard. I haven’t had any crashes related to those as far as I can tell.

**Don’t know if it would make a difference, but the CANBus terminal on my M8P is just a simple molex socket like for a fan.

What are the ground wires to your EBB36? If you have both power and USB going to the EBB36 it can be tricky to get things right and could be contributing to your undervoltage errors.

When we talked previously, there was no mention of having both a CAN line and a USB going to your EBB36.

Hey Myke!

The USB cable was just recently added as I already had the umbilical apart to replace the CANBUs cable and didn’t want to disassemble it again to try USB later, so figured I’d run the USB cable while I have it open.

The shield on the CANBus cable I still plan to tie to 24VDC-Negative, but the USB cable I can leave disconnected for the moment on either/both ends.

If the new CANBus cable resolves the problem I can remove the USB cable later when I try out a Berd-Air cooler and, if not, can try the USB and see if that makes it work.

If neither resolve the issue, then I’ll likely look at replacing the CB2 with a Raspberry CM4 and installing a 32-bit image instead of BTT’s 64-bit image (…I wonder if I could request a 32-bit image from them actually?).

*Alright, new cable is wired in, the USB cable is unplugged at both ends, and I have the shield tied directly to the PSU negative in the same crimp as 24VDC-Negative. I’ve also retied the shields in the stepper wires over to the Motor (-) supply, which should be 48VDC-Negative. The only things tied to AC-Ground should be the chassis and the PSUs’ AC-Ground Connections.

Fingers crossed!

Fair enough - from what you said above, it sounded like you were running them both together and that can be problematic in terms of grounding.

I’m wondering about the latest CB1/CB2 OS images for download. I’m running a CB1 and CB2 on two of my printers and have been for about a year without any issues. But, I’m helping out somebody with putting on an SPI display on a CB1 and the latest OS images with Klipper are difficult to work with - I don’t know what’s going on there. The older images didn’t have Klipper and I was able to set them up without issues.

If you want a 32bit OS image for the CB2, I suspect that you’re going to have to build that yourself. Probably not impossible but some work.

I’m having good luck with the Orange Pi Debian image right now - you may want to consider that.

Mine too - and I managed a couple of toes.

Where can it be seen? Attached is my klippy.log

I am using MKS SKIPR board with a THR36 toolhead board, connected via CAN.

klippy.log (4.4 MB)

@engineer-50

As I wrote above:

Could you start this as a new thread in “General Discussion” with all the requested information including your klippy.log ?

Please do not hijack somebody else’s thread - it is unlikely that the issues are related as you are using completely different hardware than the Original Posters.

Thank you for your understanding.

Hey Myke,

I went and sent an email to BTT’s support, who knows, maybe they can get a 32-bit version spooled off. Although that is an interesting mention about using an older version. Might need to ask them if they’re getting more reports with the latest version…

I think a CM4 is like $85, but I could probably sell a Raspberry Pi 4 to cover that. Wouldn’t be able to use it with the M8p anyways.

So next step is now fixing my water-cooling system as the PVC tubing I put in does not work with my hose clamps. Fortunately, I put the water-system components on the left side of the printer away from the electronics, so just need to get new clamps, put in a better part cooler, hope the CANBus cable solves it, and then see if BTT support can help me sort out a 32-bit board or even try reflashing with an older image.

Thank you very much for correcting me, sorry to miss your initial reply and that it looked as an hijack attempt.

I have started a new topic here.

1 Like

New CANBus cable is wired in, shields tied together to DC-Negative, and I also eliminated a ground-loop between the 48V and 24V supplies.

Printer was running through a several hour print and stopped with a “Lost communication with MUC ‘mcu’” (In the klippy.log the error is first “Timeout with MCU ‘mcu’” and then reports “Lost communication…”).

Ran the log through https://sineos.github.io/ and it looks like the shutdown happened at 0:37am, couple hours into the print, but there’s spikes, dips, or abnormalities that I can see…

Original klippy.log is 23MB on dropbox here, this is a trimmed copy, but should still have all the relevant errors:

klippy(1)trimmed.log (1.0 MB)

I have no ideas on this one. Every forum post I’ve seen relates to USB-connected PI’s and not a CB2 physically connected to the MCU nor did it happen immediately…

Looking through th elog I saw these, don’t know if they’re relevant, but it’s the first I’ve seen the stepper_x phase change (was 1 now 3) only on the most recent klippy.log and well before the crash happened:

This shows up sometimes in the logs, but also well before any crashes if at all:
EBBCan got old

As for the original issue, I’ve also been emailing BTT support, but they’ve been focused on the undervoltage reports. Was waiting on the CANBus cable before removing the board to check for shorts on the stepper socket pins, but in the meantime I found:

-The undervoltage error happens only on the first G28 homing after the printer is turned on. Firmware restart, Klipper restart, and motors_off does not cause the undervotlage issue to reappear.
-With an oscilloscope there were no spikes nor drops on the 48V, 24V, and even the M5V (Motor 5V) rails.
-During this first homing, there’s a banging sound as the steppers start to move that doesn’t happen consistently even after a motors_off command.

I’ve tried:

-Switching some and all the steppers to just the 24V rail
-Reducing their run currents
-Disconnecting the steppers at the Manta M8P, then running G28

*I was just able to get the undervoltage error to appear on just one stepper at a time by using force_move. G28 activates all steppers at once, but force_move only activates one (x, y, or z’s). The error still only shows up the first time the stepper activates on startup.

I have some TMC2225’s and some TMC2209’s (though the latter were on a board that popped as I had them soldered backwards, which shorted 24V’s to where it shouldn’t be…) I’m going to install and test, we’ll see how that works out.

**Just installed the TMC2225’s in UART mode and there’s no undervoltage errors showing up in the klippy.log. So I’ll go ahead and run a test print.

Lots to process here.

Are you connecting the shields at both ends? It should just be one (I recommend at the main controller end).

Are you asking for analysis/help here? If you’re seeing problems then it sounds like you’re moving forward on this on your own?

Please don’t post a partial klippy.log - if it’s too big, then .zip it and post that.

I presume you’re still talking about the Manta M8P V2 and CB2 you reference in your original post, correct?

When you plug in a CM4, CB2 or whatever CM4 equivalent to a Manta, you’re making a USB connection. I don’t know what you mean by being “physically connected” - it’s a USB connection.

Looking through th elog I saw these, don’t know if they’re relevant, but it’s the first I’ve seen the stepper_x phase change (was 1 now 3) only on the most recent klippy.log and well before the crash happened:

That sounds like the right things to be focused on.

I’m not really sure where things are - I don’t know what you mean by soldering in TMC2209s backwards?

It would be helpful to get a description of your hardware, the latest .zipped klippy.log and a prioritization of your issues.

From what you’ve written above, it sounds like you’re getting an undervoltage with the first stepper movements you’re trying and BTT is looking at that. Is that correct?

If it is, that sounds like the logical place to start.

Let us know how your test print goes.

Well, even with the new CANBus cable, shields only on one end, and the TMC2225’s it still threw a “Missed Scheduling…” error after a coupe hours of printing…

By “physical” connection I just mean there isn’t a cable and it’s socketed into the M8P. The threads I was reading were PI4’s that were connected via USB cables, rather than sockets.

The TMC2209’s came without the header pins soldered and I had those on the wrong side. It’s what killed the V1.0 board I had that was replaced by the 5160T’s and V2.0 board that’s currently installed.

With regards to the undervoltage error, that I’ve been emailing BTT about, though they seem

Next thing is to wipe everything and run it in USB mode I guess…

Again, I really appreciate the help!

I don’t think you got what I was saying about the “physical connection” - when you plug in a CM4 or CB2 into a Manta M8P V2 they connect via USB. If you look at the schematic, you’ll see that the USB pins of the CM4/CB2 run to a USB Hub that’s built onto the Manta M8P V2 and connects via USB.

So, whatever you see for a USB connection is valid when you plug a CB2 into your Manta M8P V2.

In your previous email you indicated that you’re getting an undervoltage error the first time you start up a stepper motor (it doesn’t seem to be specific to a stepper) and you’re bypassing this and concentrating on the “Missed Scheduling… error”. Honestly, I would be very concerned about the initial undervoltage error as that could be related to the later execution issues.

Regardless, I’m still confused as to what your current hardware configuration is and exactly the issues are. You’re talking about '2209s, '2225s and '5160s so I have no idea what you’re running.

Again, could you give a description of your current hardware, a klipyy.log that is run on this hardware and a list of exactly what your issues are.

Wiped everything and reinstalled/reflashed the boards to use USB. It was able to complete a long print overnight and I plan on running another tonight to see how it does, though I need to sort out a better part cooler as a big blower doesn’t produce enough static pressure for a CPAP hose.

With regards to the undervoltage error that was showing up in the klippy logs I did do some testing and swapping of drivers. The TL;DR is that I think it may be normal behavior as it only happens during the first activation of the drivers when they reset. My other drivers and other printer resets the drivers on first activation, but only the 5160T’s will indicate an undervoltage. I suspect the first reset probably allows the operating voltage to go below their threshold, perhaps a capacitor charging or something, but doesn’t do it again until the power is cycled.

For now, I’m going to test out USB and see if it leads to any crashing as well as run the 5160T’s at 48V’s. Pretty disappointed I spent all this time on CANBus and wires for it to just not work, perhaps a CM4 with a different distro of linux would behave, or maybe there’s some device in my home that causes the crashes, but I really just want to move on to getting this now $3k printer working.

I’ll plan on updating the thread if there are any crashes or if USB seems to behave, so I’d like to keep the thread open for a bit if that’s alright?

No problem - we’re here.

I’ve asked repeatedly for:

Could you give a description of your current hardware, a klipyy.log that is run on this hardware and a list of exactly what your issues are.

When I ask for this, it’s generally because the person I’m directing it to has a complex system that has been reworked and I’m not sure they fully understand it which means there could be inadvertent wiring errors that they discover on their own when you try to document the system.

Please take the time to go through your printer and create a wiring diagram as accurately as possible and post it to see if there is anything that looks suspicious.

Apologies Myke, I can try and sort out a schematic for it all

1 Like

I don’t don’t if this is enough detail to describe the printer, but I spent a few hours today and mapped out the current wiring configuration for the printer:

Any details that need magnified or clarified better?

*Also, for the steppers, I have some TMC2209’s and TMC2225’s as spares from my other printer as well as the TMC5160T Pro’s. I had tried swapping steppers around (and the configs of course) just to see if the undervoltage issue stayed with the Manta M8P or if it moved with the TMC5160T’s and it was the TMC5160T’s. The Octopus Pro is the board on my other printer that I was trying the TMC5160T’s in, which had the undervoltage message show up.

Yes, that’s exactly what I’m looking for.

Two questions back:

  1. What is “To Endstop Switch B”? It doesn’t seem to be connected to anything. It looks like you use the Left Power Element Module as an On/Off Switch but I’m also confused by the E-Stop.
  2. You’ve marked “USBC” on both the Octopus and the EBB36 - are these connected at the same time as the CAN cable?

Have you done a power budget for 24V? You have under 15 Amps coming out of the 24V supply and it’s powering a water pump, CPAP blower, LED Strip along with the usual 3D printer stuff - are you comfortable that you’ll never draw too much power?

The next question I have is how much MCU load are you putting on the Octopus’ MCU. How many microsteps are you specifying with your steppers and how are you controlling the various accessories (as well as monitoring the two chamber thermistors).

Regardless, you have an interesting and impressive printer!

Hello Myke,

  1. E-Stop Switch A/B is an Emergency-Stop Button. It has two isolated switches which I have wired to each AC-power input respectively. One AC-Input (A) powers the PSU’s and Bed, while the other (B) only feeds the chamber heater. Just meant to reduce the current on each plug and allow me to remove a side panel without shutting the entire machine down.

  2. The USBC is connected along with the CANBus cable with the latter supplying 24V and the USB-power jumper removed (Power should only be coming from the CANBus cable). That does remind me, I should disconnect the CANBus H&L plug, though the firmware(s) shouldn’t be using it.

Power should be squared away with the 2 PSU’s. The steppers are all fed from the 48V alone (About 6A) while the rest should be well within 15A as both the Bed and Chamber heaters are AC powered.

  • The Manta M8P is set for 32-Microsteps on the X, Y, and Extruder and 16-Microsteps on the Z’s.

  • So the Chamber heater is set to use the top chamber thermistor, while the bottom thermistor is just for personal monitoring.

  • The LED lightbar is 12V and runs on a fan output (Can’t recall what the power usage was, but I do remember checking it to ensure it was well within what the output was capable of.

  • Water pump basically just turns on and off based on the hotend temperature, powered through a 4-pin fan port.

  • The Chamber and Bed heaters are powered and controlled by Solid-State relays which are controlled by the M8P via a heater-output and Bed-control port respectively.

  • The stepper cooler fans are just given 24V continuously and sit over the drivers to keep them cool.

  • Y-Endstop connects to the M8P, while the X-Endstop goes through the EBB36

On the Octopus-printer:

Here’s a klippy.log for the Octopus-printer:
klippyCube.log (3.8 MB)

Octopus averages around 1.5% load:

Octopus steppers X and Y are set for 8-Microsteps at 1A and Steppers Z/Z1/Z2 at 16 Microsteps at 0.34A. It’s also using 0.9* steppers on the XY.

This has the potential for an issue as you have two Ground lines (literally a rather large ground loop).

It’s always a concern when people want to use USB for the toolhead but don’t know how to provide 24V to it. The simplest solution is to go with CAN.

It’s not the average which causes the problems. It’s the bumps, which is why I was asking about microsteps (which should be fine at 16) and macros.