Issues with CANable

General Discussion
1

Hi,

I can’t post all the links as new user, so I put them all here: Kit: https://www.3dptronics.com/electronics/56-621-voron-24-complete-ptfe-wiring - Pastebin.com. I will refer to them in the body of this post. Sorry for the inconvenience.

I have Voron V2.4 350x350 with recent Canbus (EBB36) upgrade, using CANable adapter (1st link). Raspberry Pi 3B+ (32 bit kernel) and Octopus board. Upon install and setup, I had issues of random error “Lost communication with MCU ‘can0’” while printing. Following this post (2nd link), I found out that it is not hardware/wiring issue, but firmware issue, because ip -details -statistics link show can0 reports no dropped bytes, but mcu can0 interface statistics on Mainsail reported lots of bytes_invalid and bytes_retransmit.

I followed the advice in that post, and flashed updated firmware(3rd link) to CANable via dfu-utils. This got rid of the bytes_invalid and bytes_retransmit (both stayed at 0 even when testing).

However, my printer still randomly shuts down during bed mesh sometimes with the same “Lost communication with MCU ‘can0’” error. Klippy.log ends with Got error -1 in can write: (11)Resource temporarily unavailable'. If I run sudo dmesg, I get several Under-voltage detected! (0x00050005) errors. However, if I run vcgencmd measure_volts core (as well as sdram_c, sdram_i, sdram_p), I get completely nominal voltages.

Here are a few screenshots: (4th link)
My klippy.log: (5th link)

I am pretty sure there should be no actual undervolt, because everything was working fine before I added CANBUS to my printer setup. Can anyone help me figure this out?

2

How are you powering the rpi?
Undervoltage is to be taken seriously.

3

I am powering it via the 5V PSU on the Voron

image
image503×675 75 KB

That PSU is supplying 5.1V. Rpi is receiving 5.1V as well (I measured at both ends).

P.S. I added an extra 5V and GND wires from the PSU to the other pins on the Rpi, just to add some redundancy in case something is shaking loose, although connections look really strong and crimping is good.

4

It’s probably not your problem but you should never have multiple power/ground wires in parallel as you can get induced voltages with the ground/Vcc loops that you have added to the system.

A better place to look is how you made up your CAN bus wiring. How was that implemented?

5

Thank you for your reply. I had this whole issue even before I added these additional wires to Rpi. I can remove one pair and leave only one pair of really thick wires. However, when I added these additional wires yesterday, I stopped getting undervoltage warnings. Bed meshes seemed to work, so I thought maybe the issue is fixed, and tried another long print - and got a new error message some 4 hours into the print:

https://imgur.com/a/kE4Iwb3

(Also attaching my Klippy.log, I trimmed the middle. But it really doesn’t show anything relevant that I can see).

Sudo dmesg shows this:

https://imgur.com/a/mg4KwF5

Some kind of “power save enabled” that I have never seen before. Not sure if this is relevant or not. This might have happened some time after the print failed, because I left it overnight and found it dead in the morning.

To answer your question about wiring, I am using the pre-crimped wires supplied with that kit I linked to. 4 wires in total. They are crimped onto a 4 pin connector, and connected to the EBB36. I am using several zip ties and strain relief gland to make absolutely sure the connector cannot move in any way when the toolhead moves. That part is rock solid. The wires then go through the umbilical sleeve into the electronics bay. CAN_LOW and CAN_HIGH are connected to the terminals of CANable and screwed down tight. 12V wire goes to the 12V PSU output, and GND goes to 12V PSU ground. Another wire goes from the third CANable to terminal also to the 12V PSU ground. CANable is connected to Rpi using a high quality 30cm USB cable. Let me know if this is what you wanted to know.

6

Thank you for your reply. I had this whole issue even before I added these additional wires to Rpi. I can remove one pair and leave only one pair of really thick wires. However, when I added these additional wires yesterday, I stopped getting undervoltage warnings. Bed meshes seemed to work, so I thought maybe the issue is fixed, and tried another long print - and got a new error message some 4 hours into the print:

https://imgur.com/a/kE4Iwb3

I can’t attach my Klippy.log because of the damn forum restrictions, I will post it in separate reply.

Sudo dmesg shows this:

https://imgur.com/a/mg4KwF5

Some kind of “power save enabled” that I have never seen before. Not sure if this is relevant or not. This might have happened some time after the print failed, because I left it overnight and found it dead in the morning.

To answer your question about wiring, I am using the pre-crimped wires supplied with that kit I linked to. 4 wires in total. They are crimped onto a 4 pin connector, and connected to the EBB36. I am using several zip ties and strain relief gland to make absolutely sure the connector cannot move in any way when the toolhead moves. That part is rock solid. The wires then go through the umbilical sleeve into the electronics bay. CAN_LOW and CAN_HIGH are connected to the terminals of CANable and screwed down tight. 12V wire goes to the 12V PSU output, and GND goes to 12V PSU ground. Another wire goes from the third CANable to terminal also to the 12V PSU ground. CANable is connected to Rpi using a high quality 30cm USB cable. Let me know if this is what you wanted to know.

P.S. Before this new error happened last night, I also added a jumper on the “120R” pin on the EBB36, because I found some info that it helped some people. Apparently it didn’t help in this case.

7

Attaching Klippy.log
klippy(7).log (480.5 KB)

8

On one of my printers I got the “Power Save Enabled” warning when I was powering my Raspberry Pi from a Mac that was nearby (I didn’t have a 5V AC/DC hooked up at the time). I replaced the PC/Mac connection with a Raspberry Pi wall wart and the issue went away.

When you say “one pair of really thick wires”, what do you mean? For my current connection, I’m using 16 AWG connected to a 5A AC/DC bulk supply soldered into a USB C prototyping connector for powering the rPi:

2023.04.22 - rPi Power Connection
2023.04.22 - rPi Power Connection1920×2560 422 KB

For a straight Raspberry Pi that isn’t powering anything, this would be overkill but in this printer the rPi is mounted to a BTT PITFT5.0 touch panel. When I measure the current on my bench supply in this configuration, it’s about 3.2A and 16 AWG is appropriate.

If you have a Raspberry Pi with no display mounted to it, the maximum current draw I’ve observed (with the rPi not powering any external devices) is 1.4A so you’ll want to use something like 20 AWG or 22 AWG to make sure your power cable losses are minimal.

I’m going through this level of detail because when I look at the schematics for the various CANable versions (1.0, 2.0 and Pro) it’s MCU is being powered by the Raspberry Pi. I’m guessing (based on the datasheet) that the CANable is drawing 200mA or so.

Now, you say that you are using an Octopus board, did you pull the USB Power jumper? If you don’t, the rPi will be providing 5V power for the entire board even when there is a bulk supply - expect the Octopus to draw at least an Amp with the stepper driver modules installed.

image
image1517×966 168 KB

So, my big question is; what do you have attached to your Raspberry Pi’s USB and what is the current rating for the bulk 5V AC/DC that you are using? If you are powering the Octopus, CANable then the total 5V current draw is going to approach 3A. If you’re also powering a display like the BTT PITFT5.0 that I’m using then total rPi current draw will be north of 5A and, depending on your bulk supply and wire AWG size, you’ll see some voltage dips and the “Power Save Enabled” warning.

If you do have the Octopus’ USB power jumper in, pull it and see if this fixes the problem.

If all you have is the CANable and the EBB36 then you should have the jumpers enabling the 120 Ohm terminating resistors installed in both boards.

Good luck!

9

Thank you for your extensive reply. To answer your questions:

  1. I am using 18 AWG wires to power RPi (during yesterday’s experiment I even had two pairs of 18 AWG wires).
  2. The RPi is connected via USB only to the Octopus board (but the jumper you showed is not installed, so the Octopus board must be getting it’s power from the larger 24V PSU (Mean Well LRS-200-24)), and the CANable adapter - I’m pretty sure RPi is powering that one, because CANable doesn’t have any power lines coming to it, only ground. So it must be getting it’s power via USB.
  3. My LCD is connected and powered from the Octopus board (so, also not related to the 5V PSU).
  4. The PSU I am using for RPi is Mean Well RS25-5 (rated at 5A). It is powering nothing else, just RPi.
  5. I had installed the 120 Ohm jumpers on both the CANable and the EBB36 when the last print failure happened. Sorry I forgot to mention it.

So, to sum up - 5V PSU is only powering RPi, and RPi is only powering CANable. RPi and PSU connected with 18 AWG wires, and the total load should be well below 5A. Octopus, EBB36, LCD and the rest are all powered from the 24V PSU.

So basically my current setup during the last print failure already matches what you suggested :confused: What do I do now?

10

Have you put a voltmeter on the output of the Mean Well 5V supply? It has an adjustment, maybe it’s a bit low from the factory.

The only other thing that I can think of is that there is a problem with your rPi’s 5V wiring - could you share a picture of the wires and how they go into the Raspberry Pi? If things got better when you added a second set of wires, that sounds like there may be an issue with the basic wiring.

Other than those two things, I can’t think of anything else. The “under-voltage” detected warning is pretty specific to the power supply going into the Raspberry Pi.

11

I measured the voltage out of 5V PSU - it is actually 5.1V. I also measured voltage at RPi - it is the same, meaning there is no drop due to wiring. If I knew when the issue is going to occur, I could measure at that time… But I don’t know how to cause it to happen on purpose, and chances are, if there is any voltage drop, it is very sudden and difficult to notice.

Here are the pictures you asked for: Imgur: The magic of the Internet

Sorry for bad cable management - I am trying to work out the issue so I’m rearranging things a lot. As you can see I still have two 5V wires going from the PSU to RPi (the two adjacent 5V pins). As for ground, I tried one to 5V PSU, one to 24V PSU (since the ground is shared, it shouldn’t matter, but I still wanted to try).

Thing is, if you remember, after adding the second pair of wires, I don’t get low voltage error anymore - just that “power save enabled”. Maybe it’s the same thing, I don’t know.

In any case, are you absolutely sure that this power supply issue is what’s causing CANable to lose connection? I really don’t know what else to do here as far as power supply goes.

12

You have a ground connection from each of the 5V and the 24V power supplies going to the Raspberry Pi? There’s a good chance that’s your problem - at the very least you have a big ground loop and, most likely, you have ground shift between the two power supplies that’s effectively lowering the 5V applied to the rPi.

Disconnect the Raspberry Pi ground lines from both the 5V and 24V supplies and run a new one, the same size as your 5V power line, from the 5V supply’s “-V” pin to the rPi’s Gnd.

It also looks like the CANable has a ground line connected to the 24V supply - you might want to disconnect that as well (I’m not impressed by the quality of the CANable circuitry).

Let’s see if this fixes your problem.

13

My apologies - I wasn’t aware that there is such a thing as ground loop. I thought it’s always best to connect all the grounds. I will try what you suggested, but then I have a question: should I connect the grounds of 5V and 24V PSU together or keep them separate? Also, what about the ground of the EBB36 and CANable - should I connect their grounds to RPi or instead to the 5V PSU “-V”?

14

No, keep the 24V and 5V grounds separate to the different devices. They’ll actually be connected through the USB connection between the rPi and the Octopus.

Honestly, I wonder if your problem is due to the thin ground wires on the rPi - the one going back to the 5V supply can’t carry all the current being provided so some of it is being shunted to the 24V supply with problems ensuing.

I’m not sure about the CANable with the third screw terminal that connects to “Ground”. I think you might have to leave that unconnected.

Let me know how things work.

15

Perhaps, I won’t pretend I understand how this all works :smiley: But in any case, I have rigged up the wiring as you suggested: 5V PSU connected only to the RPi, CANable connected only to RPi (via USB) and to EBB via CAN_HIGH and CAN_LOW, EBB only to 24V PSU. No ground shared between 5V and 24V PSUs, except through the USB cable between RPi and Octopus. Printer boots up so far, I will attempt a previously failed print (without filament) overnight. If anything I listed here is wrong, let me know. Otherwise, I will report the results tomorrow.

16

Okay, apparently we won’t have to wait. My printer halted during QGL, with the same error “Lost communication with MCU ‘can0’”. Klippy log shows nothing of interest at the end. “Sudo dmesg” shows “power save enabled” again. So basically everything the same as before, so I’m skipping screenshots.

I am at a complete loss. Like I wrote, RPi 5V and GND are connected to +5V and -5V on the PSU. No ground loops this time. Nothing else connected to these lines. The only other connections are via USB to CANable and Octopus.

EDIT: It seems that whatever we did, made things worse. I now get printer shutdown every time I try to QGL. Can’t even make a single one succeed. Klipper log now ends with b'Got error -1 in can write: (11)Resource temporarily unavailable'. Firmware restart doesn’t work, I am forced to use on/off switch to restart the printer.

17

I know it’s of little consolation but a consistent error like this is generally easier to find than an intermittent one.

Can you:

  1. Share your klippy.log
  2. Draw out your wiring? I feel like we’re missing something here.

Thanx.

18

Yeah, you are right. It’s always better when the issue is reproducible. Attaching my klippy.log after the last shutdown during QGL.
klippy(9).log (1.2 MB)

I have drawn out the wiring. Very crudely so, sorry for that. But it is accurate:

image
image1120×758 36.3 KB

P.S. If you check my Klippy.log, something really interesting is happening at around line 8152 and onwards.

19

Power schematic looks good -that’s what I would expect. Based on what I’m seeing in the klippy(9).log and it not having any Under-voltage detected errors, I would say that the cleaned up wiring has fixed that problem.

Now, when I go through your klippy.log your Quad Gantry Level execution looks messed up. You have a metric shit tonne of macros in your printer.cfg and I can’t begin to follow them but it looks like you’ve set some really aggressive movements when you start the QGL (at line 8001). I guess you’re running with Klicky (based on the Probe “Dock” operation) and doing four probes for each point (based on what I’m seeing). I’m really not an expert at decoding klippy.log but I think you’re problem is being flagged at line 8127 with a Lost communication with MCU 'can0'.

Did you write all these macros or did you pick them up somewhere? Have they ever worked (with the CAN Bus) before?

20

Yes, I would expect the undervoltage issue is gone too, but like I said, I still get the “power save enabled” message when running “sudo dmesg” after printer has halted.

I have the UnKlicky probe on the toolhead. Yes, there are a lot of macros, I picked them up from various places. For example, QGL macro came with default Klipper install I think. Klicky related macros came from Klicky guides. Nozzle cleaning macro came from the wire brush guide. There are some macros for controlling LEDs and for sounds. I wrote a few maintenance macros myself, like bringing toolhead to various positions, etc.

Basically when I start new print, this is the sequence:

  1. Set bed temperature to target, heat soak for 5 minutes (I print PLA, so it is enough most of the time)
  2. Home all
  3. QGL
  4. Home Z
  5. Bed mesh
  6. Heat nozzle to 180°C
  7. Clean nozzle on the brush
  8. Auto-calibrate Z offset
  9. Heat nozzle to target temp
  10. Clean nozzle on the brush
  11. Move tool head to the center of the bed
  12. Print

To answer your question, yes, this setup worked really well for me. It worked without any issues before I installed CANBUS, and after installing it, sequence still works without issues on the occasions when I don’t get lost connection with mcu can0. So I don’t think this sequence is what triggers the lost connection, especially because like I said, if printer does get through this sequence, I still get print terminations several hours into the print.

Attaching my whole config directory if you want to take a look. I admit I could tidy it up a bit, so sorry for the mess!
2023 04 23.zip (34.2 KB)