PSA: CanBus on 32 Bit vs 64 Bit Pi's - "Communication timeout" errors

G’day All, I ran into something in the last couple of days that doesn’t seem to be well documented - CanBus on 64 Bit Pi OS’s, eg Bullseye, is highly error-prone.

My HW: SKR v2 → Pi 3B+ → U2C → EBB36

Context: I was running an old version of MainSali OS and needed to start a fresh OS build to update to the latest versions of Crowsnest and a couple of other extensions. So I thought I’d try starting with the 64 Bit version of MainSail OS 1.3.2.

Anyway, after 2 days of constant “Communication timeout” errors when running a BedMesh etc, I found more reports of ppl having these issues on 64 Bit Pi OS’s than any other config. I also noticed that the “bytes_invalid” counter was consistently increasing no matter what I tried, and that had never been an issue previously. However, there was no consistent advice that the OS version was the root cause.

So I decided to throw the 32 Bit version of MainSail OS 1.3.2 onto an SD card, and after putting my config (same config as used on 64 Bit) back on I started testing. Low and behold, all the errors have gone away, and I’m 2 hours into a print with Zero “bytes_invalid” on my EBBCan MCU.

SCR-20240818-ktkv

Recommendation: Do NOT use 64 Bit versions of Pi Operating Systems and CanBus - you will have issues with errors and timeouts.

@koconnor perhaps it’s worth creating an article in the KB warning ppl off running 64 Bit OS’s? I’ve found folk running Pi 5’s + 64 Bit Bullseye blaming Crowsnest’s USB WebCam handling for these errors, but I strongly suspect their issues are actually CANBus (Katapult) not working well with a 64 Bit OS.

PS. This is the Guide I recommend for configuring CanBus on Klipper, it works perfectly on every 32 Bit Pi OS I’ve tried.

1 Like

I’ve had a few problems with running the rPi’s Mainsail OS.

Before you start writing up a ticket, could I suggest that you repeat the experiment with the basic rPi 32bit OS Lite and rPi 64bit OS lite with Klipper/Moonraker/Mainsail/Katapult (if used) installed using KIAUH?

1 Like

Howdy, yes I can definitely do that. It’ll likely take me a week or so before I’ll have the time. Now that I’ve got my little printer working, I need to finish configuring and tuning a Magneto X for my son’s school.

These things are very cool, and so darn big I have to keep it in my garage! :rofl:

1 Like

This is known and at least briefly mentioned here: Installing Klipper with KIAUH

2 Likes

That is a well written guide, perhaps the OS issue should be called out more prominently?

By which I mean the official Klipper wiki - there’s no mention of 32 Bit vs 64 Bit being a potential issue, even in the troubleshooting section:

https://www.klipper3d.org/CANBUS.html

The reason I raised the issue here, is I couldn’t find a single location that mentioned it via Google, including on this forum.

Admittedly Google search has devolved significantly in recent years, but if someone like myself, who finds this out the hard way, and is intentionally searching to see if others have too, can’t find the information, someone deep in troubleshooting hell has almost no chance of figuring it out.

I do not think it is a general problem, but it might be a problem. Such a pointer would be suited for CANBUS Troubleshooting - Klipper documentation (it briefly already mentions Kernel woes).

Feel free to raise a pull request to amend the documentation :wink:

At least here on the forum, this has been mentioned multiple times already.

1 Like

@dJOS_500, I see you call out using an RPI 3B+. Have you tried an RPI 4 or 5 to ensure it is not an issue of overtasking 3B+ with the 64-bit RPIOS?

We need more information before we point to the OS or RPI processing speed, memory, USB, etc… debugging requires removing components in the setup to drill down to the problem.

You are also missing information on the equipment you are using, U2C (V1.x [STMF072] or V2.x [STMG0B1]) these MPUs have different processing speeds.

EBB36 (V1.0 [STMF072], V1.1 or V1.2 [STMG0B1]) these MPUs have different processing speeds.

The CPU utilisation was never above 25%, and everything looked pretty normal, aside from the errors in data transmission on the CanBus to the EBB36 MCU.

The only other issue I noticed was crowsnest getting low frame rates. This was how I stumbled across users of Pi4’s & 5’s blaming it for their CanBus errors on 64 Bit OS.

I don’t have any of the more powerful Pi’s on hand to try, but consorting @Sineos has pointed out that this is a known issue, I’m not sure it’s worth chasing down i on my system. Especially as it’s working flawlessly on Bullseye 32 Bit.

If it helps, I have the U2C 1.x and the EBB36 v1.2.

1 Like

I would suggest changing out your U2C V1.x for a V2.x. The V1.x F072 seems slow to push the CAN bus at fast speeds.

Note that the U2C is a USB to CAN bus adaptor and is not a Klipper device so it wouldn’t be included in Klipper load information.

Considering Klipper and all of the extensions I use appear to be native 32 Bit code, and most Pi’s have less than 4GB of ram, there’s really no good reason to use a 64 bit OS.

The only reason I went with 64Bit Mainsail OS, was the Pi imager hiding the 32 Bit options when I selected my Pi model. I discovered later that if you don’t select a model, Mainsail OS 32 Bit version is visible and has “(recommended)” alongside it.

What? This response has nothing to do with my last comment.

I would suggest changing out your U2C V1.x for a V2.x. The V1.x F072 seems slow to push the CAN bus at fast speeds

It does, think about it, the v1.x works perfectly fine connected to a Pi running a 32 Bit OS, so what reason do I have to spend money on a hardware solution, when there is no real reason to use a 64 bit OS?

I have just re-setup a couple of my machines, and forgot all about this 64bit issue, I have non stop timeout’s, I get tmcuart timeout, and heaters shutting down randomly on several machines, its like a little gremlin is running around inside the dam things.

One is an RPI5 + MKS-SKIPR + MKS-THR36 all on CAN, the other is an RPI4 + MKS-SKIPR + MKS-THR36 all on CAN, no U2C style boards involved.

I’m about to switch the RPI4 over to 32bit and see if the issue persists.

2 Likes

Which Kernel version are you using? There are reports that the versions 6.6.x are improving it.

I was running 64bit Kernel 6.6 Bookworm, testing 32bit Kernel 6.1 Bullseye now, so far so good.

1 Like

I still had an error, so I changed my CAN MCU’s from 500,000 to 1000,000 and also used the following in /etc/network/interfaces.d/can0

allow-hotplug can0
auto can0
iface can0 can static
bitrate 1000000
up ifconfig $IFACE txqueuelen 2048

Then applied the mcu_timing.sh from this repo zippy-klipper_config/scripts at master · rootiest/zippy-klipper_config · GitHub

So far seems to be printing fine, but time will tell, this timing bug seems to keep showing its ugly head randomly just like a cold sore…

1 Like

I’ve found the CanBus settings in this guide to be flawless at 1Mbit.

Both the interfaces and network files make the difference IME.

1 Like

Okay so most of this I already know, but I dont have these two lines:

pre-up ip link set can0 type can bitrate 1000000
pre-up ip link set can0 txqueuelen 1024

Not sure what they do, and I have not had to add theses before

[Match]
Type=can

[Link]
TransmitQueueLength=1024

and

[Match]
Name=can*

[CAN]
BitRate=1M

Once I have finished punisihing my machine with an oversized benchy, I’ll add these and see what happens :slight_smile:

2 Likes

Mind Use an appropriate txqueuelen setting

2 Likes