I am trying to use multi-mcu homing on a single axis (y), but I am running into the following error at the moment when the homing move should start:
Communication timeout during homing y
The motor is moving for a very brief period before the printer shuts down with this error. Without multi-mcu homing it seems to work.
Is this a regression, or are there any additional limitations on multi-mcu homing which are not mentioned in the documentation (or which I might have missed)? My two MCUs are very different, I have one old ATMega2560 as a primary MCU (that’s where the endswitch is connected to) and a Raspberry Pi Pico as secondary with the y-axis motor connected. Could this asymmetry be an issue?
Should Klipper detect a communication issue between micro-controllers during multi-mcu homing then it will raise a “Communication timeout during homing” error.
The issue I had was obviously this one:
Note that an axis with multiple steppers (eg, stepper_z and stepper_z1) need to be on the same micro-controller in order to use multi-mcu homing. For example, if an endstop is on a separate micro-controller from stepper_z then stepper_z1 must be on the same micro-controller as stepper_z.
Unfortunately reducing the homing speed does not change anything. I went down to 1.0 and still get the same error. I also updated my branch to the latest master (my branch mainly adds some more modules for load cell probing etc.) and re-programmed both MCUs, still it does not work…
At least it is an interesting setup to have the driver’s SPI bus on one MCU and the stepper’s control pins on the other MCU.
I never thought about such thing to even be possible, and maybe this is your issue’s root cause. In any case, way beyond my pay grade. Ping @koconnor
Edit:
Again just spitballing: Besides some software issues, could such a setup also be causing trouble on an electrical level, like the driver seeing different potentials, ground-loops etc.?
Ah sorry, that driver is not actually used. I commented it now and still get the same issue.
Background info: I am trying to replace my X and Y steppers with servo motors, which have integrated drivers. I have started with the Y motor and a “flying” wiring on a breadboard for testing. Later the setup will change, but for now it would be easiest to keep the endswitch connected to the original mainboard.
To use multi-mcu homing one needs to have low latency to all the micro-controllers involved in homing. This is necessary to ensure that the toolhead is stopped within 25ms of an endstop event (as described at Multiple Micro-controller Homing and Probing - Klipper documentation ).
Your log indicates the MCUs have very high latency (10+ milliseconds and erratic latency) - see the srtt, rttvar, and rto reports in the log Stats messages. This is very unusual for a USB connection.
It looks like you are running custom code - you may want to verify the multi-mcu homing on the pristine code to see if the issue persists.
Finally, a good debugging tip is to run M112 immediately after the undesirable event. This will cause Klipper to log recently sent and received messages, which can provide more information on the details of the messages that got delayed.
I tried a different USB cable and port (even different host controller - I have PC as a host) without any change.
The ATMega does not have a real USB connection but uses a FTDI USB to serial bridge. Could that cause such issues? That would mean that 8-bit MCUs cannot do multi-mcu homing - which might even be reasonable. On the other hand, I tried increasing the baud rate to 1M (seems to be the maximum for the ATMega2560) and this again makes no difference in the srtt, rttvar and rto values.
I cannot really do that, since I need some minimal custom code to get the motors moving… I have no modifications in the C part of the code, only added the stepper driver module and some load-cell probing code, which is totally unrelated…
As far as I know it works over serial and with avr chips, but admittedly that’s not a common setup for multi-mcu homing. As before, you could issue an M112 immediately after you get the event, run the resulting logs through logextract.py and see what messages were being sent and their timings during the event.
10 years or so ago, I 'scoped out the USB signals on an Arduino Mega 2560 (with the basic FTDI chip) and found that the data rate on the wires is 1.5Mbps; it doesn’t matter what the incoming data rate was that was the speed of the bits running through the lines. I did this because I was having problems with Octoprint not communicating with the printer and I had access to an oscilloscope with USB probes.
Along with that, the maximum data packet size is 64 bytes plus 40 bytes USB overhead. Now, the amount of data sent was a bit tricky - it seemed that if there was nothing following the stop bit of a byte (ie a start bit for the next byte coming through) then the chip assumes that’s the packet and would send it. As noted size, the maximum data packet size is 64 bytes and the data packet is a multiple of 8 bytes (I believe that this is part of the USB standard). So, if you are sending only one byte, the data would be rounded up to 8 bytes and the complete data packet would be 48 bytes.
So, if you’re saying you’re maxing out at about 1Mbps, that sounds about right as if you are sending full 64 byte data packets (which are actually 104 bytes) at 1.5Mbps then, with the overhead, the actual data rate would be 920kbps.
As for latency, if I remember correctly, it’s pretty good with the packet starting to go out very soon after the last byte was passed to the FTDI chip. You don’t say if you have a direct connection between the rPi and the Arduino as if you are using a USB hub then you could see your response packets latencies go 'way up. I can’t comment on any other sources of long latencies.
The basic FTDI chips (I’m using plural here because there are multiple companies that fab the chips) used in Arduinos are cheap, reliable and were designed in the days of USB 1.0. There are more modern and capable versions of the chips but I don’t think Arduino has updated the bill of materials on any of their parts after it (or a set version) was released.
If you’re just using the Arduino for your Y axis, could I suggest you use something like an EBB 42 and use its USB connection? I know of one person that’s using one each for the A/B (X & Y) stepper motors in a Voron 2.4 without any issues. The EBB (and other toolhead controllers) are really cheap for the capability you get with them.