Prusa showed off the XL printer which has a load cell in the tool head used for Z homing, bed meshing and X/Y nozzle alignment. These seem like significant advantages of a load cell probe that are not easy to replicate with any other probing technology. Strain gauge based probes are the industry standard in the machine tool world (e.g. Renishaw). This isn’t new technology, its just new to 3D printers.
Straight to the relevant bits:
So far we have had one implementation written by Martin (@mhier) that uses an ADC connected to the Klipper host via I2C. This works but it is a lot slower than what we see in the commercial products. This thread is about trying to make it fast and integrate it with Klipper’s existing probing routines. Anybody that wants to contribute to the effort is welcome.
Note 1: the digital endstop pins are sampled at 66.6k but they use 4 consecutive readings to trigger an event. So the effective triggering rate is ~16.6k leading to the lower than expected resolution.
Note 2: the effective resolution @ 5mm/s with multi-mcu homing is 0.125mm but timing based compensation is applied to improve that.
Note 3: The resolution for the ADC chips is theoretical and doesn’t include oversampling, which we may well need to do.
From this chart, only the ADS1256 and ADS1263 have a data rate that approaches the digital endstop pin. The sensors with an 80 SPS speed would only allow for 2 samples before they moved farther than the multi-mcu homing distance. That is a very narrow margin for success.
I’m going to develop for the ADS1263 because:
it supports the highest sample rate
Its available on a relatively cheap board that can be connected to a Pi or an MCU via SPI
It supports standard SPI (supposedly)
It has 5 differential inputs, I have a tool changer with 4 tools so that’s “enough” inputs
The ADS1256 board is more expensive for no obvious advantage to us
There probably is an ideal chip for this application but finding it on a ready made board is going to be tough.
It is an absolute cool technology and touch plates / probes are standard in the CNC industry for ages.
Looking at my own hotends, I sincerely doubt that this (ref to Prusa) is practical in reality. Each bit of plastic sticking at your nozzle will throw off the results.
I would like to make a few general additions at first:
The technology is much older than the Prusa XL. My printer is a Renkforce RF1000 which is already originally equipped with load cells for probing. It has been released I think in 2015 already (I brought mine in 2016 and started to improve its firmware soon). It is using an ADS1100 ADC which can only do 8 Hz at the full resolution (16 bit).
I have developed my implementation based on the experience I have gathered on that printer model. I believe it is very robust. It can even be used to probe in X/Y direction, which in principle is mechanically not foreseen. This feature is important for me, since I am using my printer also for CNC milling (the printer is solid enough for light milling operation).
From that experience I can tell a few things:
A simple threshold on the ADC value (i.e. the measured force) is insufficient. Depending on the exact mechanical setup, there may be mechanical tension (especially if a heated bed is already switched on) which can lead to slow drifts of the measured ADC value even when actually no external force is applied on the hotend. This needs to be compensated by performing a tare measurement quite frequently (measure ADC value without bed contact and subtract this value from subsequent measurements). Frequently here means that this is necessary multiple times even within a single probe measurement.
Even if the the tare compensation is done, a simple threshold has the downside of adding an offset to the result. The threshold will correspond to some real force which needs to be applied to the nozzle before recognising it as a contact situation. This force corresponds to a certain displacement of the nozzle (load cells need to deform to measure anything), which leads to an error. It can be compensated for, but this needs to be done manually.
Hence I have introduced a linear regression fit to determine the position at which the force is just zero. This works perfectly (precision is around a few micrometers). For this to work so good and reliably, it is important to do the tare compensation for each single ADC measurement. This is realised by hopping in z direction up and down multiple times (upper position for tare measurement, lower position for the real force measurement).
When performing just linear movement (without hopping around in z direction) and doing measurements along at regular intervals, the measurements are often very inconsistent. I found that e.g. reversing the direction and taking measurements at the same positions again on the way back leads often to completely different results. I think this is due to the before-mentioned mechanical tensions.
Taking measurements while any axis moves, is completely unreliable. Hence one should always stop any movement, even wait a bit, and only then take the measurement. I have found it useful to take at least two measurements to average over and also determine whether they are comparable - if not, there was still some mechanical vibrations and the measurement needs to be repeated. Without tricks like this, the results were way to unreliable.
Maybe Prusa has found a smart way to improve this on the mechanical side. Still I could imagine having a more robust algorithm in Klipper is very useful especially for people who want to build this themselves.
My current implementation is slow (as it relies purely on Python code), but actually it is not much slower than the original implementation my printer was sold with. So maybe it is a bit unfamiliar how long a bed mesh calibration will take if you are coming from a different printer model, but I can integrate these 15-20 minutes quite easily in my usual workflow. Of course I would be happy to improve speed, but I think it is already usable as it is.
Using a faster ADC is important to improve speed, but there will be a limit what can still make a difference. Mechanical movements will not be faster with faster ADCs. Also the necessary wait time after stopping a move and before taking a measurement are determined purely by mechanics, so a faster ADC won’t help.
The implementation currently has 4 phases, which need to be optimised differently:
“Fast” rough approach (essentially a simple threshold with a single tare before commencing)
Verify contact (if not verified, continue with 1 after re-doing the tare)
Backup to a position above the bed but as close as possible
Take measurements for linear regression
Especially the fast approach could be much quicker with a faster ADC. Probably here we can move while measure, since we anyway confirm the result in the 2nd step.
The 2nd step is fast anyway, not much to improve here
The 4rd step might again be improved. It is currently heavily optimised to deal with the long latency of the measurements. I could imagine the same technique as for the fast approach could be used, just in the other direction and a slower for improved precision.
The last step will be harder to optimise, but it takes around half of the total time. If we do not want to move the entire logic into the microcontroller code, we need to compensate for the latency by other means. If the ADC measurements can be scheduled just like the moves can be, we could queue some movements and measurements ahead and hence improve the waiting time here. I am not sure though how much we can gain in the end, since the waiting time also is important for the mechanical vibrations to settle.
I can only recommend to use my experience I have gained over several years and not redo the same mistakes. I have spend a lot of time trying to improve the precision and reliability way before thinking about Klipper. The original firmware is Repetier-based (forked from a very old version which makes it very hard to maintain - hence I decided to switch to Klipper), so it was not suffering from any latencies. The implementation there was a lot simpler, but it also was less precise (especially the absolute precision was not so good, we always had to account for some static offset).
PS: My personal next step will be to get some MCU driver for the HX711. In the long run it should support both a virtual endstop pin based on a runtime configurable threshold and scheduled measurements. Of course I will start at first with a much simpler implementation to just replace by ADS1100. I like to do these improvements step by step, otherwise I will just produce a big mess
This seems like using an ADC / Comparator combo would be best. The comparator would give the high speed and low latency, whilst the ADC could be used to trim the Comparator threshold to avoid false positives.
It might even be possible to use something like this with a strain gauge frontend added to it
There’s a number of other similar options in chip form such as:
Thanks @mhier! I have learned a lot from your work:
The sample rate will make your design choices for you
We must compensate for noise and temperature drift
Event detection wont be purely a pre-configured threshold.
Some post processing is likely to be needed to identify the exact position of contact.
But I want to challenge this idea that we need to stop the movement to take a measurement. That might well be true for your printer and electronics but it isn’t true for all printers. Its not true for strain gauge based CNC probes and its not true for the Prusa XL. There is plenty of video to show that’s not how its done. If I cant get continuous probing moves to work I wont submit anything to the Klipper codebase. I would not have made a meaningful improvement over what you have submitted.
I want to keep adding data to this thread so I lets take a look at this screenshot from the XL video:
This is a plot of their load cell data for a probing move. I added those red vertical lines. Here is what I can see:
They are doing a continuous move with no pauses
The probe move is really short and pretty slow, maybe 1mm @ 1mm/s
The y axis is in grams.
The noise before the contact is about +/-25g.
The max force applied is 150g.
There are approximately 80 samples points per second in the graph. (hard to see but at least that many)
The contact event is very easy to identify from the noise. Signal to Noise ratio is ~6:1. A simple threshold would have worked here if the magnitude of the noise was known before the move started. 2 samples > 2x noise ==> trigger.
They get the machine stopped in what looks like 2-4 samples. That’s 25ms to 50ms which is actually pretty slow.
If time to trigger is only 1 data point the best accuracy you can get is 0.025mm. That seems high. Maybe they are using something like the linear regression @mhier proposed to improve this?
There are black vertical bars on the right that look like data being dropped. The tool head also seems to stop at this time. Maybe the MCU is overloaded? Maybe this is an early version and still has bugs?
This screenshot lends some support to the idea of using an 80SPS ADC. It hard to say what the real data rate is, the graph could be down-sampled for display purposes.
It would certainly make mounting those bulky strain gauges easier! But you would give up X/Y probing, nozzle jam detection etc. I want to focus on the software side of this so if you want to experiment you have something in Klipper that you can use.
What are the pros and cons of a load-cell-based probe compared to a piezo probe on the tool head? This conversation reminded me of the Precision Piezo Orion I’d read about some time ago. Used as an endstop and probe it seems it solves the direct-probing problem, but its binary nature doesn’t allow for measurement of force on the nozzle while extruding (although that’s not really in scope for what’s being discussed here).
Ugh, I spelled “Strain Gauge” wrong in the thread title and now I cant edit it
@blalor I guess pros for the Piezo is that its a switch so its easy to integrate. Cons are that its a switch so it cant tell you about strain. The attraction is doing things with the force data during the print: collision/layer shift detection, clogged nozzle detection, print tuning. Plus it doubles as a X/Y/Z sensor. Getting the data out of the ADC and back to Klippy will enable all of this extra stuff to be developed by others. But if we did only that its not a great solution for homing moves.
I did a little more poking around this morning and it does seem that piezos need some temperature compensation, as well. The Precision Piezo probe has trimpots for doing that adjustment. The Pyr0-Piezo attempts to build that compensation into another circuit that sits between the sensors and the controller board. I get the impression that strain gauges are also not immune to temperature changes, but perhaps that’s more easily accounted for.
Thanks for entertaining my questions. I would really love to explore the use of a reliable method of direct probing; I’ll be watching this project!
Actually I am not even 100% sure if this part is really necessary even on my hardware. I think I never tried this differently. It was always implemented like this in the original firmware, and with a pure Python Klipper module it was a technical need since we have to know at which position the measurement is taken.
And I want to add another thought I got recently: Increasing speed may actually help mitigating issues with the “baseline drift” (i.e. the measured value without force changes over time). Since I believe the baseline drift is physical (cause by thermal effects and mechanical tension), it won’t drift faster if we increase readout speed. On the contrary, the drift effect will be smaller if we complete the scan faster, since it has less time to drift.
I would suggest to make such things configurable. This allows everyone to try out a configuration which is as fast as possible for the given hardware setup. First we need of course the fundamental possibility to read out the ADC synchronised to the movement, as this is otherwise simply not implementable.
In any case, 80 Hz we can get from the HX711 is already 10x faster what my hardware currently can do. My guess is this will do for now and it will be hard to make use of faster readouts anyway.
I have unfortunately not a lot of time at the moment to dig into writing a HX711 MCU “driver”. I will do it, but it will take some time…
I think it would still be helpful if someone could try out what we have right now in my branch (with the preliminary HX711 support or by using an ADS1100 or by writing another driver module, which is not so difficult if it has a standard I2C or true SPI interface) on different hardware. We might learn a lot from that, since so far nobody has tried this to my knowledge. I am happy to help with any kind of problems
Yes all of that! I think temperature drift is less of a problem and mechanical “drift” will be more of an issue, especially for x/y probing. I expect we will be able to see:
Noise - with the printer stationary we should see some level of noise from the ADC and load cell
Temperature Drift - with the machine still the average reading should drift with temperature changes at the load cell. Also the noise amplitude may increase.
Mechanical Noise - with the machine moving at a constant speed vibrations caused by the stepper motors could be picked up.
Mechanical Offset - With the machine at a constant speed, depending on the direction of movement, the average value should shift as there is essentially a constant force on the load cell.
Mechanical acceleration - as the machine accelerates from 0 to probing speed both the mechanical noise and offset will ramp up suddenly.
Its the the last one, the acceleration that I think will be the largest factor. I would expect it would plot like a shift in the graph right at the start of the probing move. That’s going to be much larger and faster than temperature drift.
I think we can solve this with smoothing and deadband. We use smoothing to work out the average input value over time. As time moves forward we can continuously adjust the zero setpoint based on the smoothed value and a maximum allowed rate of change. The smoothing rate and the max rate of change should be configurable.
We define a deadband that is +/- n counts from the setpoint value. This deadband contains all of the noise we expect to see. We can compute the size of the deadband by looking at the (max - min) over some recent time period. The size of the deadband can be calculated based on previous measurements of the sensor with the printer in motion in the direction and speed of the homing move (i.e. a calibration move). Then for each homing move of some speed and direction we can select an appropriate deadband value.
Finally triggering is done when we get a number of counts outside the deadband. They should all be either + or - of the deadband (i.e. not an oscillation). That number of counts should be configurable as well for every probing move.
I plan to wrap all of this up in an adc_endstop.c module that we can feed with data from any ADC sensor module that gets written. Here is a diagram:
The deadband is exactly what I mean with threshold. As I said, the direction of the expected change is known, so even a simple threshold (and a direction in which it will be crossed) is in principle sufficient. A deadband of course will do as well.
Please keep in mind when designing the interface between the adc_endstop and the Klippy host that the threshold resp. deadband must be reconfigurable on the fly, not just when configuring the firmware (not sure if special precautions are necessary here, I have not yet worked on that level in Klipper).
Btw: The effects of acceleration won’t be visible strongly on my printer model normally, as the Z axis moves the bed, not the print head. Still, even when probing in X direction (which moves the toolhead even with a heavy milling motor) I did not see this as an issue. Hence my guess/hope is we might come away without caring about acceleration effects much.
Yes! I think this is even more clear if we read endstop.c. The endstop_home command takes 3 things that should interest us:
sample_count - this is the number of samples before a trigger
trsync_oid - the ID of the trsync that will be triggered
trigger_reason - the reason that will be send with the trigger
This gets invoked at the start of very homing move so you can pick any trigger, reason and sample count per-move. adc_endstop would just add to those options:
set_point - the starting smoothed value of the ADC
smoothing - the amount of smoothing to apply to compute the new set_point
deadband - the width of the deadband
On my test printer this is how my Z axis works as well. One of the first things I want to investigate is X/Y vs Z movement. To really see whats gong on I need a way to graph the sensor output in real time. I’m hoping I can do this through Moonraker the way its done for the ADXL345. But some custom graph UI needs to be built. I will look at eather its easier to extend Fluidd or do a separate tool. Maybe there is some code already for the visualizing the ADXL345 that we could work from?
FWIW, homing with a strain gauge sounds like a lot of work to avoid a $2 z-max switch. And, it notably increases the risk of nozzle crashes.
After homing, once the toolhead z position is roughly known, I’d guess the code/complexity of strain gauge probing would be easier.
That said, feel free to work on it if you want to.
FYI, it’s possible to extract adxl345 measurements and then graph them using the “motan” tool ( Debugging - Klipper documentation ). It’s more “batch” than live though.
FYI, Klipper has an infrastructure for precision timing of adxl345 measurements.
At a high-level, the adxl345.py code works with the sensor_adxl345.c code to read measurements from the adxl345’s buffer in such a way that we can estimate the sample time with very good precision. The “motan” tool ( Debugging - Klipper documentation ) demonstrates how one can then correlate these measurement times with past stepper positions.
We’ve used this system in various accelerometer tests and there is good confidence that the timing correlation works well.
Indeed it does. I have an E3D Toolchanger that has 4 tools that I want to align in X, Y & Z. It still has the “$2” endstop switchs for homing. I called the thread ‘strain gauge endstops’ because probing uses endstops and trsync under the hood to stop the steppers and I want to plug directly into that system. If I can accomplish this I expect that everything else will just be math and python.
