Strain Gauge/Load Cell based Endstops

[garethky]

I integrated the code from the Jupyter notebook and I thought I would be stuck in integration hell for a week. But no, it worked on like the 10th re-compile :

// PROBE_ACCURACY (samples=50 retract=2.000 speed=5.0 lift_speed=10.0)
// 9 point bed sweep, 50 samples, 5mm/s

probe accuracy results: maximum 0.022973, minimum 0.018063,
                        range 0.004910, average 0.021614,
                        median 0.021707, standard deviation 0.000854
probe accuracy results: maximum 0.055126, minimum 0.052617,
                        range 0.002509, average 0.053295,
                        median 0.053291, standard deviation 0.000366
probe accuracy results: maximum -0.017186, minimum -0.019178,
                        range 0.001992, average -0.018102,
                        median -0.018080, standard deviation 0.000481
probe accuracy results: maximum -0.023697, minimum -0.025266,
                        range 0.001569, average -0.024345,
                        median -0.024314, standard deviation 0.000357
probe accuracy results: maximum 0.006352, minimum 0.004198,
                        range 0.002155, average 0.005163,
                        median 0.005244, standard deviation 0.000550
probe accuracy results: maximum -0.039901, minimum -0.040901,
                        range 0.001000, average -0.040235,
                        median -0.040200, standard deviation 0.000217
probe accuracy results: maximum -0.013751, minimum -0.016480,
                        range 0.002728, average -0.014947,
                        median -0.014835, standard deviation 0.000547
probe accuracy results: maximum 0.047511, minimum 0.046222,
                        range 0.001289, average 0.046718,
                        median 0.046701, standard deviation 0.000238
probe accuracy results: maximum -0.005060, minimum -0.007647,
                        range 0.002587, average -0.006149,
                        median -0.006065, standard deviation 0.000578

All but one of those is under 3 microns. It super consistent.

I set the number of probe samples to 1 in the config and pushed the gantry out of alignment. QGL solved it on 2 retries:

Retries: 2/5 Probed points range: 0.004072 tolerance: 0.007500

With my switch probe it would have take 3 probes on each corner and probably 4 retries.

I have some debugging to do and I need to restore the code that emits the probe info over the websocket. But this should get pushed this weekend.

[inkwhite]

hello, i was confused by this problem for a few days.

[Sineos]

Do a:

sudo apt install libopenblas-base
~/klippy-env/bin/pip install -v numpy

[inkwhite]

it works, thanks

[alexb]

I know this is a creality board but any idea why you would want four HX711 instead of just wiring 4 load cells to one HX711/HX717? I can’t think of any advantages. I have purchased some nice loadcells, half bridge and full bridge and I’ll try some underbed configurations soon. I have a HX711 here and I’m about to order a custom HX717 pcb for testing. My initial thought was I’ll try to put four half bridge loadcells together so that I’ll have one full bridge. One HX717 should work with garethky’s current branch if I’m not mistaken

[zarboz]

I can only speculate based on how it’s used

They put a cell in each corner of the bed and seem to take a median from all 4 to determine the point of which the nozzle presses the bed lightly

They have other deployments of this setup on like the Ender 3 SE which has a single cell in one corner instead

[inkwhite]

i am doing the same thing like you, have you already resolved the problem ?

[garethky]

Use LOAD_CELL_DIAGNOSTIC to check the function of the load cell before attempting to home or probe with it. If you see that all the values are 0, that is extremely unlikely and usually indicates some sort of wiring issue. You can tap the load cell while the diagnostic is running and you should see different results that if you do not.

I just want to VERY STRONGLY discourage you all from using those cheap HX711 boards that sample at 10SPS for probing. They are not fit for that use. They are only supported because we want to use them in weigh filament scales. Your Z axis moves a long distance in 2/10ths of a second (the default number of samples to stop is 2). This will subject the nozzle to a lot of force. (I’m considering making 80SPS the minimum for the probe and having it throw an error if you even try this. If you insist you can change the code yourself)

Also for these boards use very short wires to connect them to the MCU. Like 2 inches/5cm or less. You can use long twisted pair wires from the sensor chip to the load cell. With long leads to the chips we have seen issues with ESD rebooting the chip. These boards were never designed to be used like this.

If you must use an HX711, get the one from Spark Fun: SparkFun Load Cell Amplifier - HX711 - SEN-13879 - SparkFun Electronics

A LoveBord from Prusa is not much more money: LoveBoard | Original Prusa 3D printers directly from Josef Prusa and you get the HX717 with 320SPS.
It was actually designed to be a toolhead board with long leads. It uses an additional coms chip to make signal transmission reliable. This is what I’m using to test. More info in this thread: Load Cell Design - #37 by garethky

We for sure need to get some open source boards designed for the HX717.

[garethky]

Major update to the branch:

Tap detection algorithm
This involves making a slow speed ‘pullback’ move and working out when the toolhead breaks contact with the build surface. This is the approach that Prusa are using in their machines, and it turns out to be more accurate because of the lower electrical and mechanical noise at these low speeds. (0.4mm/s for 0.1mm is the default, its configurable). This also allows the probe approach speed to be fast so the overall time to probe is quick.

I did a bunch or work to generalize the code from the Jupyter notebeook. It now computes a full representation of the tap with intersections calculated with linear regression at all of the elbows.

This works when Probing but not Homing. This keeps code changes out of homing.py but means you cant home with the same level of accuracy. I’m sure we can write a macro to probe after homing and reset the toolhead position to solve this. If you use Bed Mesh (and you should be) this isn’t even an issue.

When probing at 5mm/s I am seeing about 3 microns of range across the bed of my Voron 2.4 when running batches of 50 samples. At 2mm/s it is slightly better.

Debugging Updates
The debugging tool was updated: Klipper Load Cell Debugging Tool / garethky | Observable
Data from the new tap detection algorithm now shows there:

Black line is the raw force graph. Red line is the result of the tap detection algorithm. If you are testing this code please use this tool to verify that your taps are being detected correctly. Tap validation is currently non-existent.

Z Position from Trapq
The Z position of the point where the nozzle breaks contact with the bed is computed from the data in the toolhead trapq. This means its as accurate as anything that klippy knows about machine position. I have an outstanding TODO to export the toolhead positions with the graph data.

Automatic Taring
Automatic taring of the load cell is in. When homing or probing the loadcell endstop will wait for the machine to come to a complete stop and grab samples across 4 x 60Hz power cycles to tare the loadcell. You no longer have to tare it on startup and you don’t have to worry about drift between probes.

Bad Taps and Nozzle Cleaning
It’s now possible to attach a bad_tap_module and nozzle_cleaner_module to the probe. There is nozzle cleaning gcode as an option as well. Right now there is no bad tap detection, its just empty. The module gets all of the data shown in the graph above to use to decide if the tap was good or not. There is outstanding work to override the way PrinterProbe counts probe attempts to have it retry ones that we decide are bad taps (I may simply lie to it about the z value). The goal here is to make further research in this area a community effort.

HX71X Safety
I added a couple of additional safety checks to the HX71x code to hopefully catch reboot events and shutdown the MCU if they occur. These are courtesy of Prusa’s safety checking code.

---

For myself, I’ve had enough python for the rest of the year. I want to start printing with it this week and see how well it really works for first layers and how much of a problem oozing/bad taps really.

[garethky]

Well, I tried to print with it and that didn’t work. Its not getting past QGL because of the sticky nozzle.

This is what a “bad tap” looks like:

• plastic is moving out of the way after bed contact which slowly released the force on the toolhead (rising line between the blue and green marks)
• plastic sticking to the bed/toolhead caused a pull on the toolhead (that bump on the right that rises to 40g)

So I don’t have a usable printer yet.

Here is a really hair brained one :

[garethky]

Finally, a small success:

That is the first print and that first layer is perfect by my standards. I did not tweak anything with the offset to get this, it just printed it on its own. (I hit cancel as it started layer 2, layer height is 0.2mm in the slicer, sheet is textured PEI)

To get there I mostly made changes to the printer:

• Reduce nozzle temp to 140 when QGL & bed meshing (was ~190). This stopped the ooze. 150 is the max temp recommended by Voron Tap, which has similar issues. No more sticky taps.
• Added a 375ms pause after the last move ends before probing. This seems to help with settling of external forces and produces cleaner taps. I think the Prusa machines are doing this pause to settle their internal noise filter and it has this nice side effect.
• Removed the bowden tube from the extruder. Wiggling the tube can cause up to 150g of force to register. Filament does this too but to a lesser extent. I checked on the XL and it shows variations when you wiggle its bowden tube. So this is not just a “my printer” thing.
• Reduced QGL speed from 300mm/s to 200mm/s. Reduces shaking of filament/bowden.
• The longer a probing move is the more likely it is going to suffer from this bowden torque issue. So I reduced the QGL moves to 5mm.

The bowden thing is probably the biggest problem. Maybe I need a guide of some kind

[garethky]

Not a fluke:

This is with 1-shot probes as well. I’m not taking averages in the QGL or the bed mesh, 1 sample only at each point.

If I’m critical this sample looks too squished, but this is PETG and very old/wet. That can cause over extrusion and make it look like its too squished. Also its not PA tuned, resulting in blobbing at the end of the infill liens and nozzle buildup. Going by the layer fusion its right. It has no apparent layer lines on the backside when removed.

[garethky]

Maybe the problem with the bowden tube is why Prusa are using a bandpass filter: Prusa-Firmware-Buddy/src/common/loadcell.hpp at 15d5ae0b317a37b69b8e7a4720711b962c25b57d · prusa3d/Prusa-Firmware-Buddy · GitHub

Cutting off the high frequencies cuts out stepper and power line noise, thats clear. But why not use a pure lowpass filter if thats all you want to do?

Cutting off the low frequencies could be blocking external forces on the toolhead. The probe move is like 1/2 the sample time, and for about 1/2 or more of that there is no force change, unless a silly plastic tube pulls on the toolhead as it moves. So blocking signals with a frequency about the length of the probe time should block the bowden tube slowly tugging on the toolhead. Its tricky though, if you pick too high a cutoff you will erase the probing signals themselves.

This is what I’m going to try next.

Edit: Prusa isn’t using the filtered data for its collision analysis. Its the raw data: Prusa-Firmware-Buddy/src/common/loadcell.cpp at 15d5ae0b317a37b69b8e7a4720711b962c25b57d · prusa3d/Prusa-Firmware-Buddy · GitHub
Looks like the filtered data is only fed to the code checking for the endstop to be triggered.

[Sineos]

I admire your persistent and meticulous pursuit of this topic.
Additionally, I always enjoy reading another entry in Jane Loadcell’s diary.

Keep it up!

[garethky]

Thank you @Sineos, that means a lot! It has been a labor of love, curiosity and sometimes frustration.

[garethky]

Docs updated: klipper/docs/Load_Cell.md at adc-endstop · garethky/klipper · GitHub

I refactored the config so you don’t have to set up the sensor separately from the [load_cell]. But you do have to configure the [load_cell_probe] separately. So no more is_probe which looked weird. This was on my todo list for a long time but it wasn’t urgent. All of the missing config options from [probe] have been added in. You can see my updated test config here: klipper-voron2.4-config/printer_data/config/machine.cfg at 33d00c469a6fed4c2ec5fe1c1c25f93b023d6454 · garethky/klipper-voron2.4-config · GitHub

I put in a set of pretty robust checks for tap validity. So the scary plot from above would for sure fail these checks. These invalid taps are treated as “bad taps” but the bad tap module is not called. These bad taps are not correctly re-tried when probing.

I’ve also calculated the additional data that Prusa uses to check for bad taps. The angles between the lines in the tap and the R² / Coefficient of determination for each elbow. These are now passed to the bad tap module with everything else.

I did them slightly differently from how the Prusa code base has it. The angles are +/- 180 degrees, with + meaning a clockwise rotation and - meaning counter-clockwise. For R² I’m using the same 20ms, 30ms, 40ms, 50ms and 60ms breakpoints, but I’m reporting it as a percentage. So 0 is average, -300% is terrible and 80% is very good.

So its maybe 90% code complete now?

I’m still testing printing with it. I had to add in some negative z_offset to get perfect results (z_offset: -0.1). That’s weird, it means the probe is reporting too low (hysteresis? displacement under load? heating elongation? all of them?). But at least its seems consistent print-to-print. If the z offset is right is prints with very consistent thickness over the 100mm square.

[garethky]

So much progress today. It might be the last day I get to work on this in 2023 though.

Homing vs Probe offset compensation

Earlier I said that homing isn’t as high of an accuracy as probing. I wrote a macro to automatically measure the difference and compensate for it:

[gcode_macro _SET_OFFSET_FROM_PROBE]
gcode:
    {% set z_offset_constant = 0.05 %}
    {% set z_offset = printer.probe.last_z_result | abs %}
    {% set z_offset = z_offset_constant + z_offset %}
    SET_GCODE_OFFSET Z={z_offset}

This accounts for ~0.04 of that 0.09mm offset I seem to need, leaving ~0.05 of offset unexplained. Within the bed mesh the offset error is very consistent. So that seems to indicate that the probe is accurate but there is a global offset error. I think I need to pull in the latest bed mesh changes to my branch to explore further.

But its already totally usable:

Better tap decomposition

I made a load of small tweaks to the way taps are decomposed into lines. The big one is switching from a scheme where I try to find 2 elbows to one where I try to find just 1 elbow at a time. I did this because I saw results like this:

That’s no bueno, the points for the second pair of elbows are obviously wrong. The big horizontal displacement of the last one causes a large error in z. The new code does better:

Mostly these changes improve consistency when the taps are bad. Testing with a cold nozzle didn’t show any consistency improvements.

Long homing moves

Homing from a large height (>100mm) fails because the bowden tube causes the sensor reading to drift more than the trigger value. This, I believe, is the real reason why Prusa has that “continuous tare” mode. This likely has to be solved in c. Computing it in klippy/python would end up sending a command base on stale data to the endstop which is risky.

I’m going to tackle this in January. I’ll look at the tradeoff off between the Butterworth bandpass filter and an exponential moving average. If we go with the filter, well have to find a way to pick the low frequency cutoff value without the user being a wizard.

[garethky]

I was using the old relative_reference_index option for bed mesh and I should not have been doing that. That cut the z_offset_constant down to ~0.025mm.

The nozzle hot zone is ~17mm long, brass jacketed stainless steel… so it should grow about 0.03mm going from 150C to 260C. If that’s correct them I’m within +/- 5 microns

I cant find a reference to thermal expansion compensation in Prusa’s codebase. Also they come up with an estimate for the position which should be low. Seems like they should have handled this somehow?

Edit: Better source: The thermal expansion rate of brass is 19 x 10⁻⁶ K⁻¹. 17mm * 110C * 0.000019 = 0.03553mm of expansion (and a found a calculator that agrees with that math)

[gcode_macro _SET_THERMAL_COMP]
gcode:
    # length of the nozzle + melt zone
    {% set hotend_length_mm = 17.0 %}
    # uncomment the line for the primary material of your nozzle + melt zone
    #{% set expansion_coefficient = 0.000019 %}  # brass
    #{% set expansion_coefficient = 0.0000231 %} # aluminum 
    {% set expansion_coefficient = 0.0000173 %} # stainless steel
    # pass in PRINT_TEMP= the final extruder temp before heating up
    {% set print_temp = params.PRINT_TEMP | float %}
    {% set temp_delta = print_temp - printer.extruder.target %}
    {% set thermal_comp_z = expansion_coefficient * hotend_length_mm * temp_delta %}
    SET_GCODE_OFFSET Z_ADJUST={thermal_comp_z} MOVE=1

If I try to measure this empirically with PROBE_ACCURACY at the 2 temperatures against the aluminum build place, I get a difference of 0.064533mm, nearly twice whats predicted.

[alexb]

Was the bed heated while probing? Maybe plastic oozing out at 250°C?

[garethky]

Nozzle was clean (no ooze, I pulled the filament and let the rest ooze out for a long time). Cold bed. I removed the PEI sheet and probed directly on the aluminum plate so I wouldn’t melt the PEI. I did a new test cycle and here are the results:

Extruder Temp C	PROBE_ACCURACY Average Z	Coefficient of Expansion mm / C-1
19	0.031552	-
150	0.096380	0.00049
260	0.169629	0.00067
150	0.113793	0.00051
30	0.042419	0.00059

I didn’t let the temps stabilize for very long, so there is some hysteresis when cooling down. But this is a consistent effect.

Now that I’ve seen this its too glaring to ignore, so I tried printing with it. This + the nozzle homing offset produced a print that’s too high. That seems right, the nozzle homing offset shouldn’t be needed. Bed Mesh is going to read the absolute offset at each point, so I shouldn’t have to add any global offset. Now all I have is this:

[gcode_macro _SET_THERMAL_COMP]
gcode:
    {% set expansion_coefficient = 0.00059 %} # measured empirically (avg from 150 -> 260 -> 150)
    # pass in PRINT_TEMP= the final extruder temp before heating up
    {% set print_temp = params.PRINT_TEMP | float %}
    {% set temp_delta = print_temp - printer.extruder.target %}
    {% set thermal_comp_z = expansion_coefficient * temp_delta %}
    { action_respond_info('Extruder thermal compensation: %.5fmm for temperature change %.1fC' % (thermal_comp_z, temp_delta)) }
    SET_GCODE_OFFSET Z_ADJUST={thermal_comp_z} MOVE=1

This is the best result yet both in terms of consistency print-to-print and the first layer not needing any adjustment. It is usable hands off:

I think this is the factor I’ve been looking for. I have been digging in Prusa’s source code because I assume something like this must be in there but I cant find it.