Shaper_calibrate.py and handling of irregular accelerometer output data rates lis2dw vs adxl345

I have two of BTT’s USB accelerometers. One is ADXL345 and the other is LIS2DW. I actually owned the latter first but after watching a negative review about it and then later confirming for myself anomalies in the PSD graphs for it, I bought the ADXL version too and did some comparisons.

TLDW: the video above is from the Klippain community and essentially talks about the lis2dw’s 200Hz Nyquist limit due to an internal (chip level) filter at 400Hz. The datasheet itself confirms something like it it in Table 41, which basically says there is a bandwidth filter on the output, and it is at most 400Hz.

So imagine my surprise when I actually looked at the output.csv doing an ACCELEROMETER_MEASURE and found that the ODR itself is limited to 400Hz, not just the bandwidth. This plot below is instantaneous data rate versus time, which is about 5 seconds of measure. I used a macro to do measure, G4P5000, then measure again. After that I did a diff/lag between the time rows and then just reciprocated it to get the instantaneous ODR. I could have left y axis as the original timestamp but I did the conversion just to see the actual number in Hz.

Short story long, its just under 400 data points per second. Around 390, which is reasonable but what I found unreasonable was the irregularity of the output intervals. Not only is there ±10% variation, there is also periodicity on the order of 0.5-2 seconds. This can’t be good for FFT.

I spent the $10 and got the ADXL345 version and it is much better. Not only is the ODR over 3000Hz, it is ±0.3%

OK. Now how is the bad ODR behavior of lis2dw going to affect us. Take a look at the actual PSD plot from graph_accelerometer.py/shaper_calibrate.py:

That low frequency peak 20x over background is solely from the time misbehavior of the lis2dw. You should trust me when I say I don’t live in a geography that has earthquakes. Or just look at this…I select the least offensive part of the raw data where the ODR seems most regular, like this:

Then I cut out that 1-second section and only PSD plot that section:

What do you know, 5 & 15 hz disappears into background. But now I have 85 hz. Am I going crazy? Well it turns out there is irregularity even there:

Umm..that looks like there is time modulation even at this level. I am not going there.

Ok now the ADXL345 is looking cleaner. There is a peak at 185 Hz 5x over background. Not as bad is lis2dw 20x fake peak. No fans are running. Motors are off.

Btw this is what it looks like when the hotend fan is running:

That 130hz is from the lower harmonic of a mosquito 2510 running at 13k rpm. It is loud as fuck. So at this point I’m thinking 185Hz could be another fake peak. It would be audible if it was real. So I look deeper into the time intervals for adxl345:

Okay, maybe the first second looks like there is some periodic modulation. I cut off the first 1 second and replot:

Oh FFS. It’s still there, it just shifted down to 150 Hz. Correct me if I’m wrong, but real vibrations do not shift in spectrum when you just delete the first 20% of it. This has to be because of the irregular time intervals. Unless adxl is also doing some internal filtering shenanigans which I doubt.

Ok here’s Klippain Shake n Tune plots. First is lis2dw, second is adxl:

The lis2dw has wideband noise below 50hz and is contributing to the higher psd at 5-20hz. It also has spurious peaks all around the main real one, not to mention one that is decending with time (resonant frequency decreasing while calibration excitation resonance is increasing? what??).

Ok sorry for the novella. But has there been a discussion about the shaper_calibrate.py having some sort of interval regularization instead of assuming it is regular i.e. dt = T/N_samples?

I hope this can be improved by code. But if not, I’m just happy to get this off my chest. At the very least I hoped i managed to convince people away from lis2dw given how lis2dw is being marketed by BTT to be superior in so many ways, yet is hot garbage.

1 Like

The last discussion about LIS2DW that I’m aware of was there: Icm20948: Fix accelerometer readings, some changes by MRX8024 · Pull Request #6868 · Klipper3d/klipper · GitHub
My bad, lis2dw is just mentioned, the discussion can be related, but not about it.

If you peek at the accelerometers’ support code, right now, it is pretty straightforward.
We know the ODR, we know there is FIFO, and we are just reading it.
It is received by the host, and the host tries to unpack them, guess timings, and all that stuff.
The sensor_bulk_status() is used for time synchronization.

So, to speak shortly, if the sensor does not provide data at fixed frequency, I’m not sure we can do something with it.
It is possible, though, that there are other issues in between (who knows).
Like missing responses or overflows, they can be detected on the Python side, though.

Hope that helps a little.

thanks for the clue about sensor_bulk_status(). i’ll go look into that.

i thought it might have been due to the usb overhead (as both of my accelerometers are usb version). but these are both designed by BTT, using the same circuit board layout with the same major components including serial interface. I doubt it is related to the data stream processing considering how different the delta_t anomalies are between lis2dw and adxl. They should be chip specific…

I’m actually confident there could be an effective mathematical countermeasure for this problem. the only obstacle is the ROI.

it would look something like refactoring the raw into regularized time bins before feeding into existing shaper_calibrate. this will require re-writing the time column as well as the three data columns via some interpolation. if that doesn’t help, the second more invasive method is to replace the current PSD implementation with a different one designed for irregular time intervals. like Lomb-Scargle periodograms.

but I can tell we are currently using Welch’s method to estimate PSD and this is not robust to varying delta_t. there is windowing and there is 1/f weighting but nothing to deal with irregular delta_t–Welch method requires uniform intervals.

I’m not of math/signals processing background so it might take me awhile and might even be going wrong direction. i’ll try to work on it.

If you are struggling with accelerometer data, it might help to post a Klipper log from the event. If you are seeing gaps in the data that generally indicates lost messages between micro-controller and host. That is, it’s generally not the accelerometer, but the communication channels.

In particular, if you are using a UART connection, make sure it is using the recommended 250000 baud. If you are using a canbus connection, make sure it is using the recommended 1Mbit/s data rate. In both cases, look through the logs to see if there are reports of retransmissions or similar errors.

Also, make sure you are using hardware i2c and/or hardware spi. The software equivalents can struggle with the high data rates needed for accelerometers.

Cheers,
-Kevin

1 Like

FWIW, I’ve seen complains about lis2dw accelerometer on QIDI Q2 with similar symptoms - the apparent data output rate is 400 Hz, e.g.

#time,accel_x,accel_y,accel_z
44132.604771,0.000000,76.570323,-9954.142016
44132.607409,0.000000,95.712904,-9992.427178
44132.610048,57.427742,133.998066,-10011.569758
44132.612687,0.000000,114.855485,-10049.854920
44132.615325,38.285162,114.855485,-10011.569758
44132.617964,19.142581,153.140646,-9992.427178

And this causes some data sampling issues, e.g. see the end of the test:

and that problematic part at the end enlarged:

and this causes issues there at low frequencies (see the bottom right corner), skewing shaper calibration:

And it certainly affects calibration of input shapers, compared to adxl345 that does not have this issue (same printer, same axis, just mounted in a different orientation):

FWIW, QIDI run fairly modified Klipper firmware on Q2, so at first I just concluded they did something to the sensor to decrease the sampling rate. However, I realized that I now have lis2dw sensor on a mellow fly sht36 v3 board, and I checked it with a stock Klipper, and I can confirm the data sampling issue (data is read at around 400 Hz, depending on the run, instead of 1600 Hz):

#time,accel_x,accel_y,accel_z
50.289891,-5723.631659,-3273.381317,7503.891674
50.292242,-5761.916821,-3273.381317,7484.749093
50.294594,-5819.344563,-3273.381317,7503.891674
50.296945,-5781.059402,-3273.381317,7503.891674
50.299296,-5781.059402,-3273.381317,7561.319416
50.301648,-5800.201982,-3330.809059,7427.321350
50.303999,-5819.344563,-3254.238736,7542.176835
50.306350,-5723.631659,-3235.096155,7446.463931
50.308702,-5704.489078,-3235.096155,7446.463931

The sensor is configured with a hardware SPI:

[lis2dw]
cs_pin: sht36_2:gpio12
spi_bus:spi0_gpio4_gpio3_gpio2

and the sht36 board is connected to the main Pi via CAN. And here’s a klippy.log of running ACCELEROMETER_MEASURE CHIP=lis2dw command.

klippy.log (444.9 KB)

Keep in mind that the peak for ADXL345 is around 100x smaller than with LIS2DW, and a typical resonance test induces vibrations 20-100 larger than this, so I wouldn’t worry about it.

1 Like

I think this very important information

isn’t mentioned in Measuring Resonances - Klipper documentation.

How to get the information in? PR?

It can be added via PR, yes. However, I’m unsure how important that is - I did not do exact measurements, and I suppose that really depends on the MCU and what else is connected to it. Plus, some USB accelerometers have weird wiring not according to the SPI specs of the MCU (I think some of Fysetc ADXL345 USB accelerometers were like that), so it is not possible to configure them other than via software SPI. But if at all possible, hardware SPI should be preferred. That said, in my case the lis2dw accelerometer is configured to use hardware SPI, so this should not be a problem, and I believe the problem somehow is elsewhere.

1 Like

Thank you,

if this is true

and

Should we start with a list of accelerometers, which support hardware i2c and/or hardware spi?

My 2 cents, there is fifo on the accelerometer side and as long as there is no fifo overruns - there is no problems.
(so either, i2c or spi are reading fast enough).

Some accelerometer support may have not perfect tracking of that (IIRC), but I would guess this is where you can put an effort if you like to.

1 Like

It looks to me like there is an error in the Klipper code. I don’t have that chip, but you might want to try:

--- a/src/sensor_lis2dw.c
+++ b/src/sensor_lis2dw.c
@@ -141,7 +141,7 @@ lis2dw_query(struct lis2dw *ax, uint8_t oid)
         if (ax->model == LIS3DH)
             fifo_empty = fifo[1] & 0x20;
         else
-            fifo_empty = fifo[1] & 0x3F;
+            fifo_empty = ((fifo[1] & 0x3F) == 0);
 
         fifo_ovrn = fifo[1] & 0x40;
 
@@ -167,7 +167,7 @@ lis2dw_query(struct lis2dw *ax, uint8_t oid)
         if (ax->model == LIS3DH)
             fifo_empty = fifo[0] & 0x20;
         else
-            fifo_empty = fifo[0] & 0x3F;
+            fifo_empty = ((fifo[0] & 0x3F) == 0);
 
         fifo_ovrn = fifo[0] & 0x40;
 

Also, looking at the specs, it seems like these chips support a “burst” mode where one can read out multiple fifo entries in a single read (like the mpu9250). That would be a useful feature to implement.

-Kevin

EDIT: Fixed typo in my patch.

1 Like

FYI, I have an experimental PR at Read in batches from lis2dw/lis3dh accelerometers by KevinOConnor · Pull Request #7077 · Klipper3d/klipper · GitHub that implements reading of multiple samples in a single read transaction. I don’t own these chips so that PR would need testing.

-Kevin

1 Like

As discussed above, some problems were found in the software support for lis2dw and lis3dh sensors. Hopefully they are now fixed with the merge of Read in batches from lis2dw/lis3dh accelerometers by KevinOConnor · Pull Request #7077 · Klipper3d/klipper · GitHub .

It is recommended that anyone using a lis2dw or lis3dh accelerometer should update to the latest version of Klipper and reflash their micro-controller.

-Kevin

@nicefig thanks for starting this off. I have confirmed that the software fixes to the LIS2DW code appear to make the LIS2DW perform the same as the ADXL345. I used the BTT345 and BTT2DW to confirm. I consider this initial verification as I would like to further test with toolhead boards to get a more complete picture.

Comparison of LIS2DW and ADXL345 - klipper update.pdf (420.0 KB)

2 Likes

So at least the graphs and the results of shaper selection now match. That is good.

Welcome Reth,
Thanks, very interesting.

no thank you, it was your yt video i was referencing in the original post after all…

ok better late than never.. here’s my testing which confirms the positive results of the fixes.

as i am late, i took the update from mainline v0.13.0-374 which is well after the fix. initially misunderstood the instruction to update the mcu and only did the mcu and mcu host, but after flashing it to the accelerometer mcu, (rp2040 on the BTT lis2dw) problems went away. thanks @koconnor and all…

v0.13.0-255(blue)→v374(red).

5 second static measure to verify the output data rate. 388-404Hz → 1592-1597Hz

klipper resonance test

shaketune

3 Likes

The accels would match better if klipper calculated the damping ratio from the generated graph, and then using said damping ratio into the acceleration calculation. Similar to what shake and tune provides