TMC Adaptive Microstep Table

[nefelim4ag]

Driver has up to 256 microsteps, so up to 200 * 256 = 51200 of possible data points.
The Encoder has a resolution of 15 bit = 32768 positions.
(I decreased hysteresis to 0.003 degrees, so it can be noisy)

I modified the angle.py a little, to sample each microstep 0…1023,
Export to JSON & etc.
In the end, I sampled 4 full steps, as before.
Export dicts like:

#mscnt, angle data
0: [128, 128, 127]
1: [129, 129, 128]
...

Made some computations to calculate mean angle, and extract diffs in angle for diffs in mscnt.
And convert it to um for easy reading (in the end we want to compare distance :D, right?)

I calculate stddev for each mscnt and use max one.

# For 128 microsteps
Maximum Standard Deviation across all (in degrees): 0.01083
# For 64 microsteps
Maximum Standard Deviation across all (in degrees): 0.011075156035949418
# for reference 1.8 / 256 = 0.00703

There are some graphs,
Y axis is um, X one is MSCNT as before.

I don’t want to be too optimistic, but it looks better than I expected.

---

Maybe it is a wrong assumption, that the encoder has better linearity than the stepper motor, but it looks like so, and it is good that the output is matched to previous graphs.

Now, I think about how to automate table tuning.

I think a good approach here will be to use direct_mode and the XTARGET interface to drive motor current directly to specific micro steps.
Those should allow us to get theoretically “perfect” sine wave for a specific motor and current for specified micro steps.
And later interpolate, encode, and validate it.

At least, I think so.

---

I can’t get direct_mode working.

---

To my surprise, full steps are also not equidistant.
I expected that at least the angle between 0, 255, 512, and 1023 would be around 1.8 ± 1%.
But in fact, it is like this:

Full Step 0 Angle expected: 1.800, actual: 1.697
Full Step 1 Angle expected: 1.800, actual: 1.839
Full Step 2 Angle expected: 1.800, actual: 1.807
Full Step 3 Angle expected: 1.800, actual: 1.829

[nefelim4ag]

I made something, this is a snapshot of PoC.

It should allow anyone interested to play around with an angle sensor and TMC SPI driver.
There are 2 commands, one is to dump micro step data as json, and another converges micro steps to “ideal”, it even allows to save it to config (at least it will try to do so).

This is a sophisticated process, but now an automated one.
Basic, a little bit naive approach for now:

0. Decode MSLUT
1. Choice the widest full step (I assume there will be more available angle data points)
2. Calculate ms angle
3. Get start angle
4. Iterate over each micro-step and compare it with the “ideal” angle.
5. If it is behind → increase current, if it overreaching - decrease current.
6. Check if target metrics are better (if not - abort)
7. Fit a 4th-degree polynomial function to smooth out (4th because we have 4 segments or 3 points where we can change the table encoding) (I don’t know what I’m doing )
8. Normalize it, cause even after smoothing, it can stop fitting to the encoder. (buggy, in a perfect scenario, it should always return a valid, encodable function)
9. Write MSLUT registers, repeat 0

After numerous iterations, it looks like it is “working” for 128 microstepping and 15-bit encoder:

• The motor still rotates like a motor.
• Different smaller or larger microstepping still works
• It converges to something and by real-time metrics it looks like the right direction

It is still not perfect, there still can be bugs (Ex: I think it will break around the 0 position of the angle encoder ).
It can work for 64 micro steps (I didn’t test it), for smaller ones, I think that calibration changes will return too weak a signal for function fit and will be ignored.

Trying to do it iteratively on different rotor positions is a good approach.
After the function gives up - move the motor by 4+ full steps and repeat.

This is what I got after the last “successful” iteration:

To my surprise, it still works, with this wild form, but can’t continue to converge because of mslut table encoder limitations or bad fitting function/normalization functions.

The ideal graph should show us a horizontal line,
Left - After calibration, it is far from the flat line, and from stddev’s point of view, it is worse than the initial one on the right.

---

There is a strange behavior, looks like the driver does not completely apply MSLUT, so after saving and reloading it behaves badly, and if I just remove modifications from the config file, simple FIRMWARE_RESTART is not enough to make it sane again.

[nefelim4ag]

I found a fix for the insane behavior of the TMC driver.
TLDR: During calibration, interpolation should be enabled. Because driver REALLY applies only when MSCNT==0, not when it is passed.

Long story:
I have no idea what happens underneath, from spi CUR_A/CUR_B are sane and right where they should be according to MSLUT.
But the driver can’t drive the motor correctly and this error accumulates and can only be fixed with a power reset. I will not theoretize here.

---

I slightly improved the code, added emergency stops & etc.
The most important are:

• Apply changes only up or down, because they all change the behavior of others. Like high stable current on the right side of the sinus quarter compensates for to “fast” incline of the left side and vice versa.
  
  
• Added some “genetic programming”, where I chose the best-generated MSLUT based on the lowest standard deviation.
• Use raw angle sensor data without interpolated compensation.
• Added code to guess/detect angle sensor resolution.
• Average changes over 4 full steps, because change has a little different impact on different full steps.
• Use angle sensor resolution as hysteresis.

We can’t directly map micro-steps to the angle sensor or match, so we can assume the motor position is aligned or in between angle sensor divisions of measurement, so ± 360 / (1 << N bits) should be enough in regards to actual microstepping which we try to calibrate

I unsuccessfully played with scipy, to make a better-fit function, but I was unsuccessful.
For now, this is already can be called positive change, I got a slightly better signal form in comparison to the default one.

Last graphs:
64 micro steps stddev 0.56us → 0.63us (slightly worse by the graph data, but calibration code thinks it is better).

32 micro steps stddev 0.83us → 0.73us, better by code and by graph data.

So, now I can safely say my brute-force methods are silly, but it is possible to make stepper behave better, I want to believe in making it almost linear.
Also, I removed screws from the belt pulley of the calibrated motor while calibration, for now, it helps to make calibration stable by reducing mechanical drag.

---

Code snapshot: GitHub - nefelim4ag/klipper at angle-tmc-calibration-20241202
I think the next step is to search for a better way to change and fit MSLUT.

From a real-world testing point, I think that for smoke testing it will be enough to compare shake tune vibration profile before and after angle calibration. If it detects differences, it is make sense to play with VFA tests.

[nefelim4ag]

Some good news, it now does try to make microsteps equal and it looks less like brute force.
Snapshot: GitHub - nefelim4ag/klipper at angle-tmc-calibration-20241215

Some graphs, left before, right after 16ms:

After several runs, it is still probabilistic in some way.

32 microsteps:

So, for now, I can safely say - you can try to play with it - it should work.

Changes:

• I incorporated linear interpolation here because in “an ideal” situation it is the simplest way to move points.
• Fit is used as a fallback because it often produces results that are too conservative.

Now it works for 16/32 micro steps on my setup, for 64 it looks like my hysteresis of ± 1.2 of encoder resolution is too high.

---

It was interesting to look at motan and linear motion here, can I spot a difference in deviation?

./scripts/motan/motan_graph.py -g '[["deviation(angle(mt6826s),stepq(stepper_x))"],["derivative(stepq(stepper_x))"],["step_phase(tmc5160 stepper_x)"]]' ./linear-motion-4 -d 25

32ms custom mslut, interpolation is on (4)

32ms default mslut, interpolation is on (5)

For me, it looks very similar.

Let’s assume that according to old vibration profiles I had resonances on CoreXY at 75-120 mm/s
I’m not sure how to properly compare speeds here with the speed of the motor and axis, so 120 / 1.44 ~ 80 mm/s.
I use 20kHz (sample_period: 0.00005), so 360 / 200 / 32 = 6400 pulses for 40 mm/s.
12800 pulses for 80 mm/s.

Let’s look under a microscope 80 mm/s:

10 mm/s:

Zoom to 4 full steps (one phase revolution):

I feel biased here, so I will not comment on it.

---

If we look at the last graph, we can spot that the form of deviation is repeating, like 1 and 3 quarters have hills and 2 and 4 quarters are more or less flat.
These are full steps, maybe, they can be better if there is possibility to control sin/cos offset like in TMC 2240. But I do not have one to test.

[koconnor]

Very interesting. Thanks for working on this and sharing your results.

Some random questions / thoughts:

• How did you validate that the magnetic encoders are properly calibrated? If the encoders have jitter (due to magnet offset, due to precision limitations, or due to timing inaccuracies) it could skew the microstep results. I doubt this would be an issue (as angle sensor calibration is typically done at the full-step level). I am curious on the methodology you used though.
• Have you observed any patterns when going in a positive stepper direction vs a negative stepper direction?
• Have you considered altering the Klipper iterative solver to account for these step deviations (instead of altering the tmc driver configuration)? That is, change the motion planning so that each step is not a constant distance, in an effort to alter the stepper timing, in an effort to account for stepper microstep cogging. I suspect the Klipper step compression code may struggle to handle high speeds if step distance is not constant, but the test results may be enlightening (even if it is only possible to test at slow/mid speeds).
• One thing I found useful was to simultaneously grab both accelerometer and angle sensor data. The two often correlated. The additional information was sometimes interesting.

Cheers,
-Kevin

[nefelim4ag]

• How did you validate that the magnetic encoders are properly calibrated? If the encoders have jitter (due to magnet offset, due to precision limitations, or due to timing inaccuracies) it could skew the microstep results. I doubt this would be an issue (as angle sensor calibration is typically done at the full-step level). I am curious on the methodology you used though.

I did not alter the current logic of Klipper, only added something aside.
My answer is completely about my approach to playing with MSLUT, based on which I do believe it is mostly positionally accurate.

Physical stuff

I feared that I would face all possible issues with jitter, non-linearity, and poor resolution.
So, I did some stuff like picking the right-sized magnet, checking magnet fields visually with a magnetic field viewer, and making the right motor mount with a slot for viewing the distance and centering of the magnet/encoder, everything according to the datasheet and after run internal chip calibration to an account of non-linearity.

That was controlled by ANGLE_CALIBRATE, so I have something like this:

// angle: stddev=15.322 (0.718 forward / 0.734 reverse) in 399936 queries
360 * 15 / (1 << 16) ~= 0.0823
360 / 200 / 16 = 0.1125

Here, I just assumed this is motor behavior and not sensor because of friction, inertia, and inequality.
I saw them when calibrating the motor with a laser (I already knew they existed).

Software stuff

I have control of the hysteresis register, so I played with it to get the idea of positional jitter.
It seems like even with minimal hysteresis (0.003 degrees) it just does not produce different results.

Moving tests, tricky cause of friction and inequality
Look how beautiful it is in the middle of the full step:

And how ugly it can be around zero:

But I assumed it will exist:

• I just collect N samples and use the average in tuning code.
• I do sweeping moves to overcome friction, like +2, -2, +1 (micro steps)
• I only measure everything in relation to the start of full steps

Maybe I’m lucky here.
Also, I have too much logging, so I’m able to see what happens during calibration and how repeatability/position accuracy looks at each run over the same 4 full steps.

BTW, I use raw samples without compensation, to detect chip resolution and allow misalignment of steps up to ± (360 / (encoder resolution).

Time jitter - it does not matter, because I measure the motor in static like it is done in ANGLE_CALIBRATE.

Cause I already have data on the accelerometer, micro-step level, full step level. I think I can mostly or less look at motan graphs and assume that forms of deviation are related to what I see beneath.

But I should notice, that this is one motor, one encoder, and one me, like we do not really know how it looks in different circumstances.

• Have you observed any patterns when going in a positive stepper direction vs a negative stepper direction?

Mostly no, from laser tests moving looks symmetrically bad .
I did not test this specifically with the encoder.
I only observe that sometimes micro step data in relation to full steps looks reversed.
Like in a perfect world with perfect motors and electronics full steps are equal and microstep inequality is the same for all of them. But the actual situation looks more random, like the micro-step distance pattern can be in order for the first 2 steps and reversed for the other two, or even/odd. Sometimes sin prevails over cos, and sometimes vice versa.

My hypothesis here, aside from the physical inequality of coils and motor is sin/cos phase shift can account for that. I will only be able to validate it when I get my hands on TMC2240.

I mostly think from a practical perspective of what tools we have to work with, so there are 2 outcomes after phase shift compensation it will be better or not, even if not - fine.

From a high level of view they seem symmetrical (mirrored) fullsteps and microsteps in relation to the stepper phase (default mslut, 32 steps, interpolation enabled, pulley/belt is connected):

Graphs

G91
SET_VELOCITY_LIMIT ACCEL=2000
G0 Y50 F600
G0 Y-50 F600

Full steps look similar, microsteps at first full step have a higher jitter, and later on, they seem better.

• Have you considered altering the Klipper iterative solver to account for these step deviations (instead of altering the tmc driver configuration)? That is, change the motion planning so that each step is not a constant distance, in an effort to alter the stepper timing, in an effort to account for stepper microstep cogging. I suspect the Klipper step compression code may struggle to handle high speeds if step distance is not constant, but the test results may be enlightening (even if it is only possible to test at slow/mid speeds).

I think about it, often, cause it “should work”.
But I’m too stupid to do that right now (meh, this sounds silly).
It should be possible to pass phase data to cffi, keep it in sync, and modify stepcompress data after itersolve, before actual compress.

Like on the paper, we have the motor phase already and can calculate it in real-time, we have an accelerometer to validate, and we have a “possible known pattern” where we want to slow down part of the phase and accelerate another part.

Eventually, I may try to make it work, but it looks complicated.

[nefelim4ag]

• One thing I found useful was to simultaneously grab both accelerometer and angle sensor data. The two often correlated. The additional information was sometimes interesting.

You did catch me, this is what I was planning for today.

this is core XY, this is sample GCode, both motors will affect accelerometer

G0 X10 F300
G0 Y20 F600
G0 X-10 F600
G0 Y-20 F900

So, I put my printer back together to check how it looks and there are graphs:

Slow speeds 20 mm/s

Default mslut (9)
(I think all lines should be vertical, and acceleration is just mirrored against deviation, my mistake)

Custom mslut (8)

*Resonance* speeds 50 mm/s

Default mslut

Custom mslut

Higher speeds 90 mm/s

default mslut

custom

Higher speeds 150 mm/s

So, looks like microsteps provide marginal benefits on slow speeds, but no difference on higher.

4 Full Steps graphs

default mslut

custom mslut

Also, we can safely say looks like the data between the accelerometer and encoder looks similar at medium speeds. Like we see full steps on accelerometer data.

[nefelim4ag]

I accidentally blinded myself (metaphorically), did several measurements, and forgot which one was which. I can’t reliably say which one is which and which one is better.
Because one looks better at one speed, the other one is at the other, there are 3: defaults, modified 2 times, modified 3 times. God knows.

I used different code to move only one motor

SET_VELOCITY_LIMIT ACCEL=2000
G91
G0 X10 Y10 F300
G0 X10 Y10 F600
G0 X10 Y10 F900
G0 X10 Y10 F1200
G0 X20 Y20 F1500
G0 X20 Y20 F1800

G0 X-20 Y-20 F2100
G0 X-20 Y-20 F2400
G0 X-20 Y-20 F2700
G0 X-20 Y-20 F3000
G0 X-20 Y-20 F3300
G0 X-20 Y-20 F3600
G0 X-20 Y-20 F3900
G0 X-20 Y-20 F4200

G0 X40 Y40 F4500
G0 X40 Y40 F4800
G0 X40 Y40 F5100
G0 X40 Y40 F5400

G0 X-40 Y-40 F5700
G0 X-40 Y-40 F6000
G0 X-50 Y-50 F6300
G0 X-50 Y-50 F6600

G0 X50 Y50 F6900
G0 X50 Y50 F7200
G0 X50 Y50 F7500
G0 X50 Y50 F7800

G0 X-50 Y-50 F8100
G0 X-50 Y-50 F8400
G0 X-50 Y-50 F8700
G0 X-50 Y-50 F9000

I can say MSLUT does not work at least for me.

I think that’s my requiem for a dream.
The basic idea was simple: if we compensate for low forces around zero crossing points, it will cancel out motor full-step “vibrations” and the motor will work more linearly and maybe it will fix VFA.

The microstep calibration tool looks to be working, even on the mounted/loaded motor (I’m surprised here). But it can fix large deviations, if the calibrated motor is already in good shape it will fail to converge.
It is not perfect, but I’m not sure it can be perfect.
The dump/visualization tool is the same and will allow you to take a peak underneath.

---

BTW, I saw one different motor (leadshine) and data from it.

This is with tuned registers (methodology unknown to me)

And with “auto-tuned table with default registers”

---

I got hands-on TMC2240, so I will try to play with it and show something.

[nefelim4ag]

Did someone know that tmc fields.set_field() does not set field? I didn't :D

and get_field() does not really read it (I’m not sure here).

    def set_field(self, field_name, field_value, reg_value=None, reg_name=None):
        # Returns register value with field bits filled with supplied value
        if reg_name is None:
            reg_name = self.field_to_register[field_name]
        if reg_value is None:
            reg_value = self.registers.get(reg_name, 0)
        mask = self.all_fields[reg_name][field_name]
        new_value = (reg_value & ~mask) | ((field_value << ffs(mask)) & mask)
        self.registers[reg_name] = new_value
        return new_value

I was not aware of that, so I should redo some testing, and make some refactoring.

I implemented TMC2240 offset calibration, snapshot: GitHub - nefelim4ag/klipper at tmc2240-offset90-20241221
It is pretty straightforward actually.

My offset is -11 and as required slightly compressed down the MSLUT table.
(it does sometimes try to be -9 or -10)

90 -> 90 - 90/127 * 11 = 82 degrees.

*SpreadCycle registers are Klipper defaults

It looks like micro steps work better in StealthChop (because of autotuning?), maybe spread on tuned registers in an ideal situation should perform the same way.
(Tuned registers allow for the SpreadCycle to control motor current more precisely).
Phase offset on the other hand seems to make full steps similar, like 1 ~= 3, 2 ~= 4
Without phase offset here, they are more like 1~=4, 2~=3

High-level view of Spread vs Spread + Offset

20 mm/s

50 mm/s

High-level view of Stealth vs Stealth + Offset

20 mm/s

50 mm/s

Under a microscope (50mm/s)

Spread vs Spread + Offset

Stealth vs Stealth + Offset

I’m a little disappointed,
it looks like the phase offset instead of decreased motor oscillation and increased alignment makes vibrations/oscillations worse.
This how it is looks from microstep calibration:

StealthChop + Offset:
FullStep 1.8 angles: 1.73 1.88 1.71 1.83 ~ 7.15
SpreadCycle + Offset:
FullStep 1.8 angles: 1.74 1.86 1.73 1.82 ~ 7.15
Just SpreadCycle for reference:
FullStep 1.8 angles: 1.81 1.81 1.79 1.75 ~ 7.16

---

My microstep calibration tool does not work here right now, tmc2240 refuses to correctly work after updating MSLUT (even if I do reg read → reg write).

[nefelim4ag]

I said above that hypothetically tuned motor registers would look at the micro-step level similar to StealthChop. Looks like - yes.
I was not successful again with the oscilloscope and tuning, cause I can’t reliably see current without a good current probe, but I used it to validate tmc table data and end the expected chopper frequency.
So, I used ~30kHz as the target, all data below are with offset90 and TMC2240.
Interpolation enabled.

There is the remark that I drive my motor with ~75% RMS, with lower values like 40% it should be harder to get it to work reliably by table.
I didn’t verify frequency with Z motors, but I was unsuccessful in making it behave in the same manner. Motors are the same, but TMC2226 on Z is just overheating with a current above 1.2A

*All microsteps graphs scaled to same values 3…8us

Spread with tuned registers and Stealth look similar, I even worried the left graph was made with StealthChop instead of SpreadCycle, so there are 2.

So 2 spreads side by side

Without interpolation tuned spread also looks more equidistant

Stealth on the other hand, without interpolation looks a little bit twitchy

Left one with interpolation enabled,

Two to check “repeatability”, from different driver initializations.

From the linear motion side, I did tests again to compare “new” data.

I did want to say, that it makes something better, but there are resonances and it will look better or worse, depending on which speed range we are looking at.

Okay, we move slowly, it really looks better.

Then faster, no difference.

There, where strange things start to appear, a little speed difference produces a large accelerometer swing.

And there again the difference is small.

I can’t say that it is possible to tune driver registers by microstep values or by accelerometer data. There is definitely a correlation, and good-tuned drivers should behave similarly to stealth and be closer to the “ideal” motor.

It would be nice to tune drivers and/or have an automatic way to reliably do so, set offset90 on TMC2240, etc. But it mostly looks marginal from a vibration perspective to my taste. Precision - maybe, but microsteps are barely visible during motion, and equidistance maybe important for very slow travel with a low gear ratio.

I will rephrase this, they of course make a difference and they do change the behavior of motor.
But my end goal was to make rotation linear and the motor closer to the “ideal” one if it is possible. For now, for me with what I have it looks like an unreliable way to do so, and changes only shift stuff around, but not like really make it linear.
Maybe someone with motors with higher inductance will experience different behavior, I hope so. Cause inductance will add some inertia to the current flow and the motor - I don’t know.
(I have LDO 2504 with 1.5mH).

So, some sort of shaper is needed here to counterforce current motor behavior at low/medium speeds. Because it possibly can cancel torque lows and highs during full-step switching, and this shaper should be highly coupled to the motor/driver and its phase.

---

Just a small notice on the current shaper and accel to deacceleration phase:

ADXL data looks awesome.
Because of while acceleration/deacceleration phase there is rotor slippage (the rotor will overrun the target position while deacceleration and will lag behind while accelerating) there is a minimal amount of full step artifacts, I think so, maybe slippage here is just a little too low.

[nefelim4ag]

Okay, I made the stepper input shaper, snapshot branch is here: GitHub - nefelim4ag/klipper at stepper-shaper-20250112

So, basic approach for now:

• Assume there are patterns at a stepper step level
• It should be synced into steps
• Shift around step timings to compensate for that pattern

Cause it is stepper and stepper phase-specific, it is done after input shaper and iter solver, just before step compress kicks in.
So, it is defined and tuned per stepper.

It is PoC, I think there at least should be some speed-related transformations and tuning.

I specifically used this to test cause I think it is easier to cancel out:

It is TMC2240 + offset90, so there are peaks on even full steps, and odd full steps are slightly shorter and weaker.
(also there is max_stepper_error: 0.0000025 at MCU, to force step compress to follow my small adjustments because otherwise there is something similar to meander)
There are parabolic functions underneath, and 3 params: period, amplitude, and offset.
Assume the first half and second half are the same here, so period = 512.
Offset is the virtual middle point, the middle point should be stretched and corners compressed.
Offset is where the most amplitude lies. 3/4 ~ 384 should fit.
Amplitude is multiplayer here, to control the power of changes.

I specifically chose the noisy piece of graph, where the motor seems to resonate, and do several iterations changing params till it looks better.

There are period 512, amp: 10, offset: 384

There are amp: 5, offset: 360

On lower speeds - there are no changes so far

Let’s try a different one, there are the same registers, but offset90 is disabled.
It looks just noisy and I’m not sure it can be canceled that simple, like there is a little difference between high/low.

First attempt, there are compression artifacts, Let tune max_stepper_error

Second, with `max_stepper_error: 0.000001`

Third, increased amplitude, not much success, only one peak is canceled

Fourth, no difference:

This is how it currently behaves with the current input shaper (btw, the stepper shaper lost sync here):

---

I don’t like things where there are a lot of configurations, it sounds like:
“If you don’t know how to configure it, how is the user supposed to do so?”

So, for now, this is completely dev/experimental stuff.
I think if I anyway compute the array to shift around step points, maybe I can do the same with the use of accelerometer data, screw the micro steps, and we have a repeated pattern, technically with this, it is possible to feed the opposite signal literally.

There are samples from different speed ranges, and there are different patterns, but they are similar to each other

---

I use sinf() to generate values, 2 sin rotations per 4 full steps. shifted by 128 (1/4) to make it opposite to vibration profile:

This one looks amazing

Also good

Medium speeds just got a little worse

[olhev]

Can you see any impact on print quality?

[nefelim4ag]

No,
this code is not near the phase where it can be tested on real print.

Ironically, in the end, I reinvented the wheel from this thread: Motion analysis by stepper phase
I read about it some time ago and have completely forgotten it since May.