Manual heater PID constant derivation by example

Background

Many people (myself included in my early 3D printing days) implicitly expect off-the-shelf near “perfection” of the heater performance of their printer. It takes time and practice to learn and appreciate that PID controller tuning is a largely empirical process without a one size fits all solution. Different printer firmware uses different implementations of the PID controllers and different methodologies to derive (approximate) the process PID constants. When this is combined with a practically infinite number of printer designs and configurations, corner cases exist where the predefined solutions can struggle or falter. The purpose of this post is to document a reasonably simple to follow approach that will allow people to experiment with manual tuning of the Klipper PID controller in those corner cases.

Basic Process Steps

  • Establish typical average PWM drive value for the heater being characterized
  • Reconfigure Klipper printer.cfg to allow manual open-loop control of the heater
  • Perform an open-loop step response test of the ‘process’
  • Use the data stored in klippy.log to model key process parameters: process gain, process time constant, process dead time
  • Use the key process parameters to derive Klipper PID constants using one of the many available tuning techniques, such as Ziegler-Nichols, Cohen-Coon, etc.
  • Verify heater response and stability using the manually derived PID constants

Detailed Procedure

The following summary outlines the steps that I took to manually derive the PID constants for my heavily modified CR-10S Pro heated bed.

  1. Establish typical average PWM value for heater when printing. For example, set bed to 70C, allow it to come up to temperature and stabilize, then note the average PWM value. This is your <TARGET_PWM> value. In my example we will use 25% or <TARGET_PWM> value of 0.25, because this is roughly the average PWM value that my bed uses to control at around 70 degrees C. This will give process gain specific to your selected target temperature.

  2. Modify your Klipper printer configuration in printer.cfg to comment out the [heater_bed] block. Define new [fan_generic] block using the same output pin. This will allow you to enter PWM drive values for the bed manually in an open-loop fashion. Define new [temperature_sensor] block. This will allow you to monitor and log the bed temperature. In my example using a Duet 3 Mini 5+ the configuration is as follows:

    #[heater_bed]
    #heater_pin: PB17  # OUT0
    #sensor_type: EPCOS 100K B57560G104F
    #sensor_pin: vref_scaled:PC0  # TEMP0
    #pullup_resistor: 2200
    #control: pid
    #pid_Kp = 51.75
    #pid_Ki = 0.823
    #pid_Kd = 813.2
    #min_temp: 0.0
    #max_temp: 110.0
    
    [fan_generic bed_power]
    # for manual open-loop PID testing only
    pin: PB17  # OUT0
    
    [temperature_sensor Heater_Bed]
    # for manual open-loop PID testing only
    sensor_type: EPCOS 100K B57560G104F
    sensor_pin: vref_scaled:PC0  # TEMP0
    pullup_resistor: 2200
    min_temp: 0.0
    max_temp: 110.0
    
  3. Restart Klipper and home your printer - homing will activate the steppers and ensures that Klipper data logging remains active throughout this exercise. Then set initial bed PWM by entering the following Klipper console commands at the same time (web interfaces like Fluidd allow multi-line console commands to be sent at once):

    SET_IDLE_TIMEOUT TIMEOUT=<MARKER>
    SET_FAN_SPEED FAN=bed_power SPEED=<START_PWM>
    

    where <MARKER> is a unique number that’s easy to correlate later to the PWM value, and where <START_PWM> is 5% lower than <TARGET_PWM>, or <START_PWM> = <TARGET_PWM> - 0.05. In my example we will use the following commands:

    SET_IDLE_TIMEOUT TIMEOUT=10020
    SET_FAN_SPEED FAN=bed_power SPEED=0.20
    

    Allow the bed temperature to completely stabilize, noting this may take a very long time (1/2 hour to an hour, or even longer).

    !!! WARNING !!!
    !!! DO NOT UNDER ANY CIRCUMSTANCES LEAVE THE PRINTER UNATTENDED !!!

    Normal closed-loop heater safety features of Klipper do not work in this configuration because Klipper is configured to power the heater as if it was a manually controlled fan!

  4. Once bed temperature is completely stable and no longer changes, increase the bed PWM by entering the following Klipper console commands at the same time:

    SET_IDLE_TIMEOUT TIMEOUT=<MARKER>
    SET_FAN_SPEED FAN=bed_power SPEED=<TARGET_PWM>
    

    In my example we will use the following commands:

    SET_IDLE_TIMEOUT TIMEOUT=10025
    SET_FAN_SPEED FAN=bed_power SPEED=0.25
    

    Allow the bed temperature to increase and completely stabilize. Once again, this may take a very long time.

  5. Enter the following command: SET_FAN_SPEED FAN=bed_power SPEED=0 then restore your original Klipper printer configuration in printer.cfg.

  6. Download klippy.log and locate the heater_bed PWM drive commands and heater_bed temperature log. You will need to extract the following column entries from the log file:

    Stats 746372.8: this is the MCU time reference in seconds
    Heater_Bed: temp=55.2 this is our custom defined open-loop Heater_Bed temperature

    You will also need to locate and note the time markers when the PWM commands were issued, particularly the second command that introduced 5% PWM step input. These markers will look similar to this in klippy.log and will be present in the log file at the appropriate MCU times:

    idle_timeout: Timeout set to 10025.00 s

    In this case the bed_power PWM control was set to 0.25 per the <MARKER> value selected earlier.

  7. Create a spreadsheet with the data extracted above. The spreadsheet will need TIME, POWER and TEMP columns. For example, it may look like this:

    MCU TIME TIME POWER TEMP
    [sec] [sec] [%] [deg.C]
    234317.3 0.0 0.20 68.2
    234318.3 1.0 0.20 68.2
    234319.3 2.0 0.20 68.1
    234320.3 3.0 0.20 68.2
    234321.3 4.0 0.20 68.1
    234322.3 5.0 0.20 68.1
    234323.3 6.0 0.20 68.2
    234324.3 7.0 0.20 68.1
    234325.3 8.0 0.20 68.2
    234326.3 9.0 0.20 68.1
    234327.3 10.0 0.20 68.1
    234328.3 11.0 0.20 68.2
    234329.3 12.0 0.20 68.1

    The POWER [%] column will need to be populated manually based on the power command time stamps extracted from klippy.log.

  8. Open https://pidtuner.com/ in your browser and complete the initial three steps of the process:

    (1) ‘Import Data’, noting that ‘Input’ is the manual PWM command (POWER [%]) and ‘Output’ is the temperature (TEMP [deg.C]).

    (2) ‘Select Step’ to pick a subset of the recorded data that corresponds to good steady initial temperature and fully stabilized final temperature

    (3) ‘Select Model’ 1st Order model type and confirm good fit between your collected data (blue) and the model prediction (orange). Record the model fit parameters highlighted below: k (process gain), tao (process time constant), theta (process dead time).

  9. Download the Control Specialists Loop Tuning Calculator spreadsheet from https://controlspecialists.co.uk/wp-content/uploads/2017/08/Control-Specialists-Ltd-Loop-Tuning-Calculations-V4.5.xlsx. Alternatively, if you would like to derive the PID constants by yourself, there are many references and papers available that describe how to obtain the PID constants from process parameters.

  10. Enter the process model parameters k, tao and theta into KP, TC and DT cells of the SELF REGULATING LOOP RULES spreadsheet as highlighted below. If desired, adjust the SAFETY FACTOR. Collect values of P (Gain), I (sec/rep) and D (secs) from the PID row of the chosen tuning technique (for example COHEN COON as highlighted below). NOTE: The calculated values of the PID constants are very sensitive to the value of the process dead time (theta or DT), because dead time is “the enemy” of closed loop control. It is therefore very important to establish the dead time as accurately as possible. The value can be confirmed manually by looking at the raw data - see Process Parameters section below.

  11. Calculate your PID constants to use in Klipper printer.cfg as follows:

    pid_Kp = P (Gain) * 255 _______________ 0.83 * 255 = 211.7 in this example
    pid_Ki = P (Gain) * 255 / I (sec/rep) _____ 211.7 / 13.5 = 15.68 in this example
    pid_Kd = P (Gain) * 255 * D (secs) ________ 211.7 * 2 = 423.4 in this example

    Note that the derived PID constants must all be multiplied by 255, as shown above, in order to be entered in the printer.cfg file. Refer to PID_PARAM_BASE in Klipper source code for more information.

  12. Test your new PID constants by modifying printer.cfg and heating the bead to a desired temperature. Note the behaviour of the temperature and especially PWM power commanded by the Klipper PID loop. Below is a comparison of the PID loop performance of my printer bed with Klipper auto-tuned PID constants and PID constants derived manually as described above:

// Klipper auto-tune PID parameters: pid_Kp = 70.14 pid_Ki = 1.218 pid_Kd = 1010.1

// Manual Cohen-Coon PID parameters: pid_Kp = 211.7 pid_Ki = 15.68 pid_Kd = 423.4

It is worth noting that the manually tuned PID constants result in substantially tighter control, as evidenced by virtually no temperature overshoot (0.1 deg.C in practice) and very aggressive power control. As shown above I obtained those using SAFETY FACTOR of 1. In some cases such aggressive loop could become unstable resulting in temperature and power oscillations. In those cases a higher safety factor value would be appropriate, for example 1.5. Feel free to experiment with your specific setup - the beauty here is that once you establish the thermal model of your setup, you can derive the PID constants using any technique of your preference and with your chosen safety factor (or effectively stability margin).

Process Parameters

The following graph illustrates the meaning of the three key process parameters: gain (change in process variable PV divided by change in controller output CO), time constant (time it takes PV to reach 0.63 of total PV change, shown as tau and calculated from the end of the dead time) and dead time (time it takes from the change in CO to the point where the maximum PV slope line intersects the original level of PV line, shown as td). Unfortunately I do not recall where I came across it, so I am unable to credit the source.

Additional Notes

I have not spent much time experimenting with the hot end PID constants because in my case the Klipper auto-tuned values work very well. In principle, identical approach can be used for the hot end but it will require some more creativity in editing printer.cfg because Klipper will not function without a defined & functional [extruder] block.

Revisions

2022-04-07: Revised content of steps 7 to 10 in order to fix my incorrect interpretation of PID_PARAM_BASE. The overall method and final results are not affected by this revision.

2022-04-11: I seem to have experienced a “senior moment” when compiling step 6. Despite copying the content of my own notes from several months ago, I am unable to reproduce the presence of Received 746305.064772: {"id": 1750268944, "method": "gcode/script", "params": {"script": "SET_FAN_SPEED FAN=bed_power SPEED=0.25"}} in the logs. Furthermore, I am not sure how my original logs included these stamps since they originate from webhooks.py I therefore revised steps 3 to 6 with a workaround that inserts time stamps in the logs using the SET_IDLE_TIMEOUT command.

2023-10-14: Included a note in step 10 to emphasize the high sensitivity of the calculated PID constants to the value of process dead time (theta / DT). Added an illustration visualizing the process parameters for those who wish to derive them manually, instead of using https://pidtuner.com/.

2024-01-14: Revised step 3 to include printer homing to keep data logging active.

8 Likes

Hi ReXT3D. I’m trying to follow your guide but when I look at my klippy.log there is a large amount of time stats missing. It seems to only record 60s after the timeout command is issued. Is there some setting I need to change to get the klippy.log to record everything?

Below is my klippy.log and the I used.

idle_timeout: Timeout set to 10022.00 s

idle_timeout: Timeout set to 10027.00 s

klippy.log (2.7 MB)

Hi and welcome to the community.

The best thing to do to prevent the logging from pausing would be to home the printer just before you issue the timeout command in order to enable the steppers. Another alternative would be to enable the hot end heater at some low temperature (like 50C). Either of these will force Klipper to continue logging every second.

I should perhaps add this to the original procedure…

Ah, ok. I’ll do that. Thank you.

Excel Link failure

Wayback Machine (a.k.a. Internet Archive) is often your friend:

http://web.archive.org/web/20220407202544/https://controlspecialists.co.uk/wp-content/uploads/2017/08/Control-Specialists-Ltd-Loop-Tuning-Calculations-V4.5.xlsx

But I would generally recommend that anyone going to this level of PID loop tuning detail to try doing it by hand :slight_smile:

Ah! I should have read the entire thread first. I too had only 60 seconds of logs for each idle_timeout. I set my extruder temp to 30C and started again. Logs look to be logging more than 1 minute now. Maybe add this to step 3? Thanks for the tutorial BTW. I will report back my results when I finish.

1 Like

Success! It is so much better than before, although there is still a big overshoot, it settles down to the setpoint and controls very well. Before it wouldn’t really settle back down to the setpoint and kept adding power.

Before:

pid_Kp: 28.414
pid_Ki: 0.071
pid_Kd: 2838.589

After:

pid_Kp: 26.78314
pid_Ki: 0.307901
pid_Kd: 347

That’s great, glad that manual tuning helped your situation. Also thanks for the reminder about truncated logging - I forgot to revise step 3 but just did it now.

You may be able to bring the degree and duration of overshoot down by additional hand tweaking of the PID constants. For example, you can try to increase your derivative term and then evaluate the results - don’t be shy and double the value, for example. Just make sure you don’t leave the printer unattended when you do this.

I would like to emphasize again how important is the correct value of the process dead time to the values of the PID constants. I personally ALWAYS derive the dead time myself “by hand”.

Yes, I looked at the raw data to validate the dead time. I came up with pretty much the same number, but there is a range to choose from. Would longer or shorter in that range be better? I will experiment with larger pid_Kd values and see what that does.

Longer dead time would generally mean less tight / more relaxed PID control as the control loop needs to “chill out” waiting for the response from the system. I was just recommending that you are not afraid to experiment a bit since tuning control loops is not exact science - most methods are largely empirical.