Change to coordinates exported as status variables

Developers
1

I think it may be necessary to change the behavior of some of the status variables available to macros and on the API Server. For example, it may be necessary to change the format of {printer.gcode_move.position}.

This message is to gather feedback on how we should go about making a change like this. @pedrolamas, @meteyou, @arksine, @dmbutyugin, I’d be interested in hearing your thoughts.

We recently added support for “extra gcode axes” to the Klipper code ( Support for additional G1 axes (6-axis support) by KevinOConnor · Pull Request #6888 · Klipper3d/klipper · GitHub ). Previously, regular G1 style moves contained four coordinates - XYZE. However, now a G1 move can have many additional axes (eg, XYZERPNU) - the number of axes and their letter designations are user defined and can change at run-time. Currently, it is possible to map “manual_stepper” motors to these additional axes, but @dmbutyugin is looking to extend that support to additional extruders and additional kinematic carriages ( Support of mapping dual_carriage(s) and extruders to custom GCode axes ).

We currently export coordinates in status reports today in a somewhat quirky way. We typically export coordinates using a Python “named tuple” - examples include gcode_move.position, gcode_move.homing_origin, gcode_move.gcode_position, toolhead.position, toolhead.axis_min, toolhead.axis_max, motion_report.live_position. A “named tuple” allows the internal code and macros to access the coordinates by name (eg, {printer.gcode_move.position.x}), but on the API Server the coordinates are transmitted as a list (eg, { "gcode_move": { "position": [1.0, 2.0, 3.0, 4.0] }, ... }).

Although the code fully supports additional axes today, it does not currently export their positions. Going forward we would like to export this information, however it isn’t clear what is the best way to do that. Extending the “named tuple” isn’t going to work well for extra axes because there isn’t any meaningful order to these axes (reporting them as a list doesn’t make sense). Also, the full axis name can be quite long and contain spaces (eg, “manual_stepper my_extra_stepper”).

I have a proposal for how we could make this change:

  1. The following status variables will continue to use an internal “named tuple”, but only provide XYZ in the future: gcode_move.homing_origin, toolhead.axis_min, toolhead.axis_max, toolhead.position, motion_report.live_position. For the first three of these, the E field was always zero anyway, so this shouldn’t be a significant change. To obtain the toolhead position of extruder (and other extra axes), we could add a new {'toolhead': {'extra_axes': ['extruder', 'manual_stepper my_stepper']}} list and export the current position in these classes (eg, {printer.exruder.position}). For motion_report we could introduce a new field for extra axes: {'motion_report': {'live_position_extra_axes': {'extruder': 123.4, 'manual_stepper my_stepper': 56.7 } } }. To facilitate a conversion, these fields can continue to report XYZE for several months prior to their removal.

  2. The remaining status variables would be replaced with alternative variables that use mappings (instead of named tuples). So, gcode_move.position would be replaced with gcode_move.axis_position - for example: {'gcode_move': { 'axis_position': {'x': 1.0, 'y': 2.0, 'z': 3.0, 'e': 4.0, 'r': 99.9}}}. Similarly, gcode_move.gcode_position could be replaced with gcode_move.gcode_axis_position. Using new status variable names could allow us to provide both versions for a few months during a phaseout period.

One challenge with making a change like this is that there isn’t a good way to provide automatic deprecation notices to users from within Klipper itself. At some point I think it would be worthwhile to remove the old support, and when we do so there’s a good chance it will catch at least some users unaware.

Anyway, does the above seem like a viable plan? Are there other approaches that should be considered?

-Kevin

1 Like
2

@koconnor thx for the pin and the detailed description. I’ve been wondering why the positions are only in an array without the axis names and i think its a good improvement to transfer these data including the names via API.

As long as the old and new data exist in parallel for a certain period of time (at least the 28 days that Moonraker has in its update interval) so that clients/guis can use a fallback to the old structure, I don’t see any problem with this change.

I think the second variant has the benefit, that all “fast changing data” are in one “object” (gcode_move). so for guis with a small cache before the state manager, you only have to exclude this object from the cache. But I have no problem with both of them. Both looks fine to me.

3

Thank you for the detailed explanation @koconnor

I do like the gcode_move.axis_position approach, but I wonder if we can do the same for toolhead and motion_report (eg. having toolhead.axis_position and motion_report.axis_position instead of having to read the position of each of the extra axis)?

Other than that, I think this should be relatively easy change for us in the frontends.

4

The proposed changes look good to me. We might want to consider mapping all “extra axes” to a shorthand name. These could be set by configuration options that default to auto-generated values. We could then provide an API endpoint that returns a map of shorthand names to full names.

This would reduce the amount of data serialized when consumers subscribe to motion_report.live_position_extra_axes. We could also consider having the toolhead report the positions directly for extra axes, ie {'toolhead': {'extra_axes': {'e0': 4.0, 'e1': 3.8, 's7': 100.85}}}.

With regard to deprecation notices we can probably come up with a way for Moonraker to facilitate that. In the short term I can create an announcement after the changes have been merged as this would largely affect Moonraker’s API consumers…although I suspect it could impact a number of gcode macros as well.

5

Thanks, that’s an interesting idea.

Internally, we do have a “short identifier” called the ea_index. We could expose that externally.

Here’s an alternate proposal:

  • toolhead.position, motion_report.live_position, gcode_move.position, gcode_move.gcode_position would still go out as lists on the API Server, but those lists may contain more than 4 values. That is, if an extra axis is added, then the above may contain 5 or more values. For example: {"toolhead": {"position": [1.0, 2.0, 3.0, 7.3, 19.2]}}

  • A new toolhead.extra_axes value would be exported containing a mapping from object name to “position index”. For example: {"toolhead": {"extra_axes": {"extruder": 3, "manual_stepper my_stepper": 4}}}. Macro and API users that know the full name of a object can find its position with something like {printer.toolhead.position[printer.toolhead.extra_axes.extruder]}.

  • A new gcode_move.axis_map map would be exported containing a mapping from gcode axis identifier to “position index” - for example: {"gcode_move": {"axis_map": {"x": 0, "y": 1, "z": 2, "e": 3, "r": 4}}}. Users that know the gcode id can then lookup its position with something like {printer.toolhead.position[printer.gcode_move.axis_map.r]}.

  • To facilitate macros, we can replace the internal Coord() Python named tuple with our own wrapper class. That wrapper class can continue to provide XYZE names, as well as support additional axes by index. Thus macro users can continue to use {printer.gcode_move.gcode_position.x}, but only for XYZE axes. For any of the “extra axes” macro users would need to use the more verbose format (eg, {printer.gcode_move.gcode_position[printer.gcode_move.axis_map.r]}).

  • We can deprecate the 4th entry for gcode_move.homing_origin, toolhead.axis_min, toolhead.axis_max. The 4th (‘e’) entry is always zero on these fields anyway. A few months after deprecation we can send these values using just a 3 entry list (instead of 4 today).

  • We can deprecate the use of coord.e in macros and ask users to use the longer form for that coordinate (eg, {printer.toolhead.position[printer.gcode_move.axis_map.e]}. We can keep the XYZ index helpers (eg, coord.y) as they are unlikely to change.

So, basically, if we export the “name to position index” mappings then we can continue to send positions as lists of numbers.

The advantage of this plan is that it is less of a change to the current behavior. It also likely to use less bandwidth on the API Server (the verbose toolhead.extra_axes and gcode_move.axis_map rarely change so don’t need to be frequently transmitted). It is slightly more involved for macro users though - at least when accessing the extra axes.

Thoughts?
-Kevin

6

The new proposal should be much easier on developers, I suspect most current macros and API clients will continue to work without changes.

The only minor thing that comes to mind is that we need to make sure the new Coord wrapper serializes to json. We could do it with an object hook, but its probably easier to just have the wrapper subclass tuple.

7

FYI, I got that working earlier today.

--- a/klippy/gcode.py
+++ b/klippy/gcode.py
@@ -1,15 +1,26 @@
 # Parse gcode commands
 #
-# Copyright (C) 2016-2024  Kevin O'Connor <kevin@koconnor.net>
+# Copyright (C) 2016-2025  Kevin O'Connor <kevin@koconnor.net>
 #
 # This file may be distributed under the terms of the GNU GPLv3 license.
-import os, re, logging, collections, shlex
+import os, re, logging, collections, shlex, operator
 
 class CommandError(Exception):
     pass
 
-Coord = collections.namedtuple('Coord', ('x', 'y', 'z', 'e'))
+# Custom "tuple" class for coordinates - add easy access to x, y, z components
+class Coord(tuple):
+    __slots__ = ()
+    def __new__(cls, x, y, z, e, *args):
+        return tuple.__new__(cls, (x, y, z, e) + args)
+    def __getnewargs__(self):
+        return tuple(self)
+    x = property(operator.itemgetter(0))
+    y = property(operator.itemgetter(1))
+    z = property(operator.itemgetter(2))
+    e = property(operator.itemgetter(3))
 
+# Class for handling gcode command parameters (gcmd)
 class GCodeCommand:
     error = CommandError
     def __init__(self, gcode, command, commandline, params, need_ack):

Cheers,
-Kevin

2 Likes
8

FYI, the “alternate proposal” was committed to the main Klipper repository ( Export location of "extra axes" by KevinOConnor · Pull Request #7086 · Klipper3d/klipper · GitHub ).

Cheers,
-Kevin