Way to post/pre process gcode through moonraker?

I’m currently trying to modify the spoolman extras module in order to enhance it’s capabilities for my needs.

  • I use the location in the db to store the host_name:slot_number and treat all printers i own as if they had an MMU system with between 1 and N slots.
  • I added lots of pre-print checks in order to avoid launching prints with bad slot configuration, not enough filament left (which is now native in spoolman i think) …
  • I made sure the spoolman db and the orca-slicer db are consistent so that i can manage all these filaments simply by adding them through the slicer.

That’s the context… I’m now trying to add a functionality that would enable dynamic relocation of a spool when launching a print, thus i need:

  • if used spools are not on correct slots, create a lookup table that maps wanted spool on correct slot :white_check_mark:
  • use this newly generated lookup table to modify the print behavior :x:

For this i thought of multiple solutions such as:

  • replacing all TN gcode calls with T{lut[N]} in the original file
  • replacing all TN gcode calls with T{lut[N]} in the loaded gcode (virtual sdcard object ?)
  • dumping the lookup table somewhere on the disk and access it via the TN macros to change the slot number at macro call.

I’m not really sure which way i should go and not really sure if I’m not missing something here.

Thanks in advance for any help, i will try some methods and keep you posted.

I’m doing something very much like this. I’ve got a 6-in-1-out setup so I have 6 filament “slots.” Each one has an extruder_stepper_* that gets synced with the extruder motion queue as needed via macros.

I have Klipper set up with save_variables, and one of the persistent variables I use is spool_registry, which is a dictionary of dictionaries, where the keys are the slot numbers and the values are dictionaries containing the filament names and spool numbers of the filaments that are currently loaded into those slots.

I hacked up the KlipperScreen module for Spoolman so it displays the 6 slots across the top and I can tap a slot to make it active, then pick a spool from the selector to map that spool to that slot. The visual is updated and it triggers a Klipper macro that updates the spool_registry dictionary accordingly.

I have custom filament loading macros that take the target filament name as a parameter from the slicer. Also, as part of my PRINT_START macro, it will take the names of every filament to be used in the print and check the spool_registry dictionary to make sure each needed filament is loaded into at least one slot. If not it will throw and warning and not start the print.

When it comes time to load a filament, the LOAD_FILAMENT macro checks spool_registry again and then activates the first slot in which it finds a filament with the requested name, by syncing extruder_stepper_{slot_number}. It will also activate the associated spool number in Spoolman to track usage. This makes it possible to relocate filament during the print, or even swap out another spool of the same filament seamlessly, as long as the changes are reflected in the spool_registry by using the hacked-up KlipperScreen menu.

I think the main difference from your setup is that I’m using Klipper’s save_variables function rather than Spoolman’s location variable to keep track of what’s where. But my spool_registry persistent variable sounds like your idea:

dumping the lookup table somewhere on the disk and access it via the TN macros to change the slot number at macro call

Ok nice, i understand what you are going for.
I’m going to keep track of the location through the location in the spoolman field anyways as i find it very usefull to track which spools are where in the farm via the spoolman web interface.

I think your solution can be implemented for me as follows:

  • dump the spool dict into a spools_generated.cfg file like {‘1’ : {‘spoolA’ : spool features …}, …‘N’ : ‘spoolN’ …}
  • use this dict at macro call to swap the spools indexes at runtime when calling TN

I like your approach as i might need to add some other features to the spools dict later.

I don’t really like moving the filament high level information to the klipper side but as it is build now i cannot use orca slicer to model filaments the way i would like …

I started a discussion on the gihub (filament modelling enhancement (ie for Pressure Advance) · SoftFever/OrcaSlicer · Discussion #4574 · GitHub) about it if you want to take a look. I might use this approach for swapping spools but i would prefer not to move too much filament db stuff to klipper if i can.

Thanks for your reply, i’ll keep you posted if need be.

Right. My variables.cfg includes (at the moment):

spool_registry = {'1': {'color': None, 'id': None, 'name': None}, '2': {'color': None, 'id': None, 'name': None}, '3': {'color': '3B7501', 'id': '21', 'name': 'Solutech PLA Green'}, '4': {'color': 'EC0000', 'id': '39', 'name': 'Elegoo PLA Red'}, '5': {'color': 'FFFFFF', 'id': '38', 'name': 'Elegoo PLA White'}, '6': {'color': 'A0A0A0', 'id': '31', 'name': 'Elegoo PLA Grey'}}

Here is a set of macros that are only used by KlipperScreen to reflect changes to spool_registry made from my hacked-up KS panel:

[gcode_macro _UNASSIGN_SPOOL]
    {% set spool_registry = printer.save_variables.variables.spool_registry %}
    {% set slot = params.SLOT %}
    {% for param in spool_registry[slot] %}
        {% set dummy = spool_registry[slot].__setitem__(param, None) %}
    {% endfor %}
    SAVE_VARIABLE VARIABLE=spool_registry VALUE="{spool_registry | pprint | replace("\n", "") | replace("\"", "\\\"")}"
    _SET_TOOL_COLOR TOOL={slot|int - 1} COLOR=None

[gcode_macro _ASSIGN_SPOOL]
    {% set spool_registry = printer.save_variables.variables.spool_registry %}
    {% set slot = params.SLOT %}
    {% set spool_id = params.SPOOL_ID %}
    {% set spool_name = printer['gcode_shell_command spoolman_client'].last_output | replace("\n", "") %}
    {% set dummy = spool_registry[slot].__setitem__('id', spool_id) %}
    {% set dummy = spool_registry[slot].__setitem__('name', spool_name) %}
    {% set color = params.COLOR|default(None) %}
    {% set dummy = spool_registry[slot].__setitem__('color', color) %}
    SAVE_VARIABLE VARIABLE=spool_registry VALUE="{spool_registry | pprint | replace("\n", "") | replace("\"", "\\\"")}"
    _SET_TOOL_COLOR TOOL={slot|int - 1} COLOR={color}

[gcode_macro SPOOLS]
variable_slot: None
variable_spool_id: None
    {% set spool_registry = printer.save_variables.variables.spool_registry %}
    {% set globals = printer["gcode_macro GLOBALS"] %}
    {% set slot = params.SLOT %}
    {% if params.REMOVE %}
        _UNASSIGN_SPOOL SLOT={slot}
    {% else %}
        {% set spool_id = params.SPOOL_ID %}
        {% for slot in spool_registry %}
            {% if spool_registry[slot]['id']|int == spool_id|int %}
                _UNASSIGN_SPOOL SLOT={slot}
            {% endif %}
        {% endfor %}
        RUN_SHELL_COMMAND CMD=spoolman_client PARAMS={params.SPOOL_ID}
        _ASSIGN_SPOOL SPOOL_ID={params.SPOOL_ID} SLOT={slot} COLOR={params.COLOR|default(None)}
    {% endif %}
    # Dummy argument block for Mainsail
    {% set dummy = None if True else "
    {% set dummy = params.SLOT|default(None)|int %}
    {% set dummy = params.SPOOL_ID|default(None)|int %}
    {% set dummy = params.REMOVE|default(False)|bool %}
    " %} # End argument block for Mainsail

Then I call these macros from my PRINT_START and FILAMENT_CHANGE macros:

[gcode_macro SET_ACTIVE_SPOOL]
  {% if params.ID %}
    {% set id = params.ID|int %}
    {action_respond_info("Spool %s is now active."|format(id))}
  {% else %}
    {action_respond_info("Parameter 'ID' is required")}
  {% endif %}

[gcode_macro _SET_ACTIVE_SPOOL]
    {% set spool_registry = printer.save_variables.variables.spool_registry %}
    {% set globals = printer["gcode_macro GLOBALS"] %}
    {% if globals.active_slot is not none %}
        {% set id = spool_registry["%s"|format(globals.active_slot)]['id'] %}
        SET_ACTIVE_SPOOL ID={id}
    {% else %}
        {action_respond_info("No slot selected. Could not set active spool.")}
    {% endif %}

[gcode_macro CLEAR_ACTIVE_SPOOL]

    {% set name = params.NAME %}
    {% set globals = printer["gcode_macro GLOBALS"] %}
    {% set spool_registry = printer.save_variables.variables.spool_registry %}
    {% set filament_list = [] %}
    {% for slot in spool_registry %}
        {% set dummy = filament_list.append(spool_registry[slot]['name']) %}
    {% endfor %}
    {% if name in filament_list %}
        { action_respond_info('Requested filament "%s" found.'|format(name))}
    {% else %}
        { action_raise_error('Load "%s" filament in a spool slot and try again!'|format(name)) }
    {% endif %}

Oh, I almost forgot. This also depends on the shell_command addon for Klipper and this block in printer.cfg:

[gcode_shell_command spoolman_client]
command: python3 /home/pi/spoolman-python-client/main.py get_spool --id
timeout: 30.
verbose: True

and it’s a version of gcode_shell_command.py that I tweaked to make the output of the shell command accessible to Klipper via printer['gcode_shell_command spoolman_client'].last_output:

# Run a shell command via gcode
# Copyright (C) 2019  Eric Callahan <arksine.code@gmail.com>
# This file may be distributed under the terms of the GNU GPLv3 license.
import os
import shlex
import subprocess
import logging

class ShellCommand:
    def __init__(self, config):
        self.name = config.get_name().split()[-1]
        self.printer = config.get_printer()
        self.gcode = self.printer.lookup_object('gcode')
        cmd = config.get('command')
        cmd = os.path.expanduser(cmd)
        self.command = shlex.split(cmd)
        self.timeout = config.getfloat('timeout', 2., above=0.)
        self.verbose = config.getboolean('verbose', True)
        self.proc_fd = None
        self.partial_output = ""
        self.last_output = None
            "RUN_SHELL_COMMAND", "CMD", self.name,

    def _process_output(self, eventime):
        if self.proc_fd is None:
            data = os.read(self.proc_fd, 4096)
        except Exception:
        data = self.partial_output + data.decode()
        if '\n' not in data:
            self.partial_output = data
        elif data[-1] != '\n':
            split = data.rfind('\n') + 1
            self.partial_output = data[split:]
            data = data[:split]
            self.partial_output = ""
        self.last_output = data

    cmd_RUN_SHELL_COMMAND_help = "Run a linux shell command"
    def cmd_RUN_SHELL_COMMAND(self, params):
        gcode_params = params.get('PARAMS','')
        gcode_params = shlex.split(gcode_params)
        reactor = self.printer.get_reactor()
            proc = subprocess.Popen(
                self.command + gcode_params, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        except Exception:
                "shell_command: Command {%s} failed" % (self.name))
            raise self.gcode.error("Error running command {%s}" % (self.name))
        if self.verbose:
            self.proc_fd = proc.stdout.fileno()
            self.gcode.respond_info("Running Command {%s}...:" % (self.name))
            hdl = reactor.register_fd(self.proc_fd, self._process_output)
        eventtime = reactor.monotonic()
        endtime = eventtime + self.timeout
        complete = False
        while eventtime < endtime:
            eventtime = reactor.pause(eventtime + .05)
            if proc.poll() is not None:
                complete = True
        if not complete:
        if self.verbose:
            if self.partial_output:
                self.partial_output = ""
            if complete:
                msg = "Command {%s} finished\n" % (self.name)
                msg = "Command {%s} timed out" % (self.name)
            self.proc_fd = None

    def get_status(self, eventtime):
        return {'last_output': self.last_output}

def load_config_prefix(config):
    return ShellCommand(config)

Then I have an overengineered python script that takes a spool ID as an argument and returns the name of the filament from Spoolman, which Klipper uses to populate the spool_registry and to check whether the filament requested by the slicer is loaded into a slot (since the slicer doesn’t know or care about spool IDs and it shouldn’t matter as long as some spool of the correct filament is loaded somewhere).

It’s funny, I saw this post the other day and was planning to respond but simply hadn’t had time yet. My thought was that most of this information would be better handled and tracked on the firmware side rather than the slicer side to keep gcode files reusable. My preference is that the slicer should just specify the parameters that its motion-generation choices were relying on (things like nozzle diameter and filament type), and then the firmware (with varying levels of user input) uses that information to decide how to drive the machine (e.g., which extruder(s) to use, what temperature(s) to set the heaters to, what speeds and acceleration limits to apply, what pressure advance parameters to use, etc.). That way if I want to reprint a file with a different filament color or a different PA configuration or something, I don’t have to reslice it.

Yes i see we converged on something similar, i just moved most of the logic from the macro side to the spoolman.py file in moonraker components (i might create a spoolman branch somtime …).

I can see where you are coming from, i understand that motion related parameters should be on the firmware side, but i have an issue with maw flow for example…

In my case i really want to be able to accurately slice the gcode for the specific combination of printer/extruder/nozzle width and filament. In this case the max flow between a .4 and a .15 mm nozzle is very diffferrent which means that the estimated times are drastically impacted…

Concerning non time impacting parameters such as temperature (which is different according to nozzle width also…) or PA and maybe others, I agree with you, i really see no real downside grabbing these values from a db i would manage (maybe spoolman) and use the correct values at runtime in the firmware.

But for max flow thus nozzle diameter it seems like we would decorrelate the slicer estimations from real print times.

I understand your reasoning behind gcode reusability, but my use case is having a slicer project for each print. (combination of model/printer/nozzle width/filament …) I keep all of those project for tracability purposes.

What’s your thought on the slicer vs print decorrelation matter ?

My slicer estimated print times have never seemed to have any meaningful relationship with actual print times so I only use the slicer estimate to get a rough idea whether different slicer settings will mean slower or faster print.
I’ve used PrusaSlicer, SuperSlicer, and now OrcaSlicer and the time estimates have never been usable for me. I even use the Klipper print time estimator and while it tends to get closer, it’s still usually at least several minutes off.

I see, in my case i might have several hours of difference for a big print, for example between a .6 and a .4 nozzle. In my case several minutes is acceptable when comparing estimated times between klipper and orca. But i still feel like when the slicer has the same max velocities than klipper it tends to be quite accurate in my case. Thanks for you insight though imma try to setup something similar as you for my auto slot swapping mechanism :wink:

I’d be interested to see what you’ve done here. While I was posting some of my macros/python/hacks above, it occurred to me that my whole approach really is little more than a patchwork of workarounds where a dedicated solution would be preferable. Spoolman already has a couple of Moonraker remote methods, maybe it would be better to build it out further to allow for this kind of functionality? Maybe projects like ERCF and SMuFF and Prusa’s MMU could benefit too. A while back @garethky posted a thread about a system to allow Klipper to have more information about filament properties, and I wonder if those objectives could also be accomplished by extending Spoolman, even on single-material printers.

I’m going to try to create the fork today, i’ll keep you posted.

Hey, i forked the moonraker repo here : GitHub - CooperGerman/moonraker_spoolman_enhancement: Web API Server for Klipper with spoolman enhancement for mmu setups and automatic filament selection...

This repo primarly updates metadata.py for filament usage extraction purposes and spoolman.py to make use of it and add functionality to the module.

Feel free to explore, here are the macros i added to use the updated spoolman.py :

[gcode_macro SET_ACTIVE_SPOOL]
description: Sets active spool to spool with given id.
  Usage: SET_ACTIVE_SPOOL [ID=<int>]
    {% if params.ID %}
        {% set id = params.ID|int %}
    {% else %}
        {action_respond_info("Parameter 'ID' is required")}
    {% endif %}

[gcode_macro SET_ACTIVE_SLOT]
description: Sets active spool at slot <SLOT> as active spool.
  Usage: SET_ACTIVE_SPOOL [SLOT=<int>]
    {% if params.SLOT %}
        {% set slot = params.SLOT|int %}
    {% else %}
        {action_respond_info("Parameter 'ID' is required")}
    {% endif %}

[gcode_macro GET_SPOOL_INFO]
description: Gets information for active spool if no id given else information of spool with given id.
  Usage: GET_SPOOL_INFO [ID=<int>]
    # Dummy block for mainsail
    {% set id = None %}
    {% if params.ID %}
        {% set id = params.ID|int %}
    {% endif %}

[gcode_macro SET_SPOOL_SLOT]
description: Assigns spool id=ID to current machine. If MMU is enabled slot number must be specified with SLOT=<int>.
  Usage: SET_SPOOL_SLOT [ID=<int>]
    {% set id = None %}
    {% set slot = None %}
    {% if params.ID %}
        {% set id = params.ID|int %}
    {% endif %}
    {% if params.SLOT %}
        {% set slot = params.SLOT|int %}
    {% endif %}
    {action_call_remote_method("spoolman_set_spool_slot",spool_id=id, slot=slot)}

[gcode_macro GET_SPOOLS]
description: Gets all spools assigned to current machine.

[gcode_macro CHECK_FILAMENT]
description: Proceeds to checks several aspects of currently loaded filaments and selected fialement for current print.
    Usage : CHECK_FILAMENT [DEBUG=<int>]

[gcode_macro CLEAR_ACTIVE_SPOOL]
description: Clears active spool.

[gcode_macro CLEAR_SPOOL_SLOTS]
description: Clears all spool slots.

Not everything is in a publish ready state but feel free to snoop around and ask for clarifications :wink:

I just noticed that you posted this. Thanks! I haven’t had an opportunity yet to really sit down with it but I’ve skimmed it over somewhat, and it looks like a huge step in the right direction. I think what we really need is the ability to have something like a printer.spoolman object that would include all the filament and spool data in the Spoolman DB, including your modifications to add the slot functionality.

At least for my setup, that would eliminate the need for my local spool “registry” using save_variables, because the same information would already be stored and accessible in Spoolman. It would also eliminate my need for gcode_shell_command and my DIY Spoolman client because again, I could just look up the same data natively in Klipper.

I actually implemented something like this after our discussion ^^ :

  • I dump the Lookup table from spoolman.py using :
async def _gen_swap_table(self, swap_table):
        await self.klippy_apis.run_gcode("SAVE_VARIABLE VARIABLE=swap_table VALUE=\"{}\"".format(swap_table))

And use it like this afterwards :

[gcode_macro _ERCF_CHANGE_TOOL]
description: Perform a tool swap
    {% set initial_tool_string = 'unknown tool' %}
    {% if printer["gcode_macro _ERCF_SELECT_TOOL"].color_selected|int != -1 %}
        {% set initial_tool_string = 'T' + printer["gcode_macro _ERCF_SELECT_TOOL"].color_selected|string %}
    {% endif %}
    # check if a swap table exists from variables
-->    {% if "swap_table" in printer.save_variables.variables %}
-->        {% set swap_table = printer.save_variables.variables.swap_table %}
-->        {% set swapped_tool = swap_table.get(params.TOOL|int, -1) %}
-->        {action_respond_info("Applied swap table to tool " ~ params.TOOL ~ " --> " ~ swapped_tool)}
-->    {% else %}
-->        {% set swap_table = None %}
-->        {% set swapped_tool = params.TOOL %}
-->    {% endif %}
    {action_respond_info("Tool swap request, from " ~ initial_tool_string ~ " to T" ~ swapped_tool)}
    M117 {initial_tool_string} -> T{swapped_tool}
    {% if printer["gcode_macro _ERCF_SELECT_TOOL"].color_selected|int != swapped_tool %}
        # Save status of the ERCF as "changing_tool" into a variable
        _SET_ERCF_STATUS STATUS="changing_tool"
        # retract a little bit to avoid stringing or oozing on filament swap
        G92 E0
        G1 E-0.5 F1200
        _ERCF_CHANGE_TOOL_INNER {rawparams}
    {% else %}
        {action_respond_info("Tool " ~ swapped_tool ~ " already selected and loaded. Nothing to do.")}
    {% endif %}

I did not validate everything yet but that’s the thought.

1 Like

Hey i just pushed some updates that might suit your needs.

I you want to give this version a try you can set it up like follows:

  • add an entry in the moonraker.cfg file like this
server: http://spoolman-serv:7912
#   URL to the Spoolman instance. This parameter must be provided.
sync_rate: 5
filament_slots: 9
#   The interval, in seconds, between sync requests with the
#   Spoolman server.  The default is 5.
  • Assign one or more spools to your machine like follows in the location field of the spools in the spoolman db
    <your klipper machine's hostname>:<slot_number>

  • Restart the moonraker service

This should update the mmu_slot variable in the variables.cfg file every time the get_spools_for_machine() function is called (Ie at initialisation and on every slot/spool modification through moonraker) and give something like :

mmu_slots = [{'id': 15, 'registered': '...', 'first_used': '...', 'last_used': '...', 'filament': {'id': 4, 'registered': '...', 'name': 'ABS Black', 'vendor': {'id': 1, 'registered': '...', 'name': 'Vendor Name', 'comment': '...'}, 'material': 'ABS', 'price': 1.00, 'density': 1.04, 'diameter': 1.75, 'weight': 500.0, 'spool_weight': 182.0, 'article_number': 'xxxx', 'comment': ''', 'settings_extruder_temp': 250, 'settings_bed_temp': 105, 'color_hex': '000000'}, 'remaining_weight': 255.08475913761387, 'used_weight': 244.91524086238613, 'remaining_length': 101972.9992443498, 'used_length': 97907.62002333665, 'location': 'hostname:slot_number', 'archived': False}, ... ]
swap_table = [None, None, None, 8, None, None, None, None, None]

Only one spool is present in mmu_slots in my above example but all assigned spools from spoolman should appear in this list.
In the example i have 9 slots with slot 3 beeing redirected to spool 8 (the swap_table is regenerated at each check_filament() call so either manually via CHECK_FILAMENT macro if you have the macros i posted earlier (Way to post/pre process gcode through moonraker? - #11 by coopergerman) or at each print start if CHECK_FILAMENT is in your start macro (slicer or klipper side)

I hope this helps for testing, i would really appriciate any feedback if able.

Not being able to update the klipper object model from Moonraker has been a dead end with these moonrker plugins. E.g. this issue blocks users attempting to find out what spool is in the printer from gcode: Get Spoolman active Spool ID inside Klipper

So basic things that most printers can do now, like verify that its PLA in the printer before you start the print, cant be done by klipper.

The sync_rate thing, I assume, is how often Moonraker will write the data to save_variables? Thats going to mostly work in practice. But its got a lot of weird edge cases. It would be better if klipper read some state directly from Moonraker. i.e. pull not push.

Ideally klipper would decorate its existing status objects by pulling data from Moonraker. Like having the spool merged into the status object for the extruder via data exposed from Moonraker. Then it could be accessed from gcode templates in a more natural way {extruder.spool.filament_type}.

I agree, on the proposed solution. In my case the mmu_slot variable that is saved, is only written at start of print or on manual update. I don’t need to monitor the updated values of usage etc on the klipper side and it seems sufficient for my use case to do the checks on moonraker’s side.

But i would indeed be interested in having a real “push pull” communication through klipper’s api for this. My thought was to limit the scope of modifications as much as possible without changing too much stuff on klipper and moonraker’s side.

My solution as i understand deals with the problems from

I also understand it is not “the intended way” of implementing it especially on the part where moonraker wants to communicate to klipper.

I can try to dig a bit further into this to try and setup a “real” communication between spoolman (moonraker side) and klipper. I imagine that connecting klipper to the moonraker socket is not an option either ? as it seems we would need a new protocol to be implement as stated here :

This is it exactly. Do you happen to know if the lack of this functionality is a design philosophy thing? Or just a nobody-has-implemented-it-yet thing?

Hey, just discussing some stuff on github and someone shared me this pull request : Feature: Add Spoolman Compatability by Ocraftyone · Pull Request #4771 · SoftFever/OrcaSlicer · GitHub

This seems very very promising to me ! I’m not sure it will resolve all me troubles with dealing with filaments but i really like the direction this PR is taking.