Palette 3 Support for Klipper?

[ReSummit]

Basic Information:

Printer Model: Ender 3 V2
MCU / Printerboard: Mellow Fly E3 Pro
Host / SBC: RPi 4
[x] klippy.log

Describe your issue:

Hey guys, first time posting here. I have a Palette 3 Pro that I was interested in hooking up to Klipper like how the Palette 2S is supported. I was able to SSH into the Palette 3 and take a look at how it operates; seems like there are some images dedicated to communicating with the MCU through a serial path and some other nodejs applications that manage the state of the device. Strangely it refers to alternate names like “liberty” and “simcoe”… the logs seem to indicate their purpose though.

Ideally, I’d like to contribute to Klipper with support for this somehow but not sure how I can do it. If it’s interesting enough, I can maybe drop some files here containing the files because the packets to control the device are different values compared to the Palette 2S.

[ReSummit]

Alright, I took a few days to look at the code and it’s really complicated. As it’s a deep rabbit hole to go through, I’ll bring some context on the Palette 3 and what I’ve found so far.

Warning: I am not responsible for anything that happens to your Palette 3 if you try this. Try at your own risk.

Part 1: Palette 3 is just a CM3

---

So it turns out that if you go through the guide linked below from Mosaic’s guide on flashing the firmware, it turns out it is simply a CM3 module with a secondary MCU that it communicates with:

Installing the OS on the Compute Module

Also, you can find that there are some ports that are open on the Palette 3 if you connect it to wifi, which are 22, 5000, and 8883. Port 5000 gives you the web interface, the same interface as on the Palette 3. Port 8883 interestingly is a websocket port, but won’t be relevant for this information. That leaves us with port 22 which some will surmise allows us to access the OS using SSH.

If you happen to download and decompress any firmware images from the website, one of the files you will find is a docker compose file, which indicates that the user is mosaic. However, we don’t get a password… so we’ll have to overwrite the password somehow. I used the chroot method using another raspberry pi. It might be possible another way but using the guide linked above and this method allowed me to access the root files and change the mosaic password.

If you get here, you should backup the data on the CM3 in case something goes wrong… Happened to me once already O.o

Part 2: How Palette 3 works

---

As hinted earlier, Palette 3 operates on docker images, of which there are 6 of them. Here is a description of what they generally do (of what limited educated guess I could make out of it):

• Apollo: Backend of the Palette 3
• Cashmere: Frontend of the Palette 3, sending MQTT requests that will be parsed by Simcoe.
• Galaxy: Handles database management. Logs information such as number of splices and other data.
• Liberty: Handles communication to the secondary MCU that controls the hardware of the Palette 3.
• Meridian: Broadcasts information regarding touchscreen input.
• Simcoe: Communicates between Palette 3 and Canvas / Mosaic on print status and details of the current status. This is the central hub for parsing MQTT events and logging things to the backend database.
• Vojvodina: Manages webcam streaming.

Quite a lot… let’s just cut to the chase and say that in reality, only “Liberty” and “Apollo” are required to get the functionality of the Palette 3. If you change the network ports to allow port access to 1883, you can simply send MQTT commands outside of the other containers and it will operate the same way, just without the nice interface that Mosaic gives us. For example, to jog the filament, the MQTT topic is palette/request/drives/jog and the corresponding data parameters can be sent as the following:

{
  "header":
  {
    "originID":"simcoe",
    "msgID":75
  },
  "payload": 
  { 
    "method":"job",
    "query": 
    {
       "drive":"out",
       "withOut":true,
       "speed":1
    }
  }
}

Intentionally, I have changed the speed to 1 but by default it is 40. The drive parameter can be from 1-4 or 1-8 (depending on if you have the pro version or not). Great! Shouldn’t be too hard I think if everything allows us to splice at will?

Part 3: “Liberty”

Well… not quite… Turns out that you can’t just set it in a continuous state of printing and splice at will. Typically for the Palette 3, you would need to set it in something called “accessory mode”, which tracks the progress of splicing using “pings” and “pongs”. However, liberty requires a file that is a compressed file containing the splice lengths and the expected pongs so that the Palette 3 machine knows where to splice. Bummer.

Underneath liberty, it actually communicates with a serial port that is linked with the MCU. Luckily, the parameters to communicate with the MCU are there named something else:

• Amarillo: The actual manager of the MQTT events and utilizes “Citra” to communicate with the secondary MCU. Also is the one that forwards the flags and commands to the secondary MCU
• Citra: A Nodejs plugin made to communicate with the secondary MCU at a baudrate of 38400 and a port of /dev/serial0… Interestingly the default is 115200 but it’s not the case here. Also this plugin indicates STM32 communication.

This then leads to the hard part… While most of the functionality is there, there needs to be a way to allow for “splice at will” during a print, as well as protections if something goes wrong. The todos are thus the following:

• Create a python program (or something similar) to communicate with the secondary MCU as it did with liberty
• Develop "splice at will" filament changing instead of relying on whether there is a .mafx file to print
• Connect this to Klipper using macros placed in gcode (problem: will require splicing to be done at an earlier point in time, I’m hoping that Klipper can parse the file beforehand to order "splice at will" but unsure if it is possible)

I’ll be continuing to look into this but again if anyone is interested, let me know!

[ReSummit]

Alright, I looked into this a bit more and this will be slightly more complicated. Some things I was able to figure out through communicating with the MCU using a test python program but other things such as “splice at will” is more complicated. I think at the end of the day, P2PP will still be needed for multi color prints using Palette 3 but for parsing the output from P2PP to Klipper, it is a bit more complicated. To clarify, I aim to minimize the use of the original implementation so that people can instead use the screen to monitor klipper prints since the interface you see on the Palette 3 is literally just an incognito chrome browser pointing to the local control interface frontend (cashmere).

Here are some updates since the last time I discussed the functionality:

MCU communications through serial

I was able to look at the logs of liberty to find the serial commands. Turns out that the serial communication is parsed as a character string, not as bytes, which makes things slightly easier. For example, to jog only the output, you would send this command to the MCU:
Jog output motor with speed 10: 0,0,00,0030,14,41200000,,

These commands can easily be found if you look through docker compose logs liberty and do any action on the frontend, since the log will print out 3,Parsing string: (<cmd contents>). Despite the relative easiness of seeing these logs, decyphering the logs to do actions is harder.

The format for this command boils down to this:
<msg type>, <msg direction>, <msg uid>, <cmd>, <params>
msg type: Differentiates between differing kinds of messages, which are in order, [JOB, REST, UPDATE, LOG, ERROR]
msg_direction: Differentiates what direction the message is heading. 0 is a request, 1 is a response.
msg uid: Similar to a network packet identifier, this is the identifier for message ordering. This is a hex value.
cmd: The command itself, which a list of commands can be found in liberty/amarillo/coms/constants/job/operations.js. This list however may not be comprehensive. This needs 2 hex values.
params: Any auxiliary params that the command needs. The MCU will tell you the missing data but some info can be found in liberty/amarillo/coms/schemas.

So back to the jog output example, the command is a job request with UID 0. The command is 0x0030. The params it takes is the index to jog and at what speed. 14 is the index to only jog the outer motor and the speed is 10 converted to a hex floating point. The index to jog is somewhat confusing as it seems that this depends on the type of Palette 3 you have but if it seems correct, the indexes are as follows (this is a hex value also):
0-7: Just jog the filament only, no outer motor jogging.
a-f, also 10 and 11: Jogs both the filamant and the outer motor
14: Jogs outer motor only

I don’t know where 8, 9, 12, and 13 went but yea these are the values. Here are some other commands; these and more constitute the full functionality of the debug interface menu:

Cancel (must be sent for jog and any other job type messages): 0,0,00,0100
Cut: 0,0,00,0010
Splice Tuning (you need more than just this command, splicing from index 0 to 1): 0,0,00,0003,0,1
Accessory print mode start: 0,0,e0,0000,2

There are certainly more commands that can be sent but I don’t want to clog this thread with a pastebin of files just for this; thus, I will bring it up when it is relevant to the thread.

Print modes

With this, it is possible to execute the same commands to obtain the same functionality as the stock Palette 3. However, when starting a accessory mode print, there are some REST requests that tell the MCU information about the print (number of splices, printer profile, etc). For instance, to actually be able to start a accessory mode print, some REST POST messages are sent. Below is what I could decypher:

HistoricalModifier
LoadingOffset
TubeLength
???
???
MaterialID 1
MaterialID 2
?
?
?
FilamentSpliceSettings
TotalLength

liberty  | 3,Parsing string: (1,1,00,0,00c8,3f800000)
liberty  | 3,Got RestResponseEvent: UID: 00, Get, param 1: 3f800000, Status: 00c8, param 1: 3f800000.
liberty  | 3,Parsing string: (1,1,01,0,00c8,43480000)
liberty  | 3,Got RestResponseEvent: UID: 01, Get, param 1: 43480000, Status: 00c8, param 1: 43480000.
liberty  | 3,Parsing string: (1,1,02,0,00c8,44898000)
liberty  | 3,Got RestResponseEvent: UID: 02, Get, param 1: 44898000, Status: 00c8, param 1: 44898000.
liberty  | 3,Parsing string: (1,1,03,0,00c8,12000000)
liberty  | 3,Got RestResponseEvent: UID: 03, Get, param 1: 12000000, Status: 00c8, param 1: 12000000.
liberty  | 3,Parsing string: (1,1,04,0,00c8,00000000)
liberty  | 3,Got RestResponseEvent: UID: 04, Get, param 1: 00000000, Status: 00c8, param 1: 00000000.
liberty  | 3,Parsing string: (1,1,05,0,00c8,1,1)
liberty  | 3,Got RestResponseEvent: UID: 05, Get, param 1: 1, param 2: 1, Status: 00c8, param 1: 1, param 2: 1.
liberty  | 3,Parsing string: (1,1,06,0,00c8,2,1)
liberty  | 3,Got RestResponseEvent: UID: 06, Get, param 1: 2, param 2: 1, Status: 00c8, param 1: 2, param 2: 1.
liberty  | 3,Parsing string: (1,1,07,0,00c8,25)
liberty  | 3,Got RestResponseEvent: UID: 07, Get, param 1: 25, Status: 00c8, param 1: 25.
liberty  | 3,Parsing string: (1,1,08,0,00c8,24)
liberty  | 3,Got RestResponseEvent: UID: 08, Get, param 1: 24, Status: 00c8, param 1: 24.
liberty  | 3,Parsing string: (1,1,09,0,00c8,1)
liberty  | 3,Got RestResponseEvent: UID: 09, Get, param 1: 1, Status: 00c8, param 1: 1.
liberty  | 3,Parsing string: (1,1,0a,0,00c8,0,1,1,00000000,00000000,00000000)
liberty  | 3,Got RestResponseEvent: UID: 0a, Get, param 1: 0, param 2: 1, param 3: 1, param 4: 00000000, param 5: 00000000, param 6: 00000000, Status: 00c8, param 1: 0, param 2: 1, param 3: 1, param 4: 00000000, param 5: 00000000, param 6: 00000000.
liberty  | 3,Parsing string: (1,1,0b,0,00c8,24,1,467639b0)
liberty  | 3,Got RestResponseEvent: UID: 0b, Get, param 1: 24, param 2: 1, param 3: 467639b0, Status: 00c8, param 1: 24, param 2: 1, param 3: 467639b0.

If there are additional filaments needed, you will need to provide additional commands detailing the splice settings and the filament type (e.g. 1,1,06,0,00c8,2,1, which sets filament 2 to PLA of index 1). I wasn’t able to know what 5 of the commands could be. Some things I also have no idea of what it could signify as well, such as why some params are specific numbers… If anyone has any idea of what this could be, advice would be appreciated.

Just as a note, it might be a while to make further progress since I have some other worries to handle so it might take a while… Hopefully I can make some python program for Klipper at least before the thread auto closes within 2 months O.o