To end off this thread, I’ve done a fairly extensive experiment to try and figure out what is the best approach to get to the ultimate goal which is to have an OS/firmware controller installation that can be updated without physically accessing the controller.
In terms of updating a controller, I believe the optimum (maybe only) way to do this is with a Katapult (formerly “CanBoot”) installation with a CAN network specified. I know that some people don’t like using a CAN bus interface to their toolheads but this approach will work even if there are no CAN devices attached to the controller. I suspect, but have not tested, that this approach will work with a controller with an unused UART in place of the CAN interface.
Some of the results surprised me and, thanks to @Sineos pointing out the “MainsailOS”, I have a better Raspberry PI OS option to load. I’m going to try and present as much information as possible so that other people can replicate my tests on different hardware combinations. I’d be interested in hearing people’s results with controllers like the Raspberry Pi Zero.
This is a long post, so rather than slog through it, I suggest you scroll down to the “Observations” and “Conclusions” at the end before reading through the procedure information.
Apparatus
In this experiment, I used the following host systems:
- Raspberry Pi 4B connected to an Octopus controller via USB. The Octopus has a fan connected to the “FAN0” Port
- Raspberry Pi CM4 connected to a Manta M8P controller via the board’s internal USB. The Manta has a fan connected to the “FAN0” Port.
- BTT CB1 attached to a “RPI 4B adapter board” and connected to an Octopus via USB
For Firmware installation I used:
For all other installations the host network connection was used and specified using SSH. I used a Linux Ubuntu Terminal for SSH access to the hosts.
Test Procedure
Tests were run on different host/controller combination in terms of using the standard (“Lite” in the case of the Raspberry PIs) Operating Systems with these different update approaches:
- “Vanilla” or no updates
- “Update/Upgrade” which consisted of running
sudo apt update
followed by sudo apt upgrade -y
- “Myke Script” which is running the script presented at the start of this thread
- “Sineos Script” which was the script presented in the previous post to this thread
The following procedure was used on each of the host/controller combinations for each of the test configurations:
- Load Selected Operating System into the host for the test using Raspberry Pi Installer. The options are:
1.1. 32-Bit Raspberry Pi OS Lite (for the Raspberry Pi 4B and CM4)
1.2. 64-Bit Raspberry Pi OS Lite (for the Raspberry Pi 4B and CM4)
1.3. CB1 Operating System provided by BTT
1.4. 32-Bit MainsailOS Operating System (for the Raspberry Pi 4B and CM4)
1.5. 64-Bit MainsailOS Operating System (for the Raspberry Pi 4B and CM4)
- Login using SSH into the host
2.1. When MainsailOS is used, check to see that the Mainsail web page was available
- When testing the CB1 host, execute the
sudo nmcli dev wifi connect [NetworkName] password "[Password]"
from an Ethernet connection to ensure CB1 wifi is working. Reboot the CB1 and login again after executing this command
- Run and record the results of
apt-cache policy udev
This is compared to after the updates to see if any changes were made
- Perform the specified update procedure:
5.1. For “Vanilla” do nothing (this includes doing the reboot specified for the other approaches)
5.2. For “Update” execute sudo apt update
followed by sudo apt upgrade -y
and then reboot
5.3. For "Myke Script’ execute curl -sSL https://raw.githubusercontent.com/mykepredko/rPi_scripts/main/debianFix.sh | bash
and then reboot
5.4. For "Sineos Script’ execute curl -sf -L https://raw.githubusercontent.com/Sineos/useful_bits/main/Linux/fix_debian_udev.sh | sudo bash
and then reboot
- If a reboot was performed in the previous step, then login to the host via SSH
6.1. Execute apt-cache policy udev
again to see what changes were made to the system
6.2. If the host is running MainsailOS, check to see that the Mainsail web page is available after reboot
- If MainsailOS is NOT running, execute the following commands to load Klipper, Moonraker and Mainsail:
7.1. sudo apt-get install git -y
7.2. git clone https://github.com/dw-0/kiauh.git
7.3. ./kiauh/kiauh.sh
7.4. Install Klipper with Python 3.x and one instance.
7.5. Install Moonraker
7.6. Install Mainsail
7.7. Exit out of KIAUH
- Install Katapult and make a firmware image with the commands:
8.1. git clone https://github.com/Arksine/Katapult
8.2. pip3 install pyserial
8.3. cd Katapult
8.4. make menuconfig
8.5. For Octopus Select STM32
, F446
, 12MHz Clock Reference
8.6. For Manta Select SMT32
, G0B1
8.7. Q
uit from menuconfig and S
ave
8.8. make
- Make a Klipper firmware image using the commands:
9.1. ~/klipper
9.2. make menuconfig
9.3. For Octopus Select STM32
, F446
, 12MHz Clock Reference
, USB to CAN bus bridge
, CAN bus (on PD0/PD1)
, 500000 CAN bus speed
9.4. For Manta Select SMT32
, G0B1
, USB to CAN bus bridge
, CAN bus (on PD12/PD13)
, 500000 CAN bus speed
9.5. Q
uit from menuconfig and S
ave
9.6. make
- Put controller into DFU mode.
10.1. For Octopus, insert the jumper on the “BOOT” pins and press reset.
10.2. For Manta, Press and hold down the “BOOT” button followed by pressing and releasing “RESET” and then releasing “BOOT”
10.3. Confirm the controller is in DFU mode by using the command lsusb
and looking for the entry “ID 0483:df11 STMicroelectronics STM Device in DFU Mode”
- Flash the Katapult firmware using the command
sudo dfu-util -a 0 -D ~/Katapult/out/katapult.bin --dfuse-address 0x08000000:force:mass-erase:leave -d 0483:df11
11.1. Check that flash operation is complete using the lsusb
command and seeing that the USB port is now labeled “OpenMoko”
- Get the MCU serial number using the command
ls /dev/serial/by-id/
. The returned string with be copy and pasted into the next command.
- Flash the Klipper firmware image into the controller using the command
python3 ~/Katapult/scripts/flash_can.py -d /dev/serial/by-id/[mcuSerial]
where “[mcuSerial]” is the string returned from the previous step.
- Setup the CAN0 device:
14.1. Create the CAN0 file using the command sudo nano /etc/network/interfaces.d/can0
14.2. Put in the following contents of the file and then Ctr-X and Save:
allow-hotplug can0
iface can0 can static
bitrate 500000
up ifconfig $IFACE txqueuelen 256
pre-up ip link set can0 type can bitrate 500000
pre-up ip link set can0 txqueuelen 256
- Execute
sudo reboot
to restart the host
- Check the operation of the system by carrying out the following tasks:
16.1. SSH into the host
16.2. Execute ls /dev/serial/by-id
and expect to see “No Such File or Directory”
16.3. Check to see that Mainsail web page is active
16.4. Execute flash_can.py -I can0 -q
and you should see a "Detected UUID: " with a 12 character hex string following and then “, Application Klipper”
If everything above is good, then the initial programming is complete.
Now, to test if the controller can be updated without accessing, change the Klipper firmware so that the fan on the FAN0 port on the controller runs on firmware startup.
- Change the firmware to turn on the FAN0:
1.1. Execute cd ~/klipper
(if not already in the folder)
1.2. Execute make clean
1.3. Execute make menuconfig
1.4. If Octopus, enter PA8
into the “GPIO pins to set at micro-controller startup”
1.5. If Manta, enter PE0
into the “GPIO pins to set at micro-controller startup”
1.6. Q
uit and S
ave
1.7. Execute make
- Get the controller CAN bus UUID using the command
~/Katapult/scripts/flash_can.py -i can0 -q
and save it for copy and pasting in the next step
- Setup the Flashing operation with the command
python3 ~/Katapult/scripts/flash_can.py -r -u [uuid]
where “[uuid]” is the value returned from the previous step
- Get the controller serial ID using the command
ls /dev/serial/by-id/
and save it for copy and pasting in the next step.
- Program the controller with the updated klipper firmware using the command
python3 ~/Katapult/scripts/flash_can.py -d /dev/serial/by-id/MCU Serial from previous
where “MCU Serial from previous” is the string returned in the previous step.
- If everything works correctly, the fan attached to the controller board will start running
After doing all this and the fan is running, I consider the operation a “PASS” .
If anything does not work as expected or return an invalid response then the process is repeated from the start and if the same error is encountered, then the result is a “FAIL”.
If the conditions of the script are not met, then the test was stopped (in the case of my script where I knew nothing would happen) or it was continued but marked as not executing as expected (@sineos’ script which indicated that the udev wasn’t updated, which is what I thought was the point of the script).
Observations
The test results were recorded in this spreadsheet: 2023.09.18 - Raspberry Pi Klipper & Katapult OS Build and Program Results.zip (19.2 KB)
Netting out the results looks like:
- The BTT is a real PITA to work with.
1.1. A big part of the issue is that the image cannot be updated in the Raspberry Pi Imager to specify a WiFi network - editing the configuration file as described in the documentation works about 50% of the time. I found that the nmcli dev wifi connect
command works 100% of the time, but needs an Ethernet connection to enter it in.
1.2. I never did figure out how to change the URL the CB1 appears on the network. “BTT-CB1.hitronhub.home” seems to be the only thing it will appear as which limits the number of devices on a network.
1.3. I never figured out how to change the primary user from “biqu”. This, along with the locked in “BTT-CB1.hitronhub.home” comes across to me as a huge potential security problem (especially if you don’t change the default password from “biqu”).
1.4. If anybody knows how to change these preset values, please let me know.
- The two scripts (@sineos’s and mine) did not change the system or the test results in the way that I expected.
- The host and controller programming process outlined here worked very reliably for all three hosts and two controller boards.
Conclusions
- I believe I have a way for doing firmware updates on CAN equipped and Katapult programmed controllers without having to access the hardware.
1.1. I’m not sure how important this is to other people but unless I have a printer designed that the controller can be easily accessed (which I’ve never seen on any commercial printers with the exeception of the Prusa i3) I’m not going to do firmware updates unless it’s deemed really important
1.2. Klipper’s printer.cfg
file eliminates probably 90% of the need for accessing a printer’s controller, being able to update the Klipper firmware this way picks up the last 10%
- When I look at the results, I feel like the most reliable option is to use MainsailOS along with
sudo apt update
and sudo apt ugrade -y
2.1. The scripts we came up with don’t seem to add to the effectiveness of the overall system
- While long, the programming process outlined here was very reliable and repeatable. I believe that there are some real opportunities here for automating the process which will make Klipper more accessible to users that are not programmers and find the host setup and controller firmware programming to be overwhelming. Automating this process would greatly reduce the amount of work setting up a new printer and make Klipper more approachable for new users
3.1. Mainsail OS further reduces the remaining workload and, I think, the resulting process could be very slick.
- I am going to reach out to BTT and see if they are receptive to working on the CB1’s OS image. Right now the supply of CM4s still seems to be recovering from the Covid shortages (the rPi 4B seems to have a good supply and reasonable prices) so the CB1 is a reasonable choice - but once the CM4 pipeline fills up and prices go down, I don’t see the CB1 being a viable competitor against the “genuine” article, which is a shame because I like the hardware and think that with a bit of work could be a pretty decent host.
Sorry for all this - it was a lot of work and I think I learned some things that will be useful to others and I am confident that I have a process in which I can update Klipper firmware without the bother of turning my printers upside down (or at least on their sides with pillows).