For future reference;
Was able to solve by going into the klipper firmware directly; Through ssh on the py, ~nano ~/klippy-env/bin/python ~/klipper/klippy/toolhead.py did the trick.
In the file klipper/klippy/toolhead.py
There are two main settings; BUFFER_TIME (high low and start) → this determines how much seconds of future g-code is allowed to enter the buffer.
And the LOOKAHEAD_FLUSH_TIME
G-code is loaded in small batches, the batch size determined by time (0.25s of gcode by default) can be changes with this variable.
I got it down to 5ms average per gcode command, with a latency of 300 ms. This seems to not stutter and be adequately responsive for my needs.
BUFFER_TIME_HIGH = 0.25
BUFFER_TIME_LOW = 0.2
LOOKAHEAD_FLUSH_TIME = 0.005
There are some other BGF_FLUSH variables, I am not sure what they do, I’ve lowered them (but I do not see a direct result), and everything seems to run fine!
Again I do not know what it all does, I just report what worked for me! So take it with a grain of salt.