It shouldn't be too much effort to extend/rewrite that function to use websockets (i wouldn't use just http for something like this). All the important function

It shouldn't be too much effort to extend/rewrite that function to use websockets (i wouldn't use just http for something like this). All the important functions used by that function (llama_eval, sample_top_p_top_k etc.. ) seem to be public anyway.