I recently found out about llama-cpp's official "Server" function: https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md https://github.c

I recently found out about llama-cpp's official "Server" function: https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md https://github.com/ggerganov/llama.cpp/blob/master/examples/... Works like a charm for my (simple) use case. I'm running it on an Always Free Oracle Ampere A1 instance with 4 cores and ~20gb of memory. (Obligatory "fuck Larry Ellison" here)