My LLM host is a Framework Desktop 128GB (Ryzen AI 395) with an RTX 5090 (Minisforum DEG1 dock, cheap Oculink card plugged into the PCIe 4.0 x4 slot). I run lla

My LLM host is a Framework Desktop 128GB (Ryzen AI 395) with an RTX 5090 (Minisforum DEG1 dock, cheap Oculink card plugged into the PCIe 4.0 x4 slot). I run llama-server (b9296 compiled for CUDA) using a systemd service, and its configured as: # server-2.env LLAMA_ARGS="--host 0.0.0.0 --port 8082 --sleep-idle-seconds 1800" LLAMA_ARG_MODELS_MAX=1 LLAMA_ARG_MODELS_PRESET=/usr/local/etc/llama.cpp/models-2.ini # models-2.ini [*] device = CUDA0 fit = on cache-type-k = q8_0 cache-type-v = q8_0 [Qwen3.6 27B It Q6] model = /mnt/data1/llm/huggingface/hub/models--unsloth--Qwen3.6-27B-GGUF/snapshots/82d411acf4a06cfb8d9b073a5211bf410bfc29bf/Qwen3.6-27B-UD-Q6_K_XL.gguf alias = qwen3.6-27b-it:q6 ctx-size = 200000 temp = 0.6 top-p = 0.8 top-k = 20 min-p = 0.0 presence-penalty = 1.5 repeat-penalty = 1.0 chat-template-kwargs = {"enable_thinking":false} I use the model from a sandboxed OpenCode using the @ai-sdk/openai-compatible plugin. And I'm using an SSH tunnel from home to work to expose llama-server to OpenCode.