2 ms·

Hi! Tried it and i'm impressed. The Mac app reports 4.4 token/s in the Mac Mini M2 with 8GB RAM. Not fast but still very much usable (my use rarely goes past fr

Hi! Tried it and i'm impressed. The Mac app reports 4.4 token/s in the Mac Mini M2 with 8GB RAM. Not fast but still very much usable (my use rarely goes past from summarizing and generating pretty documentation). However, that mac sits in the rack cabinet and i ssh into it, so i would love to chat with it from the terminal, but because i generally use ollama i don't really know how to do that. Can someone help?

gitpusher42 2mo ago
Thank you for testing and sharing results! I think I might understand your use case. You ssh the Mac and want something like `ollama run` with an interactive chat in terminal. Am I right? There is already experimental OpenAI-compatible server in this repo: ``` swift build -c release --product TurboFieldfareServer .build/release/TurboFieldfareServer \ --model scratch/gemma4.gturbo ``` After that a small terminal client can run inside the same ssh session and talk to `/v1/chat/completions` The client needs to keep a messages array, add each user message, send the full array with `stream:true`, print SSE chunks until `[DONE]`, then add the response back to the array. `/reset` can clear it There is a python example in the server docs. (https://github.com/drumih/turbo-fieldfare/blob/main/docs/OPENAI_SERVER.md https://github.com/drumih/turbo-fieldfare/blob/main/docs/OPE...) It is non-streaming, but can be used as a starting point. The server is still experimental and I am fixing some problems currently. But you can try to vibecode a simple terminal client around it. If not, create an issue on Github and describe desired behaviour
- monegator 2mo ago
  Yes, that is the idea. did read the paragraph about the cli server but of course that goes beyon what i know about these tools, that's why i asked. Having a streaming chat via command line would be awesome, because then multiple users could use the machine at the same time (i think 2, max 3 on a 8GB device. It would really make our old mini useful again, instead of sitting in the corner taking space)

gitpusher42 2mo ago

Thank you for testing and sharing results! I think I might understand your use case. You ssh the Mac and want something like `ollama run` with an interactive chat in terminal. Am I right? There is already experimental OpenAI-compatible server in this repo: ``` swift build -c release --product TurboFieldfareServer .build/release/TurboFieldfareServer \ --model scratch/gemma4.gturbo ``` After that a small terminal client can run inside the same ssh session and talk to `/v1/chat/completions` The client needs to keep a messages array, add each user message, send the full array with `stream:true`, print SSE chunks until `[DONE]`, then add the response back to the array. `/reset` can clear it There is a python example in the server docs. (https://github.com/drumih/turbo-fieldfare/blob/main/docs/OPENAI_SERVER.md https://github.com/drumih/turbo-fieldfare/blob/main/docs/OPE...) It is non-streaming, but can be used as a starting point. The server is still experimental and I am fixing some problems currently. But you can try to vibecode a simple terminal client around it. If not, create an issue on Github and describe desired behaviour

monegator 2mo ago
Yes, that is the idea. did read the paragraph about the cli server but of course that goes beyon what i know about these tools, that's why i asked. Having a streaming chat via command line would be awesome, because then multiple users could use the machine at the same time (i think 2, max 3 on a 8GB device. It would really make our old mini useful again, instead of sitting in the corner taking space)