3 ms·
LM Studio has quickly become the best way to run local LLMs on an Apple Silicon Mac: no offense to vllm/ollama and other terminal-based approaches, but LLMs hav
by minimaxir 1y ago
LM Studio has quickly become the best way to run local LLMs on an Apple Silicon Mac: no offense to vllm/ollama and other terminal-based approaches, but LLMs have many levers for tweaking output and sometimes you need a UI to manage it. Now that LM Studio supports MLX models, it's one of the most efficient too.
I'm not bullish on MCP, but at the least this approach gives a good way to experiment with it for free.
- nix0n 1y agoLM Studio is quite good on Windows with Nvidia RTX also.
- boredemployee 1y agocare to elaborate? i have rtx 4070 12gb vram + 64gb ram, i wonder what models I can run with it. Anything useful?
- nix0n 1y agoLM Studio's model search is pretty good at showing what models will fit in your VRAM. For my 16gb of VRAM, those models do not include anything that's good at coding, even when I provide the API documents via PDF upload (another thing that LM Studio makes easy). So, not really, but LM Studio at least makes it easier to find that out.
- boredemployee 1y agook, ty for the reply!
- Eupolemos 1y agoIf you go to huggingface.co, you can tell it what specs you have and when you go to a model, it'll show you what variations of that model are likely to run well. So if you go to this[0] random model, on the right there is a list of quantifications based on bits, and those you can run will be shown in green. [0] https://huggingface.co/unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF https://huggingface.co/unsloth/Mistral-Small-3.1-24B-Instruc...
- pzo 1y agoI just wish they did some facelifting of UI. Right now is too colorfull for me and many different shades of similar colors. I wish they copy some color pallet from google ai studio or from trae or pycharm.
- chisleu 1y ago> I'm not bullish on MCP You gotta help me out. What do you see holding it back?
- minimaxir 1y agotl;dr the current hype around it is a solution looking for a problem and at a high level, it's just a rebrand of the Tools paradigm.
- mhast 1y agoIt's "Tools as a service", so it's really trying to make tool calling easier to use.
- ijk 1y agoNear as I can tell it's supposed to make calling other people's tools easier. But I don't want to spin up an entire server to invoke a calculator. So far it seems to make building my own local tools harder, unless there's some guidebook I'm missing.
- xyc 1y agoIt's a protocol that doesn't dictate how you are calling the tool. You can use in-memory transport without needing to spin up a server. Your tool can just be a function, but with the flexibility of serving to other clients.
- ijk 1y agoAre there any examples of that? All the documentation I saw seemed to be about building an MCP server, with very little about connecting an existing inference infrastructure to local functions.
- xyc 1y agoFor TypeScript you can refer to https://github.com/modelcontextprotocol/typescript-sdk/blob/main/src/inMemory.ts https://github.com/modelcontextprotocol/typescript-sdk/blob/... There isn't much documentation available right now but you can ask coding agent eg. Claude Code to generate an example.
- zackify 1y agoOllama doesn’t even have a way to customize the context size per model and persist it. LM studio does :)
- Anaphylaxis 1y agoThis isn't true. You can `ollama run {model}`, `/set parameter num_ctx {ctx}` and then `/save`. Recommended to `/save {model}:{ctx}` to persist on model update
- truemotive 1y agoThis can be done with custom Modelfiles as well, I was pretty bent when I found out that 2048 was the default context length. https://ollama.readthedocs.io/en/modelfile/ https://ollama.readthedocs.io/en/modelfile/
- zackify 1y agoAs of 2 weeks back if I did this, it would reset back the moment cline made an api call. But lm studio would work correctly. I’ll have to try again. Even confirmed cline was not overriding num context