9 ms·
LM Studio 0.3 – Discover, download, and run local LLMs
- navaed01 2y agoCongrats! I’m a big fan of the existing product and the are some great updates to make the app even more accessible and powerful
- navaed01 2y agoWhy did this get down voted so much? At all?
- mythz 2y agoOriginally started out with LM Studio which was pretty nice but ended up switching to Ollama since I only want to use 1 app to manage all the large model downloads and there are many more tools and plugins that integrate with Ollama, e.g. in IDEs and text editors
- a1o 2y agoWhat is the recommended system settings for this?
- gymbeaux 2y agoIt depends on the model you run but generally speaking you want an NVIDIA GPU of some substance. I’d say like a 3060 at minimum. CPU inference is incredibly slow versus my RTX 3090, but technically it will work.
- mark_l_watson 2y agoQuestion for everyone: I am using the MLX version of Flux to generate really good images from text on my M2 Mac, but I don’t have an easy setup for doing text + base image to a new image. I want to be able to use base images of my family and put them on Mount Everest, etc. Does anyone have a recommendation? For context: I have almost ten years experience with deep learning, but I want something easy to set up in my home M2 Mac, or Google Colab would be OK.
- MacsHeadroom 2y agoTry Diffusion Bee's latest release https://github.com/divamgupta/diffusionbee-stable-diffusion-ui/releases https://github.com/divamgupta/diffusionbee-stable-diffusion-...
- dgreensp 2y agoI filed a GitHub issue two weeks ago about a bug that was enough for me to put it down for a bit, and there’s been not even a response. Their development velocity seems incredible, though. I’m not sure what to make of it.
- yags 2y agoWe probably just missed it. Can you please ping me on it? “@yagil” on GitHub
- IronWolve 2y agoBeen using LM studio for months on windows, its so easy to use, simple install, just search for the LLM off huggingface and it downloads and just works. I dont need to setup a python environment in conda, its way easier for people to play and enjoy. Its what I tell people who want to start enjoying LLM's without the hassle.
- webprofusion 2y agoCool, it's a bit weird that the Windows download is 32-bit, it should be 64-bit by default and there's no need for a 32-bit windows version at all.
- webprofusion 2y agoIt's probably 64-bit and they just call it x86 on their website. Needs an option to choose where models get downloaded to as your typically C: drive is an SSD with limited space.
- Jedd 2y agoHas an option to choose where models get downloaded - in the Models tab you can pick the target path.
- diggan 2y ago> Needs an option to choose where models get downloaded to as your typically C: drive is an SSD with limited space. You can already do this? https://i.imgur.com/BpF3K9t.png https://i.imgur.com/BpF3K9t.png
- fcukdei 2y ago[dead]
- pcf 2y agoIn some brief testing, I discovered that the same models (Llama 3 7B and one more I can't remember) are running MUCH slower in LM Studio than in Ollama on my MacBook Air M1 2020. Has anyone found the same thing, or was that a fluke and I should try LM Studio again?
- viccis 2y agoJust chiming in with others to help out: By default LM Studio doesn't fully use your GPU. I have no idea why. Under the settings pane on the right, turn the slider under "GPU Offload" all the way to 100%.
- cma 2y agoMaybe so the web browser etc. still has some GPU without swapping from main memory? What % does it default to?
- pcf 2y agoThat froze the whole computer, and even disabled the possibility of clicking both the internal and external trackpad. The model is Dolphin 2.9.1 Llama 3 8B Q4_0. I set it to 100% and wrote this: "hi, which model are you?" The reply was a slow output of these characters, a mouse cursor that barely moved, and I couldn't click on the trackpads: "G06-5(D&?=4>,.))G?7E-5)GAG+2;BEB,%F=#+="6;?";/H/01#2%4F1"!F#E<6C9+#"5E-<!CGE;>;E(74F=')FE2=HC7#B87!#/C?!?,?-%-09."92G+!>E';'GAF?08<F5<:&%<831578',%9>.='"0&=6225A?.8,#8<H?.'%?)-<0&+,+D+<?0>3/;HG%-=D,+G4.C8#FE<%=4))22'*"EG-0&68</"G%(2(" Help?
- christkv 2y agoMake sure you turn on the use of the GPU using the slider. By default it does not leverage the full speed.
- smcleod 2y agoDon’t forget to tune your num_batch
- Terretta 2y agoTwo replies to parent immediately suggest tuning. Ironically, this release claims to feature auto-config for best performance: “Some of us are well versed in the nitty gritty of LLM load and inference parameters. But many of us, understandably, can't be bothered. LM Studio 0.3.0 auto-configures everything based on the hardware you are running it on.” So parent should expect it to work. I find the same issue: using a MBP with 96GB (M2 Max with 38‑core GPU), it seems to tune by default for a base machine.
- alok-g 2y agoSee also: Msty.app It allows both local and cloud models. * Not associated with them in any way. Am a happy user.
- grigio 2y agocan somebody share benchmarks on AMD ryzen AI with and without NPU ?
- Jedd 2y agoIt's using llama.cpp, so it's going to be the same benchmarks as almost all other apps (given almost everything uses llama.cpp under the hood).
- pornlover 2y agoLM Studio is great, although I wish recommended prompts were part of the data of each LLM. I probably just don't know enough but I feel like I get hunk of magic data and then I'm mostly on my own. Similarly with images, LLMs and ML in general feel like DOS and config.sys and autoexec.bat and qemm days.
- Tepix 2y agoNeat! Can i use it with Brave browser‘s local LLM festure?
- qwertox 2y agoYesterday I wanted to find a conversation snippet in ChatGPT of a conversation I had maybe 1 or 2 weeks ago. Searching for a single keyword would have been enough to find it. How is it possible that there's still no way to search through your conversations?
- potatoman22 2y agoTry exporting your data and searching the JSON/HTML.
- Jedd 2y agoAre you complaining about OpenAI's ChatGPT's web UI interface?
- code51 2y agoFor Mac and iOS, you can install ChatGPT app. Why they won't enable search for their main web user crowd is beyond me. Perhaps they are just afraid of scale. With all their might, it's still possible that they can't estimate the scale and complexity of queries they might receive.
- nilsherzig 2y agoThey did staged rollouts for almost every recent feature. I think it might be in their interest if you just ask the LLM again? Old answers might not be up to their current standards and they don't gain feedback from you looking at old answers
- xyc 2y agoA user's personal data really does not have that much scale. Worst case they can cache everything locally. I've imported thousands of chat sessions into a local AI chat app's database, total storage is under 30MB. Full text search (with highlights and all) is almost instant.
- BaculumMeumEst 2y agoThere are lots of ways to search through your conversations, just not through OpenAI's web interface. If you don't want to explore alternatives because you don't want to lose access to your conversations, I would argue you've just demonstrated to yourself why you should avoid proactively avoid vendor lock-in.
- smcleod 2y agoNice, it’s a solid product! It’s just a shame it’s not open source and its license doesn’t permit work use.
- yags 2y agoThanks! We actually totally permit work use. See https://lmstudio.ai/enterprise.html https://lmstudio.ai/enterprise.html
- jdboyd 2y agoAn email us link is a bit discouragement for using it work purposes. I want a clearly defined price list, at least for some entry levels of commercial use.
- fragmede 2y agoOr even just a ballpark. Are we talking $500, $5,000, $50,000 or $500,000?
- e-clinton 2y agoThis. When companies don’t list prices, it automatically gives me a “they want to rip you off” vibe. Put in the effort and define enterprise pricing. If you later find that it isn’t right, change it.
- smcleod 2y agoThanks, what license is it under? This means that anyone that wants to try it at work has to fill that out though right?
- TeMPOraL 2y agoDoes anyone know if there's a changelog/release notes available for all historical versions of this? This is one of those programs with the annoying habit to surface only the list of changes in the most recent version, and their release cadence is such that there are some 3 to 5 updates between the times I run, and then I have no idea what changed.
- flear 2y agoSame. I found their Discord announcement Channel [1] and they may have started to use their blog for a full version changelog [2] [1] https://discord.gg/aPQfnNkxGC https://discord.gg/aPQfnNkxGC [2] https://lmstudio.ai/blog https://lmstudio.ai/blog
- swalsh 2y agoI LOVE LM studio, it's super convenient for testing model capabilities, and the OpenAI server makes it really easy to spin up a server and test. My typical process is to load it up in LM studio, test it, and when I'm happy with the settings, move to vllm.
- xeromal 2y agoI never could get anything local working a few years ago and someone on reddit told me about LM Studio and I finally managed to "run an AI" on my machine. Really cool and now I'm tinkering with it using the built in HTTP server
- yags 2y agoHello Hacker News, Yagil here- founder and original creator of LM Studio (now built by a team of 6!). I had the initial idea to build LM Studio after seeing the OG LLaMa weights ‘leak’ (https://github.com/meta-llama/llama/pull/73/files https://github.com/meta-llama/llama/pull/73/files) and then later trying to run some TheBloke quants during the heady early days of ggerganov/llama.cpp. In my notes LM Studio was first “Napster for LLMs” which evolved later to “GarageBand for LLMs”. What LM Studio is today is a an IDE / explorer for local LLMs, with a focus on format universality (e.g. GGUF) and data portability (you can go to file explorer and edit everything). The main aim is to give you an accessible way to work with LLMs and make them useful for your purposes. Folks point out that the product is not open source. However I think we facilitate distribution and usage of openly available AI and empower many people to partake in it, while protecting (in my mind) the business viability of the company. LM Studio is free for personal experimentation and we ask businesses to get in touch to buy a business license. At the end of the day LM Studio is intended to be an easy yet powerful tool for doing things with AI without giving up personal sovereignty over your data. Our computers are super capable machines, and everything that can happen locally w/o the internet, should. The app has no telemetry whatsoever (you’re welcome to monitor network connections yourself) and it can operate offline after you download or sideload some models. 0.3.0 is a huge release for us. We added (naïve) RAG, internationalization, UI themes, and set up foundations for major releases to come. Everything underneath the UI layer is now built using our SDK which is open source (Apache 2.0): https://github.com/lmstudio-ai/lmstudio.js https://github.com/lmstudio-ai/lmstudio.js. Check out specifics under packages/. Cheers! -Yagil
- fallinditch 2y agoDoes anyone know what advantages LM Studio has over Ollama, and vise versa?
- barrkel 2y agoOllama doesn't have a UI.
- vunderba 2y agoA better question would be over something like Jan or LibreChat. Ollama's is CLI/API/backend for easily downloading and running models. https://github.com/janhq/jan https://github.com/janhq/jan https://github.com/danny-avila/LibreChat https://github.com/danny-avila/LibreChat Jan's probably the closest thing to a open-source LLM chat interface that is relatively easy to get started with. I personally prefer Librechat (which supports integration with image generation) but it does have to spin up some docker stuff and that can make it a bit more complicated.
- himhckr 2y agoThere is also Msty (https://msty.app https://msty.app), which I find much easier to get started with and it comes with interesting features such as web search, RAG, Delve mode, etc.
- BaculumMeumEst 2y agoIf you're hopping between these products instead of learning and understanding how inference works under the hood, and familiarizing yourself with the leading open source projects (i.e. llama.cpp), you are doing yourself a great disservice.
- hnuser123456 2y agoI know how training and inference works under the hood, I know the activation functions and backprop and MMUL, and I know some real applications I really want to build. But there's still plenty of room in the gap between that LM studio helps fill. I also already have software built around the openai api, and the lmstudio openai api emulator is hard to beat for convenience. But if you can outline a process I could follow (or link good literature) to shift towards running LLMs locally with FOSS but still interact with them through an API, I'll absolutely give it a try.
- gastonmorixe 2y agoHave you tried Jan? https://github.com/janhq/jan https://github.com/janhq/jan
- hnuser123456 2y agoFantastic, thank you.
- BaculumMeumEst 2y ago"hopping between these products instead of learning and understanding" was intended to exclude people who already know how they work, because I think it is totally fine to use them if you know exactly what all the current knobs and levers do.
- barrkel 2y agoWhy would someone expect interacting with a local LLM to teach anything about inference? Interacting with a local LLM develops one's intuitions about how LLMs work, what they're good for (appropriately scaled to model size) and how they break, and gives you ideas about how to use them as a tool in a bigger applications without getting bogged down in API billing etc.
- 2browser 2y agoRunning this on Windows on an AMD card. Llama 3.1 Instruct 7B runs really well on this if anyone wants to try.