7 ms·

This article does not touch on the thing which worries me the most with respect to LLMs: the dependence. Unless you can run the LLM locally, on a computer you

by cesarb 1y ago

This article does not touch on the thing which worries me the most with respect to LLMs: the dependence. Unless you can run the LLM locally, on a computer you own, you are now completely dependent on a remote centralized system to do your work. Whoever controls that system can arbitrarily raise the prices, subtly manipulate the outputs, store and do anything they want with the inputs, or even suddenly cease to operate. And since, according to this article, only the latest and greatest LLM is acceptable (and I've seen that exact same argument six months ago), running locally is not viable (I've seen, in a recent discussion, someone mention a home server with something like 384G of RAM just to run one LLM locally). To those of us who like Free Software because of the freedom it gives us, this is a severe regression.

ImaCake 1y ago
>the dependence. Sure, but that is not the point of the article. LLMs are useful. The fact that you are dependent on someone else is a different problem like being dependent on microsoft for your office suite.
aaron_m04 1y ago
Yes, and it's even worse: if you think LLMs may possibly make the world a worse place, you should not use any LLMs you aren't self-hosting, because your usage information is being used by the creators to make LLMs better.
- inadequatespace 1y ago
  I think that’s a bit of a leap; if you think LLMs make the world a worse place, there are many actions that you might take or not take to try to address that.
  - aaron_m04 1y ago
    It's true that there could be other more impactful actions. I'd love to hear your thoughts on what else can be done.
    - inadequatespace 1y ago
      To be fair, I don't have any great specific ideas, but "Work Without the Worker" for example talks about how a lot of LLMs are fueled by neo-colonialist exploitation. So I guess broadly speaking there could be strategies involving attempting to influence governmental policy rather than by consumer choice. Or more radically, trying to change the structure of the government in general such that the above influences actually are more tractable for the common person.
- MetaWhirledPeas 1y ago
  > you should not use any LLMs you aren't self-hosting, because your usage information is being used by the creators to make LLMs better This sounds a bit like bailing out the ocean.
  - aaron_m04 1y ago
    > This sounds a bit like bailing out the ocean. If it's one individual doing this, sure. I am posting this in the hopes that others follow suit.
  - mrtesthah 1y ago
    so does voting.
Hilift 1y ago
That's going full speed ahead though. Every major cloud provider has an AI offering, and there are now multiple AI-centric cloud providers. There is a lot of money and speculation. Now Nvidia has their own cloud offering that "democratize access to world-class AI infrastructure. Sovereign AI initiatives require a new standard for transparency and performance".
79a6ed87 1y ago
>To those of us who like Free Software because of the freedom it gives us, this is a severe regression. It's fair to be worried about depending on LLM. But I find the dependance on things like AWS or Azure more problematic, if we are talking about centralized and proprietary
mrheosuper 1y ago
I disagree. Self-hosting has always have a lot of drawbacks compared with commercial solutions. I bet my self-host file server has worse reliability than Google Drive, or my self-host git server has worse number of concurrent user than github. It's one thing you must accept when self-host. So when you self-host LLM, you must either accept a drop in output quality, or spend a small fortune on hardware
- deleted 1y ago
  [deleted]
- kortilla 1y ago
  Those aren’t good analogies because it costs nearly nothing to make that availability tradeoff and run things on your computer for your own fun. Raspberry pi was a huge step forward, the move to LLMs is two steps back.
Aeolun 1y ago
It's not like the code is suddenly elsewhere right? If the LLM disappears I'll be annoyed, not helpless.
- nessbot 1y ago
  Not if they only way you know how to code is vibe coding.
- brailsafe 1y ago
  Well, I'd think of it like being car-dependent. Sure, plenty of suburbanites know how to walk, they still have feet, but they live somewhere that's designed to only be practically traversable by car. While you've lived that lifestyle, you may have gained weight and lost muscle mass, or developed an intolerance for discomfort to a point where it poses real problems. If you never got a car, or let yourself adapt to life without one, you have to work backwards from that constraint. Likewise with the built environment around us; the cities many people under the age of 40 consider to be "good" are the ones that didn't demolish themselves in the name of highways and automobiles, in which a car only rarely presents what we'd think of as useful technology. There are all kinds of trades that the car person and the non-car person makes for better or worse depending on the circumstance. The non-car person may miss out on a hobby, or not know why road trips are neat, but they don't have the massive physical and financial liabilities that come with them. The car person meanwhile—in addition to the aforementioned issues—might forget how to grocery shop in smaller quantities, or engage with people out in the world because they just go from point A to B in their private vessel, but they may theoretically engage in more distant varied activities that the non-car person would have to plan for further in advance. Taking the analogy a step further, each party gradually sets different standards for themselves that push the two archetypes into diametrically opposed positions. The non-car owner's life doesn't just not depend on cars, but is often actively made worse by their presence. For the car person, the presence of people, especially those who don't use a car, gradually becomes over-stimulating; cyclists feel like an imposition, people walking around could attack at any moment, even other cars become the enemy. I once knew someone who'd spent his whole life commuting by car, and when he took a new job downtown, had to confront the reality that not only had he never taken the train, he'd become afraid of taking it. In this sense, the rise of LLM does remind of the rise of frontend frameworks, bootcamps thay started with React or React Native, high level languages, and even things like having great internet; the only people who ask what happens in a less ideal case are the ones who've either dealt with those constraints first-hand, or have tried to simulate it. If you've never been to the countryside, or a forest, or a hotel, you might never consider how your product responds in a poor connectivity environment, and these are the people who wind up getting lost on basic hiking trails having assumed that their online map would produce relevant information and always be there. Edit: To clarify, in the analogy, it's clear that cars are not intrinsically bad tools or worthwhile inventions, but had excitement for them been tempered during their rise in commodification and popularity, the feedback loops that ended up all but forcing people to use them in certain regions could have been broken more easily.
keutoi 1y ago
I think the same argument could be made about search engines. Most people are not too worried about them.
- thayne 1y ago
  Maybe they should be.
  - rerdavies 1y ago
    You could stop using them, I suppose.
    - schaefer 1y ago
      "Say, have you seen any good websites lately?" Is a conversation I've literally never started. And it feels strange, because I am constantly asking people what books they're reading.
underdeserver 1y ago
Good thing it's a competitive market with at least 5 serious, independent players.
- nosianu 1y ago
  That will work until there has been a lot of infrastructure created to work with a particular player, and 3rd party software. See the Microsoft ecosystem as an example. Nothing they do could not be replicated, but the network effects they achieved are strong. Too much glue, and 3rd party systems, and also training, and what users are used to, and what workers you could hire are used to, now all point to the MS ecosystem. In this early mass-AI-use phase you still can easily switch vendors, sure. Just like in the 1980s you could still choose some other OS or office suite (like Star Office - the basis for OpenOffice, Lotus, WordStar, WordPerfect) without paying that kind of ecosystem cost, because it did not exist yet. Today too much infrastructure and software relies on the systems from one particular company to change easily, even if the competition were able to provide a better piece of software in one area.
- shaky-carrousel 1y ago
  Until they all merge, or form a cartel.
- rpigab 1y ago
  Good thing it's funded by generous investors or groups who are okay with losing money on every sale (they'll make it up in volume), and never stop funding, and never raise prices, insert ads or enshittify.
underdeserver 1y ago
You can also make this argument to varying degrees about your internet connection, cloud provider, OS vendor, etc.
- 0x1ceb00da 1y ago
  ... search engine
- simoncion 1y ago
  I'm not the OP but: * Not even counting cellular data carriers, I have a choice of at least five ISPs in my area. And if things get really bad, I can go down to my local library to politely encamp myself and use their WiFi. * I've personally no need for a cloud provider, but I've spent a lot of time working on cloud-agnostic stuff. All the major cloud providers (and many of the minors) provide compute, storage (whether block, object, or relational), and network ingress and egress. As long as you don't deliberately tie yourself to the vendor-specific stuff, you're free to choose among all available providers. * I run Linux. Enough said.
  - underdeserver 1y ago
    * You might have a choice of carriers or ISPs, but many don't. * Hmm, what kind of software do you write that pays your bills? * And your setup doesn't require any external infrastructure to be kept up to date?
    - simoncion 1y ago
      > ...but many don't. And many do. The US isn't the entire world, you know. > ...what kind of software do you write that pays your bills? B2B software that allows anyone to run their workloads with most any cloud provider, and most any on-prem "cloud". The entire point of this software is to abstract out the underlying infrastructure so that businesses can walk away from a particular vendor if that vendor gets too stroppy. > ...your setup doesn't require any external infrastructure... It's Gentoo Linux, so it runs largely on donated infra (and infra paid for with donations). But -unlike Windows or OS X users- if I get sick of what the Gentoo steering committee are doing, I can go to another distro (or just fucking roll my own should things get truly dire). That's the point of my comment.
wiseowise 1y ago
Wake up, you’re already dependent on everything, unless you stick exclusively to Python std and no outside batteries. Maven central is gone and you have no proxy setup or your local cache is busted? Poof, you’re fucking gone, all your Springs, Daggers, Quarkuses and every third party crap that makes up your program is gone. Same applies to bazillion JS, Rust libraries.
- pxnicksm 1y ago
  There are multiple organizations with mirrors for packages, and I doubt if the cost of a mirror is the same as a cost of 384GB memory server. A guy says here you need 4TB for a PyPi mirror, 285 GB for npm https://stackoverflow.com/questions/65995150/is-it-possible-to-find-out-the-total-size-of-all-existing-npm-modules-in-the-npm https://stackoverflow.com/questions/65995150/is-it-possible-...
- wolvesechoes 1y ago
  If PyPI goes out and I cannot use NumPy, I can still roll-out my own implementation of linear algebra library, because I've got the required knowledge, and I've got it because I had to learn it instead rely on LLMs.
  - roywiggins 1y ago
    I definitely couldn't, because I just use NumPy. Maybe I could roll my own matrix multiplication, but anything more would require cracking open a textbook that I haven't looked at for a decade. And this was true before I touched an LLM.
- miloignis 1y ago
  Panamax works great for mirroring all of crates.io in 300-400GB, which is big but easily small enough for enthusiasts. I've got it on an external USB drive myself, and it's saved my bacon a few times. We're not yet to that same point for performance of local LLM models afaict, though I do enjoy messing around them.
eleveriven 1y ago
It's also why local models, even if less powerful, are so important. The gap between "state of the art" and "good enough for a lot of workflows" is narrowing fast
- dabockster 1y ago
  Especially with MCP programs that can run in Docker containers.
- mplanchard 1y ago
  Yeah I am very excited for local models to get good enough to be properly useful. I’m a bit of an AI skeptic I’ll admit, but I’m much more of a SV venture-backed company skeptic. The idea of being heavily reliant on such a company, plus needing to be online, plus needing to pay money just to get some coding done is pretty unpalatable to me.
rvnx 1y ago
With the Mac Studio you get 512 GB of unified memory (shared between CPU and GPU), this is enough to run some exciting models. In 20 years, memory has doubled 32x It means that we could have 16 TB memory computers in 2045. It can unlock a lot of possibilities. If even 1 TB is not enough by then (better architecture, more compact representation of data, etc).
- fennecbutt 1y ago
  Yeah, for £10,000. And you get 512GB of bandwidth starved memory. Still, I suppose that's better than what nvidia has on offer atm (even if a rack of gpus gives you much, much higher memory throughput).
  - theshrike79 1y ago
    AKCSHUALLY the M-series CPU memory upgrades are expensive because the memory is on-chip and the bandwidth is a lot bigger than on comparable PC hardware. In some cases it's more cost effective to get M-series Mac Minis vs nVidia GPUs
    - lolinder 1y ago
      They know that, but all accounts I've read acknowledge that the unified memory is worse than dedicated VRAM. It's just much better than running LLMs on CPU and the only way for a regular consumer to get to 64GB+ of graphical memory.
    - hu3 1y ago
      It's still magnitude slower than AI GPUs. And with $10k I could pay 40 years of Claude subscription. A much smarter and faster model.
      - onemoresoop 1y ago
        Claude subscription is $20 but is that all you're using? If you want to get the best of Claude it'll cost you a few hundred+ a month.
amadeuspagel 1y ago
I can't run google on my computer on my own, but I'm totally dependent on it.
- zelphirkalt 1y ago
  There are many alternatives though. It is not like Google has a search monopoly or office product monopoly, or e-mail provider monopoly. It is quite possible to cut out a lot of Google from one's life, and not even complicated to do that.
  - pkilgore 1y ago
    Is your argument there are no LLM alternatives?
    - zelphirkalt 1y ago
      Not really, no. Though I would argue if Google disappeared tomorrow, as a private person you would probably do mostly fine. The point being, that your dependence is most likely not that strong actually. Unless you got important mail arriving at only your gmail mailbox. That would be dangerous. I lost several good accounts on other websites that way in the past. Now I don't register anything useful on gmail addresses any longer, in fact don't actually use gmail any longer, unless I still have some old accounts that I still didn't migrate away out of laziness.
- _heimdall 1y ago
  Is your entire job returning google results? The point being made here is that a developer that can only do their primary job of coding via a hosted LLM is entirely dependent on a third party.
  - scotty79 1y ago
    How much of useful programming work are you able to do without google? I don't think I even tried to do do any for the last 20 years. You make a good point of course that independence is important. But primo, this ship sailed long ago, secundo, more than one party provides the service you depend on. If one failes you still have at least some alternatives.
imhoguy 1y ago
Even FOSS-based development depends on walled gardens, it is evident every time when GitHub is down.
- zelphirkalt 1y ago
  Sensibly hosted FOSS doesn't go to GitHub for hosting though. There are other options for people who care. I personally like Codeberg.
- neop1x 1y ago
  IMO Github doesn't matter for FOSS because you have a lot of local clones, it won't disappear forever if Github goes down or deletes the repo there. Self-hosted alts are not 100% up either. And I actually find collaboration functions / easy PR contribution on Github highly beneficial. At the same time I hate the friction of all those private Gitlabs, Giteas or, God forbid, gitweb.
  - skydhash 1y ago
    > And I actually find collaboration functions / easy PR contribution on Github highly beneficial. At the same time I hate the friction of all those private Gitlabs, Giteas or, God forbid, gitweb. FOSS is more about: 1. Finding some software you can use for your problem 2. Have an issue for your particular use case 3. Download the code and fix the issue. 4. Cleanup the patch and send a proposal to the maintainer. PR is easy, but email is ok. You can even use a pastebin service and post it on a forum (suckless does that in part). 5. The maintainer merges the patch and you can revert to the official version, or they don't and you decides to go with your fork.
Flemlo 1y ago
I think 384gb of ram is surprisingly reasonable tbh. 200-300$/month are already 7k in 3 years. And I do expect some hardware chip based models in a few years like a GPU. AiPU we're you can replace the hardware ai chip.
- BoiledCabbage 1y ago
  > I think 384gb of ram is surprisingly reasonable tbh. > 200-300$/month are already 7k in 3 years. Except at current crazy rates of improvement, cloud based models will in reality likely be ~50x better, and you'll still have the same system.
  - Flemlo 1y ago
    I'm surprised that it's even possible running big models locally. I agree we will see how this plays out but I hope models might start to become more efficient and it might not matter that much for certain things to run some parts locally. I could imagine a LLM model with a lot less languages and optimized for one programming language to happen. Like 'generaten your model'
  - simonw 1y ago
    I've had the same system (M2 64GB MacBook Pro) for three years. 2.5 years ago it could just about run LLaMA 1, and that model sucked. Today it can run Mistral Small 3.1, Gemma 3 27B, Llama 3.3 70B - same exact hardware, but those models are competitive with the best available cloud-hosted model from two years ago (GPT-4). The best hosted models (o3, Claude 4, Gemini 2.5 etc) are still way better than the best models I can run on my 3-year-old laptop, but the rate of improvements for those local models (on the same system) has been truly incredible.
  - bravesoul2 1y ago
    Yes LLMs are a funny workload. They require high amounts if processing but are very bursty. Therefore using your own bare metal is a low of expensive redundancy. For the cloud provider they can utilise the GPU to make it pay. They can also subsidise it with VC money :)
nkotov 1y ago
The truth is that majority of people do not care about this. It's why AWS exists. It's why Fly.io exists.
0j 1y ago
I don't feel like being dependent on LLM coding tools is much of an issue, you can very easily switch between different vendors. And I hope that open weight models will be "good enough" until we get a monopoly. In any case, even if you are afraid of getting too dependent on AI tools, I think everyone needs to stay up to date on what is happening. Things are changing very quickly right now, so no matter what argument you may have against LLMs, it may just not be valid any more in a few months
- mplanchard 1y ago
  > I think everyone needs to stay up to date on what is happening. Things are changing very quickly right now, so no matter what argument you may have against LLMs, it may just not be valid any more in a few months This actually to me implies the opposite of what you’re saying here. Why bother relearning the state of the art every few months, versus waiting for things to stabilize on a set of easy-to-use tools?
benced 1y ago
You can run LLMs locally pretty easily, especially if you have a Mac (the unified memory architecture of Macs is really good at this). It's a niche thing but caring about Free Software is niche.
dabockster 1y ago
You can get 90%+ of the way there with a tiny “coder” LLM running on the Ollama backend with an extension like RooCode and a ton of MCP tools. In fact, MCP is so ground breaking that I consider it to be the actual meat and potatoes of coding AIs. Large models are too monolithic, and knowledge is forever changing. Better just to use a small 14b model (or even 8b in some cases!) with some MCP search tools, a good knowledge graph for memory, and a decent front end for everything. Let it teach itself based on the current context. And all of that can run on an off the shelf $1k gaming computer from Costco. It’ll be super slow compared to a cloud system (like HDD vs SSD levels of slowness), but it will run in the first place and you’ll get *something* out of it.
- macrolime 1y ago
  Which MCPs do recommend?
  - FinalDestiny 1y ago
    DesktopCommander and Taskmaster are great to start with. With just these, you may start to see why OP recommends a memory MCP too (I don’t have a recommendation for that yet)
- esaym 1y ago
  Why don't you elaborate on your setup then?
  - xandrius 1y ago
    Because you can look it easily up. Jan, gtp4all, etc. It's not black magic anymore.
rsanheim 1y ago
We will have the equivalent of Claude Sonnet 4 in a local LLM that can run well on a modern Mac w/ 36+ GB of ram in a year or two. Maybe faster. The local/open models are developing very fast in terms of quantization and how well they can run on consumer hardware. Folks that are local LLMs everyday now will probably say you can basically emulate at least Sonnet 3.7 for coding if you have an real AI workstation. Which may be true, but the time and effort and cost involved is substantial.
peab 1y ago
do you use GitHub? Vs code? GCP or AWS? The internet, perhaps?All work is dependent on other services, that's the modern world
rco8786 1y ago
> Unless you can run the LLM locally, on a computer you own, you are now completely dependent on a remote centralized system to do your work. To be fair, the entire internet is basically this already.
sanex 1y ago
You think an LLM provider has a bigger moat than an IDE (say pre vs code for a better parallel). MSDN and Jetbrains licenses are far more expensive than Cursor or Windsurf.
marcofloriano 1y ago
Best observation so far. Specially the cost side of using all those APIs ... i pay in dollars, but earn in reais (brazil), the cost scares me.

by cesarb 1y ago

ImaCake 1y ago

>the dependence. Sure, but that is not the point of the article. LLMs are useful. The fact that you are dependent on someone else is a different problem like being dependent on microsoft for your office suite.

aaron_m04 1y ago

Yes, and it's even worse: if you think LLMs may possibly make the world a worse place, you should not use any LLMs you aren't self-hosting, because your usage information is being used by the creators to make LLMs better.

inadequatespace 1y ago
I think that’s a bit of a leap; if you think LLMs make the world a worse place, there are many actions that you might take or not take to try to address that.
- aaron_m04 1y ago
  It's true that there could be other more impactful actions. I'd love to hear your thoughts on what else can be done.
  - inadequatespace 1y ago
    To be fair, I don't have any great specific ideas, but "Work Without the Worker" for example talks about how a lot of LLMs are fueled by neo-colonialist exploitation. So I guess broadly speaking there could be strategies involving attempting to influence governmental policy rather than by consumer choice. Or more radically, trying to change the structure of the government in general such that the above influences actually are more tractable for the common person.
MetaWhirledPeas 1y ago
> you should not use any LLMs you aren't self-hosting, because your usage information is being used by the creators to make LLMs better This sounds a bit like bailing out the ocean.
- aaron_m04 1y ago
  > This sounds a bit like bailing out the ocean. If it's one individual doing this, sure. I am posting this in the hopes that others follow suit.
- mrtesthah 1y ago
  so does voting.

Hilift 1y ago

That's going full speed ahead though. Every major cloud provider has an AI offering, and there are now multiple AI-centric cloud providers. There is a lot of money and speculation. Now Nvidia has their own cloud offering that "democratize access to world-class AI infrastructure. Sovereign AI initiatives require a new standard for transparency and performance".

79a6ed87 1y ago

>To those of us who like Free Software because of the freedom it gives us, this is a severe regression. It's fair to be worried about depending on LLM. But I find the dependance on things like AWS or Azure more problematic, if we are talking about centralized and proprietary

mrheosuper 1y ago

I disagree. Self-hosting has always have a lot of drawbacks compared with commercial solutions. I bet my self-host file server has worse reliability than Google Drive, or my self-host git server has worse number of concurrent user than github. It's one thing you must accept when self-host. So when you self-host LLM, you must either accept a drop in output quality, or spend a small fortune on hardware

deleted 1y ago
[deleted]
kortilla 1y ago
Those aren’t good analogies because it costs nearly nothing to make that availability tradeoff and run things on your computer for your own fun. Raspberry pi was a huge step forward, the move to LLMs is two steps back.

Aeolun 1y ago

It's not like the code is suddenly elsewhere right? If the LLM disappears I'll be annoyed, not helpless.

nessbot 1y ago
Not if they only way you know how to code is vibe coding.
brailsafe 1y ago
Well, I'd think of it like being car-dependent. Sure, plenty of suburbanites know how to walk, they still have feet, but they live somewhere that's designed to only be practically traversable by car. While you've lived that lifestyle, you may have gained weight and lost muscle mass, or developed an intolerance for discomfort to a point where it poses real problems. If you never got a car, or let yourself adapt to life without one, you have to work backwards from that constraint. Likewise with the built environment around us; the cities many people under the age of 40 consider to be "good" are the ones that didn't demolish themselves in the name of highways and automobiles, in which a car only rarely presents what we'd think of as useful technology. There are all kinds of trades that the car person and the non-car person makes for better or worse depending on the circumstance. The non-car person may miss out on a hobby, or not know why road trips are neat, but they don't have the massive physical and financial liabilities that come with them. The car person meanwhile—in addition to the aforementioned issues—might forget how to grocery shop in smaller quantities, or engage with people out in the world because they just go from point A to B in their private vessel, but they may theoretically engage in more distant varied activities that the non-car person would have to plan for further in advance. Taking the analogy a step further, each party gradually sets different standards for themselves that push the two archetypes into diametrically opposed positions. The non-car owner's life doesn't just not depend on cars, but is often actively made worse by their presence. For the car person, the presence of people, especially those who don't use a car, gradually becomes over-stimulating; cyclists feel like an imposition, people walking around could attack at any moment, even other cars become the enemy. I once knew someone who'd spent his whole life commuting by car, and when he took a new job downtown, had to confront the reality that not only had he never taken the train, he'd become afraid of taking it. In this sense, the rise of LLM does remind of the rise of frontend frameworks, bootcamps thay started with React or React Native, high level languages, and even things like having great internet; the only people who ask what happens in a less ideal case are the ones who've either dealt with those constraints first-hand, or have tried to simulate it. If you've never been to the countryside, or a forest, or a hotel, you might never consider how your product responds in a poor connectivity environment, and these are the people who wind up getting lost on basic hiking trails having assumed that their online map would produce relevant information and always be there. Edit: To clarify, in the analogy, it's clear that cars are not intrinsically bad tools or worthwhile inventions, but had excitement for them been tempered during their rise in commodification and popularity, the feedback loops that ended up all but forcing people to use them in certain regions could have been broken more easily.

keutoi 1y ago

I think the same argument could be made about search engines. Most people are not too worried about them.

thayne 1y ago
Maybe they should be.
- rerdavies 1y ago
  You could stop using them, I suppose.
  - schaefer 1y ago
    "Say, have you seen any good websites lately?" Is a conversation I've literally never started. And it feels strange, because I am constantly asking people what books they're reading.

underdeserver 1y ago

Good thing it's a competitive market with at least 5 serious, independent players.

nosianu 1y ago
That will work until there has been a lot of infrastructure created to work with a particular player, and 3rd party software. See the Microsoft ecosystem as an example. Nothing they do could not be replicated, but the network effects they achieved are strong. Too much glue, and 3rd party systems, and also training, and what users are used to, and what workers you could hire are used to, now all point to the MS ecosystem. In this early mass-AI-use phase you still can easily switch vendors, sure. Just like in the 1980s you could still choose some other OS or office suite (like Star Office - the basis for OpenOffice, Lotus, WordStar, WordPerfect) without paying that kind of ecosystem cost, because it did not exist yet. Today too much infrastructure and software relies on the systems from one particular company to change easily, even if the competition were able to provide a better piece of software in one area.
shaky-carrousel 1y ago
Until they all merge, or form a cartel.
rpigab 1y ago
Good thing it's funded by generous investors or groups who are okay with losing money on every sale (they'll make it up in volume), and never stop funding, and never raise prices, insert ads or enshittify.

underdeserver 1y ago

You can also make this argument to varying degrees about your internet connection, cloud provider, OS vendor, etc.

0x1ceb00da 1y ago
... search engine
simoncion 1y ago
I'm not the OP but: * Not even counting cellular data carriers, I have a choice of at least five ISPs in my area. And if things get really bad, I can go down to my local library to politely encamp myself and use their WiFi. * I've personally no need for a cloud provider, but I've spent a lot of time working on cloud-agnostic stuff. All the major cloud providers (and many of the minors) provide compute, storage (whether block, object, or relational), and network ingress and egress. As long as you don't deliberately tie yourself to the vendor-specific stuff, you're free to choose among all available providers. * I run Linux. Enough said.
- underdeserver 1y ago
  * You might have a choice of carriers or ISPs, but many don't. * Hmm, what kind of software do you write that pays your bills? * And your setup doesn't require any external infrastructure to be kept up to date?
  - simoncion 1y ago
    > ...but many don't. And many do. The US isn't the entire world, you know. > ...what kind of software do you write that pays your bills? B2B software that allows anyone to run their workloads with most any cloud provider, and most any on-prem "cloud". The entire point of this software is to abstract out the underlying infrastructure so that businesses can walk away from a particular vendor if that vendor gets too stroppy. > ...your setup doesn't require any external infrastructure... It's Gentoo Linux, so it runs largely on donated infra (and infra paid for with donations). But -unlike Windows or OS X users- if I get sick of what the Gentoo steering committee are doing, I can go to another distro (or just fucking roll my own should things get truly dire). That's the point of my comment.

wiseowise 1y ago

Wake up, you’re already dependent on everything, unless you stick exclusively to Python std and no outside batteries. Maven central is gone and you have no proxy setup or your local cache is busted? Poof, you’re fucking gone, all your Springs, Daggers, Quarkuses and every third party crap that makes up your program is gone. Same applies to bazillion JS, Rust libraries.

pxnicksm 1y ago
There are multiple organizations with mirrors for packages, and I doubt if the cost of a mirror is the same as a cost of 384GB memory server. A guy says here you need 4TB for a PyPi mirror, 285 GB for npm https://stackoverflow.com/questions/65995150/is-it-possible-to-find-out-the-total-size-of-all-existing-npm-modules-in-the-npm https://stackoverflow.com/questions/65995150/is-it-possible-...
wolvesechoes 1y ago
If PyPI goes out and I cannot use NumPy, I can still roll-out my own implementation of linear algebra library, because I've got the required knowledge, and I've got it because I had to learn it instead rely on LLMs.
- roywiggins 1y ago
  I definitely couldn't, because I just use NumPy. Maybe I could roll my own matrix multiplication, but anything more would require cracking open a textbook that I haven't looked at for a decade. And this was true before I touched an LLM.
miloignis 1y ago
Panamax works great for mirroring all of crates.io in 300-400GB, which is big but easily small enough for enthusiasts. I've got it on an external USB drive myself, and it's saved my bacon a few times. We're not yet to that same point for performance of local LLM models afaict, though I do enjoy messing around them.

eleveriven 1y ago

It's also why local models, even if less powerful, are so important. The gap between "state of the art" and "good enough for a lot of workflows" is narrowing fast

dabockster 1y ago
Especially with MCP programs that can run in Docker containers.
mplanchard 1y ago
Yeah I am very excited for local models to get good enough to be properly useful. I’m a bit of an AI skeptic I’ll admit, but I’m much more of a SV venture-backed company skeptic. The idea of being heavily reliant on such a company, plus needing to be online, plus needing to pay money just to get some coding done is pretty unpalatable to me.

rvnx 1y ago

With the Mac Studio you get 512 GB of unified memory (shared between CPU and GPU), this is enough to run some exciting models. In 20 years, memory has doubled 32x It means that we could have 16 TB memory computers in 2045. It can unlock a lot of possibilities. If even 1 TB is not enough by then (better architecture, more compact representation of data, etc).

fennecbutt 1y ago
Yeah, for £10,000. And you get 512GB of bandwidth starved memory. Still, I suppose that's better than what nvidia has on offer atm (even if a rack of gpus gives you much, much higher memory throughput).
- theshrike79 1y ago
  AKCSHUALLY the M-series CPU memory upgrades are expensive because the memory is on-chip and the bandwidth is a lot bigger than on comparable PC hardware. In some cases it's more cost effective to get M-series Mac Minis vs nVidia GPUs
  - lolinder 1y ago
    They know that, but all accounts I've read acknowledge that the unified memory is worse than dedicated VRAM. It's just much better than running LLMs on CPU and the only way for a regular consumer to get to 64GB+ of graphical memory.
  - hu3 1y ago
    It's still magnitude slower than AI GPUs. And with $10k I could pay 40 years of Claude subscription. A much smarter and faster model.
    - onemoresoop 1y ago
      Claude subscription is $20 but is that all you're using? If you want to get the best of Claude it'll cost you a few hundred+ a month.

amadeuspagel 1y ago

I can't run google on my computer on my own, but I'm totally dependent on it.

zelphirkalt 1y ago
There are many alternatives though. It is not like Google has a search monopoly or office product monopoly, or e-mail provider monopoly. It is quite possible to cut out a lot of Google from one's life, and not even complicated to do that.
- pkilgore 1y ago
  Is your argument there are no LLM alternatives?
  - zelphirkalt 1y ago
    Not really, no. Though I would argue if Google disappeared tomorrow, as a private person you would probably do mostly fine. The point being, that your dependence is most likely not that strong actually. Unless you got important mail arriving at only your gmail mailbox. That would be dangerous. I lost several good accounts on other websites that way in the past. Now I don't register anything useful on gmail addresses any longer, in fact don't actually use gmail any longer, unless I still have some old accounts that I still didn't migrate away out of laziness.
_heimdall 1y ago
Is your entire job returning google results? The point being made here is that a developer that can only do their primary job of coding via a hosted LLM is entirely dependent on a third party.
- scotty79 1y ago
  How much of useful programming work are you able to do without google? I don't think I even tried to do do any for the last 20 years. You make a good point of course that independence is important. But primo, this ship sailed long ago, secundo, more than one party provides the service you depend on. If one failes you still have at least some alternatives.

imhoguy 1y ago

Even FOSS-based development depends on walled gardens, it is evident every time when GitHub is down.

zelphirkalt 1y ago
Sensibly hosted FOSS doesn't go to GitHub for hosting though. There are other options for people who care. I personally like Codeberg.
neop1x 1y ago
IMO Github doesn't matter for FOSS because you have a lot of local clones, it won't disappear forever if Github goes down or deletes the repo there. Self-hosted alts are not 100% up either. And I actually find collaboration functions / easy PR contribution on Github highly beneficial. At the same time I hate the friction of all those private Gitlabs, Giteas or, God forbid, gitweb.
- skydhash 1y ago
  > And I actually find collaboration functions / easy PR contribution on Github highly beneficial. At the same time I hate the friction of all those private Gitlabs, Giteas or, God forbid, gitweb. FOSS is more about: 1. Finding some software you can use for your problem 2. Have an issue for your particular use case 3. Download the code and fix the issue. 4. Cleanup the patch and send a proposal to the maintainer. PR is easy, but email is ok. You can even use a pastebin service and post it on a forum (suckless does that in part). 5. The maintainer merges the patch and you can revert to the official version, or they don't and you decides to go with your fork.

Flemlo 1y ago

I think 384gb of ram is surprisingly reasonable tbh. 200-300$/month are already 7k in 3 years. And I do expect some hardware chip based models in a few years like a GPU. AiPU we're you can replace the hardware ai chip.

BoiledCabbage 1y ago
> I think 384gb of ram is surprisingly reasonable tbh. > 200-300$/month are already 7k in 3 years. Except at current crazy rates of improvement, cloud based models will in reality likely be ~50x better, and you'll still have the same system.
- Flemlo 1y ago
  I'm surprised that it's even possible running big models locally. I agree we will see how this plays out but I hope models might start to become more efficient and it might not matter that much for certain things to run some parts locally. I could imagine a LLM model with a lot less languages and optimized for one programming language to happen. Like 'generaten your model'
- simonw 1y ago
  I've had the same system (M2 64GB MacBook Pro) for three years. 2.5 years ago it could just about run LLaMA 1, and that model sucked. Today it can run Mistral Small 3.1, Gemma 3 27B, Llama 3.3 70B - same exact hardware, but those models are competitive with the best available cloud-hosted model from two years ago (GPT-4). The best hosted models (o3, Claude 4, Gemini 2.5 etc) are still way better than the best models I can run on my 3-year-old laptop, but the rate of improvements for those local models (on the same system) has been truly incredible.
- bravesoul2 1y ago
  Yes LLMs are a funny workload. They require high amounts if processing but are very bursty. Therefore using your own bare metal is a low of expensive redundancy. For the cloud provider they can utilise the GPU to make it pay. They can also subsidise it with VC money :)

nkotov 1y ago

The truth is that majority of people do not care about this. It's why AWS exists. It's why Fly.io exists.

0j 1y ago

I don't feel like being dependent on LLM coding tools is much of an issue, you can very easily switch between different vendors. And I hope that open weight models will be "good enough" until we get a monopoly. In any case, even if you are afraid of getting too dependent on AI tools, I think everyone needs to stay up to date on what is happening. Things are changing very quickly right now, so no matter what argument you may have against LLMs, it may just not be valid any more in a few months

mplanchard 1y ago
> I think everyone needs to stay up to date on what is happening. Things are changing very quickly right now, so no matter what argument you may have against LLMs, it may just not be valid any more in a few months This actually to me implies the opposite of what you’re saying here. Why bother relearning the state of the art every few months, versus waiting for things to stabilize on a set of easy-to-use tools?

benced 1y ago

You can run LLMs locally pretty easily, especially if you have a Mac (the unified memory architecture of Macs is really good at this). It's a niche thing but caring about Free Software is niche.

dabockster 1y ago

You can get 90%+ of the way there with a tiny “coder” LLM running on the Ollama backend with an extension like RooCode and a ton of MCP tools. In fact, MCP is so ground breaking that I consider it to be the actual meat and potatoes of coding AIs. Large models are too monolithic, and knowledge is forever changing. Better just to use a small 14b model (or even 8b in some cases!) with some MCP search tools, a good knowledge graph for memory, and a decent front end for everything. Let it teach itself based on the current context. And all of that can run on an off the shelf $1k gaming computer from Costco. It’ll be super slow compared to a cloud system (like HDD vs SSD levels of slowness), but it will run in the first place and you’ll get *something* out of it.

macrolime 1y ago
Which MCPs do recommend?
- FinalDestiny 1y ago
  DesktopCommander and Taskmaster are great to start with. With just these, you may start to see why OP recommends a memory MCP too (I don’t have a recommendation for that yet)
esaym 1y ago
Why don't you elaborate on your setup then?
- xandrius 1y ago
  Because you can look it easily up. Jan, gtp4all, etc. It's not black magic anymore.

rsanheim 1y ago

We will have the equivalent of Claude Sonnet 4 in a local LLM that can run well on a modern Mac w/ 36+ GB of ram in a year or two. Maybe faster. The local/open models are developing very fast in terms of quantization and how well they can run on consumer hardware. Folks that are local LLMs everyday now will probably say you can basically emulate at least Sonnet 3.7 for coding if you have an real AI workstation. Which may be true, but the time and effort and cost involved is substantial.

peab 1y ago

do you use GitHub? Vs code? GCP or AWS? The internet, perhaps?All work is dependent on other services, that's the modern world

rco8786 1y ago

> Unless you can run the LLM locally, on a computer you own, you are now completely dependent on a remote centralized system to do your work. To be fair, the entire internet is basically this already.

sanex 1y ago

You think an LLM provider has a bigger moat than an IDE (say pre vs code for a better parallel). MSDN and Jetbrains licenses are far more expensive than Cursor or Windsurf.

marcofloriano 1y ago

Best observation so far. Specially the cost side of using all those APIs ... i pay in dollars, but earn in reais (brazil), the cost scares me.