3 ms·

Of course they are. I wouldn't expect otherwise :) But the price we're paying (and I don't mean money) is very high, imho. We all talk about how good engineers

by dakiol 1y ago

Of course they are. I wouldn't expect otherwise :) But the price we're paying (and I don't mean money) is very high, imho. We all talk about how good engineers write code that depends on high-level abstractions instead of low-level details, allowing us to replace third party dependencies easily and test our apps more effectively, keeping the core of our domain "pure". Well, isn't it time we started doing the same with LLMs? I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. That would at least allow us to switch to a free and opensource version if the companies behind the private LLMs go rogue. I'm afraid tho that wouldn't be enough, but it's a starting point. To put an example: what would you think if you need to pay for every single Linux process in your machine? Or for every Git commit you make? Or for every debugging session you perform?

azan_ 1y ago
> I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. That would at least allow us to switch to a free and opensource version if the companies behind the private LLMs go rogue. I'm afraid tho that wouldn't be enough, but it's a starting point. There are open source tools that do exactly that already.
- dakiol 1y ago
  Ah, well that's nice. But every single post I read don't mention them? So, I assume they are not popular for some reason. Again, my main point here is: the normalization of using private LLMs. I don't see anyone talking about it; we are all just handing over a huge part of what it means to build software to a couple of enterprises whose goal is, of course, to maximize profit. So, yeah, perhaps I'm overthinking I don't know; I just don't like that now these companies are so ingrained in the act of building software (just like AWS is so ingrained in the act of running software)
  - rahimnathwani 1y ago
    But every single post I read don't mention them? Why would they? Does every single post about a Jetbrains feature mention that you can easily switch from Jetbrains to an open source editor like VS Code or vim?
  - deleted 1y ago
    [deleted]
  - Arainach 1y ago
    >every single post I read don't mention them Because the models are so much worse that people aren't using them. Philosophical battles don't pay the bills and for most of us they aren't fun. There have been periods of my life where I stubbornly persisted using something inferior for various reasons - maybe I was passionate about it, maybe I wanted it to exist and was willing to spend my time debugging and offer feedback - but there a finite number of hours in my life and often I'd much rather pay for something that works well than throw my heart, soul, time, and blood pressure at something that will only give me pain.
airstrike 1y ago
None of that applies here since we could all easily switch to open models at a moment's notice with limited costs. In fact, we switch between proprietary models every few months. It just so happens that closed models are better today.
ghm2180 1y ago
> I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. Has someone computed/estimated what is at cost $$$ value of utilizing these models at full tilt: several messages per minute and at least 500,000 token context windows? What we need is a wikipedia like effort to support something truly open and continually improving in its quality.
simonw 1y ago
> I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. I have been building that for a couple of years now: https://llm.datasette.io https://llm.datasette.io
vunderba 1y ago
> an open source tool that can plug into either free and open source LLMs or private ones Fortunately there are many of these that can integrate with offline LLMs through systems like LiteLLM/Ollama/etc. Off the top of my head, I'd look into Continue, Cline and Aider. https://github.com/continuedev/continue https://github.com/continuedev/continue https://github.com/cline/cline https://github.com/cline/cline https://github.com/Aider-AI/aider https://github.com/Aider-AI/aider
IanCal 1y ago
> Well, isn't it time we started doing the same with LLMs? I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. Almost all providers and models can be used with the OpenAI api and swapping between them is trivial.

by dakiol 1y ago

azan_ 1y ago

> I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. That would at least allow us to switch to a free and opensource version if the companies behind the private LLMs go rogue. I'm afraid tho that wouldn't be enough, but it's a starting point. There are open source tools that do exactly that already.

dakiol 1y ago
Ah, well that's nice. But every single post I read don't mention them? So, I assume they are not popular for some reason. Again, my main point here is: the normalization of using private LLMs. I don't see anyone talking about it; we are all just handing over a huge part of what it means to build software to a couple of enterprises whose goal is, of course, to maximize profit. So, yeah, perhaps I'm overthinking I don't know; I just don't like that now these companies are so ingrained in the act of building software (just like AWS is so ingrained in the act of running software)
- rahimnathwani 1y ago
  But every single post I read don't mention them? Why would they? Does every single post about a Jetbrains feature mention that you can easily switch from Jetbrains to an open source editor like VS Code or vim?
- deleted 1y ago
  [deleted]
- Arainach 1y ago
  >every single post I read don't mention them Because the models are so much worse that people aren't using them. Philosophical battles don't pay the bills and for most of us they aren't fun. There have been periods of my life where I stubbornly persisted using something inferior for various reasons - maybe I was passionate about it, maybe I wanted it to exist and was willing to spend my time debugging and offer feedback - but there a finite number of hours in my life and often I'd much rather pay for something that works well than throw my heart, soul, time, and blood pressure at something that will only give me pain.

airstrike 1y ago

None of that applies here since we could all easily switch to open models at a moment's notice with limited costs. In fact, we switch between proprietary models every few months. It just so happens that closed models are better today.

ghm2180 1y ago

> I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. Has someone computed/estimated what is at cost $$$ value of utilizing these models at full tilt: several messages per minute and at least 500,000 token context windows? What we need is a wikipedia like effort to support something truly open and continually improving in its quality.

simonw 1y ago

> I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. I have been building that for a couple of years now: https://llm.datasette.io https://llm.datasette.io

vunderba 1y ago

> an open source tool that can plug into either free and open source LLMs or private ones Fortunately there are many of these that can integrate with offline LLMs through systems like LiteLLM/Ollama/etc. Off the top of my head, I'd look into Continue, Cline and Aider. https://github.com/continuedev/continue https://github.com/continuedev/continue https://github.com/cline/cline https://github.com/cline/cline https://github.com/Aider-AI/aider https://github.com/Aider-AI/aider

IanCal 1y ago

> Well, isn't it time we started doing the same with LLMs? I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. Almost all providers and models can be used with the OpenAI api and swapping between them is trivial.