5 ms·

A large contention of this essay (which I’m assuming the talk is based on or is transcribed from depending on order) I do think that open source models will eve

A large contention of this essay (which I’m assuming the talk is based on or is transcribed from depending on order) I do think that open source models will eventually catch up to closed source ones, or at least be “good enough” and I also think you can already see how LLMs are augmenting knowledge work. I don’t think it’s the 4th wave of pioneering a new dawn of civilization but it’s clear LLMs will remain useful when applied correctly.

umeshunni 1y ago
> I do think that open source models will eventually catch up to closed source ones It felt like that was the direction for a while, but in the last year or so, the gap seems to have widened. I'm curious whether this is my perception or validated by some metric.
- msgodel 1y ago
  Already today I can use aider with qwen3 for free but have to pay per token to use it with any of the commercial models. The flexibility is worth the lower performance.
  - QRY 1y ago
    Do you have anything to share on that workflow? I've been trying to get a local-first AI thing going, could use your insights!
    - msgodel 1y ago
      It's super easy. I already had llama.cpp/llama-server set up for a bunch of other stuff and actually had my own homebrew RAG dialog engine, aider is just way better. One crazy thing is that since I keep all my PIM data in git in flat text I now have essentially "siri for Linux" too if I want it. It's a great example of what Karpathy was talking about where improvements in the ML model have consumed the older decision trees and coded integrations. I'd highly recommend /nothink in the system prompt. Qwen3 is not good at reasoning and tends to get stuck in loops until it fills up its context window. My current config is qwen2.5-coder-0.5b for my editor plugin and qwen3-8b for interactive chat and aider. I use nibble quants for everything. 0.5b is not enough for something like aider, 8b is too much for interactive editing. I'd also recommend shrinking the ring context in the neovim plugin if you use that since the default is 32k tokens which takes forever and generates a ton of heat.
      - QRY 1y ago
        I really appreciate you going into such technical specificity, thank you! I'll have to steal that siri for linux setup, that sounds awesome. Exploring ways to make use of compute people have lying around to do useful things without the vendor dependencies. But I'm relatively new to the AI scene, so your input really boosts my learning speed, thank you again!
- no_wizard 1y ago
  This was how early browsers felt too, the open source browser engines were slower at adapting than the ones developed by Netscape and Microsoft, but eventually it all reversed and open source excelled past the closed source software. Another way to put it, is that over time you see this, it usually takes a little while for open source projects to catch up, but once they do they gain traction quite quickly over the closed source counter parts.
  - tayo42 1y ago
    Those were way simpler projects in the beginning when that happened. Like do you think a new browser would catch up today chrome now?
    - no_wizard 1y ago
      The tech behind LLMs has been open source for a very long time. Look at DeepSeek and LLAMA for example. They aren’t yet as capable as say Gemini but they aren’t “miles behind” either, especially if you know how to tune the models to be purpose built[0]. The time horizons will be different as they always are, but I believe it will happen eventually. I’d also argue that browsers got complicated pretty fast, long cry from libhtml in a few short years. [0]: of which I contend most useful applications of this technology will not be the generalized ChatGPT interface but specialized highly tuned models that don’t need the scope of a generalized querying
  - umeshunni 1y ago
    That's a good analogy. Makes a lot of sense. The only caveat I see is that there is a lot of context locked up in proprietary data sets (e.g. YT, books, podcasts) and I'm not sure how OSS models get access to that.
bix6 1y ago
Why would open source outpace? Isn’t there way more money in the closed source ones and therefore more incentive to work on them?
- no_wizard 1y ago
  I didn’t say outpace, but I do believe the collective nature of open source will allow it to catch up, much like it did with browser tech, and at which point you’ll see a shift of resources toward that by major companies. It’s a collective works thing. I think it also is attractive to work on in open source, much like Linux or web browsers (hence the comparison to one) and that will also help it along over time. I stick by my general thesis that OSS will eventually catch up or the gap will be so small only frontier applications will benefit from using the most advanced models
- oblio 1y ago
  They didn't say "outpace", they said "catch up to good enough levels".

umeshunni 1y ago

> I do think that open source models will eventually catch up to closed source ones It felt like that was the direction for a while, but in the last year or so, the gap seems to have widened. I'm curious whether this is my perception or validated by some metric.

msgodel 1y ago
Already today I can use aider with qwen3 for free but have to pay per token to use it with any of the commercial models. The flexibility is worth the lower performance.
- QRY 1y ago
  Do you have anything to share on that workflow? I've been trying to get a local-first AI thing going, could use your insights!
  - msgodel 1y ago
    It's super easy. I already had llama.cpp/llama-server set up for a bunch of other stuff and actually had my own homebrew RAG dialog engine, aider is just way better. One crazy thing is that since I keep all my PIM data in git in flat text I now have essentially "siri for Linux" too if I want it. It's a great example of what Karpathy was talking about where improvements in the ML model have consumed the older decision trees and coded integrations. I'd highly recommend /nothink in the system prompt. Qwen3 is not good at reasoning and tends to get stuck in loops until it fills up its context window. My current config is qwen2.5-coder-0.5b for my editor plugin and qwen3-8b for interactive chat and aider. I use nibble quants for everything. 0.5b is not enough for something like aider, 8b is too much for interactive editing. I'd also recommend shrinking the ring context in the neovim plugin if you use that since the default is 32k tokens which takes forever and generates a ton of heat.
    - QRY 1y ago
      I really appreciate you going into such technical specificity, thank you! I'll have to steal that siri for linux setup, that sounds awesome. Exploring ways to make use of compute people have lying around to do useful things without the vendor dependencies. But I'm relatively new to the AI scene, so your input really boosts my learning speed, thank you again!
no_wizard 1y ago
This was how early browsers felt too, the open source browser engines were slower at adapting than the ones developed by Netscape and Microsoft, but eventually it all reversed and open source excelled past the closed source software. Another way to put it, is that over time you see this, it usually takes a little while for open source projects to catch up, but once they do they gain traction quite quickly over the closed source counter parts.
- tayo42 1y ago
  Those were way simpler projects in the beginning when that happened. Like do you think a new browser would catch up today chrome now?
  - no_wizard 1y ago
    The tech behind LLMs has been open source for a very long time. Look at DeepSeek and LLAMA for example. They aren’t yet as capable as say Gemini but they aren’t “miles behind” either, especially if you know how to tune the models to be purpose built[0]. The time horizons will be different as they always are, but I believe it will happen eventually. I’d also argue that browsers got complicated pretty fast, long cry from libhtml in a few short years. [0]: of which I contend most useful applications of this technology will not be the generalized ChatGPT interface but specialized highly tuned models that don’t need the scope of a generalized querying
- umeshunni 1y ago
  That's a good analogy. Makes a lot of sense. The only caveat I see is that there is a lot of context locked up in proprietary data sets (e.g. YT, books, podcasts) and I'm not sure how OSS models get access to that.

bix6 1y ago

Why would open source outpace? Isn’t there way more money in the closed source ones and therefore more incentive to work on them?

no_wizard 1y ago
I didn’t say outpace, but I do believe the collective nature of open source will allow it to catch up, much like it did with browser tech, and at which point you’ll see a shift of resources toward that by major companies. It’s a collective works thing. I think it also is attractive to work on in open source, much like Linux or web browsers (hence the comparison to one) and that will also help it along over time. I stick by my general thesis that OSS will eventually catch up or the gap will be so small only frontier applications will benefit from using the most advanced models
oblio 1y ago
They didn't say "outpace", they said "catch up to good enough levels".