4 ms·

You can run a model that can beat 4o which was released less than 6 months ago _locally_! I know this requires a ton of hardware but OpenAI will not be the lead

by msoad 2y ago

You can run a model that can beat 4o which was released less than 6 months ago _locally_! I know this requires a ton of hardware but OpenAI will not be the leader in 2025 I can assume. Always bet on open source (or rather somewhat more open development strategies) The math and coding performance is what we really care about. I am paying for o1 Pro and also Sonnet, in my experience beside Sonnet being faster, it is also better at many tasks. In a few instances I got answers from o1 Pro but it's not justifying the price so I am cancelling and going back to $20/mo. I am currently paying for Cursor, Claude, ChatGPT and v0! The productivity I am gaining from those tools are totally worth it (except for o1 Pro). But I am really hoping at some point those tools converge so I can pay less. For instance I am looking forward to VSCode Copilot improvements so I can go back to VSCode and once Claude has no limits I rather pay for one AI system.

chvid 2y ago
OpenAi toppled as LLM leader by an open source / open weight company? OpenAi has much more capital and compute than any of its competitors (especially deepseek); if that was to happen it would demonstrate that capital and compute doesn't matter as much as it is assumed ... (and it just might be the thing that pops the current ai bubble).
- xnx 2y ago
  > OpenAi has much more capital and compute than any of its competitors Isn't openai still losing money? I don't think they own any data centers.
- cloverich 2y ago
  Until the models can host themselves theyll always need a company to make the experience good enough for typical users; OpenAI can always host open source models instead of their own and their user base will mostly stick around, especially if they can leverage their existing base into a network effect. I wouldnt be surprised it they are investing heavily into this vs pure model hosting running. Im thinking their real challenge will be surviving Apple (once they go all in) or Google (if they can figure out how to make a good product). Or something along those lines.
polotics 2y ago
well yes, locally, if you assume that someone's got about 300'000 dollars of hardware at hand... right? as you are not paying for Gemini, may I ask why, did you try it and find it inferior?
- msoad 2y ago
  Gemini for coding does not work for me. It gets so many things wrong
  - sumedh 2y ago
    Which Gemini AI model did you use?
  - xnx 2y ago
    You should try again. Gemini rates highest on coding at lmarena.
- KTibow 2y ago
  You actually can't pay for the latest models, they're only available as free with limits
- apexalpha 2y ago
  I bought two (relatively) old datacenter GPUs with 48gb VRAM total for €200 that gets me 7 token/s for a 70b model.
  - 383toast 2y ago
    which GPUs?
    - zargon 2y ago
      Not the GP, but I bought a few P40s over the summer for $150 each. Last I checked they're more expensive now, but it's still cheap vram and fast enough at inference for me.
    - 2y ago

by msoad 2y ago

chvid 2y ago

OpenAi toppled as LLM leader by an open source / open weight company? OpenAi has much more capital and compute than any of its competitors (especially deepseek); if that was to happen it would demonstrate that capital and compute doesn't matter as much as it is assumed ... (and it just might be the thing that pops the current ai bubble).

xnx 2y ago
> OpenAi has much more capital and compute than any of its competitors Isn't openai still losing money? I don't think they own any data centers.
cloverich 2y ago
Until the models can host themselves theyll always need a company to make the experience good enough for typical users; OpenAI can always host open source models instead of their own and their user base will mostly stick around, especially if they can leverage their existing base into a network effect. I wouldnt be surprised it they are investing heavily into this vs pure model hosting running. Im thinking their real challenge will be surviving Apple (once they go all in) or Google (if they can figure out how to make a good product). Or something along those lines.

polotics 2y ago

well yes, locally, if you assume that someone's got about 300'000 dollars of hardware at hand... right? as you are not paying for Gemini, may I ask why, did you try it and find it inferior?

msoad 2y ago
Gemini for coding does not work for me. It gets so many things wrong
- sumedh 2y ago
  Which Gemini AI model did you use?
- xnx 2y ago
  You should try again. Gemini rates highest on coding at lmarena.
KTibow 2y ago
You actually can't pay for the latest models, they're only available as free with limits
apexalpha 2y ago
I bought two (relatively) old datacenter GPUs with 48gb VRAM total for €200 that gets me 7 token/s for a 70b model.
- 383toast 2y ago
  which GPUs?
  - zargon 2y ago
    Not the GP, but I bought a few P40s over the summer for $150 each. Last I checked they're more expensive now, but it's still cheap vram and fast enough at inference for me.
  - 2y ago