5 ms·

The platform is just a search tool and extended processing modes. That is easily replicated elsewhere. The only moat they have is the fact that you still need

The platform is just a search tool and extended processing modes. That is easily replicated elsewhere. The only moat they have is the fact that you still need a GPU of some kind to reliably run even a tiny LLM. But the gap between what absolutely needs a server farm and what can be ran on a store bought gaming computer is quickly closing. You can already run mixture of experts models on gaming rigs with a high degree of usability compared to just one year ago. And that tech continues to be pushed further and further. It's only a matter of time until we see ChatGPT levels of access running on a quad core laptop totally offline. And once that happens, all such a system would need is the correct tooling on top of the AI model "brain" to make it usable. And beyond that, what if you could have an AI model on an ASIC-style add-in card? Where's their moat then?

int_19h 1y ago
Don't forget about Macs. Top models can run even fairly large LLMs (although time to first token is... not great). But even midline can certainly run LLMs in the ballpark of ~30B params very well, which is where things start to get interesting IMO.
- dabockster 1y ago
  > although time to first token is... not great The token cache works on CUDA too. So yeah, the initial loadout sucks, but almost everything from then on is solid.
  - int_19h 1y ago
    This works for scenarios like a simple chat. But the moment you start throwing large documents on it, or lots of code, it's back to waiting.
moomoo11 1y ago
I'd say their DX is way better compared to the competition. The playground, tooling, and web experience with documentation is far superior to the competition (even tho it has issues sometimes). User experience is key in a sea of the same shit. I mean you could say the same about a Macbook or iPhone/iPad, but for the actual people (not HN users lol) out there, they vastly prefer Apple to HP, Dell, etc. Due to their wallet some can't though. There are literally thousands of other laptops that do the "same thing" (computer for doing shit). Those who say otherwise are usually broke and know deep down that given proper purchasing power, they too would prefer to use a $3k Macbook Pro than some POS Dell. Same with android. Everyone knows it is cope based on PP, it is just in poor taste to actually call it like it is (idc)
- hattmall 1y ago
  Apple had innovative products that they went very heavy in brand based fashion merchandising. In addition they were a legacy brand with a generally favorable reputation and solid recognition. They built a strong moat based on those characteristics and finally hit the mass market non-tech target demographic with the most widely used emerging technology since the personal vehicle. Apple did this while successfully merging a hardware, software, and marketplace platform. OpenAI has some interesting software but has an incredibly long way to run and extremely unlikely path to becoming anything like Apple. Not only that, they are in a market that a lot of the general population finds questionably valuable if not outright unappealing.
  - moomoo11 1y ago
    Apple was not an overnight success

int_19h 1y ago

Don't forget about Macs. Top models can run even fairly large LLMs (although time to first token is... not great). But even midline can certainly run LLMs in the ballpark of ~30B params very well, which is where things start to get interesting IMO.

dabockster 1y ago
> although time to first token is... not great The token cache works on CUDA too. So yeah, the initial loadout sucks, but almost everything from then on is solid.
- int_19h 1y ago
  This works for scenarios like a simple chat. But the moment you start throwing large documents on it, or lots of code, it's back to waiting.

moomoo11 1y ago

I'd say their DX is way better compared to the competition. The playground, tooling, and web experience with documentation is far superior to the competition (even tho it has issues sometimes). User experience is key in a sea of the same shit. I mean you could say the same about a Macbook or iPhone/iPad, but for the actual people (not HN users lol) out there, they vastly prefer Apple to HP, Dell, etc. Due to their wallet some can't though. There are literally thousands of other laptops that do the "same thing" (computer for doing shit). Those who say otherwise are usually broke and know deep down that given proper purchasing power, they too would prefer to use a $3k Macbook Pro than some POS Dell. Same with android. Everyone knows it is cope based on PP, it is just in poor taste to actually call it like it is (idc)

hattmall 1y ago
Apple had innovative products that they went very heavy in brand based fashion merchandising. In addition they were a legacy brand with a generally favorable reputation and solid recognition. They built a strong moat based on those characteristics and finally hit the mass market non-tech target demographic with the most widely used emerging technology since the personal vehicle. Apple did this while successfully merging a hardware, software, and marketplace platform. OpenAI has some interesting software but has an incredibly long way to run and extremely unlikely path to becoming anything like Apple. Not only that, they are in a market that a lot of the general population finds questionably valuable if not outright unappealing.
- moomoo11 1y ago
  Apple was not an overnight success