3 ms·
> I’m seriously confused why anybody thinks anyone will even remotely care about any kind of pause is asked for. I can say that Google and Microsoft would defi
by mlinsey 4y ago
> I’m seriously confused why anybody thinks anyone will even remotely care about any kind of pause is asked for.
I can say that Google and Microsoft would definitely care, just as two specific instances of "anybody" that I have worked for and seen what their legal review process gating new product launches is like.
You seem much more optimistic that the community efforts are <6 months behind those companies than I am; I would be happy to be wrong here and see an open-source self-hosted LLM with better than GPT4 performance by September.
- alchemist1e9 4y agoI’ve read OpenAI only used around 4000 high end GPUs ($30K each) for around 4 months to train GPT4. Yes there are some reinforcement tricks that are still unclear but input data is basically known. I think people underestimate how many entities have the ability to easily acquire that amount of resources. It’s less than $200M to purchase that amount of hardware. Militaries spend $2-4M per missile. This is nothing for a lot of organizations. We already have self hosted LLM inference that reaches gpt 3.5 turbo level. Google and Microsoft are notoriously slow moving and probably less relevant than smaller faster operations like OpenAI.