4 ms·
It does seem like the (misnamed because it’s not open) OpenAI is very far ahead of most other efforts, especially at the edges in areas like instruction trainin
by api 4y ago
It does seem like the (misnamed because it’s not open) OpenAI is very far ahead of most other efforts, especially at the edges in areas like instruction training and output filtering.
Playing with Llama 65G gave me a sense for what the median raw effort is probably like. It seems to take a lot of work to fine tune and harness these systems and get them reliably producing useful output.
- CuriouslyC 4y agoI don't think it's possible to build a moat around models at all. The model architectures are public, and there are already distributed group training projects so the compute isn't a barrier. The only moat is data.
- levidos 4y agoOpenAI stopped publishing the architecture of GPT-4, so I'm worried that architecture availability will not be as available in the futire