3 ms·

ML in general has been in production in many places/companies for a while now. Specifically GPt-4 is useful as a coding assistant and a reference tool. Howeve

ML in general has been in production in many places/companies for a while now. Specifically GPt-4 is useful as a coding assistant and a reference tool. However, it's hard to know whether what it's telling you is accurate or fake and if you want to be thorough you need to double check it. So, it's already useful but in a limited way and it remains to be seen how much better it's likely to get in the near future. But let's not confuse that in general with AI or Machine Learning which is already used heavily in lots of places. the specific type of architecture that gpt-4 uses for next word prediction is not the only possible architecture and is not what's used for many real world tasks. There a lot of different problems being addressed by ML and next word prediction is just one of them, although quite important.

eladgil 3y ago
Agreed on ML being in production in a lot of places and has been quite valuable, particularly for large incumbents. I write about this a bit here: https://blog.eladgil.com/p/ai-startup-vs-incumbent-value https://blog.eladgil.com/p/ai-startup-vs-incumbent-value I think the difference this time is the types of capabilities provided by transformers vs prior waves of AI are sufficiently different to allow many more types of startups to emerge, as well as big changes in some types of enterprise software by incumbents - in ways that were not enabled by pre-existing ML approaches.
- vasilipupkin 3y ago
  The jury is still out on how useful these additional capabilities provided by transformers are. The question is what is the degree to which it’s possible to reduce the frequency and severity of hallucinations. If that degree is limited without major changes in architecture or major new breakthroughs, then usefulness of gpt-4 style models will be limited. And we just don’t know the answer yet. So far, usefulness of gpt-4 is real but extremely limited. Another issue is this approach means models are costly to train and don’t easily incorporate latest info about the world. In short, it’s way too early to hype this up.

eladgil 3y ago

Agreed on ML being in production in a lot of places and has been quite valuable, particularly for large incumbents. I write about this a bit here: https://blog.eladgil.com/p/ai-startup-vs-incumbent-value https://blog.eladgil.com/p/ai-startup-vs-incumbent-value I think the difference this time is the types of capabilities provided by transformers vs prior waves of AI are sufficiently different to allow many more types of startups to emerge, as well as big changes in some types of enterprise software by incumbents - in ways that were not enabled by pre-existing ML approaches.

vasilipupkin 3y ago
The jury is still out on how useful these additional capabilities provided by transformers are. The question is what is the degree to which it’s possible to reduce the frequency and severity of hallucinations. If that degree is limited without major changes in architecture or major new breakthroughs, then usefulness of gpt-4 style models will be limited. And we just don’t know the answer yet. So far, usefulness of gpt-4 is real but extremely limited. Another issue is this approach means models are costly to train and don’t easily incorporate latest info about the world. In short, it’s way too early to hype this up.