3 ms·

I work in this space and think that it's far more rational to accept the axiom that LLM progress will not be significant than to bank product work on assuming i

I work in this space and think that it's far more rational to accept the axiom that LLM progress will not be significant than to bank product work on assuming it will increase drastically. I do think we still have yet to squeeze the most value out of current LLMs, but most people's radical AI dreams are completely out-of-touch with reality for anyone working closely on these problems. My biggest fear in this space is that disappointment in the inability of these tools to live up to the hype will cause people to irrationally abandon exploring the spaces where they do work.

6D794163636F756 3y ago
I think people assume that the current LLM release cycle is similar to other software releases which is to say, MVP followed by full product. ChatGPT is an actual product though, it's not the beta version of the technology. I think the dramatic increases in the product have likely mostly happened before launch but that you're right and we will see minor improvements.
ambrozk 3y ago
It may be rational and it may be irrational. All I'm saying is I want to see an argument. My impression of the AI space generally is that there are many, many obvious ideas which are simply waiting to be picked up off the ground and tested, and that it's not clear to anyone how much better (what are we even quantifying this with -- log loss?) the base LLM capabilities have to be before they're suitable for making tools that automate large quantities of work. Even if you assume that 240B is the limit for how many parameters number of engineers who have the ability to fine-tune GPT-4 or whatever Google is about to come out with is vanishingly small compared to the number of engineers who are participating in the open-source ML community, and the number of engineers in the open-source ML community is vanishingly small compared to the number of engineers in the long-tail of app developers who will ultimately adopt LLMs to their use-cases. Even assuming that GPT-4 is the best an LLM can possibly be, (which, again, I've seen no argument for), the widening of LLM availability and the building of practical tooling is a strong reason to believe that the utility of LLMs to concrete products will dramatically increase in the next 4-5 years.

6D794163636F756 3y ago

I think people assume that the current LLM release cycle is similar to other software releases which is to say, MVP followed by full product. ChatGPT is an actual product though, it's not the beta version of the technology. I think the dramatic increases in the product have likely mostly happened before launch but that you're right and we will see minor improvements.

ambrozk 3y ago

It may be rational and it may be irrational. All I'm saying is I want to see an argument. My impression of the AI space generally is that there are many, many obvious ideas which are simply waiting to be picked up off the ground and tested, and that it's not clear to anyone how much better (what are we even quantifying this with -- log loss?) the base LLM capabilities have to be before they're suitable for making tools that automate large quantities of work. Even if you assume that 240B is the limit for how many parameters number of engineers who have the ability to fine-tune GPT-4 or whatever Google is about to come out with is vanishingly small compared to the number of engineers who are participating in the open-source ML community, and the number of engineers in the open-source ML community is vanishingly small compared to the number of engineers in the long-tail of app developers who will ultimately adopt LLMs to their use-cases. Even assuming that GPT-4 is the best an LLM can possibly be, (which, again, I've seen no argument for), the widening of LLM availability and the building of practical tooling is a strong reason to believe that the utility of LLMs to concrete products will dramatically increase in the next 4-5 years.