3 ms·

No. Something stuck out to me today: Instagram presented me with Meta’s AI prompt. In the interface it prompted me to ask it anything. It’s a blank text box th

by dangus 2y ago

No. Something stuck out to me today: Instagram presented me with Meta’s AI prompt. In the interface it prompted me to ask it anything. It’s a blank text box that I’m supposed to interact with. But why am I using that in something like Instagram? What’s my motivation to ask this AI something within Instagram? It dawned on me that even the most sophisticated AI companies out there have no idea what to do with LLMs and where they are best applied. Later on in the day I talked to a colleague and they work at a company that is demanding they make an AI product. They have no specifications or vision on what it should do. Every company that does this is going to make a glorified chat bot, which of course they’ll inevitably find out nobody wants. Every company that has needed “AI” has already been using machine learning and data analysis techniques that have nothing to do with LLMs for years now. What we see now are a bunch of companies with no vision trying to figure out how to make the next ChatGPT. But hey I like your post as a science fiction book. Do I think AI will help surveillance states? Sure. Do I think that we are going to get the worst case scenario sci-fi novel version of the future? Probably not.

defrost 2y ago
Yep. For now at least there's an infuriating amount of "Do you want AI with that?" That aside, technically I see AI results as a (one of many many possibilities) regression fit | "best match" (by some criteria or another) result. That's a good start, but having dealt with such things in applied domans for decades there's still a whole other level of "it's close, but is it correct", what if we fudge the metrics and measures, etc. that still calls for informed human judgement. Potentially a great tool but still needing a craftsperson. For now.
atleastoptimal 2y ago
You are assuming the issues with LLM's today will persist into the future and that they will not improve. Perhaps consider that the mass adoption of genAI by major tech companies isn't entirely due to an assumption that plugging in whatever is available now will be useful, but rather that their efforts are for the sake of establishing footing and infrastructure in a domain which will prove useful in the near future. They are building highways when cars can travel 25mph because they know soon they'll be able to travel 50mph. All economic activity is based on the assumption that the future is a real thing.
- pokerface_86 2y ago
  i think there is a very very real limit to LLM technology that hasn’t been publicly solved yet. fundamentally, how do you get it to do something new? openAI’s moat is their training data, but again, all machine learning techniques fundamentally don’t get a model to produce something novel. they quite literally have to do something that’s been done before, maybe spelled a little differently. until that’s solved, i feel LLM’s will always have a peak below a skilled human.
  - atleastoptimal 2y ago
    That depends on what counts as "new". It's very easy to humans and LLM to produce something new. Just ask them "make up a new word and tell me its definition" or "write a story about (random thing a) and (random thing b). What I'm sure you're referring to is "new" and "useful", which is the standard by which we judge the utility and force of innovation. Very few humans could even do this. The problem is partially due to many of the "low-hanging fruit" innovations having already been discovered, along with the fact that creating something of value requires an intersecting intuition over many domains, modalities, considerations, etc. It is a test of general intelligence, and AI has not gotten there yet, but there's no reason to think that this is a fundamental limit to LLM's.
- dangus 2y ago
  I'm assuming that LLMs will hit the plateau that all emerging technology hits. Just because cars used to travel at 25 MPH and their speed increased to 75 MPH doesn't mean we are going to drive them at 300 MPH anytime in the future.
  - atleastoptimal 2y ago
    The point is that the plateau could be 6 months from now or decades from now, you don't know, and can't make a judgement solely on the idea that all tech hits plateaus. Moore's law held steady for decades. Nvidia's chip improvements, the main driver behind exponential gains in AI compute per dollar, are not close to slowing down. Even if the current generation of chips is the last one forever (almost 0% chance this is the case), all the SOTA models today were trained on chips from 1-2 years ago, there's still a 2-4x gain per dollar to be made for free. I'll believe LLM's plateau when they do plateau, but judging on the rate of development in algorithmic advances, chip capacity, multi-modal integration to broaden training data, investment, etc, it seems silly to just assume it will just plateau at a level no more significant that the level we are at now.

by dangus 2y ago

defrost 2y ago

Yep. For now at least there's an infuriating amount of "Do you want AI with that?" That aside, technically I see AI results as a (one of many many possibilities) regression fit | "best match" (by some criteria or another) result. That's a good start, but having dealt with such things in applied domans for decades there's still a whole other level of "it's close, but is it correct", what if we fudge the metrics and measures, etc. that still calls for informed human judgement. Potentially a great tool but still needing a craftsperson. For now.

atleastoptimal 2y ago

You are assuming the issues with LLM's today will persist into the future and that they will not improve. Perhaps consider that the mass adoption of genAI by major tech companies isn't entirely due to an assumption that plugging in whatever is available now will be useful, but rather that their efforts are for the sake of establishing footing and infrastructure in a domain which will prove useful in the near future. They are building highways when cars can travel 25mph because they know soon they'll be able to travel 50mph. All economic activity is based on the assumption that the future is a real thing.

pokerface_86 2y ago
i think there is a very very real limit to LLM technology that hasn’t been publicly solved yet. fundamentally, how do you get it to do something new? openAI’s moat is their training data, but again, all machine learning techniques fundamentally don’t get a model to produce something novel. they quite literally have to do something that’s been done before, maybe spelled a little differently. until that’s solved, i feel LLM’s will always have a peak below a skilled human.
- atleastoptimal 2y ago
  That depends on what counts as "new". It's very easy to humans and LLM to produce something new. Just ask them "make up a new word and tell me its definition" or "write a story about (random thing a) and (random thing b). What I'm sure you're referring to is "new" and "useful", which is the standard by which we judge the utility and force of innovation. Very few humans could even do this. The problem is partially due to many of the "low-hanging fruit" innovations having already been discovered, along with the fact that creating something of value requires an intersecting intuition over many domains, modalities, considerations, etc. It is a test of general intelligence, and AI has not gotten there yet, but there's no reason to think that this is a fundamental limit to LLM's.
dangus 2y ago
I'm assuming that LLMs will hit the plateau that all emerging technology hits. Just because cars used to travel at 25 MPH and their speed increased to 75 MPH doesn't mean we are going to drive them at 300 MPH anytime in the future.
- atleastoptimal 2y ago
  The point is that the plateau could be 6 months from now or decades from now, you don't know, and can't make a judgement solely on the idea that all tech hits plateaus. Moore's law held steady for decades. Nvidia's chip improvements, the main driver behind exponential gains in AI compute per dollar, are not close to slowing down. Even if the current generation of chips is the last one forever (almost 0% chance this is the case), all the SOTA models today were trained on chips from 1-2 years ago, there's still a 2-4x gain per dollar to be made for free. I'll believe LLM's plateau when they do plateau, but judging on the rate of development in algorithmic advances, chip capacity, multi-modal integration to broaden training data, investment, etc, it seems silly to just assume it will just plateau at a level no more significant that the level we are at now.