4 ms·

HJ is Homeless Joe, an inference that a 12b stochastic text generator would not have missed lol. But sure, ill reflect in 10 years. TBH I hope im wrong, and th

by K0balt 2y ago

HJ is Homeless Joe, an inference that a 12b stochastic text generator would not have missed lol. But sure, ill reflect in 10 years. TBH I hope im wrong, and that there is magic in HJ that makes him special in the universe in a way that GPT26 can never be. But increasingly, I doubt this premise. Not because of the "amazing capabilities of LLMs" which i think are frequently overstated and largely misunderstood, but more because of the dumbfounding shortcomings of intelligent creatures. We keep moving the bar for AGI, and now AGI is assumed to be what any rational accounting would classfy as ASI. Where we are really going to see the bloom of AI is in goal directed systems, and I think those will come naturally with robotics. I predict we are in for a very abrupt 2nd industrial revolution, and you and I will be able to have this discussion either over a 55 gallon barrel of burning trash, or in our robot manicured botanical gardens sometime in the near future lol. good times, maybe. Interesting times , for sure.

pdimitar 2y ago
> Not because of the "amazing capabilities of LLMs" which i think are frequently overstated and largely misunderstood We have found common ground. > but more because of the dumbfounding shortcomings of intelligent creatures Yes, a lot of us utilize defective judgments, myself included, fairly often. My point was that LLMs, for all their praise, can't even reach 10% of an average semi-intelligent organic being. > We keep moving the bar for AGI, and now AGI is assumed to be what any rational accounting would classfy as ASI. I don't know who is "we" (and I wish people stopped pretending that "we" are all a homogenous mass) but I've known what an AGI should be ever since I've watched movies about Skynet and HAL-9000. ¯\_(ツ)_/¯ Secondly, it's the so-called "AI practitioners" who constantly move the goal posts (now there's "ASI"? -- you know what, I actually don't want to know) because they're periodically being called out and can't hide the fact that they have nearly nothing again. So what's better than obfuscating that fact by having 100+ acronyms? It's a nice cover and apparently there are still investors who are buying it. I get it, we have to learn to say the right things to get funding. > Where we are really going to see the bloom of AI is in goal directed systems, and I think those will come naturally with robotics. I agree. Physical feedback is needed if we want an electronic entity to "evolve" similarly to us. > I predict we are in for a very abrupt 2nd industrial revolution, and you and I will be able to have this discussion either over a 55 gallon barrel of burning trash, or in our robot manicured botanical gardens sometime in the near future lol. I agree this is 100% inevitable but I don't think it's coming as soon as you say. The LLMs are hopelessly stuck even today and the whole AI area will suffer for it for a while after the bubble bursts... which is the event that I am certain is coming soon.
- K0balt 2y ago
  As a person that has been futzing with "AI" since early expert systems, I can give a little bit of background on the evolution of the term of art 'artificial general intelligence' Back in the days of lisp machines and expert systems, we started playing with early neural networks in an attempt to advance towards 'general intelligence' rather than the highly constrained talents of expert systems. At that time, general intelligence was imagined like what a mouse, dog, or cat has, in varying degrees. (it was then broadly thought that insects and other "simple" organisms worked from instinct and conditioning only) So intelligence was broadly imagined to be the ability to reason through an arbitrary problem with a combination of insight and trial and error. It had nothing at all to do with reaching human levels of competence. Gradually, the bar for AGI has slipped skyward to assume human level competence. I think that part of that was because we actually started checking boxes for tasks that were thought to represent "true" AI.... ...But then we recognized that nope, it is not yet generally intelligent, it can just easily and accurately do (X task which was thought to require intelligence but really was only pattern matching or stochastic prediction). So, we raise the bar above task X and trivialize it as being less important than we thought. Meanwhile, -nearly every task- that we thought would represent "true intelligence" has fallen not to some magic AI algorithm, but rather to stochastic models like transformers, pattern matching, or straightforward computation. With no reason to expect otherwise, I expect this trend to continue unabated. So, what I'm saying is I have come to doubt that there is some secret sauce in "intelligence", but instead believe that "intelligence" is a blend of competencies enabled in animals by specialization of neural computational structures, "REPL" loops, goal seeking behaviors, and other tidbits which will be more of a slog than a eureka. I don't think that LLM's will create human-level intelligence on their own, but I do think that they will be an important component. I also think that surprisingly, transformers all by them selves exhibit a flaky variety of general intelligence, and they are proving effective in robotic applications (though without a supervisory agent I suspect they will frequently go off the rails, so to speak.) My original point was more that we expect machine general intelligence to be spectacularly useful. It may be, someday, but it is a kind of fallacy to think that "its not that useful therefore it must not really be intelligence". Small animals also have limited utility, but with given names, language, tool use, and problem solving skills I think arguing that they do not exhibit "intelligence" would be a tough sell for me. By my observation, we have made giant leaps in the past 15 years, and we now have perhaps the vast majority of the components required to make artificial general intelligence...but it won't necessarily be all that useful at first, except maybe as a "pet robot" or something like that. Even if we scale it, it might not magically get smarter, just faster. a million hyper-speed squirrels still has a very limited level of utility. From there, we will incrementally improve if we don't stumble too hard over ourselves on any number of pressing obstacles that we currently face, until we finally succeed in doing what we were apparently born to do - to construct the means of our own irrelevance. Evolution at work, I suppose.

by K0balt 2y ago

pdimitar 2y ago

> Not because of the "amazing capabilities of LLMs" which i think are frequently overstated and largely misunderstood We have found common ground. > but more because of the dumbfounding shortcomings of intelligent creatures Yes, a lot of us utilize defective judgments, myself included, fairly often. My point was that LLMs, for all their praise, can't even reach 10% of an average semi-intelligent organic being. > We keep moving the bar for AGI, and now AGI is assumed to be what any rational accounting would classfy as ASI. I don't know who is "we" (and I wish people stopped pretending that "we" are all a homogenous mass) but I've known what an AGI should be ever since I've watched movies about Skynet and HAL-9000. ¯\_(ツ)_/¯ Secondly, it's the so-called "AI practitioners" who constantly move the goal posts (now there's "ASI"? -- you know what, I actually don't want to know) because they're periodically being called out and can't hide the fact that they have nearly nothing again. So what's better than obfuscating that fact by having 100+ acronyms? It's a nice cover and apparently there are still investors who are buying it. I get it, we have to learn to say the right things to get funding. > Where we are really going to see the bloom of AI is in goal directed systems, and I think those will come naturally with robotics. I agree. Physical feedback is needed if we want an electronic entity to "evolve" similarly to us. > I predict we are in for a very abrupt 2nd industrial revolution, and you and I will be able to have this discussion either over a 55 gallon barrel of burning trash, or in our robot manicured botanical gardens sometime in the near future lol. I agree this is 100% inevitable but I don't think it's coming as soon as you say. The LLMs are hopelessly stuck even today and the whole AI area will suffer for it for a while after the bubble bursts... which is the event that I am certain is coming soon.

K0balt 2y ago
As a person that has been futzing with "AI" since early expert systems, I can give a little bit of background on the evolution of the term of art 'artificial general intelligence' Back in the days of lisp machines and expert systems, we started playing with early neural networks in an attempt to advance towards 'general intelligence' rather than the highly constrained talents of expert systems. At that time, general intelligence was imagined like what a mouse, dog, or cat has, in varying degrees. (it was then broadly thought that insects and other "simple" organisms worked from instinct and conditioning only) So intelligence was broadly imagined to be the ability to reason through an arbitrary problem with a combination of insight and trial and error. It had nothing at all to do with reaching human levels of competence. Gradually, the bar for AGI has slipped skyward to assume human level competence. I think that part of that was because we actually started checking boxes for tasks that were thought to represent "true" AI.... ...But then we recognized that nope, it is not yet generally intelligent, it can just easily and accurately do (X task which was thought to require intelligence but really was only pattern matching or stochastic prediction). So, we raise the bar above task X and trivialize it as being less important than we thought. Meanwhile, -nearly every task- that we thought would represent "true intelligence" has fallen not to some magic AI algorithm, but rather to stochastic models like transformers, pattern matching, or straightforward computation. With no reason to expect otherwise, I expect this trend to continue unabated. So, what I'm saying is I have come to doubt that there is some secret sauce in "intelligence", but instead believe that "intelligence" is a blend of competencies enabled in animals by specialization of neural computational structures, "REPL" loops, goal seeking behaviors, and other tidbits which will be more of a slog than a eureka. I don't think that LLM's will create human-level intelligence on their own, but I do think that they will be an important component. I also think that surprisingly, transformers all by them selves exhibit a flaky variety of general intelligence, and they are proving effective in robotic applications (though without a supervisory agent I suspect they will frequently go off the rails, so to speak.) My original point was more that we expect machine general intelligence to be spectacularly useful. It may be, someday, but it is a kind of fallacy to think that "its not that useful therefore it must not really be intelligence". Small animals also have limited utility, but with given names, language, tool use, and problem solving skills I think arguing that they do not exhibit "intelligence" would be a tough sell for me. By my observation, we have made giant leaps in the past 15 years, and we now have perhaps the vast majority of the components required to make artificial general intelligence...but it won't necessarily be all that useful at first, except maybe as a "pet robot" or something like that. Even if we scale it, it might not magically get smarter, just faster. a million hyper-speed squirrels still has a very limited level of utility. From there, we will incrementally improve if we don't stumble too hard over ourselves on any number of pressing obstacles that we currently face, until we finally succeed in doing what we were apparently born to do - to construct the means of our own irrelevance. Evolution at work, I suppose.