4 ms·

> Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. That's a pretty early

> Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. That's a pretty early definition of what we now call ASI (Artificial Super Intelligence). In the next paragraph the author goes to describe what we today call the "singularity" (ASI designing better ASI). But that term seems to be associated to some very weird communities, so the concept is relegated to sci-fi. Even though we're already seeing signs of things we have working towards this. Interesting to see that in the past "Man" was more optimistic :) > It is more probable than not that, within the twentieth century, an ultraintelligent machine will be built and that it will be the last invention that man need make, since it will lead to an “intelligence explosion.” Well, that didn't happen. > The first ultraintelligent machine will need to be ultraparallel, and is likely to be achieved with the help of a very large artificial neural net. Right on, that we have. > The required high degree of connectivity might be attained with the help of microminiature radio transmitters and receivers. Hahaha, this is straight out of 60s-70s sci-fi, where their best futuristic interfaces were smaller CRT screens / flashy keys, etc. > The first ultraintelligent machine will be educated partly by means of positive and negative reinforcement. The task of education will be eased if the machine is somewhat of a robot, sinae the activity of a robot is concrete. [...] the machine will be able to lem from experience, by means of positive and negative reinforcement, and the instruction of the machine will resemble that of a child. Heh, nice early insights. They missed the how, but RL is the thing that ultimately made it "click" and be useful. And there is increasing talk about embodiment and how that'll help the next iteration of models. So there's that. Overall a cool read. I skipped most of the middle part, only skimming for things here and there.

classified 3mo ago
> “intelligence explosion” > Well, that didn't happen. Not only that, but in my cynical eyes the proliferation of LLMs has triggered a stupidity explosion. Either that, or it just made it blatantly obvious by how much stupidity we have been surrounded the whole time without realizing it. No other development demonstrated so clearly that the dark ages where we believed in sorcery and miracles have never really ended.
- antonvs 3mo ago
  When I read this, there was a comment right below yours saying that they bet the military had LLMs in the 1980s. Consider your point proved.
- bsenftner 3mo ago
  I think it is exposing how weakly people grasp communications. The amount of implied everything in most people's communications is extremely high, and those same people do not understand they are using implied information that a non-human construct really struggles to figure out what the human is talking about. I help a lot of non-technical people use AI, and asking them not to imply and to explain what they mean... many cannot parse, the implied version is all they have ever held in their head, and my telling them that they are using implied information confuses them, they don't know or never realized the full meaning of their usage of language.
  - eszed 3mo ago
    Yes. Furthermore, when I write precisely - because it's an important subject (read: will cost the company money) where we must be on the same page - people's eyes glaze over, and they ask for a meeting or a phone call instead. So, we do that, and I answer all the questions, and then everyone goes away with a different idea about what we're talking about. The only person who doesn't do that is our controller, who'll tell me "I had to read this three times, but now I understand", and I love her for it. Sadly, she doesn't make decisions, just pays the bills. I'm sure it's my fault, somehow - but I'm a damn good writer; I just don't know how to communicate with people who won't put forth the effort to engage with detail. The latest advice is "put in through [LLM]". Which, yeah, takes out the details, and makes bullet points, but sometimes the details are the point... Right?
    - bsenftner 3mo ago
      I realize this may sounds like a lot of effort, but I find telling stories of how a person would use the system, with theatrics of confusion and failure, tend to penetrate the dull unwilling to think participants I must engage to get my work completed. I'm not an actor or a good speaker, I grew up with a severe stutter, and I find I have to "put on a show" to generate the dawn of comprehension in others.
dwoldrich 3mo ago
> Well, that didn't happen. The military had stealth aircraft in the 70's. I'll bet they had LLM or better in the 80's and the tech we have now is the consumer-grade version they seeded into industry in the 2020's.
- not-a-llm 3mo ago
  the same military is using computers with Windows and Xbox controllers and got upset when Anthropic didn't want to let them use their LLM
  - applfanboysbgon 3mo ago
    While I think GP comment was a bit looney, the first sentence is a fairly weak counterargument. Xbox controllers especially are used specifically for familiarity purposes. The US military can and does produce significantly more advanced control planes, but they aren't necessary for all purposes and if something is simple enough it's easier to not have to train people on something new. Likewise creating an own general-purpose OS would require significant time into training people how to use it.
    - not-a-llm 3mo ago
      sure, I was just talking about this Hollywood myth that the military is far ahead of everyone else, including in computing matters when we all know the military procurement birrocracy
  - dwoldrich 3mo ago
    I'm not saying the movie War Games was a limited hangout for a super-intelligent secret military computer system/network that already existed. I'm just saying it could have been. I see those trillion dollar data centers we're building in the US are classed as military installations or at least appear to be a public-private partnerships where the data centers are being built on air force bases. Perhaps the consumer compute tech has advanced to the point that it's good enough to host the next phase of intelligence for the military. Maybe all the water AI datacenters need is for energy, not cooling.
Nevermark 3mo ago
>> It is more probable than not that, within the twentieth century, an ultraintelligent machine will be built and that it will be the last invention that man need make, since it will lead to an “intelligence explosion.” > Well, that didn't happen. Predictions are not false because their claims are not in the past yet. That bar would make all predictions wrong. The faster things happen, the higher the speed-of-progress expectation bar gets raised. This is how objectively compounding progress gets interpreted subjectively as linear, or even as a stall. Despite the dramatically increased rate of progress compared to the context of decades or centuries of speculations about cognitive machines. Models being used to write a lot of the code for new models is a strong suggestion of compounding capability. With new models achieving higher scores. Not proof, but a high bar for evidence that we may be in that explosion now. The fact that models trained to match SOTA model behavior (i.e. distillation), now learn much faster and more cheaply, than models trained on human behavior, is also strong evidence that capabilities are compounding.
- NitpickLawyer 3mo ago
  > Not proof, but a high bar for evidence that we may be in that explosion now. I agree, that's why I said "we're seeing things we have working towards this". I think that the "jagged intelligence" that is often used to describe our curent models is confusing a lot of people. On the one hand you have models failing basic "trick" questions that a 5yo would get, but on the other hand you also have models that write better/faster kernels that can run faster/better models, and are currently used to improve the next generation (via dataset prep, filtering, RL environment creation/curation and so on).
- AnimalMuppet 3mo ago
  >>> It is more probable than not that, within the twentieth century, an ultraintelligent machine will be built and that it will be the last invention that man need make, since it will lead to an “intelligence explosion.” >> Well, that didn't happen. > Predictions are not false because their claims are not in the past yet. "Within the twentieth century" places it definitely in the past, and definitely false.
  - 3mo ago
nialse 3mo ago
> > The required high degree of connectivity might be attained with the help of microminiature radio transmitters and receivers. > Hahaha, this is straight out of 60s-70s sci-fi, where their best futuristic interfaces were smaller CRT screens / flashy keys, etc. Actually that is not far off from how high bandwidth interconnects work, but through other media.
- zulux 3mo ago
  Agreed. It's basically RF guided by wires, or light guided by glass/plastic. And... I hate to say it, but I've been interacting with computers connected to WiFi about 90% of the time over the last week. Rarely used a physical Ethernet link.
  - shermantanktop 3mo ago
    Funny to think that WiFi is in fact just a specialized variant of the radio technology explosion that started with Marconi. Technically of course the path is continual evolution, but the applications are wildly different.
awinter-py 3mo ago
Yeah the hyperparallelism and the fact that it would emerge from machine translation are shockingly good calls. There are things in there that resemble the bag of words model, his 'clumps' sound a lot like tokenization, and he has concepts like RAG. Also, of the early AI writers, I think he was the least doom-y. Norbert wiener and I think von neumann were very concerned about the social effects of the singularity (von neumann's word for it). Vernor Vinge has an 80s essay calling down to Good a lot, and credits him with a 'meta golden rule' which aspirationally says that what comes after us will care about their origins.
usefulcat 3mo ago
> It is more probable than not that, within the twentieth century, an ultraintelligent machine will be built and that it will be the last invention that man need make, since it will lead to an “intelligence explosion.” It’s not uncommon to find these kinds of predictions from the twentieth century, which of course with hindsight can seem wildly optimistic. But I think there are understandable reasons for that optimism. Imagine you were born in the early twentieth century, say roughly 1910 +/- 10 years. If you had been born then, you would probably have lived through all of the following: - The transition from horses to motor vehicles - The invention of the airplane, from first flight to supersonic - Not one but two world wars - The splitting of the atom - Human spaceflight, including traveling to the moon If you’d been born slightly earlier, you could also add electrification to that list. When I compare what I’ve seen in my life so far, I don’t think anything comes close (for better and worse) to even one of those things. So yeah, of course they were optimistic about technological progress—look at what they just lived through.
- Timwi 3mo ago
  > When I compare what I’ve seen in my life so far, I don’t think anything comes close (for better and worse) to even one of those things. I dunno, I personally consider the internetification of the world to be on a level with the developments you've listed. We've transitioned a lot of things away from the real world into cyberspace: banking, shopping, postal communication... and now even recent technologies such as radio and TV are being replaced by it. I don't know how old you are, but I've lived through that and it has been quite a ride — for better and worse, as you say. Nevertheless, I agree that little has changed since. Maybe this puts you at < 20 years of age? I don't want to be presumptuous.
  - usefulcat 3mo ago
    No, I'm much older than that. I was in college in the early 90s when the internet first became known to the general public. I guess I just don't consider internetification to be that impressive. I mean, sure, it's nice that I can pay for things electronically and Amazon is very convenient, but compared to being able to fly? Or travel to the moon? Or split the atom? To say nothing of the many, many millions who were directly affected by two world wars. As for the replacement of radio and TV, I think that's one of the least impressive aspects. Broadcasting via the internet enables on-demand consumption. Which is nice, but hardly revolutionary compared to the ability to broadcast audio and video at all, which we had already been doing for many decades. There's also the fact that it's easier for individuals to publish/broadcast, which I suspect is more impactful. However there is one major exception related to my opinion of the impact of the internet. The huge number of 'information sources' that exist today (previously these would have been called news sources) means that it's pretty trivial for every person to listen to only those sources that cater to their existing preferences. Which is exactly what nearly everyone does. IMO this has had a huge impact on driving political polarization, which is in turn having a very large negative impact on governance and society as a whole. At least in the US, people were far less politically divided when there were fewer sources of news, probably because there was less disagreement on basic facts.
jcelerier 3mo ago
> The required high degree of connectivity might be attained with the help of microminiature radio transmitters and receivers. I mean pretty much everything has a wifi chip these days

classified 3mo ago

> “intelligence explosion” > Well, that didn't happen. Not only that, but in my cynical eyes the proliferation of LLMs has triggered a stupidity explosion. Either that, or it just made it blatantly obvious by how much stupidity we have been surrounded the whole time without realizing it. No other development demonstrated so clearly that the dark ages where we believed in sorcery and miracles have never really ended.

antonvs 3mo ago
When I read this, there was a comment right below yours saying that they bet the military had LLMs in the 1980s. Consider your point proved.
bsenftner 3mo ago
I think it is exposing how weakly people grasp communications. The amount of implied everything in most people's communications is extremely high, and those same people do not understand they are using implied information that a non-human construct really struggles to figure out what the human is talking about. I help a lot of non-technical people use AI, and asking them not to imply and to explain what they mean... many cannot parse, the implied version is all they have ever held in their head, and my telling them that they are using implied information confuses them, they don't know or never realized the full meaning of their usage of language.
- eszed 3mo ago
  Yes. Furthermore, when I write precisely - because it's an important subject (read: will cost the company money) where we must be on the same page - people's eyes glaze over, and they ask for a meeting or a phone call instead. So, we do that, and I answer all the questions, and then everyone goes away with a different idea about what we're talking about. The only person who doesn't do that is our controller, who'll tell me "I had to read this three times, but now I understand", and I love her for it. Sadly, she doesn't make decisions, just pays the bills. I'm sure it's my fault, somehow - but I'm a damn good writer; I just don't know how to communicate with people who won't put forth the effort to engage with detail. The latest advice is "put in through [LLM]". Which, yeah, takes out the details, and makes bullet points, but sometimes the details are the point... Right?
  - bsenftner 3mo ago
    I realize this may sounds like a lot of effort, but I find telling stories of how a person would use the system, with theatrics of confusion and failure, tend to penetrate the dull unwilling to think participants I must engage to get my work completed. I'm not an actor or a good speaker, I grew up with a severe stutter, and I find I have to "put on a show" to generate the dawn of comprehension in others.

dwoldrich 3mo ago

> Well, that didn't happen. The military had stealth aircraft in the 70's. I'll bet they had LLM or better in the 80's and the tech we have now is the consumer-grade version they seeded into industry in the 2020's.

not-a-llm 3mo ago
the same military is using computers with Windows and Xbox controllers and got upset when Anthropic didn't want to let them use their LLM
- applfanboysbgon 3mo ago
  While I think GP comment was a bit looney, the first sentence is a fairly weak counterargument. Xbox controllers especially are used specifically for familiarity purposes. The US military can and does produce significantly more advanced control planes, but they aren't necessary for all purposes and if something is simple enough it's easier to not have to train people on something new. Likewise creating an own general-purpose OS would require significant time into training people how to use it.
  - not-a-llm 3mo ago
    sure, I was just talking about this Hollywood myth that the military is far ahead of everyone else, including in computing matters when we all know the military procurement birrocracy
- dwoldrich 3mo ago
  I'm not saying the movie War Games was a limited hangout for a super-intelligent secret military computer system/network that already existed. I'm just saying it could have been. I see those trillion dollar data centers we're building in the US are classed as military installations or at least appear to be a public-private partnerships where the data centers are being built on air force bases. Perhaps the consumer compute tech has advanced to the point that it's good enough to host the next phase of intelligence for the military. Maybe all the water AI datacenters need is for energy, not cooling.

Nevermark 3mo ago

>> It is more probable than not that, within the twentieth century, an ultraintelligent machine will be built and that it will be the last invention that man need make, since it will lead to an “intelligence explosion.” > Well, that didn't happen. Predictions are not false because their claims are not in the past yet. That bar would make all predictions wrong. The faster things happen, the higher the speed-of-progress expectation bar gets raised. This is how objectively compounding progress gets interpreted subjectively as linear, or even as a stall. Despite the dramatically increased rate of progress compared to the context of decades or centuries of speculations about cognitive machines. Models being used to write a lot of the code for new models is a strong suggestion of compounding capability. With new models achieving higher scores. Not proof, but a high bar for evidence that we may be in that explosion now. The fact that models trained to match SOTA model behavior (i.e. distillation), now learn much faster and more cheaply, than models trained on human behavior, is also strong evidence that capabilities are compounding.

NitpickLawyer 3mo ago
> Not proof, but a high bar for evidence that we may be in that explosion now. I agree, that's why I said "we're seeing things we have working towards this". I think that the "jagged intelligence" that is often used to describe our curent models is confusing a lot of people. On the one hand you have models failing basic "trick" questions that a 5yo would get, but on the other hand you also have models that write better/faster kernels that can run faster/better models, and are currently used to improve the next generation (via dataset prep, filtering, RL environment creation/curation and so on).
AnimalMuppet 3mo ago
>>> It is more probable than not that, within the twentieth century, an ultraintelligent machine will be built and that it will be the last invention that man need make, since it will lead to an “intelligence explosion.” >> Well, that didn't happen. > Predictions are not false because their claims are not in the past yet. "Within the twentieth century" places it definitely in the past, and definitely false.
- 3mo ago

nialse 3mo ago

> > The required high degree of connectivity might be attained with the help of microminiature radio transmitters and receivers. > Hahaha, this is straight out of 60s-70s sci-fi, where their best futuristic interfaces were smaller CRT screens / flashy keys, etc. Actually that is not far off from how high bandwidth interconnects work, but through other media.

zulux 3mo ago
Agreed. It's basically RF guided by wires, or light guided by glass/plastic. And... I hate to say it, but I've been interacting with computers connected to WiFi about 90% of the time over the last week. Rarely used a physical Ethernet link.
- shermantanktop 3mo ago
  Funny to think that WiFi is in fact just a specialized variant of the radio technology explosion that started with Marconi. Technically of course the path is continual evolution, but the applications are wildly different.

awinter-py 3mo ago

Yeah the hyperparallelism and the fact that it would emerge from machine translation are shockingly good calls. There are things in there that resemble the bag of words model, his 'clumps' sound a lot like tokenization, and he has concepts like RAG. Also, of the early AI writers, I think he was the least doom-y. Norbert wiener and I think von neumann were very concerned about the social effects of the singularity (von neumann's word for it). Vernor Vinge has an 80s essay calling down to Good a lot, and credits him with a 'meta golden rule' which aspirationally says that what comes after us will care about their origins.

usefulcat 3mo ago

> It is more probable than not that, within the twentieth century, an ultraintelligent machine will be built and that it will be the last invention that man need make, since it will lead to an “intelligence explosion.” It’s not uncommon to find these kinds of predictions from the twentieth century, which of course with hindsight can seem wildly optimistic. But I think there are understandable reasons for that optimism. Imagine you were born in the early twentieth century, say roughly 1910 +/- 10 years. If you had been born then, you would probably have lived through all of the following: - The transition from horses to motor vehicles - The invention of the airplane, from first flight to supersonic - Not one but two world wars - The splitting of the atom - Human spaceflight, including traveling to the moon If you’d been born slightly earlier, you could also add electrification to that list. When I compare what I’ve seen in my life so far, I don’t think anything comes close (for better and worse) to even one of those things. So yeah, of course they were optimistic about technological progress—look at what they just lived through.

Timwi 3mo ago
> When I compare what I’ve seen in my life so far, I don’t think anything comes close (for better and worse) to even one of those things. I dunno, I personally consider the internetification of the world to be on a level with the developments you've listed. We've transitioned a lot of things away from the real world into cyberspace: banking, shopping, postal communication... and now even recent technologies such as radio and TV are being replaced by it. I don't know how old you are, but I've lived through that and it has been quite a ride — for better and worse, as you say. Nevertheless, I agree that little has changed since. Maybe this puts you at < 20 years of age? I don't want to be presumptuous.
- usefulcat 3mo ago
  No, I'm much older than that. I was in college in the early 90s when the internet first became known to the general public. I guess I just don't consider internetification to be that impressive. I mean, sure, it's nice that I can pay for things electronically and Amazon is very convenient, but compared to being able to fly? Or travel to the moon? Or split the atom? To say nothing of the many, many millions who were directly affected by two world wars. As for the replacement of radio and TV, I think that's one of the least impressive aspects. Broadcasting via the internet enables on-demand consumption. Which is nice, but hardly revolutionary compared to the ability to broadcast audio and video at all, which we had already been doing for many decades. There's also the fact that it's easier for individuals to publish/broadcast, which I suspect is more impactful. However there is one major exception related to my opinion of the impact of the internet. The huge number of 'information sources' that exist today (previously these would have been called news sources) means that it's pretty trivial for every person to listen to only those sources that cater to their existing preferences. Which is exactly what nearly everyone does. IMO this has had a huge impact on driving political polarization, which is in turn having a very large negative impact on governance and society as a whole. At least in the US, people were far less politically divided when there were fewer sources of news, probably because there was less disagreement on basic facts.

jcelerier 3mo ago

> The required high degree of connectivity might be attained with the help of microminiature radio transmitters and receivers. I mean pretty much everything has a wifi chip these days