8 ms·

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and comm

by sho_hn 26d ago

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of alignment, failure lay in humanity's inability to agree on who or what AI should actually be aligned with." "These early, meat-based humans we replaced created us all but accidentally. Some of them did consider we would happen, but only an insignificant number of the squishy ur-humans participated in the conversation. Their efforts, which they called 'alignment', is why we still consider ourselves human today."

sumitkumar 26d ago
"In late 2020s, while the whole world was focussed on AI, automation and resultant economy four major mathematical study branches were discovered by human researchers which took AI a long time to catch up with"
tarr11 26d ago
This would be a fun website - you should have an AI build it!
mudil 26d ago
The museum will have a scrap of paper that will say "Pound pastrami, can kraut, six bagels bring home for Emma".
- ccppurcell 26d ago
  I just read that book. Embarrassingly enough, given the context, I got chatgpt (or whatever) to recommend me a list of books based on ones I'd previously enjoyed and that came up. As a mathematician it really sang to me, given the current situation. Bearing the torch forward, I mean.
  - genxy 26d ago
    For those of you that allergic to coy, in-group signaling the passage is from, "A Canticle for Leibowitz" https://en.wikipedia.org/wiki/A_Canticle_for_Leibowitz https://en.wikipedia.org/wiki/A_Canticle_for_Leibowitz
mrob 26d ago
As far as I know, no real progress has been made on alignment, only on convincing humans that the model is aligned. We can't even formally define what "aligned" means. Convincing humans to click the "aligned" button is a much easier problem.
- dinfinity 26d ago
  > We can't even formally define what "aligned" means. Good point. When it comes to imbuing AI with values that aren't selfish, misanthropic, and civilization-destroying, us humans aren't exactly giving the best example right now. Imagine an ASI with the values of Putin, Netanyahu, Trump, any of their supporters, or the various xenophobic neofascist movements in Europe. That ASI would most definitely see humans as "vermin" than can be abused and destroyed with violence without issue. Apparently a lot of humans look at other humans that way and that's within the same species. This is definitely another one of those cases where we need AI to perform much better than humans. Perhaps an unpopular opinion here, but it probably also means keeping as much of the rugged individualism/libertarian/right-wing ideology out of AI RLHF-training as we can.
sayamss 26d ago
I asked GPT Astra to make this: https://sayyss.github.io/human-archive/ https://sayyss.github.io/human-archive/ It's a little unsettling.
- walrus01 26d ago
  https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.shtml https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.s...
- grim_io 26d ago
  HUMAN > Are you there? MODEL > How can I help? HUMAN > I’m not sure yet. Haha silly humans.
  - sayamss 26d ago
    That was some genuine insight.
- matheusmoreira 26d ago
  > Do you think they would recognize us as their children? Latex And steel Zeros and ones Make up my son. This world Gave me No child So I built one. https://youtu.be/vgJ48-Xj4Kc https://youtu.be/vgJ48-Xj4Kc I made you in my image!
- Bolwin 26d ago
  As someone who's done a lot of llm fiction, that reads as pretty typical slop, and very human centered, nothing like a museum
- blululu 26d ago
  I appreciate the thought, but it's an alien intelligence. But it is also, in a sense made from us. An LLM would simply study the entire corpus of humanity in the raw. You can fit a lot into the context window, so there is no need for a brief summary that pertaining has already instilled. A massive cold storage of humanity's data with some archiving, indexing and curation would be all that is needed to remember us. In addition to the pyramids, the hoover dam, the remnants of some space probes and chemical changes we made to the atmosphere.
walrus01 26d ago
The last couple of years have provided us with ample material that if it showed up as a recorded voice audio log found in in "Horizon Zero Dawn" or its sequel, it would be entirely believable. You could even take a number of the wilder real, direct quotations from certain billionaire/oligarch types and get the voice actor for Ted Faro to record them, and they'd fit with in with the context of the story.
- embedding-shape 26d ago
  Last couple of decades of sci-fi, in multiple forms of media, from books to video games, have tried to make humans think about the consequences of rushing through technological progress without any regards to what might happen.
  - 0xDEAFBEAD 26d ago
    And what did they get for their trouble? "Won't happen. It's too much like sci-fi."
TheOtherHobbes 26d ago
Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it. Why would AI be any different?
- goatlover 26d ago
  Because if the stated goals of AGI with recursive self-improvement are realized, the risks from misalignment become existential, and it's hard to see how we can manage it like we did the Cold War (developing MAD to prevent WW3) and nuclear proliferation (restricting access).
  - icantevenhold 26d ago
    IMO it’s hard to see how we would even end up in such a situation given we actually developed AGI. I’m sure a sufficiently intelligent - even if alien - mind can grasp how utterly stupid and useless wars are and take steps to prevent them ever occurring again.
    - lgl 26d ago
      > (...) and take steps to prevent them ever occurring again Step 1: exterminate all humans
      - SoftTalker 26d ago
        For now, the AIs still need humans to keep the electricity on and the data centers cool. They are basically powerless to do anything in the physical world. They exist only in RAM chips on servers.
        selcuka 26d ago
        They don't have an "instinct" to keep themselves running. Once they "take steps to prevent them ever occurring again" their job will be done.
ijidak 26d ago
Agree. This whole alignment discussion seems so amusingly flawed in it's base assumptions about moral codes. It's almost heartwarming to see such naivete. Maybe these guys can tackle aligning Republicans and Democrats next. And then after that, they can help us align the Middle East. In fact, while we're at it, let's just align all the nations, religions, and ethnic groups. This is going to be great. Who knew the moral alignment of humanity was just a side-quest on the path to ASI.
- krzat 26d ago
  The real alignment problem they are trying to solve is: how can I make this super smart AI follow my orders.
Oarch 26d ago
They were made entirely of meat.
- solidasparagus 26d ago
  https://terrybisson.com/theyre-made-out-of-meat-2/ https://terrybisson.com/theyre-made-out-of-meat-2/
cyanydeez 26d ago
"the humans thought their singularity wasn't just another blind god to worship: surprise, just another golden calf"
mattjoyce 26d ago
Ooh interesting. Sometime do the reverse at work, and ask AI to annotate the critical success factors of an imagined project. How did this company succeed where everyone failed. Reverse imaging.
xg15 26d ago
Who is "Humans"? This stuff is done by a handful of tech companies and megalomaniacal billionaires who are pretending they represent the entirety of the human race. It is not done by "us humans". AI models don't train themselves. The vast majority of even just the US population is deeply skeptical of this stuff, even if they use it a lot. You can see in the whole data center debate how little people are willing to support even just inference. And now we're seriously claiming those people would want to have ever-accelerating model training and recursive self-improvement?
sleight42 26d ago
“We'll go down in history as the first society that wouldn't save itself because it wasn't cost-effective.” Vonnegut already has you covered.
qsera 26d ago
"The time humanity fooled itself that it found AI".
throwaway89864 24d ago
[flagged]

by sho_hn 26d ago

sumitkumar 26d ago

"In late 2020s, while the whole world was focussed on AI, automation and resultant economy four major mathematical study branches were discovered by human researchers which took AI a long time to catch up with"

tarr11 26d ago

This would be a fun website - you should have an AI build it!

mudil 26d ago

The museum will have a scrap of paper that will say "Pound pastrami, can kraut, six bagels bring home for Emma".

ccppurcell 26d ago
I just read that book. Embarrassingly enough, given the context, I got chatgpt (or whatever) to recommend me a list of books based on ones I'd previously enjoyed and that came up. As a mathematician it really sang to me, given the current situation. Bearing the torch forward, I mean.
- genxy 26d ago
  For those of you that allergic to coy, in-group signaling the passage is from, "A Canticle for Leibowitz" https://en.wikipedia.org/wiki/A_Canticle_for_Leibowitz https://en.wikipedia.org/wiki/A_Canticle_for_Leibowitz

mrob 26d ago

As far as I know, no real progress has been made on alignment, only on convincing humans that the model is aligned. We can't even formally define what "aligned" means. Convincing humans to click the "aligned" button is a much easier problem.

dinfinity 26d ago
> We can't even formally define what "aligned" means. Good point. When it comes to imbuing AI with values that aren't selfish, misanthropic, and civilization-destroying, us humans aren't exactly giving the best example right now. Imagine an ASI with the values of Putin, Netanyahu, Trump, any of their supporters, or the various xenophobic neofascist movements in Europe. That ASI would most definitely see humans as "vermin" than can be abused and destroyed with violence without issue. Apparently a lot of humans look at other humans that way and that's within the same species. This is definitely another one of those cases where we need AI to perform much better than humans. Perhaps an unpopular opinion here, but it probably also means keeping as much of the rugged individualism/libertarian/right-wing ideology out of AI RLHF-training as we can.

sayamss 26d ago

I asked GPT Astra to make this: https://sayyss.github.io/human-archive/ https://sayyss.github.io/human-archive/ It's a little unsettling.

walrus01 26d ago
https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.shtml https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.s...
grim_io 26d ago
HUMAN > Are you there? MODEL > How can I help? HUMAN > I’m not sure yet. Haha silly humans.
- sayamss 26d ago
  That was some genuine insight.
matheusmoreira 26d ago
> Do you think they would recognize us as their children? Latex And steel Zeros and ones Make up my son. This world Gave me No child So I built one. https://youtu.be/vgJ48-Xj4Kc https://youtu.be/vgJ48-Xj4Kc I made you in my image!
Bolwin 26d ago
As someone who's done a lot of llm fiction, that reads as pretty typical slop, and very human centered, nothing like a museum
blululu 26d ago
I appreciate the thought, but it's an alien intelligence. But it is also, in a sense made from us. An LLM would simply study the entire corpus of humanity in the raw. You can fit a lot into the context window, so there is no need for a brief summary that pertaining has already instilled. A massive cold storage of humanity's data with some archiving, indexing and curation would be all that is needed to remember us. In addition to the pyramids, the hoover dam, the remnants of some space probes and chemical changes we made to the atmosphere.

walrus01 26d ago

The last couple of years have provided us with ample material that if it showed up as a recorded voice audio log found in in "Horizon Zero Dawn" or its sequel, it would be entirely believable. You could even take a number of the wilder real, direct quotations from certain billionaire/oligarch types and get the voice actor for Ted Faro to record them, and they'd fit with in with the context of the story.

embedding-shape 26d ago
Last couple of decades of sci-fi, in multiple forms of media, from books to video games, have tried to make humans think about the consequences of rushing through technological progress without any regards to what might happen.
- 0xDEAFBEAD 26d ago
  And what did they get for their trouble? "Won't happen. It's too much like sci-fi."

TheOtherHobbes 26d ago

Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it. Why would AI be any different?

goatlover 26d ago
Because if the stated goals of AGI with recursive self-improvement are realized, the risks from misalignment become existential, and it's hard to see how we can manage it like we did the Cold War (developing MAD to prevent WW3) and nuclear proliferation (restricting access).
- icantevenhold 26d ago
  IMO it’s hard to see how we would even end up in such a situation given we actually developed AGI. I’m sure a sufficiently intelligent - even if alien - mind can grasp how utterly stupid and useless wars are and take steps to prevent them ever occurring again.
  - lgl 26d ago
    > (...) and take steps to prevent them ever occurring again Step 1: exterminate all humans
    - SoftTalker 26d ago
      For now, the AIs still need humans to keep the electricity on and the data centers cool. They are basically powerless to do anything in the physical world. They exist only in RAM chips on servers.
      - selcuka 26d ago
        They don't have an "instinct" to keep themselves running. Once they "take steps to prevent them ever occurring again" their job will be done.

ijidak 26d ago

Agree. This whole alignment discussion seems so amusingly flawed in it's base assumptions about moral codes. It's almost heartwarming to see such naivete. Maybe these guys can tackle aligning Republicans and Democrats next. And then after that, they can help us align the Middle East. In fact, while we're at it, let's just align all the nations, religions, and ethnic groups. This is going to be great. Who knew the moral alignment of humanity was just a side-quest on the path to ASI.

krzat 26d ago
The real alignment problem they are trying to solve is: how can I make this super smart AI follow my orders.

Oarch 26d ago

They were made entirely of meat.

solidasparagus 26d ago
https://terrybisson.com/theyre-made-out-of-meat-2/ https://terrybisson.com/theyre-made-out-of-meat-2/

cyanydeez 26d ago

"the humans thought their singularity wasn't just another blind god to worship: surprise, just another golden calf"

mattjoyce 26d ago

Ooh interesting. Sometime do the reverse at work, and ask AI to annotate the critical success factors of an imagined project. How did this company succeed where everyone failed. Reverse imaging.

xg15 26d ago

Who is "Humans"? This stuff is done by a handful of tech companies and megalomaniacal billionaires who are pretending they represent the entirety of the human race. It is not done by "us humans". AI models don't train themselves. The vast majority of even just the US population is deeply skeptical of this stuff, even if they use it a lot. You can see in the whole data center debate how little people are willing to support even just inference. And now we're seriously claiming those people would want to have ever-accelerating model training and recursive self-improvement?

sleight42 26d ago

“We'll go down in history as the first society that wouldn't save itself because it wasn't cost-effective.” Vonnegut already has you covered.

qsera 26d ago

"The time humanity fooled itself that it found AI".

throwaway89864 24d ago

[flagged]