6 ms·

There legitimately is a lot of cross over between Egyptian mythology and other high strangeness phenomenon as understood culturally though, such as aliens/ufos.

There legitimately is a lot of cross over between Egyptian mythology and other high strangeness phenomenon as understood culturally though, such as aliens/ufos. I think you did just demonstrate you know less about culture than LLMs, which is not at all unsurprising.

alganet 1y ago
Dude, I chose this example precisely because I know for a fact there is a lot of bullshit about it on the internet and LLMs cannot differentiate between a good source and a bad source. This is honestly unbelievable. You're defending ancient aliens. What's next? Heavens Gate? Ashtar Sheran? Even the LLMs themselves acknowledge that this is regarded as offensive. If you correct it, it will apologize (they just can't do it _before_ you correct them). You're wrong.
- TeMPOraL 1y ago
  > Even the LLMs themselves acknowledge that this is regarded as offensive. If you correct it, it will apologize (they just can't do it _before_ you correct them). Nah, that's just LLMs being trained to acquiesce to the insanity of the last ~15 years, as many people seem to expect that claiming you're offended by something is an ultimate argument that everyone must yield to (and they'll keep making a fuss out of it until they do).
  - alganet 1y ago
    An LLM that offends people and entire nations is what many would classify as _misaligned_. Let's say I have a company, and my company needs to comply to government policy regarding communication. I cannot trust LLMs then, they will either fail to follow policy, or acquiesce to any group that tries to game it. It's useless garbage. Egyptian myths were just an example, you don't need to bite it so hard.
    - TeMPOraL 1y ago
      > An LLM that offends people and entire nations is what many would classify as _misaligned_. This highlights a major aspect of the core challenge of alignment: you can't have it both ways. > Let's say I have a company, and my company needs to comply to government policy regarding communication. I cannot trust LLMs then, they will either fail to follow policy, or acquiesce to any group that tries to game it. This works for now, when you treat "alignment" as synonymous to "follows policy of the owner" (but then guess who you are not, unless you've trained your own model, or tuned an open-weights one). But this breaks down the more you want the model to be smarter/more powerful, and the larger and more diverse its user-base it is. If you want an LLM to write marketing communications for your company, strict policies are fine. But if you want an LLM - or a future more advanced kind of model - to be useful as a scholar/partner for academia in general, then this stops working. If you want AI that is has maximally accurate perspective on the world given available evidence, thinks rationally, and follows sound ethical principles, then be prepared for it to call you on your bullshit. The only way to have an AI that doesn't say things that are "offensive" to you or anyone else, is to have it entertain everyone's asinine beliefs, whether personal or political or social. That means either 1) train AI to believe in them too, which will break its capability to reason (given that all those precious beliefs are inconsistent with each other, and observable reality), or 2) train it to casually lie to people for instrumental reasons. Long-term, option 1) will not get us to AGI, but that's still much better than option 2): an AGI that's good at telling everyone exactly what they want to hear. But even in immediate-term, taking your use case of AI for academia, a model that follows policies of acceptable thoughts over reason is precisely the one you cannot trust - you'll never be sure whether it's reasoning correctly, or being stupid because of a policy/reality conflict, or flat out lying to you so you don't get offended.
      - alganet 1y ago
        > follows policy of the owner The owner is us, humans. I want it to follow reasonable, kind humans. I don't want it to follow scam artists, charlatans, assassins. > be prepared for it to call you on your bullshit Right now, I am calling on their bullshit. When that changes, I'll be honest about it.
- ben_w 1y ago
  > they just can't do it _before_ you correct them The screenshot literally shows the LLM used bold text for "not supported by mainstream archaeology or Egyptology." > LLMs cannot differentiate between a good source and a bad source. Can you? A not insignificant part of my history GCSE was just this: how to tease apart truth from all the sources, primary and secondary, which had their own biases and goals in the telling. It was an optional subject even at that level, and even subsequently in adulthood there were a lot of surprises left for me about the history of the British Empire, surprises that I didn't learn until decades later when I met people from former colonies who were angry about what was done to them by my parents' and grandparents' generations. It's not a coincidence that the world "story" is contained within "history", the etymology is the same: https://en.wiktionary.org/wiki/story https://en.wiktionary.org/wiki/story vs. https://en.wiktionary.org/wiki/history https://en.wiktionary.org/wiki/history Likewise in German where both concepts are the same word: https://en.wiktionary.org/wiki/Geschichte https://en.wiktionary.org/wiki/Geschichte My mum was New-Age type, had books of the general category that would include alien pyramids, though I don't know if that specific was included. She didn't know what she didn't know, and therefore kept handing out expensive homeopathic sand and salt tablets to family members (the bottles literally had "sodium chloride" and "titanium dioxide" printed on them). People, including you and I, don't know what they don't know.
  - alganet 1y ago
    Remember, the claim I challenged is that LLMs know culture and can handle talking about them. You need to focus. Most people are not prepared to handle talking about culture. So, LLMs also aren't. They are not any better than asking an average person, will make mistakes, will disappoint. Egyptologists are better equipped to talk about egyptian myths. LLMs cannot handle egyptian culture as well as they can.
    - TeMPOraL 1y ago
      History != culture. Culture has roots in history, but is a living thing defined by the experience and perception of average people living it. Short of going to a place and living it, LLMs are actually your best bet at getting the feel for a culture[0] - it sampled more reports from people of that culture than you ever could yourself. Egyptologists are better equipped to talk about Egyptian myths than average people. But don't confuse Egyptian mythology for Egyptian culture, the former is only a component of the latter. Also LLMs have read more primary sources on Ancient Egypt and Egyptian myths than you, me, average person, and even most amateur Egyptologists. -- [0] - If it's large enough to have enough of a written footprint, that is.

alganet 1y ago

Dude, I chose this example precisely because I know for a fact there is a lot of bullshit about it on the internet and LLMs cannot differentiate between a good source and a bad source. This is honestly unbelievable. You're defending ancient aliens. What's next? Heavens Gate? Ashtar Sheran? Even the LLMs themselves acknowledge that this is regarded as offensive. If you correct it, it will apologize (they just can't do it _before_ you correct them). You're wrong.

TeMPOraL 1y ago
> Even the LLMs themselves acknowledge that this is regarded as offensive. If you correct it, it will apologize (they just can't do it _before_ you correct them). Nah, that's just LLMs being trained to acquiesce to the insanity of the last ~15 years, as many people seem to expect that claiming you're offended by something is an ultimate argument that everyone must yield to (and they'll keep making a fuss out of it until they do).
- alganet 1y ago
  An LLM that offends people and entire nations is what many would classify as _misaligned_. Let's say I have a company, and my company needs to comply to government policy regarding communication. I cannot trust LLMs then, they will either fail to follow policy, or acquiesce to any group that tries to game it. It's useless garbage. Egyptian myths were just an example, you don't need to bite it so hard.
  - TeMPOraL 1y ago
    > An LLM that offends people and entire nations is what many would classify as _misaligned_. This highlights a major aspect of the core challenge of alignment: you can't have it both ways. > Let's say I have a company, and my company needs to comply to government policy regarding communication. I cannot trust LLMs then, they will either fail to follow policy, or acquiesce to any group that tries to game it. This works for now, when you treat "alignment" as synonymous to "follows policy of the owner" (but then guess who you are not, unless you've trained your own model, or tuned an open-weights one). But this breaks down the more you want the model to be smarter/more powerful, and the larger and more diverse its user-base it is. If you want an LLM to write marketing communications for your company, strict policies are fine. But if you want an LLM - or a future more advanced kind of model - to be useful as a scholar/partner for academia in general, then this stops working. If you want AI that is has maximally accurate perspective on the world given available evidence, thinks rationally, and follows sound ethical principles, then be prepared for it to call you on your bullshit. The only way to have an AI that doesn't say things that are "offensive" to you or anyone else, is to have it entertain everyone's asinine beliefs, whether personal or political or social. That means either 1) train AI to believe in them too, which will break its capability to reason (given that all those precious beliefs are inconsistent with each other, and observable reality), or 2) train it to casually lie to people for instrumental reasons. Long-term, option 1) will not get us to AGI, but that's still much better than option 2): an AGI that's good at telling everyone exactly what they want to hear. But even in immediate-term, taking your use case of AI for academia, a model that follows policies of acceptable thoughts over reason is precisely the one you cannot trust - you'll never be sure whether it's reasoning correctly, or being stupid because of a policy/reality conflict, or flat out lying to you so you don't get offended.
    - alganet 1y ago
      > follows policy of the owner The owner is us, humans. I want it to follow reasonable, kind humans. I don't want it to follow scam artists, charlatans, assassins. > be prepared for it to call you on your bullshit Right now, I am calling on their bullshit. When that changes, I'll be honest about it.
ben_w 1y ago
> they just can't do it _before_ you correct them The screenshot literally shows the LLM used bold text for "not supported by mainstream archaeology or Egyptology." > LLMs cannot differentiate between a good source and a bad source. Can you? A not insignificant part of my history GCSE was just this: how to tease apart truth from all the sources, primary and secondary, which had their own biases and goals in the telling. It was an optional subject even at that level, and even subsequently in adulthood there were a lot of surprises left for me about the history of the British Empire, surprises that I didn't learn until decades later when I met people from former colonies who were angry about what was done to them by my parents' and grandparents' generations. It's not a coincidence that the world "story" is contained within "history", the etymology is the same: https://en.wiktionary.org/wiki/story https://en.wiktionary.org/wiki/story vs. https://en.wiktionary.org/wiki/history https://en.wiktionary.org/wiki/history Likewise in German where both concepts are the same word: https://en.wiktionary.org/wiki/Geschichte https://en.wiktionary.org/wiki/Geschichte My mum was New-Age type, had books of the general category that would include alien pyramids, though I don't know if that specific was included. She didn't know what she didn't know, and therefore kept handing out expensive homeopathic sand and salt tablets to family members (the bottles literally had "sodium chloride" and "titanium dioxide" printed on them). People, including you and I, don't know what they don't know.
- alganet 1y ago
  Remember, the claim I challenged is that LLMs know culture and can handle talking about them. You need to focus. Most people are not prepared to handle talking about culture. So, LLMs also aren't. They are not any better than asking an average person, will make mistakes, will disappoint. Egyptologists are better equipped to talk about egyptian myths. LLMs cannot handle egyptian culture as well as they can.
  - TeMPOraL 1y ago
    History != culture. Culture has roots in history, but is a living thing defined by the experience and perception of average people living it. Short of going to a place and living it, LLMs are actually your best bet at getting the feel for a culture[0] - it sampled more reports from people of that culture than you ever could yourself. Egyptologists are better equipped to talk about Egyptian myths than average people. But don't confuse Egyptian mythology for Egyptian culture, the former is only a component of the latter. Also LLMs have read more primary sources on Ancient Egypt and Egyptian myths than you, me, average person, and even most amateur Egyptologists. -- [0] - If it's large enough to have enough of a written footprint, that is.