2 ms·

So you see, if you address this black box in a baby voice, on a Tuesday, during full moon, while standing on one foot, then your chances of a better answer are

So you see, if you address this black box in a baby voice, on a Tuesday, during full moon, while standing on one foot, then your chances of a better answer are increased! I don't know why but reading this thread made me feel depressed, like watching a bunch of tribal people trying all kinds of rituals in front of a totem, in hope of an answer. Say the magic incantation and watch the magic unfurl! Not saying it doesn't work, I did witness the magic myself, just saying the whole thing it's very depressing from a rationalist/scientific point of view.

erulabs 2y ago
It gets worse if you imagine a future AGI which just tells us new novel implementations of previously unknown physics but it either isn’t willing or can’t explain the rationale.
ManuelKiessling 2y ago
Isn’t that one of the cornerstones of the Mechwarrior universe, that thousands(?) of years in the future, there is a guild(?) that handles all the higher-level technology, but the actual knowledge has been long forgotten, and so they approach it in a quasi-religious way with chanting over cobbled-together systems or something like that? (Purely from memory from reading some Mechwarrior books about 30 years ago)
- GeoAtreides 2y ago
  Sounds more like the Adeptus Mechanicus from Warhammer 40K: https://warhammer40k.fandom.com/wiki/Adeptus_Mechanicus https://warhammer40k.fandom.com/wiki/Adeptus_Mechanicus
booleandilemma 2y ago
I agree. Whatever this is, it's not engineering (not software engineering, anyway), and it does feel like a regression to a more primitive time. Can ChatGPT Omni read? I can't wait for future people to be illiterate and just ask the robot to read things for them, Ancient Roman slave style.
- ClassyJacket 2y ago
  It reads text from images very well
TastyDucks 2y ago
The use of this sort of anthropomorphic and "incantation" style prompting is a workaround while mechanistic interpretability and monosemanticity work[1] is done to expose the neuron(s) that have larger impacts on model behavior -- cf Golden Gate Claude. Further, even if end-users only have access to token input to steer model behavior, we likely have the ability to reverse engineer optimal inputs to drive desired behaviors; convergent internal representations[2] means this research might transfer across models as well (particularly, Gemma -> Gemini, as I believe they share the same architecture and training data). I suspect we'll see understandable super-human prompting (and higher-level control) emerge from GAN and interpretability work within the next few years. [1]: https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html https://transformer-circuits.pub/2024/scaling-monosemanticit... [2]: https://arxiv.org/abs/2405.07987 https://arxiv.org/abs/2405.07987

erulabs 2y ago

It gets worse if you imagine a future AGI which just tells us new novel implementations of previously unknown physics but it either isn’t willing or can’t explain the rationale.

ManuelKiessling 2y ago

Isn’t that one of the cornerstones of the Mechwarrior universe, that thousands(?) of years in the future, there is a guild(?) that handles all the higher-level technology, but the actual knowledge has been long forgotten, and so they approach it in a quasi-religious way with chanting over cobbled-together systems or something like that? (Purely from memory from reading some Mechwarrior books about 30 years ago)

GeoAtreides 2y ago
Sounds more like the Adeptus Mechanicus from Warhammer 40K: https://warhammer40k.fandom.com/wiki/Adeptus_Mechanicus https://warhammer40k.fandom.com/wiki/Adeptus_Mechanicus

booleandilemma 2y ago

I agree. Whatever this is, it's not engineering (not software engineering, anyway), and it does feel like a regression to a more primitive time. Can ChatGPT Omni read? I can't wait for future people to be illiterate and just ask the robot to read things for them, Ancient Roman slave style.

ClassyJacket 2y ago
It reads text from images very well

TastyDucks 2y ago

The use of this sort of anthropomorphic and "incantation" style prompting is a workaround while mechanistic interpretability and monosemanticity work[1] is done to expose the neuron(s) that have larger impacts on model behavior -- cf Golden Gate Claude. Further, even if end-users only have access to token input to steer model behavior, we likely have the ability to reverse engineer optimal inputs to drive desired behaviors; convergent internal representations[2] means this research might transfer across models as well (particularly, Gemma -> Gemini, as I believe they share the same architecture and training data). I suspect we'll see understandable super-human prompting (and higher-level control) emerge from GAN and interpretability work within the next few years. [1]: https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html https://transformer-circuits.pub/2024/scaling-monosemanticit... [2]: https://arxiv.org/abs/2405.07987 https://arxiv.org/abs/2405.07987