4 ms·

All these policy prompts remind me laws of robotics by Asimov [1], and definitely our current 'robots' frequently violate them. Asimov's laws are more logical s

by nuccy 3y ago

All these policy prompts remind me laws of robotics by Asimov [1], and definitely our current 'robots' frequently violate them. Asimov's laws are more logical since those are hierarchical with high-to-low prioritization and self-referencing. Can't those LLM/text-to-image model rules be embedded in the training/alighnment process instead of being injected before user input? 1. https://en.m.wikipedia.org/wiki/Three_Laws_of_Robotics https://en.m.wikipedia.org/wiki/Three_Laws_of_Robotics

ilaksh 3y ago
Following rules is part of the reinforcement learning tuning process I believe. In reference to the Three Laws, see also GATO framework: https://github.com/daveshap/GATO_Framework https://github.com/daveshap/GATO_Framework
Chabsff 3y ago
If you read Asimov's short stories and novels, you'll find that the point being made over and over again is that despite them sounding ironclad at first, the laws are naïve, futile, fraught with unexpected ambiguity, and ultimately cause more trouble than they solve. People have this idea that Asimov envisioned a world where robotics was based on the rules, but it's the opposite really. He was claiming that there is no such thing as absolute rules once intelligence starts getting involved, and that nuance and grey areas are inevitable. The three laws were never more than a straw man to be taken down, and it's really weird to me whenever anyone uses them as some kind of north star wrt/ to AI ethics. So in that sense, the comparison is definitely apt :)
- KineticLensman 3y ago
  Yes exactly. I also enjoyed charles Stross’s take on the laws of robotics in Saturns Children, an SF which explores the problems that robots face with the laws after humankind has gone extinct.
cypress66 3y ago
> Can't those LLM/text-to-image model rules be embedded in the training/alighnment process instead of being injected before user input? Absolutely. The model would fairly easily learn these rules with enough training even if you don't include such prompt. But the prompt helps with training stability, and with not hurting other tasks.

by nuccy 3y ago

ilaksh 3y ago

Following rules is part of the reinforcement learning tuning process I believe. In reference to the Three Laws, see also GATO framework: https://github.com/daveshap/GATO_Framework https://github.com/daveshap/GATO_Framework

Chabsff 3y ago

If you read Asimov's short stories and novels, you'll find that the point being made over and over again is that despite them sounding ironclad at first, the laws are naïve, futile, fraught with unexpected ambiguity, and ultimately cause more trouble than they solve. People have this idea that Asimov envisioned a world where robotics was based on the rules, but it's the opposite really. He was claiming that there is no such thing as absolute rules once intelligence starts getting involved, and that nuance and grey areas are inevitable. The three laws were never more than a straw man to be taken down, and it's really weird to me whenever anyone uses them as some kind of north star wrt/ to AI ethics. So in that sense, the comparison is definitely apt :)

KineticLensman 3y ago
Yes exactly. I also enjoyed charles Stross’s take on the laws of robotics in Saturns Children, an SF which explores the problems that robots face with the laws after humankind has gone extinct.

cypress66 3y ago

> Can't those LLM/text-to-image model rules be embedded in the training/alighnment process instead of being injected before user input? Absolutely. The model would fairly easily learn these rules with enough training even if you don't include such prompt. But the prompt helps with training stability, and with not hurting other tasks.