37 ms·
I think this is the point where the field has just entered pseudoscientific nonsense. If this stuff were properly understood, these rules could be part of the
by dwroberts 3y ago
I think this is the point where the field has just entered pseudoscientific nonsense.
If this stuff were properly understood, these rules could be part of the model itself. The fact that ‘prompts’ are being used to manipulate its behaviour is, to me, a huge red flag
- andrewpolidori 3y agoInteresting! What do you think it's a red flag for? If it's easier to use the prompt why not do that at the level of abstraction that makes sense?
- layer8 3y agoIt means we don’t understand the mapping between the model and the prompt-level semantics. It also means we can’t cleanly separate (1) controlling the conversational behavior of the model from (2) having a conversation with the model. It means we have a black box that we can experiment on, but we can’t plan or predict how to control and customize it, nor can we reliably predict and control its behavior, because we can’t accurately reason about it.
- kenjackson 3y agoNone of your conclusions are true. We can plan and predict just fine with it. It’s not fully deterministic, but still more predictable than people. And there are a lot of things we don’t understand but are fairly predictable, like the use of pain medications. They aren’t 100% predictable, but good enough to be able to administer as part of a well thought out regimen. And given the fact that we can test it in a somewhat closed system gives us much more ability to predict its behavior than many things “in real life”.
- Tactician_mark 3y agoSure, it's a sign that we don't "understand this stuff properly", but you can say the same about human brains. Is it a red flag that we use language to communicate with each other instead of manipulating nerve impulses directly?
- LadyCailin 3y agoWe have no choice but to have and use our brains, not so with LLMs. We don’t have to start building core technologies off of fundamentally flawed models.
- manojlds 3y agoYeah, if we wanted to flap wings to fly, we would have never got the airplane.
- Filligree 3y agoThe ornithopter is real, mind you.
- stevenhuang 3y ago> fundamentally flawed Citation needed
- diydsp 3y agoGreat point. Btw: The problem is corporate irresponsibility: When self-driving cars were first coming out a professor of mine said "They only have to be as a good as humans." It took a while but now i can say why that's insufficient: human errors are corrected by discipline and justice. Corporations dissipate responsibility by design. When self-driving cars kill, no one goes to jail. Corporate fines are notoriously ineffective, just a cost of doing business. And even without the legal power, most people do try to drive well enough to bit injure each other which is a different calculus from prematurely taking products to market for financial gain.
- cdogl 3y agoVery succinctly put - this captures my view but I couldn’t have put it in these words; thanks!
- kenjackson 3y agoHuman errors are not corrected by discipline and justice. Drunk driving is still a huge problem.
- izacus 3y agoYeah, I wonder how much of the same could be achieved by just adding a few keywords. Like in Ultima Online, where people could either do "Dear Sir, may I buy your wares?" which would be equivalent to "buy wares".
- mentos 3y agovendor buy the bank guards
- cornholio 3y agoIt's not pseudosience if the prompts are engineered according to the scientific method: formulate a hypothesis, experiment, reincorporate the results into your knowledge. But it's a very fuzzy and soft science, almost on par with social sciences: your experimental results are not bounded by hard, unchanging physical reality; rather, you poke at a unknown and unknowable dynamic and self-reflexive system that in the most pathological cases behaves in an adversarial manner, trying to derail you or appropriating your own conclusions and changing its behavior to invalidate them. See, for example, economics as a science.
- lgas 3y ago> But it's a very fuzzy and soft science, almost on par with social sciences There's no need for name-calling.
- Davidzheng 3y agoI think what you're asking for--this total scientific control and understanding of how the model will behave was never going to happen for any models close to our own intelligence.
- rabbits_2002 3y agoI think thats a bit pessimistic. I am sure it would be possible to bake these prompts into the model but if this is equally effective and much less effort to do it this way, why is it an issue?