4 ms·
Norbert Wiener in 1960: "As is now generally admitted, over a limited range of operation, machines act far more rapidly than human beings and are far more prec
by RGS1811 2mo ago
Norbert Wiener in 1960:
"As is now generally admitted, over a limited range of operation, machines act far more rapidly than human beings and are far more precise in performing the details of their operations. This being the case, even when machines do not in any way transcend man's intelligence, they very well may, and often do, transcend man in the performance of tasks. An intelligent understanding of their mode of performance may be delayed until long after the task which they have been set has been completed. This means that though machines are theoretically subject to human criticism, such criticism may be ineffective until long after it is relevant. To be effective in warding off disastrous consequences, our understanding of our man-made machines should in general develop _pari passu_ with the performance of the machine. By the very slowness of our human actions, our effective control of our machines may be nullified. By the time we are able to react to information conveyed by our senses and stop the car we are driving, it may already have run head on into a wall."
"In neurophysiological language, ataxia can be quite as much of a deprivation as paralysis. A patient with locomotor ataxia may not suffer from any defect of his muscles or motor nerves, but if his muscles and tendons and organs do not tell him exactly what position he is in, and whether the tensions to which his organs are subjected will or will not lead to his falling, he will be unable to stand up. Similarly, when a machine constructed by us is capable of operating on its incoming data at a pace which we cannot keep, we may not know, until too late, when to turn it off."
Source: https://www.cs.umd.edu/users/gasarch/BLOGPAPERS/moral.pdf https://www.cs.umd.edu/users/gasarch/BLOGPAPERS/moral.pdf
- icebxrg 2mo ago"Car accidents occur therefore we shouldn't have cars" isn't very compelling.
- doc_ick 2mo agoIt’d be more like “car accidents occur, so let’s add seat belts, air bags, etc…”.
- TSiege 2mo agoYou’re not understanding what he’s saying and your argument likewise isn’t very compelling. He’s arguing that given the speed of computers we need to change what our expectations of better than human are. Furthermore one could presume from his description of needing to change human perceptions of the machines agility it is likely we need to change how we use them.
- RunSet 2mo agoA while ago I noticed that car crashes were the leading cause of death for age ranges too old for infant mortality and too young for heart failure. I checked again before making this reply and found that in many cases "accidental poisoning" has overtaken car crashes. Accidental poisoning is overwhelmingly "drugs". I do find your argument compelling even if you do not.
- Ancv123 2mo agoMaybe they didn't have proper debuggers in 1960? For a language model you need (RNG state, context, prompt). So if they wrote an LLM step by step debugger, it would be all deterministic. But they prefer rapid sales, chaos and mystique.
- itopaloglu83 2mo agoWe also have engineer blindness, so having human in the loop confirming thousands of requests would quickly start to confirm everything without looking. It would become just another system to hack through, and slow the development process as well. The OpenAI video in the article recommends an autonomous defense mechanism. For rapid reaction, but I don’t know how sustainable or effective that would be, or if as humans we will be able to keep up.
- andai 2mo agoI'm not sure I understand. Are you going to debug the neurons? They are trying to do that, but there are too many of them, so they're building new AIs to help them do that...
- efficax 2mo agollms are not strictly deterministic in the sense that even if you had the RNG state, context, and prompt you would likely not get an identical output even if there was no other randomness involved, because the concurrent scheduling of the massive amounts of floating point calculations can produce different results, since floating point arithmetic is not truly associative [(a+b)+c can differ from a+(b+c)] and the order in which these operations happen can result in subtly different final tensors. To reproduce it deterministically you'd have to also reproduce the exact scheduling of all matrix calculations among all the GPU cores (across different physical gpus!) that it took place on, which afaik is currently impossible.
- Bjartr 2mo agoThat's not inherent, that's a consequence of performance optimizations. It's absolutely a choice to run those matrix calculations in a way that fails to have predictable execution ordering. It's just that the speed benefits to allowing that are considerable. You can make it trivially deterministic by running single threaded on a cpu, but it's becomes too slow for practical applications if you do that.
- pmarreck 2mo agoWhat a paper! And you missed an even MORE relevant excerpt!! Man and Slave The problem, and it is a moral prob- lem, with which we are here faced is very close to one of the great problems of slavery. Let us grant that slavery is bad because it is cruel. It is, how- ever, self-contradictory, and for a reason which is quite different. We wish a slave to be intelligent, to be able to assist us in the carrying out of our tasks. However, we also wish him to be subservient. Complete subservience and complete intelligence do not go together. How often in ancient times the clever Greek philosopher slave of a less intelligent Roman slaveholder must have dominated the actions of his master rather than obeyed his wishes! Similarly, if the machines become more and more efficient and operate at a higher and higher psychological level, the catastrophe foreseen by Butler of the dominance of the ma- chine comes nearer and nearer.
- satvikpendem 2mo agoCan you unformat this, it's quite annoying to read on mobile
- layer8 2mo agoIt’s fine in landscape for me, but here you go: “The problem, and it is a moral problem, with which we are here faced is very close to one of the great problems of slavery. Let us grant that slavery is bad because it is cruel. It is, however, self-contradictory, and for a reason which is quite different. We wish a slave to be intelligent, to be able to assist us in the carrying out of our tasks. However, we also wish him to be subservient. Complete subservience and complete intelligence do not go together. How often in ancient times the clever Greek philosopher slave of a less intelligent Roman slaveholder must have dominated the actions of his master rather than obeyed his wishes! Similarly, if the machines become more and more efficient and operate at a higher and higher psychological level, the catastrophe foreseen by Butler of the dominance of the machine comes nearer and nearer.” I used https://www.textfixer.com/tools/remove-line-breaks.php https://www.textfixer.com/tools/remove-line-breaks.php.
- Wowfunhappy 2mo ago
- 3abiton 2mo agoThis was so well beautifully written, and poignant for our times. Almost 70 years old paper.