12 ms·
The Commission for Stopping Further Improvements
- mnd999 4y ago> The interpretation of Brunel’s opinions, and applicability to today, are left to the reader I don’t think Brunel would have much liked SOC2.
- IshKebab 4y agoInteresting, and I guess this applies to plenty of modern standards (e.g. my work won't use wireguard because it isn't approved by some nonsense security standards. But on the other hand I don't think the sensible solution is to leave everything unregulated! I am very glad that building regulations exist!
- nathan_compton 4y agoYeah, this "we can't build anymore" shit is _often_ just people complaining about not being able to externalize costs that they don't want to pay for.
- zwieback 4y agoAgreed, it's a fine anecdote but easy to draw the wrong conclusion of "rules hamper progress and counterproductive". I think the main challenge is to attract the right talent to the rule-making body.
- 364758483 4y agoThis is the exact direction I think we should reshape political systems. Perhaps even make empowering experts the main point. Too much is legislated and enforced by people who are overly self-interested or out of touch. It's a huge missed opportunity that experts are forced to watch representatives fumble around or make wrong decisions. A long those lines, governance power should not require being a celebrity either. Applying this post to the topic of AI alignment, I'd like a democratic option for large entitlements of tax funds to be applied as Yudkowsky sees fit, if enough people vote to appoint him as AI Minister, and vote to give him governance teeth against big tech and the thousands of startups driving Moloch via the standard economic paradigm.
- lazyasciiart 4y agoYou have that option. It’s called have Yudkowsky run for office.
- 364758483 4y agoMost public offices he could run for would put him in a generalist position, with various public facing and non-technical aspects that he is not suited to or qualified to handle. I want him to be able to completely and wholly run the show for just his one narrow specialty. This isn't an option at all because our institutions don't adapt to the territory we often find ourselves in. We have a political alignment problem that exacerbates the technical alignment problems. I would like to have something like the concept of senators, but not as location-based representatives, who are strictly assigned nothing more than to lead senate committees related to issues that we separately apportion funds and power to via direct democracy.
- m463 4y agoThere must be ways to write regulations in meaningful ways. Like a train bridge must meet specification a, b and c and pass tests d, e and f, instead of saying it must be made of wrought iron.
- jasoncrawford 4y agoI think regulations can work when they enforce very well-supported, long-established best practices, such that if you don't do them it amounts to negligence. They might also work better if they say “you can't do it that way, which is known to be unsafe,” as opposed to “you must do it this way, which is the only safe thing.” Note also that regulatory standards are not the only mechanism in the law to create safety. Liability law can be very effective at creating safety, by giving the right incentives to the right parties, but liability law doesn't tell anyone what to do—only what will happen to you if you cause harm.
- hgsgm 4y agoCompanies are very good at making slight changes to bad ideas and calling them "new ideas". The plastics industry runs on this.
- netbioserror 4y agoIn economics, the closest analogy might be economic substitutes. Who was to predict that electricity would displace oil lamps but the electric pioneers? The car to replace the horse-drawn carriage? Internal combustion to replace steam? The standard shipping container to replace irregular loading? Integrated circuits to replace mazes of wires and vacuum tubes? Spreadsheets to replace calculation workers? The Internet to quite nearly replace all uses of technical books? Government planners are not these pioneers and can never make universally applicable rules which hold their fairness into an unknowable future.
- narrator 4y agoThe worst outcome is when government promotes and protects bad science for ideological reasons. The classic examples being Lysenkoism in the Soviet Union and Mao's terrible ideas during The Great Leap Forward and subsequent famine.
- carls 4y agoIt seems to me that there's a natural tension in emerging fields of science and engineering between establishing clear guidelines and regulations early on to minimize harms, or instead allowing practitioners to experiment, tinker, build and create outcomes that may be potentially harmful. What are some frameworks for how to think about navigating this tension in emerging scientific or engineering fields? Some ones I'm mulling over: 1. Rate of innovation: In rapidly evolving fields, imposing strict regulations too early can hinder innovation and progress. In such cases, it might be better to minimize restrictions early on to allow practitioners to explore new ideas. Then, as the field matures, regulations and standards can be gradually introduced. 2. Adaptive regulation: Implement a flexible regulatory framework that can be updated as new information becomes available. 3. Self-regulation: In some cases, maybe we should expect and encourage the industry to use self-regulation via developing guidelines and codes of conduct. This may be one way to try and strike a balance between responsible innovation while minimizing bureaucratic obstacles. What do others think?
- TechBro8615 4y agoI think if I want to write some software, and I have the knowledge and compute power to do what I want, then I really don't care whether you or some authoritarian committee tells me I'm allowed to run my software. I'm going to do it anyway.
- nine_k 4y agoNow think of synthesizing and distributing novel chemical compounds, or using novel medical devices.
- jkhdigital 4y agoOr performing gain-of-function research on pathogens.
- fallingknife 4y agoWhat kind of chemical compounds? Drugs? Fine. You can only hurt yourself with those. Same goes for medical devices. Explosives? Now that's a different matter.
- mortenjorck 4y agoMuch like a Commission for Stopping Further Improvements, a Coalition for Stopping Further Regulation is equally short-sighted. Brunel's contention that structural engineers should be qualified to make their own decisions unencumbered by regulation makes sense on the face of it. But then, what qualifies a structural engineer? Licensing, a product of regulation. Like many things, regulation is, in isolation, neutral. It can serve the common good, or it can serve the interests of an unrepresentative cabal; it can be forward-looking and adaptable, or it can be hastily drafted by unqualified functionaries. Why couldn’t the regulators in Brunel's day have carved out an open-ended “unless otherwise demonstrated” around the use of iron structural members?
- its_ethan 4y agoIt feels like you answered your own question in a way.. "Why couldn’t the regulators in Brunel's day have carved out an open-ended “unless otherwise demonstrated” around the use of iron structural members?" The answer is partly in your statement: "regulation is, in isolation, neutral." Looking around at regulatory/ bureaucratic bodies, "in isolation" is rarely (never?) achieved. Satisfying ambiguous "unless otherwise demonstrated"-esque clauses that are enforced by unelected paper pushers is very much part of Brunel's concern.
- makeitdouble 4y ago> unelected paper pushers The core of your argument is that regulators are unelected (vs engineers ???) and incompetent, when in reality they either get counsel from people in the field, or are former or partially active members of the profession themselves.
- jkhdigital 4y agoEngineers, at least at that point in history, paid the consequences of their failures. It’s not about competence, it’s about “skin in the game”. For further reading, Nassim Taleb wrote a whole book on this idea.
- mannykannot 4y agoWhile some counter-examples may be found, subsequent events have generally repudiated Brunel's concerns.
- zeteo 4y agoBrunel was probably right that the work of the commission would be detrimental to the rate of innovation. But is that the only thing that matters? We would probably get a faster rate of innovation in personal transportation if we allowed any unlicensed vehicles to use the public highways at any speed. But there would be a terrible cost to pay in terms of lives lost or ruined. It doesn't sound like the commission was planning to forbid any and all usage of cast iron, just not in places where public safety was at risk. An innovative engineer could always build a novel cast iron bridge in a test location and produce some dramatic demonstration of its strength and reliability that didn't involve running fully loaded passenger trains over it. The trade-off between the rate of innovation and the casualty rate does not need to be chosen at either of the extremes.
- hgsgm 4y agoThe lives of billonna5 today today are worth less than the trillions who will live in the future. We owe it to them to sacrifice ourselves. -- Effective Altruism
- d0mine 4y agoGiven that we are very limited in our ability to predict the future, the thesis does not matter much. Better to discuss specific actions here and now that we can predict the consequences of [with some certainty into some time into the future].
- eximius 4y agoSo, 'update regulations when new information comes to light'? It is frustrating that common action often precludes such common sense sentiments due to the friction of interests or bad actors.
- dgs_sgd 4y agoAh, but regulatory updates come at a rate much much slower than new information and innovation. For example, the main securities law in the US was written 90 years ago (Securities Act of 1933).
- JackFr 4y agoThe Securities Act of 1933 is a tremendously successful law. (And it’s been amended a half dozen times so it’s not really trapped in amber.)
- TulliusCicero 4y agoIn principle everyone agrees with that, but the system tends to work against it. Especially corporations or other large groups that specialize in X, where X is the currently mandated thing to do.
- flybrand 4y agoI feel like we’re living in the age of an unidentified, “Committee to Stop Maintenance of Civil Infrastructue.”
- krisoft 4y agoBut what is Brunel’s solution to the underlying problem? A bridge has fallen down. Presumably the engineers who designed it had no murderous intent. They tried their best to make a solid bridge. Do we just let engineers build bridges however they please and then be angry if they fall down? How is that going to help anybody?
- sfink 4y agoI would guess that he would feel that the commission was the wrong instrument to fix the underlying problem in the first place, and so giving specific suggestions to that same body would only weaken his argument. There is a hint of how he would prefer to solve the problem: > the free exercise of engineering skill in this country, subjected as it ever is, under the present system, to the severe and unerring control and test of competing skill and of public opinion. The ultimate goal should not be solely "never let X happen ever again" because that would ignore any harmful effects of prioritizing that goal over everything else. In the limit, the most effective approach would be to build no more bridges. (Lest you think I am being extreme, how many nuclear power plants do you see being built outside of China?) With the commission in the article, would it have been better or worse if they had then produced no regulations? We can't really know. Presumably the engineers of the time weren't stupid, so they would take steps to avoid repeating the mistake. How well would that information be disseminated and internalized by the profession? I don't know, but I also don't know the effect of regulations and red tape. It was a different time and with a vastly different scale of things happening (and poorer communications mechanisms too, but also a smaller community to communicate within). I wouldn't want to just abandon regulations today, but I also think that regulations have stifled a lot of potential progress and cost a lot of lives. We did not find an optimal path, sadly, and we seem to stray further from the optimal every day. External regulation helps except in the ways it doesn't. Self-regulation works until it doesn't. I don't think there's a simple "just do X" fix to this sort of thing.
- doctor_eval 4y ago> The ultimate goal should not be solely "never let X happen ever again" because that would ignore any harmful effects of prioritizing that goal over everything else. I can’t help but think that aviation safety is a counter example to this. The 737 Max disasters were almost entirely a failure or regulation in an industry which has otherwise produced a huge amount of both innovation and safety improvements over the last 50 years - leading to the recent launch of the worlds largest rocket just a few days ago. This rocket was launched with regulatory approval - and still went kaboom! - demonstrating at least that innovation and rism is still possible within a regulatory framework that prioritises safety.
- vjust 4y agoAKA "best practices" or "you'll never be fired for buying a Microsoft product"
- rdiddly 4y ago"...he suggested that unknown developments in the future might make cast iron strong and safe..." Of course that proved true - we call it steel - and if you go by the date of Bessemer's patent (after which steel became cheap enough to use on structures), they only had to wait 9 years.
- kragen 4y agono, steel is not cast iron ductile cast iron does now exist but bessemer didn't invent it
- rdiddly 4y agoYou should be like the steel, a little less rigid, in your thinking. Obviously steel isn't cast iron, because hey look at that, different names. Also if steel were cast iron, it wouldn't be an improvement on cast iron as I'm saying, it would just be cast iron. On the other hand, it is primarily composed of iron and is literally cast. Finally the person is speaking in 1847. I don't demand that he exactly predict the future. If he predicts there will be improvements to cast iron (the primary material of his time), I say he's right even if he fails to foresee that the resulting iron product, that is cast, is known by another name and ends up making what he knows and we know as cast iron, obsolete for structural use.
- kragen 4y ago[flagged]
- rdiddly 4y agoI went to engineering school and studied this shit my dude. Stress-strain curves, metallurgy, the whole bit. I know what steel is. Your response is weak. I'm making an apt metaphor about the brittleness of your hard insistence on shallow terminology that snaps instead of yielding to the superior force.
- kragen 4y ago
- not2b 4y agoThe article seems to assume that regulations, once enacted, never change and impede progress forever, while in real life, technical regulatory bodies update regulations when conditions change. They might change the rules more slowly than some entrepreneurs desire, but when the purpose of the rule is safety, better safe than sorry.
- AmericanChopper 4y ago> They might change the rules more slowly than some entrepreneurs desire, but when the purpose of the rule is safety, better safe than sorry. The larger the authority, the slower they move, and the less sophisticated the technical understanding they have (because in a large bureaucratic authority, many non-experts have to be convinced of a change in addition to the experts). It’s also not something that only impacts laissez faire capitalists. Information Security experts understood for decades that conventional password complexity requirements produced harmful outcomes. How long did it take an authority to produce an alternative standard? How much reputational risk did they take on by depreciating a security standard? Thank god NIST and the NCSC had the courage and competence to adopt this change, but how many lesser authorities have failed to keep pace with it? How many individual organisations? Alternatively, how much do you think the European Parliament has slowed development of new USB connectors? Are we all going to love USB-C as much in 20 years as we do today? How could a new and improved connector even distinguish itself for consideration? Experimenting with improvements in the market is prohibited by law. If an improved connector became popular in an outside market, how long would it take the European Parliament to notice? How long would it take them to make a change?
- zokier 4y ago> Alternatively, how much do you think the European Parliament has slowed development of new USB connectors? Are we all going to love USB-C as much in 20 years as we do today? How could a new and improved connector even distinguish itself for consideration? Before USB-C, EU had already standardized Micro-USB for phones. Yet there was no difficulty in transitioning to USB-C. https://www.engadget.com/2010-12-29-european-standardization-bodies-formalize-micro-usb-cellphone-ch.html?guccounter=1 https://www.engadget.com/2010-12-29-european-standardization...
- worik 4y agoWhat a tosser Brunel was. Essentially he says: "Trust me, there are many idiots, but lucky you, there is me". Reminds me of Elon Musk. Just because you are a genius does not mean others are idiots, or that your opinion is worth more. Elon proves this every day with his idiotic pronouncements and hie genius results Reminds me of the old saying: A camel is a horse designed by a committee. Meaning committees are idiotic. But a camel goes further, carrying more, on less food and less water. Yes.
- kragen 4y agoyeah, that brunel guy, what did he know about engineering anyway probably just some loser trying to cut corners on his highway offramp planning huh what's this https://en.wikipedia.org/wiki/Brunel https://en.wikipedia.org/wiki/Brunel or uhhh maybe not who is considered "one of the most ingenious and prolific figures in engineering history,"[3] "one of the 19th-century engineering giants",[4] and "one of the greatest figures of the Industrial Revolution, [who] changed the face of the English landscape with his groundbreaking designs and ingenious constructions".[5] Brunel built dockyards, the Great Western Railway (GWR), a series of steamships including the first purpose-built transatlantic steamship, and numerous important bridges and tunnels. His designs revolutionised public transport and modern engineering. yeah what a fucking tosser
- brabel 4y agoI think the parent comment's point was exactly that, to idiots, the opinions of a genius look idiotic. Just as Elon's and Brunel's opinions and methods look stupid to their contemporaries (as we see today with Elon, and in the 19th century with Brunel).
- worik 4y agoMy point is that just because you are a brilliant genius does not stop you being, also, a fool. Brunel is a nineteenth century example, Elon Must is a twenty first century example....
- QuadmasterXLII 4y agoRegulations are not written by the regulators- they are written by the bastards who put melamine in baby formula to make a dollar.
- Kalium 4y agoIn the fullness of time, both generally get to grab for the pen. Plus other groups, such as those primarily interested in making sure there are not too may accredited professionals. It is perhaps not as simple as any of us might like.
- dash2 4y agoA point I don't see mentioned is that regulation isn't just slow and bureaucratic, and it isn't necessarily imposed on unwilling capitalists. It's often, as Stigler put it, "purchased by the industry". In other words, you regulate competitors out of business, or put up barriers to them even coming into existence.
- martinclayton 4y agoBrowsing the links in the wikipedia article about the accident https://en.m.wikipedia.org/wiki/Dee_Bridge_disaster https://en.m.wikipedia.org/wiki/Dee_Bridge_disaster it’s apparent the commission report didn’t stop people dying in similar rail bridge collapses. The designer, Robert Stephenson, and bridge inspector, Major-General Paisley, were not found liable at the inquest. It seems this may have been partly due to expert testimony in an early case of “alternative facts”. Stephenson himself was apparently ready to admit liability, but was “persuaded” to claim a derailment was the cause. [1] Similar bridge collapses occurred in the late 19th century. Only after the 1891 Norwood Junction accident was inspection of all bridges mandated. [2] Some cast iron deployed in bridges built in that period is still in place. Weak bridges have weight restrictions where needed. [1] https://en.wikipedia.org/wiki/Robert_Stephenson#cite_note-149 https://en.wikipedia.org/wiki/Robert_Stephenson#cite_note-14... [2] https://en.wikipedia.org/wiki/Norwood_Junction_rail_accident https://en.wikipedia.org/wiki/Norwood_Junction_rail_accident
- lifeisstillgood 4y agoThis comes up fairly often but who would support a chief software engineer signing a document that says "I the Chief SE verify this will meet the stated requirements will not fail collapse, cause bankruptcy, will scale will meet needs of .. else I go to jail and you can sue to recover all salaries paid for everyone involved" it would certainly chnage the nature of software developement Not sure it's a win. But it is interesting Edit: I think it is as simple (?) as a law saying software warranties are unenforceable ?
- onos 4y agoSounds to me like he was an elite engineer concerned with how regulation would affect him and others like him, while the regulations would primarily be aimed at constraining the output of lesser beings. He argued that the lesser beings would follow the rules blindly and that these would not guarantee safety anyways.
- nforgerit 4y agoEuropean speaking. Dear EU politician, regulating things would be a fair strategy, but don't BS me that it would serve for creating a level playing field. In fact, it's the opposite, it favors those already big enough to (1) handle the additional bureaucracy and (2) to lobby for regulation changes. E.g. if they start regulating the data sources used for ML then little startups will have zero chance to compete with Big Tech.