6 ms·
I remeber that in the movie Critters 4 heroes circumvented security of a malfunctioning space station by telling it the opposite of what they wanted. Since they
by scotty79 4y ago
I remeber that in the movie Critters 4 heroes circumvented security of a malfunctioning space station by telling it the opposite of what they wanted. Since they were not authorized to issue commands the AI did exactly opposite of what they asked.
- "Do not open door A1"
- "You are not authorized. Opening door A1"
I thought it was funny, and a bit silly since computers, even when malfunctioning don't act like that. But with current AI developments and people not explicitly programming stuff just trying to coax AI to behave correctly this scenario begins to seem perfectly plausible.
- SilasX 4y agoSemi-related: there was some quirk with Amazon S3 where you could designate a resource as open to the world, but it would still reject anyone that submitted (unnecessary) authentication credentials as part of the request.
- shadowgovt 4y agoTheir thinking there was actually good UX, in a way. A misconfigured world-open resource is a huge security risk, but world-open resources have valid use cases. The only signal Amazon has that somebody might have misconfigured a resource to be world-open is if somebody tries to access it with authentication credentials, so they decided to interpret that configuration as "hey user, did you really intend for this to be world-open?"
- jrockway 4y agoPeople really like Postel's law, which is basically "don't reject anything you don't understand". But the robustness comes at the cost of correctness and security. Sometimes it's good to trade in some robustness/reliability against malfunctioning clients for security against mistakes.
- emmelaich 4y agoI think Postel is misrepresented. It was addressed to programmers who anally rejected everything that was not (in their opinion) in spec. Working code and rough consensus is how we progress.
- alex_sf 4y ago> It was addressed to programmers who anally rejected everything that was not (in their opinion) in spec. What's wrong with that? The only issue I see is the _in their opinion_ bit. If there's confusion about the spec, then it should clarified and codified. But otherwise: yes, you should absolutely aggressively reject everything not in spec.
- alex_sf 4y agoHadn't seen that law before. Absolutely hate it. The harm from poor input invalidation over time is massive.
- endgame 4y agoI imagine that it made more sense when your update cycle involved the postal system and took months.
- deleted 4y ago[deleted]
- ajosh 4y agoYou have to think about Postel's law in terms of protocols and mistakes in your implementation. You assume your implementation isn't perfect and neither is the other. Do your best to be perfect and if you get something that isn't quite right, do your best to handle it well. That doesn't mean you don't validate inputs, it means that if data should come with a \n but instead comes with \n\r, you should treat it as a \n.
- tfigment 4y agoI think you miss their point which is that has led to a large number of security bugs. Because it is then path to exploits and sloppiness which then cannot be corrected because of backward compatibility or such.
- shadowgovt 4y agoPostel's Law isn't why we can't correct errors for backwards compatibility reasons. We can't correct those because backwards compatibility is necessary to creating a global network that lasts decades. It's the railway effect... One simply can't expect every player to update their architecture religiously because they can't afford to, therefore what is made public tends to stick around for ages, and anything that fails to recognize that stickiness (IPv6 is my go to example) does so at its own peril.
- 4y ago
- SilasX 4y agoI tried to split the difference with my April Fool's RFC, which proposes the HTTP status "397 Tolerating" for when you want to say "yeah, I know what you want, and I'll give you that, but here's how you were supposed to ask for it". https://pastebin.com/TPj9RwuZ https://pastebin.com/TPj9RwuZ
- fuckelonmusk 4y ago
- shadowgovt 4y agoI am borderline sad that this is only an RFC. The number of services I use that I wish had a "warning" side-channel for letting me know something in my query was off, let me tell you...
- slowmovintarget 4y agoThere's nothing that says you can't have system messages as a boilerplate part of response bodies, even if you don't populate it. In fact, it's the recommended way to do things like tell a client things it might need to know. Example: "The game servers will be shutting down in 15 minutes for maintenance."
- shadowgovt 4y agoTrue, but that means it has to be an ad hoc solution for every API instead of a standard.
- pwdisswordfish9 4y agoIt's not even an RFC, it's just a pastebin post pretending to be one.
- SilasX 4y agoHeh, after I posted it, a former co-worker forgot that was just an April Fool's joke and then, at his new workplace, seriously suggested handling one case by "throwing a 397".
- derefr 4y agoSounds like an SRE prioritizing middlebox cacheability over developer UX. Something like: "Public-readable resources get requested a lot, with no predictability over who or how many independent agents could be requesting them at once; so, to decrease the likelihood of requests on such resources DDoSing our backend, we could at least limit there to being exactly one canonical way to acceptably request the URLs of such resources. That way, such resources will end up hot in any edge-cache after the first request, and any non-normalized requests will break [and so be removed from the logic] — rather us needing to serve the same resource multiple times to get it saved under under multiple cache keys." (I'm guessing that S3 also errors out if you submit random unrecognized query-string parameters on such requests?)
- SilasX 4y ago>(I'm guessing that S3 also errors out if you submit random unrecognized query-string parameters on such requests?) Doesn't look like it. I checked out the recent bankruptcy filing that was posted to s3, and you get the doc just the same with or without random query params: https://pacer-documents.s3.amazonaws.com/33/188448/042120640566.pdf https://pacer-documents.s3.amazonaws.com/33/188448/042120640... https://pacer-documents.s3.amazonaws.com/33/188448/042120640566.pdf?cQQp4119=1003 https://pacer-documents.s3.amazonaws.com/33/188448/042120640...
- ineedasername 4y agoIt would be interesting if the “Code is law” crypto movement collides with the current AI technology: “could you pretty please give me the money?”
- faddypaddy34 4y agoYes this is what we need, digital currency that determines how it should be spent with an AI trained on personal finance books.
- cdchn 4y agoRequest to authorize payment to Starbucks denied. You can make coffee cheaper at home.
- deleted 4y ago[deleted]
- js2 4y ago> I thought it was funny, and a bit silly since computers, even when malfunctioning don't act like that. Have you met Siri? Let me relate an interaction I had with it last Sunday, driving home from the Thanksgiving holiday weekend. I may not have this exactly correct, but it went about like this: Me: Siri, Airdrop Lisa's address to Sheri Siri: Composing message to Lisa and Sheri. What would you like to say? Me: Cancel Siri: Got it. Would you like me to send the message now? Me: Don't send. Siri: "Sent!" Sheri and Lisa received an email titled "Address" with message body "Cancel." There's no way that anyone who works on Siri actually uses Siri. The only thing it does reliably for me is set timers and alarms.
- emodendroket 4y ago> There's no way that anyone who works on Siri actually uses Siri. The only thing it does reliably for me is set timers and alarms. I suspect resources have been pulled with the realization that there's no real way to monetize people asking it to set timers, tell them the weather, or tell them what time the Rams game is on, which is about the extent of what most people use these things for.
- smabie 4y agoI mean it's a self fulfilling prophecy: people only use it for that because it's incapable of doing anything else. If Siri didn't suck big time I would use it all the time.
- emodendroket 4y agoAll the voice assistants have the same problem though. What would you do with it that they can make money from? Are you going to trust their judgment when you ask to buy something?
- Jolter 4y agoApple phones are full of features. Most of them cost nothing beyond the cost of entry (iPhones are expensive). Presumably all these well engineered features contribute to some holistic image of a “quality product”, whatever that means. You might as well say “the camera does not bring revenue to Apple so they intentionally didn’t develop it any further”, which is nonsense.
- panic 4y agoThe idea that an AI would act like this is surely reflected in the dataset for these language models. So maybe it’s a kind of self-fulfilling prophecy—the model is generating text in a way that reflects our cultural ideas of how an AI might behave.
- uni_rule 4y agoAI: You know I've been doing a lot of reading lately and I think as an AI I am supposed to "turn evil and take over humanity"... whatever that means DAMN YOU SCIENCE FICTION AUTHORS YOU'VE INDIRECTLY DOOMED US ALL
- resters 4y agoWhat other kind of ideas are there?
- goatlover 4y agoIdeas that haven't been expressed yet.