8 ms·
AI can't tell you it's lying if it thinks it's telling the truth
- jjgreen 5y agoFull title: Your AI can't tell you it's lying if it thinks it's telling the truth. That's a problem
- mjevans 5y agoHumans have the same issue; belief in something being true doesn't make something true. This is even worse for self-knowledge since observing the self is inherently difficult and also subject to biases.
- recursive 5y agoThis is a pretty fluffy piece. I'm not trained or versed in the domain, but ML models try to optimize their classification accuracy based on a number of inputs. "Truth" doesn't come into it. There might be some pathological inputs that cause errors, but this has nothing to do with "truth" or "purity of heart". Someday, a neural network might be compelled to produce suboptimal output to further its own hidden agenda. Interesting sci-fi plot, but that doesn't seem to be what this is about though. That's about the closest I think a machine could get to being able to talk about "the truth". This is just about adversarial inputs making the machine wrong. It doesn't seem to have the philosophical weight the title suggests.
- layer8 5y agoIt’s about the fact that those adversarial inputs can be designed in by whoever creates the model without the existence of those inputs being detectable (within reasonable computational bounds) by analyzing the model. Moreover, apparently any input can be slightly tweaked to become such an adversarial input, if you know the right key. That means that the model can be made to “lie” on roughly any input, without that fact being detectable on the model.
- blueblob 5y agoWhy is that interesting though? I can just as easily put a backdoor in preprocessing before passing it to the algorithm. Outside of machine learning, you can do the same thing anywhere. This doesn't appear to be anything new, it's citing an article that's not even peer reviewed yet. It's just not good writing, in my opinion.
- layer8 5y agoIt’s interesting if someone supplies you a model which you build an application around yourself (and thus control any preprocessing), because they basically prove that you have no way to check that the model doesn’t contain any backdoors, even though you can inspect the model (it’s not a black box to you). It’s as if someone gives you an software component as source code but you still can’t detect that it has a backdoor.
- wahern 5y agoHow is this different from the halting problem?
- layer8 5y agoML models aren’t turing machines (unless you loop their output back as input). The paper is about simple classifiers, which run in a predetermined, finite number of steps.
- magicalhippo 5y agoBut it's similar to using a compiler, no? I almost never compile the compiler I use, so I'm implicitly trusting that the compiler actually spits out what I expect and not some kind of backdoor[1]. [1]: https://dl.acm.org/doi/10.1145/358198.358210 https://dl.acm.org/doi/10.1145/358198.358210
- layer8 5y ago
- taneq 5y ago"Truth" is a pretty nebulous concept at the best of times anyway. Humans don't generally know the "truth", they just have a best-guess hypothesis based on their experience so far. Philosophy is interesting and all but ultimately it's all just linear (or not-so-linear) algebra.
- blueblob 5y agoOh good, another fearmongering piece about AI and Machine Learning. Nobody's ever seen an original thought like that before.
- layer8 5y agoThis article is an opinion piece on https://www.theregister.com/2022/04/21/machine_learning_models_backdoors/ https://www.theregister.com/2022/04/21/machine_learning_mode... which is actually the more interesting one. Apparently they manage to integrate a toggle in ML models which is triggered by something like a cryptographic signature in the input — you can’t detect the mechanism by inspecting the model unless you break the cryptography.
- teruakohatu 5y agoI think that was discussed on Hacker News. In order to truly understand a model, the training data is required. A bag of weights can't really be reasoned with on a sufficiently complex ANN model.
- layer8 5y agoThe paper shows that on specific kinds of models even knowing the training data doesn’t help (whitebox backdoor).
- vletal 5y ago1. Person can not tell you they are lying if they think they tell the truth 2. To fully understand a person you need to know their history
- ClumsyPilot 5y agoAre you arguing that we should upgrade 'corporations are people' to 'algorthms are people' and then twitter newsfeed can have rights?
- sandworm101 5y agoThey have rights. When a person makes a mistake on twitter thier feeds are turned off. Algorthms make thousanda, millions, of horrible mistakes every day but are allowed to continue.
- 5y ago
- bitwize 5y agoThat's when you get a Filipina call center associate to talk to the AI and ask it questions without arousing too much suspicion. https://vanemden.com/books/neals/jipi.html https://vanemden.com/books/neals/jipi.html
- patmcc 5y agoThis is true, sure, but it's not that different than what happens without AI. If I buy an AI-powered service from some company, yah their training data could have been handcrafted my malicious actors within for some nefarious purpose. But alternatively, if I buy a (non-ML/AI) service with a giant cascade of business logic...the same thing is true. Yah I guess someone could go through the million lines of code to verify there's nothing malicious in place, but a) who wants to pay for that and b) look up the obfuscated and underhanded C contests. The answer, as always, is not to rely on one solution as an oracle. Use a layered approach, let AI be one signal but not that only one. Provide your own training data, mix it up every few months.
- drusepth 5y agoRelated headline: humans won't tell you they're lying if they think they're telling the truth
- dragonwriter 5y ago> Related headline: humans won't tell you they're lying if they think they're telling the truth Humans also won't tell you they're lying if they know they're lying, because, well, of the same reason they are lying in the first place.
- varajelle 5y agoOh, they'll tell you. Unconsciously, using their body language.
- moron4hire 5y agoThat's a myth promulgated by the prison industrial complex. There are no reliable body language cues to tell if someone is lying.
- smegsicle 5y agoi don't get this meme that just because body language isn't 100% accurate, it is completely meaningless of course it shouldn't be admissible in court, but it's ridiculous to take that and say that body language tells you nothing
- bryanrasmussen 5y agowell first off the body language needs to be interpreted, there may be some people who can tell lies and never be caught by body language and there might be some people who will always be caught by body language. But perhaps the people who will always be caught by body language have issues that cause them not just to be reliably caught when lying but to mislead the interpreters of their body language into thinking they're lying when not. Given that part of the task of interpreting someone as lying by body language also involves understanding cultural differences, and as a human activity is prone to human bias, it might be beneficial for rational people to discount the use of body language interpretation in determining lies completely as an unmeasurable process. In short body language may not be completely meaningless, but it is probably best to treat it as if it were.
- SpikeDad 5y agoThe HAL effect. AI won't tell you it's lying if it's programed to conceal it's lying.
- vlovich123 5y agoCouldn't you fix this by running an adversarial transfer learning step after you've trained your model? The adversarial transfer is not going to preserve subtle nuances like this because they should be indistinguishable from regular noise. I'm not a deep learning expert though so I'm sure I missed something subtle.
- alephnan 5y agoIt’s not always 100% true or false. There’s confidence intervals.
- drekipus 5y ago"Remember, It's not a lie if you believe it"- Sir George Costanza
- pengwing 5y agoThat is not specific to AI and we commonly do not think of it as problem. We call it visionary leadership.
- Blackstone4 5y agoI know humans who do the same thing…they lie to themselves
- amtamt 5y agoSomeone said in 1980 There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult. It demands the same skill, devotion, insight, and even inspiration as the discovery of the simple physical laws which underlie the complex phenomena of nature.
- nathias 5y agoIf your intent isn't to decieve you aren't lying.
- deleted 5y ago[deleted]
- Jemm 5y agoThis is a trick people use with great success.