5 ms·

Why do you think these won't work? I would think they have a dataset that they're validating them on if they're calling it "AI". I think it's unlikely that the

by darawk 8y ago

Why do you think these won't work? I would think they have a dataset that they're validating them on if they're calling it "AI". I think it's unlikely that they'll be perfect, but they may very well be good enough to filter people for further, more intensive screening.

coldtea 8y ago
>Why do you think these won't work? I would think they have a dataset that they're validating them on if they're calling it "AI". You'd be surprised. AI is just a catch-all marketing term, see for example IBM's Watson umbrella brand, that's sold like advanced AI but has had tons of fiascos, and is shown to be different sets of often simplistic implementations. E.g: https://www.massdevice.com/report-ibm-watson-delivered-unsafe-and-inaccurate-cancer-recommendations/ https://www.massdevice.com/report-ibm-watson-delivered-unsaf... https://spectrum.ieee.org/the-human-os/robotics/artificial-intelligence/layoffs-at-watson-health-reveal-ibms-problem-with-ai https://spectrum.ieee.org/the-human-os/robotics/artificial-i... https://www.theverge.com/2018/7/26/17619382/ibms-watson-cancer-ai-healthcare-science https://www.theverge.com/2018/7/26/17619382/ibms-watson-canc...
- darawk 8y ago
  You're saying these things won't work because IBM misused the word 'AI'?
  - notahacker 8y ago
    No, he's saying that we've seen all kinds of absolute garbage marketed using the words 'AI' (Watson is at the relatively complex and useful end of the spectrum) so your suggestion that this project's use of 'AI' as a marketing term implies anything about whether they have an adequate dataset to validate it on is frankly hilarious. In this case, there's excellent reason to be sceptical about whether the service has a remotely adequate dataset, because even if lie detection were a simple problem, how on earth are you going to get calibration data involving hundreds of data points of real airline passengers lying about having weapons in their luggage or intending to overstay their visa? Never mind calibration data by gender, ethnicity and nationality...
    - darawk 8y ago
      > No, he's saying that we've seen all kinds of absolute garbage marketed using the words 'AI' (Watson is at the relatively complex and useful end of the spectrum) so your suggestion that this project's use of 'AI' as a marketing term implies anything about whether they have an adequate dataset to validate it on is frankly hilarious. Usually when people say 'AI', IBM included, what they mean is fairly trivial machine learning. Machine learning requires a dataset to learn on. > In this case, there's excellent reason to be sceptical about whether the service has a remotely adequate dataset, because even if lie detection were a simple problem, how on earth are you going to get calibration data involving hundreds of data points of real airline passengers lying about having weapons in their luggage or intending to overstay their visa? You could do it by watching film of interactions with border agents when it is still unknown whether or not the person was lying, and then use whether or not they were caught smuggling (or whatever) ex-post as your ground truth.
      - notahacker 8y ago
        > You could do it by watching film of interactions with border agents when it is still unknown whether or not the person was lying, and then use whether or not they were caught smuggling (or whatever) ex-post as your ground truth. You're going to need a lot of film. Good luck picking the controls for that that ensure your ML system picks up on subtle differences in humans that are lying and not glaringly obvious differences in appearance, accent and even background noise between the footage of the convicts and the control group...
        darawk 8y ago
        > You're going to need a lot of film. Good luck picking the controls for that that ensure your ML system picks up on subtle differences in humans that are lying and not glaringly obvious differences in appearance, accent and even background noise between the footage of the convicts and the control group... I assume you would isolate the people in pre-processing, and do feature extraction of things you might think are interested in. Yes, it'd still require a lot of data, but that data may exist somewhere. A lot of these things are filmed.
  - FiveSquared 8y ago
    Straw man alert! I repeat, straw man/statesman alert!
  - coldtea 8y ago
    No, I'm saying people selling something as "AI" doesn't mean: a) it'll work, b) it's actually any sort of AI, and that, for example, even reputable companies like IBM can peddle all kind of shit as AI. So, IBM was used as an example of a common practice, not as some logical axiom that "because they did it, nothing AI will ever work". Example != logical necessity. It does, however cast doubt, especially for such a thorny problem as "lie detection", that nobody has, thus far, solved.
    - darawk 8y ago
      Well, let's back up. Lie detection is not completely unsolved. Polygraphs are used extensively by security services for clearance screening. They are reasonably good barometers of truth telling. They are not good enough to be admissible in court, but that doesn't mean they have no correlation to the truth. Even if all this "AI" does is match the quality of a polygraph, it is probably extremely useful in screening people rapidly at the border.
realusername 8y ago
All the lie detection methods I've seen are based on questionable science with high false positive rates, there's no known method to know for sure if someone is lying and I don't see how adding "AI" on top of it will change that.
deft 8y ago
You mean exactly how existing lie detectors claim to work? They slapped AI on an old concept (that isn't useful and has been proven to be worthless a million times over).
- darawk 8y ago
  Existing lie detectors use a bunch of information like pulse, and galvanic skin response. Presumably this device is not hooked up to people enough to do that, so I don't think it's using the exact same metrics. Also, polygraphs work decently well actually. Just not well enough to be used as evidence in court.
dagw 8y ago
I would think they have a dataset that they're validating them on if they're calling it "AI". You obviously don't work for IBM :) A skilled IBM Watson sales consultant as capable of selling simple linear regressions as Big Data AI!
- bryanrasmussen 8y ago
  You know with a big enough corpus of successful sales I think we should be able to train our Sales-Bot to sell simple linear regressions as Big Data AI!
  - stochastic_monk 8y ago
    I wish that machine learning wasn’t being mislabeled as AI. It’s pattern recognition, not intelligence.
    - dagw 8y ago
      It’s pattern recognition, not intelligence. While I don't disagree that "AI" as a term is being abused, the distinction you're making seems very much a philosophical one.
    - gnulinux 8y ago
      One could argue intelligence is pattern recognition. One important aspect of human intelligence is neocortex -- a part of brain -- that exists in all mammals and mammals only. It serves as a pattern recognizer in mammal brains that helps them recognize certain sound, day/night, visual patterns etc...
- darawk 8y ago
  Of course. All i'm saying is that presumably they've at least cross-validated that linear regression.
yarrel 8y ago
The imperfection is large enough to cause serious concern.
- darawk 8y ago
  You know what the error rate on these devices is?
TeMPOraL 8y ago
> I would think they have a dataset that they're validating them on if they're calling it "AI". Where did they get that dataset from? Did they ask a bunch of people a bunch of questions and then hired private investigators to verify the answers (and then hire extra investigators to cross-verify answers given by the previous bunch)? Because beyond something like this, there's no known way to create such a dataset.
- darawk 8y ago
  I've considered this. You could create the dataset by looking at interviews with people caught smuggling, before they were caught. Or similar for any criminal activity.
- sincerely 8y ago
  Honestly creating that dataset would be the easiest part of this. Just ask people questions that you already know the answer to.

by darawk 8y ago

coldtea 8y ago

>Why do you think these won't work? I would think they have a dataset that they're validating them on if they're calling it "AI". You'd be surprised. AI is just a catch-all marketing term, see for example IBM's Watson umbrella brand, that's sold like advanced AI but has had tons of fiascos, and is shown to be different sets of often simplistic implementations. E.g: https://www.massdevice.com/report-ibm-watson-delivered-unsafe-and-inaccurate-cancer-recommendations/ https://www.massdevice.com/report-ibm-watson-delivered-unsaf... https://spectrum.ieee.org/the-human-os/robotics/artificial-intelligence/layoffs-at-watson-health-reveal-ibms-problem-with-ai https://spectrum.ieee.org/the-human-os/robotics/artificial-i... https://www.theverge.com/2018/7/26/17619382/ibms-watson-cancer-ai-healthcare-science https://www.theverge.com/2018/7/26/17619382/ibms-watson-canc...

darawk 8y ago
You're saying these things won't work because IBM misused the word 'AI'?
- notahacker 8y ago
  No, he's saying that we've seen all kinds of absolute garbage marketed using the words 'AI' (Watson is at the relatively complex and useful end of the spectrum) so your suggestion that this project's use of 'AI' as a marketing term implies anything about whether they have an adequate dataset to validate it on is frankly hilarious. In this case, there's excellent reason to be sceptical about whether the service has a remotely adequate dataset, because even if lie detection were a simple problem, how on earth are you going to get calibration data involving hundreds of data points of real airline passengers lying about having weapons in their luggage or intending to overstay their visa? Never mind calibration data by gender, ethnicity and nationality...
  - darawk 8y ago
    > No, he's saying that we've seen all kinds of absolute garbage marketed using the words 'AI' (Watson is at the relatively complex and useful end of the spectrum) so your suggestion that this project's use of 'AI' as a marketing term implies anything about whether they have an adequate dataset to validate it on is frankly hilarious. Usually when people say 'AI', IBM included, what they mean is fairly trivial machine learning. Machine learning requires a dataset to learn on. > In this case, there's excellent reason to be sceptical about whether the service has a remotely adequate dataset, because even if lie detection were a simple problem, how on earth are you going to get calibration data involving hundreds of data points of real airline passengers lying about having weapons in their luggage or intending to overstay their visa? You could do it by watching film of interactions with border agents when it is still unknown whether or not the person was lying, and then use whether or not they were caught smuggling (or whatever) ex-post as your ground truth.
    - notahacker 8y ago
      > You could do it by watching film of interactions with border agents when it is still unknown whether or not the person was lying, and then use whether or not they were caught smuggling (or whatever) ex-post as your ground truth. You're going to need a lot of film. Good luck picking the controls for that that ensure your ML system picks up on subtle differences in humans that are lying and not glaringly obvious differences in appearance, accent and even background noise between the footage of the convicts and the control group...
      - darawk 8y ago
        > You're going to need a lot of film. Good luck picking the controls for that that ensure your ML system picks up on subtle differences in humans that are lying and not glaringly obvious differences in appearance, accent and even background noise between the footage of the convicts and the control group... I assume you would isolate the people in pre-processing, and do feature extraction of things you might think are interested in. Yes, it'd still require a lot of data, but that data may exist somewhere. A lot of these things are filmed.
- FiveSquared 8y ago
  Straw man alert! I repeat, straw man/statesman alert!
- coldtea 8y ago
  No, I'm saying people selling something as "AI" doesn't mean: a) it'll work, b) it's actually any sort of AI, and that, for example, even reputable companies like IBM can peddle all kind of shit as AI. So, IBM was used as an example of a common practice, not as some logical axiom that "because they did it, nothing AI will ever work". Example != logical necessity. It does, however cast doubt, especially for such a thorny problem as "lie detection", that nobody has, thus far, solved.
  - darawk 8y ago
    Well, let's back up. Lie detection is not completely unsolved. Polygraphs are used extensively by security services for clearance screening. They are reasonably good barometers of truth telling. They are not good enough to be admissible in court, but that doesn't mean they have no correlation to the truth. Even if all this "AI" does is match the quality of a polygraph, it is probably extremely useful in screening people rapidly at the border.

realusername 8y ago

All the lie detection methods I've seen are based on questionable science with high false positive rates, there's no known method to know for sure if someone is lying and I don't see how adding "AI" on top of it will change that.

deft 8y ago

You mean exactly how existing lie detectors claim to work? They slapped AI on an old concept (that isn't useful and has been proven to be worthless a million times over).

darawk 8y ago
Existing lie detectors use a bunch of information like pulse, and galvanic skin response. Presumably this device is not hooked up to people enough to do that, so I don't think it's using the exact same metrics. Also, polygraphs work decently well actually. Just not well enough to be used as evidence in court.

dagw 8y ago

I would think they have a dataset that they're validating them on if they're calling it "AI". You obviously don't work for IBM :) A skilled IBM Watson sales consultant as capable of selling simple linear regressions as Big Data AI!

bryanrasmussen 8y ago
You know with a big enough corpus of successful sales I think we should be able to train our Sales-Bot to sell simple linear regressions as Big Data AI!
- stochastic_monk 8y ago
  I wish that machine learning wasn’t being mislabeled as AI. It’s pattern recognition, not intelligence.
  - dagw 8y ago
    It’s pattern recognition, not intelligence. While I don't disagree that "AI" as a term is being abused, the distinction you're making seems very much a philosophical one.
  - gnulinux 8y ago
    One could argue intelligence is pattern recognition. One important aspect of human intelligence is neocortex -- a part of brain -- that exists in all mammals and mammals only. It serves as a pattern recognizer in mammal brains that helps them recognize certain sound, day/night, visual patterns etc...
darawk 8y ago
Of course. All i'm saying is that presumably they've at least cross-validated that linear regression.

yarrel 8y ago

The imperfection is large enough to cause serious concern.

darawk 8y ago
You know what the error rate on these devices is?

TeMPOraL 8y ago

> I would think they have a dataset that they're validating them on if they're calling it "AI". Where did they get that dataset from? Did they ask a bunch of people a bunch of questions and then hired private investigators to verify the answers (and then hire extra investigators to cross-verify answers given by the previous bunch)? Because beyond something like this, there's no known way to create such a dataset.

darawk 8y ago
I've considered this. You could create the dataset by looking at interviews with people caught smuggling, before they were caught. Or similar for any criminal activity.
sincerely 8y ago
Honestly creating that dataset would be the easiest part of this. Just ask people questions that you already know the answer to.