3 ms·

<With 90 minutes driving data or monitoring more car components, they could pick out the correct driver fully 100 percent of the time.> With 90 minutes of data

by DrScump 10y ago

<With 90 minutes driving data or monitoring more car components, they could pick out the correct driver fully 100 percent of the time.> With 90 minutes of data, they could pick out one driver out of a sample size of 15. How meaningful is that, really?

Terribledactyl 10y ago
I think they meant 15/15 drivers which is a little better, but yeah.
- DrScump 10y ago
  "...could let them correctly distinguish the correct driver out of 15 individuals about nine times out of ten, after just 15 minutes of driving." So, the claim is 90% accuracy in determining which one of the fifteen drivers they are looking at. So, they were correct on 13.5 drivers. Um, what?
  - rosser 10y ago
    Try parsing it as "90% of drives", not of drivers.
    - DrScump 10y ago
      That doesn't make sense to me. The constraint is 15 minutes total. If drives vs drivers give different data, that could only imply that (a) each driver's 15 minutes spans multiple vehicles, or (b) they make conflicting conclusions for the same driver across multiple trips.
      - rosser 10y ago
        You're reading constraints into the experiment that don't appear to exist. My take is that after 15 minutes of driving the algorithm could correctly identify which driver was behind the wheel. That means: I get in, start driving, and within 15 minutes or so, the system analyzing the car's internal network will, with 90% specificity, say, "The driver was 'rosser." I don't see where you're coming up with only 15 minutes total driving. That isn't implied anywhere in The Fine Article. But even leaving that detail aside, it would actually be more impressive than the reported finding if they were correctly identifying people that well after only 15 minutes total!
      - function_seven 10y ago
        It's (b). If each driver is run in twenty 15-minute (or longer) trials, you have 300 total data points to analyze. If the car was correct in its fingerprinting 270 times, then it was correct 90% of the time.
        DrScump 10y ago
        Yes, but the only way you get a number not divisible by 15 is if, for some number of given drivers, it must have been correct for some trips but incorrect for others. I would not consider those drivers to be "successfully identified". Remember, the goal of this is to isolate individual drivers.
        function_seven 10y ago
        > it must have been correct for some trips but incorrect for others. No, it could have been inconclusive for the other 10% of the trips, simply stating, "Insufficient data for meaningful answer". So 90% of the time it said, "Oh, this must be DrScump" and the other 10% of the time it said, "I don't know who this is yet, let me keep analyzing the braking patterns" EDIT: Which is different than the way I stated it in my initial comment. Your reply helped me realize that.
        DrScump 10y ago
        But this is a binary problem (they did or did not ID the driver); "inconclusive" = did not.
        function_seven 10y ago
        Yes, 10% of the time. The other 90% it did. So.. 90%. Not being snarky or passive aggressive, I just don't understand the issue you're having with that statistic.
bllguo 10y ago
Not sure it's as trivial as you seem to be implying, but extending this to more drivers would certainly be a logical next step. Honestly it seems like everyone reading these kinds of papers/articles does a ctrl-f "sample size" before anything else. Bit cynical.
- bravura 10y ago
  For good reason. It's been easy to do face identification for a very small population for a while. But scaling that to thousands of people has proven quite difficult.
  - clort 10y ago
    Its perhaps surprising to those within the tech industry, to find out how fast progress has been -- as I recently read https://www.theguardian.com/technology/2016/may/17/findface-face-recognition-app-end-public-anonymity-vkontakte https://www.theguardian.com/technology/2016/may/17/findface-... and the HN discussion https://news.ycombinator.com/item?id=11712197 https://news.ycombinator.com/item?id=11712197
krashnburn200 10y ago
Damn man... how many people do you let drive your car?
- DrScump 10y ago
  Only me. And given how buggy-as-all-hell the Ford software is for driver displays/interfaces, I wouldn't let it drive itself either.
  - GoToRO 10y ago
    FYI, those are different projects with vastly different quality requirements... But they do shoot themselves in the foot when they put buggy software right in front of their customer.
skywhopper 10y ago
Agreed. A 10% false positive rate is huge. Expand the dataset to millions and I doubt you can identify individuals quite so certainly.
- ehnto 10y ago
  I don't believe the application is in surveillance or tracking. It's just useful to know who is driving out of say a few people so you can adjust vehicle parameters to them. Modern cars will already learn about driving characteristics so they can better manage the car to match, but I don't know if they try to match individuals.
  - jlgaddis 10y ago
    I think you're correct, re: adjusting parameters based upon the driver. My truck has three "memory settings". I can adjust everything the way I want it -- different seat positions, mirrors, etc. -- and save those in "A". My girlfriend can then get everything the way she wants it (she's much shorter and smaller than me) and save her settings in "B". When I get in to drive after she has been driving, I just hit the "A" button to have everything adjusted back to exactly how I want it. It's very handy when you have multiple people who drive a single vehicle.
  - galdosdi 10y ago
    Speaking of which, it's really maddening how the latest automatic transmissions try to learn about you, because, guess what -- that impedes ME learning about IT! I rented a new Chrysler 200 a few months ago and just when I thought I had the throttle figured out to where I could trick the transmission into not upshifting when I wanted it not to, upshifting when I did want it to, etc... it stopped working. It was a bizarre experience, like some sort of arms race or adaptive-difficulty video game between it and me, with my goal to actually control the RPM and its goal being to prevent RPM from getting out of its preferred small range no matter how increasingly erratic-seeming my throttle foot behavior became. As a result, I am unable to develop any intuition about how the engine will actually behave in response to my inputs, and in a split-second dangerous situation will be unable to do anything useful. It's advertised as "helping fuel economy" but all it really is is "adaptively learning to ignore the operator's inputs" which might be nice for the first few thousand miles a brand new driver drives, but seems outright dangerous afterwards.
superuser2 10y ago
How often are you going to want to identify who's driving a car with no clue who it is? This has implications for "which family member is it?" or "you claimed you lent it to your friend, is that really true?" etc.

by DrScump 10y ago

Terribledactyl 10y ago

I think they meant 15/15 drivers which is a little better, but yeah.

DrScump 10y ago
"...could let them correctly distinguish the correct driver out of 15 individuals about nine times out of ten, after just 15 minutes of driving." So, the claim is 90% accuracy in determining which one of the fifteen drivers they are looking at. So, they were correct on 13.5 drivers. Um, what?
- rosser 10y ago
  Try parsing it as "90% of drives", not of drivers.
  - DrScump 10y ago
    That doesn't make sense to me. The constraint is 15 minutes total. If drives vs drivers give different data, that could only imply that (a) each driver's 15 minutes spans multiple vehicles, or (b) they make conflicting conclusions for the same driver across multiple trips.
    - rosser 10y ago
      You're reading constraints into the experiment that don't appear to exist. My take is that after 15 minutes of driving the algorithm could correctly identify which driver was behind the wheel. That means: I get in, start driving, and within 15 minutes or so, the system analyzing the car's internal network will, with 90% specificity, say, "The driver was 'rosser." I don't see where you're coming up with only 15 minutes total driving. That isn't implied anywhere in The Fine Article. But even leaving that detail aside, it would actually be more impressive than the reported finding if they were correctly identifying people that well after only 15 minutes total!
    - function_seven 10y ago
      It's (b). If each driver is run in twenty 15-minute (or longer) trials, you have 300 total data points to analyze. If the car was correct in its fingerprinting 270 times, then it was correct 90% of the time.
      - DrScump 10y ago
        Yes, but the only way you get a number not divisible by 15 is if, for some number of given drivers, it must have been correct for some trips but incorrect for others. I would not consider those drivers to be "successfully identified". Remember, the goal of this is to isolate individual drivers.
        function_seven 10y ago
        > it must have been correct for some trips but incorrect for others. No, it could have been inconclusive for the other 10% of the trips, simply stating, "Insufficient data for meaningful answer". So 90% of the time it said, "Oh, this must be DrScump" and the other 10% of the time it said, "I don't know who this is yet, let me keep analyzing the braking patterns" EDIT: Which is different than the way I stated it in my initial comment. Your reply helped me realize that.
        DrScump 10y ago
        But this is a binary problem (they did or did not ID the driver); "inconclusive" = did not.
        function_seven 10y ago
        Yes, 10% of the time. The other 90% it did. So.. 90%. Not being snarky or passive aggressive, I just don't understand the issue you're having with that statistic.

bllguo 10y ago

Not sure it's as trivial as you seem to be implying, but extending this to more drivers would certainly be a logical next step. Honestly it seems like everyone reading these kinds of papers/articles does a ctrl-f "sample size" before anything else. Bit cynical.

bravura 10y ago
For good reason. It's been easy to do face identification for a very small population for a while. But scaling that to thousands of people has proven quite difficult.
- clort 10y ago
  Its perhaps surprising to those within the tech industry, to find out how fast progress has been -- as I recently read https://www.theguardian.com/technology/2016/may/17/findface-face-recognition-app-end-public-anonymity-vkontakte https://www.theguardian.com/technology/2016/may/17/findface-... and the HN discussion https://news.ycombinator.com/item?id=11712197 https://news.ycombinator.com/item?id=11712197

krashnburn200 10y ago

Damn man... how many people do you let drive your car?

DrScump 10y ago
Only me. And given how buggy-as-all-hell the Ford software is for driver displays/interfaces, I wouldn't let it drive itself either.
- GoToRO 10y ago
  FYI, those are different projects with vastly different quality requirements... But they do shoot themselves in the foot when they put buggy software right in front of their customer.

skywhopper 10y ago

Agreed. A 10% false positive rate is huge. Expand the dataset to millions and I doubt you can identify individuals quite so certainly.

ehnto 10y ago
I don't believe the application is in surveillance or tracking. It's just useful to know who is driving out of say a few people so you can adjust vehicle parameters to them. Modern cars will already learn about driving characteristics so they can better manage the car to match, but I don't know if they try to match individuals.
- jlgaddis 10y ago
  I think you're correct, re: adjusting parameters based upon the driver. My truck has three "memory settings". I can adjust everything the way I want it -- different seat positions, mirrors, etc. -- and save those in "A". My girlfriend can then get everything the way she wants it (she's much shorter and smaller than me) and save her settings in "B". When I get in to drive after she has been driving, I just hit the "A" button to have everything adjusted back to exactly how I want it. It's very handy when you have multiple people who drive a single vehicle.
- galdosdi 10y ago
  Speaking of which, it's really maddening how the latest automatic transmissions try to learn about you, because, guess what -- that impedes ME learning about IT! I rented a new Chrysler 200 a few months ago and just when I thought I had the throttle figured out to where I could trick the transmission into not upshifting when I wanted it not to, upshifting when I did want it to, etc... it stopped working. It was a bizarre experience, like some sort of arms race or adaptive-difficulty video game between it and me, with my goal to actually control the RPM and its goal being to prevent RPM from getting out of its preferred small range no matter how increasingly erratic-seeming my throttle foot behavior became. As a result, I am unable to develop any intuition about how the engine will actually behave in response to my inputs, and in a split-second dangerous situation will be unable to do anything useful. It's advertised as "helping fuel economy" but all it really is is "adaptively learning to ignore the operator's inputs" which might be nice for the first few thousand miles a brand new driver drives, but seems outright dangerous afterwards.

superuser2 10y ago

How often are you going to want to identify who's driving a car with no clue who it is? This has implications for "which family member is it?" or "you claimed you lent it to your friend, is that really true?" etc.