7 ms·

> This is a current limitation The thing with any AI/ML tech is that current limitations are always underplayed by proponents. Self-driving cars will come out

by Strom 4y ago

> This is a current limitation The thing with any AI/ML tech is that current limitations are always underplayed by proponents. Self-driving cars will come out next year, every year. I'd say that until the tech actually exists, this is a great way to detect live deepfakes. Not using the technique just because maybe sometime in the future it won't work isn't very sound. For an extreme opponent you may need additional steps. So this sideways trick probably isn't enough for CIA or whatnot, but that's about as fringe as you can get and very little generic advice applies anyway.

esotericimpl 4y ago
technothrasher 4y ago
It sounded to me like the parent poster wasn't saying not to use it, but simply that it cannot be relied upon. In other words, a deepfake could fail a 'turn sideways' test and that would be useful, but you shouldn't rely on a 'passing' test.
- kbenson 4y ago
  Another way to think of it might be that it can be relied on - until it can't. Be ready and wary of that happening, but until then you have what's probably a good mitigation of the problem.
  - hosh 4y ago
    I think the concern is complacency, and the inertia that existing security practices leads to security gaps in the future. "However, I don't know one organisation that doesn't have some outdated security guideline that they cling to, e.g. old school password rules and rotations." Or put another way, humans can't be ready and wary, constantly and indefinitely. At some point, fatigue sets in. People move in and out of the organization. Periodic reviews of security practices don't always catch everything. Why something was implemented was forgotten by institutional memory. And then there's the cost for retraining people.
    - kbenson 4y ago
      The flip side of that is people feeling/assuming there's nothing they can really do with the resources they have therefore they choose to do nothing. Also, those that are actively using mitigations that are going to be outdated at some point are probably far more likely to be aware of how close they are to being outdated by encountering more ambiguous cases, as seeing the state of the art progress right in front of them. As for people sticking to outdated security practices? That's a problem of people and organizations being introspective and examining themselves, and is not linked to any one thing. We all have that problem to a lesser or greater degree in all aspects of what we do, so either you have systems in place to mitigate it or you don't.
      - hosh 4y ago
        Therefore, developing and customizing a proper framework for security and privacy starts by accurately assessing statutory, regulatory, and contractual obligations, and the organization's appetite for risks in balance with the organization's mission and vision, before developing the policies and and specific practices that organizational members should be doing. To use a Go (the game, not the language) metaphor, skilled players always assess the whole board rather than automatically make a local move in response to a local threat. What's right for one organization is not going to be right for another. Asking the caller to turn sideways to protect against deepfakes should be considered within the organization's own framework, along with the various risks involved with deepfakes, and many other risks aside from deep fake video calls.
        eru 4y ago
        Asking the caller to turn sideways is also a cheap countermeasure without serious side-effects. So there's low risk to adopting it.
        hosh 4y ago
        If that is conclusion that is considered within the organization’s custom security and privacy framework, sure. If there is no such framework, this is no different than yoloing lines of code in a production app by a team that does not have at least some grasp of the architectural principles and constraints at play. Or worse, not understanding the “job to be done” and building the wrong product and solving for the wrong problem.
  - Sakos 4y ago
    How do you find out that it doesn't work?
- williamscales 4y ago
  Exactly. Even the article gave a couple cases of convincing profile deepfakes. Admittedly they’re exceptional cases, but in general progress tends to be made.
Nowado 4y ago
In terms of this particular tech previous obvious limitation, namely no blinking, worked for something like a quarter from discovery. Venn diagram of people who someone wants to trick by this particular tech, those who read any security guidelines and those worthy of applying this kind of approach to in the first place is however pretty narrow for the foreseeable future. It's more of a narrative framing device to talk about 'what to do to uncover deepfake video call' as a way to present interesting current tech limitations - not that I particularly mind it.
- anticristi 4y ago
  Exactly! Our SecOps includes seeing people regularly. Until deep fakes can fake accents, tone, body language and jokes, we're safe. :)
make3 4y ago
Self-driving cars are a million times harder than this, this is a terrible comparison. Getting a model to work with images turned sideways is a few lines of code (just turn image sideways at training time).
- kreeben 4y ago
  >> images turned sideways Instead of pictures of faces, now they're just vertical lines.
pclmulqdq 4y ago
The only person who is promising self driving cars next year (and has done so every year for the past 5 years) is Elon Musk. Most respectable self-driving car companies are both further along than Tesla and more realistic about their timelines.
- Strom 4y ago
  Let's take a look at some of those realistic timelines. A quick googling gave me a very helpful listicle by VentureBeat from 2017, titled Self-driving car timeline for 11 top automakers. [1] Some examples: Ford - Level 4 vehicle in 2021, no gas pedal, no steering wheel, and the passenger will never need to take control of the vehicle in a predefined area. Honda - production vehicles with automated driving capabilities on highways sometime around 2020 Toyta - Self-driving on the highway by 2020 Renault-Nissan - 2020 for the autonomous car in urban conditions, probably 2025 for the driverless car Volvo - It’s our ambition to have a car that can drive fully autonomously on the highway by 2021. Hyundai - We are targeting for the highway in 2020 and urban driving in 2030. Daimler - large-scale commercial production to take off between 2020 and 2025 BMW - highly and fully automated driving into series production by 2021 Tesla - End of 2017 It certainly wasn't just Tesla who was promising self-driving cars any second now. Tesla was definitely the most agressive, but failed to meet its goals just like every other manufacturer. -- [1] https://venturebeat.com/2017/06/04/self-driving-car-timeline-for-11-top-automakers/ https://venturebeat.com/2017/06/04/self-driving-car-timeline...
  - ghaff 4y ago
    There was definitely a period when everyone (for certain values of same) felt they needed to get into a game of topper with increasingly outlandish claims. Because if they didn't people on, say, forums like this one (and more importantly the stock market) would see them as hopelessly behind.
  - throwawaylinux 4y ago
    Wow they all really got suckered by the AI grifters didn't they?
- 4y ago
owl57 4y ago
> Self-driving cars will come out next year, every year. "Come out" could mean different things in different contexts. Deepfake defence context is analogous to something like: there are cars on public roads with no driver at the wheel. And this is already true in multiple places in the world.
- verdverm 4y ago
  Waymo in Arizona is an example
jksmith 4y ago
This may be like a proof of work cryptography issue, except the burden of work is on the deep fake. Just ask a battery of questions, just like out of a Bladerunner scene or whatever. This is still the problem with AI. It depends on tons of datasets and connectivity. Human data and human code are kind of the same. Even individually, we can start with jackshit and still come up with an answer, whether right or wrong. Ah, Lisp.
iancmceachern 4y ago
I think it's odd we don't think of other limitations of products the same way. Put another way, why don't we just say it can't do it. Example, we don't say a jet ski has a current speed limitation of 80 mph, we say it can go 80, but not 81. It's a simple fact. No promise that it will be faster tomorrow, because that's not what it is, it's not its future self. It's like they're combining startup it will always be better after you invest more money with the reality of what "is" means.
fsckboy 4y ago
> The thing with any AI/ML tech is that current limitations are always underplayed by proponents if you don't worry about deepfakes, ok. But if you worry about deepfakes, you should not be reassured that this glitch is going to save you. I'm not a proponent, just think your argument in this context doesn't work.
TomK32 4y ago
The self driving car of next year arrives just in time for the Iranian atomic bomb :-D which is ready in two years for about as long as I'm around. https://www.theatlantic.com/international/archive/2015/04/iran-has-been-two-years-away-from-a-nuclear-bomb-since-the-1980s/389333/ https://www.theatlantic.com/international/archive/2015/04/ir... If all the money on self driving cars would have been put into public transport (driverless on rails is a solved issue) and pushing shared car ownership instead, we might actually get somewhere towards congestion-free cities.
- eru 4y ago
  We can already have congestion free-cities today, no new technology nor public transport required. We had the technology for quite a while now: congestion charging. It works really well in Singapore to control congestion, and also worked well in London when they adopted it afterwards. Public transport also works quite well in many places around the world. It also used to work really well in North America in the past. A past when the continent was much poorer. (I'm mostly talking about USA plus Canada here.) Public transport only works when after you step off the bus or train, you can get to your destination on foot. Density is outlawed in much of the USA and Canada. https://www.youtube.com/watch?v=MnyeRlMsTgI&t=416s https://www.youtube.com/watch?v=MnyeRlMsTgI&t=416s starts a good section about Fake London, Ontario. At great expense, they built a new train line. But approximately no one uses it, because you can't get anywhere when leaving the stations. The video shows an example of a station where the closest other building is about 150m away. And that's just a single building. The next ones are even further. Land use restrictions and minimum parking requirements are a major roadblock. And just throwing money at public transit directly won't solve those. Shared car ownership is an interesting idea. Uber can be seen as one implementation of this concept. It can be done profitably, but I'm not sure it has much impact on the shape of cities? In the grand scheme of things, there's not much money being put into self-driving cars so far. A quick Googling gives a Forbes article that suggests about 200 billion USD.
sigg3 4y ago
The technique can in principle be defeated today so it should not be employed as a single test, but rather another arrow in the quiver.
sean2 4y ago
One thing that I haven't seen mentioned is that many of the recent articles I've seen misuse the phrase "deep fake" and usually mean "face-swap algorithm" or "look-alike". The former, I believe has been able to defeat this test for 10 years at least and the latter has always been able defeat this trick.

by Strom 4y ago

esotericimpl 4y ago

technothrasher 4y ago

It sounded to me like the parent poster wasn't saying not to use it, but simply that it cannot be relied upon. In other words, a deepfake could fail a 'turn sideways' test and that would be useful, but you shouldn't rely on a 'passing' test.

kbenson 4y ago
Another way to think of it might be that it can be relied on - until it can't. Be ready and wary of that happening, but until then you have what's probably a good mitigation of the problem.
- hosh 4y ago
  I think the concern is complacency, and the inertia that existing security practices leads to security gaps in the future. "However, I don't know one organisation that doesn't have some outdated security guideline that they cling to, e.g. old school password rules and rotations." Or put another way, humans can't be ready and wary, constantly and indefinitely. At some point, fatigue sets in. People move in and out of the organization. Periodic reviews of security practices don't always catch everything. Why something was implemented was forgotten by institutional memory. And then there's the cost for retraining people.
  - kbenson 4y ago
    The flip side of that is people feeling/assuming there's nothing they can really do with the resources they have therefore they choose to do nothing. Also, those that are actively using mitigations that are going to be outdated at some point are probably far more likely to be aware of how close they are to being outdated by encountering more ambiguous cases, as seeing the state of the art progress right in front of them. As for people sticking to outdated security practices? That's a problem of people and organizations being introspective and examining themselves, and is not linked to any one thing. We all have that problem to a lesser or greater degree in all aspects of what we do, so either you have systems in place to mitigate it or you don't.
    - hosh 4y ago
      Therefore, developing and customizing a proper framework for security and privacy starts by accurately assessing statutory, regulatory, and contractual obligations, and the organization's appetite for risks in balance with the organization's mission and vision, before developing the policies and and specific practices that organizational members should be doing. To use a Go (the game, not the language) metaphor, skilled players always assess the whole board rather than automatically make a local move in response to a local threat. What's right for one organization is not going to be right for another. Asking the caller to turn sideways to protect against deepfakes should be considered within the organization's own framework, along with the various risks involved with deepfakes, and many other risks aside from deep fake video calls.
      - eru 4y ago
        Asking the caller to turn sideways is also a cheap countermeasure without serious side-effects. So there's low risk to adopting it.
        hosh 4y ago
        If that is conclusion that is considered within the organization’s custom security and privacy framework, sure. If there is no such framework, this is no different than yoloing lines of code in a production app by a team that does not have at least some grasp of the architectural principles and constraints at play. Or worse, not understanding the “job to be done” and building the wrong product and solving for the wrong problem.
- Sakos 4y ago
  How do you find out that it doesn't work?
williamscales 4y ago
Exactly. Even the article gave a couple cases of convincing profile deepfakes. Admittedly they’re exceptional cases, but in general progress tends to be made.

Nowado 4y ago

In terms of this particular tech previous obvious limitation, namely no blinking, worked for something like a quarter from discovery. Venn diagram of people who someone wants to trick by this particular tech, those who read any security guidelines and those worthy of applying this kind of approach to in the first place is however pretty narrow for the foreseeable future. It's more of a narrative framing device to talk about 'what to do to uncover deepfake video call' as a way to present interesting current tech limitations - not that I particularly mind it.

anticristi 4y ago
Exactly! Our SecOps includes seeing people regularly. Until deep fakes can fake accents, tone, body language and jokes, we're safe. :)

make3 4y ago

Self-driving cars are a million times harder than this, this is a terrible comparison. Getting a model to work with images turned sideways is a few lines of code (just turn image sideways at training time).

kreeben 4y ago
>> images turned sideways Instead of pictures of faces, now they're just vertical lines.

pclmulqdq 4y ago

The only person who is promising self driving cars next year (and has done so every year for the past 5 years) is Elon Musk. Most respectable self-driving car companies are both further along than Tesla and more realistic about their timelines.

Strom 4y ago
Let's take a look at some of those realistic timelines. A quick googling gave me a very helpful listicle by VentureBeat from 2017, titled Self-driving car timeline for 11 top automakers. [1] Some examples: Ford - Level 4 vehicle in 2021, no gas pedal, no steering wheel, and the passenger will never need to take control of the vehicle in a predefined area. Honda - production vehicles with automated driving capabilities on highways sometime around 2020 Toyta - Self-driving on the highway by 2020 Renault-Nissan - 2020 for the autonomous car in urban conditions, probably 2025 for the driverless car Volvo - It’s our ambition to have a car that can drive fully autonomously on the highway by 2021. Hyundai - We are targeting for the highway in 2020 and urban driving in 2030. Daimler - large-scale commercial production to take off between 2020 and 2025 BMW - highly and fully automated driving into series production by 2021 Tesla - End of 2017 It certainly wasn't just Tesla who was promising self-driving cars any second now. Tesla was definitely the most agressive, but failed to meet its goals just like every other manufacturer. -- [1] https://venturebeat.com/2017/06/04/self-driving-car-timeline-for-11-top-automakers/ https://venturebeat.com/2017/06/04/self-driving-car-timeline...
- ghaff 4y ago
  There was definitely a period when everyone (for certain values of same) felt they needed to get into a game of topper with increasingly outlandish claims. Because if they didn't people on, say, forums like this one (and more importantly the stock market) would see them as hopelessly behind.
- throwawaylinux 4y ago
  Wow they all really got suckered by the AI grifters didn't they?
4y ago

owl57 4y ago

> Self-driving cars will come out next year, every year. "Come out" could mean different things in different contexts. Deepfake defence context is analogous to something like: there are cars on public roads with no driver at the wheel. And this is already true in multiple places in the world.

verdverm 4y ago
Waymo in Arizona is an example

jksmith 4y ago

This may be like a proof of work cryptography issue, except the burden of work is on the deep fake. Just ask a battery of questions, just like out of a Bladerunner scene or whatever. This is still the problem with AI. It depends on tons of datasets and connectivity. Human data and human code are kind of the same. Even individually, we can start with jackshit and still come up with an answer, whether right or wrong. Ah, Lisp.

iancmceachern 4y ago

I think it's odd we don't think of other limitations of products the same way. Put another way, why don't we just say it can't do it. Example, we don't say a jet ski has a current speed limitation of 80 mph, we say it can go 80, but not 81. It's a simple fact. No promise that it will be faster tomorrow, because that's not what it is, it's not its future self. It's like they're combining startup it will always be better after you invest more money with the reality of what "is" means.

fsckboy 4y ago

> The thing with any AI/ML tech is that current limitations are always underplayed by proponents if you don't worry about deepfakes, ok. But if you worry about deepfakes, you should not be reassured that this glitch is going to save you. I'm not a proponent, just think your argument in this context doesn't work.

TomK32 4y ago

The self driving car of next year arrives just in time for the Iranian atomic bomb :-D which is ready in two years for about as long as I'm around. https://www.theatlantic.com/international/archive/2015/04/iran-has-been-two-years-away-from-a-nuclear-bomb-since-the-1980s/389333/ https://www.theatlantic.com/international/archive/2015/04/ir... If all the money on self driving cars would have been put into public transport (driverless on rails is a solved issue) and pushing shared car ownership instead, we might actually get somewhere towards congestion-free cities.

eru 4y ago
We can already have congestion free-cities today, no new technology nor public transport required. We had the technology for quite a while now: congestion charging. It works really well in Singapore to control congestion, and also worked well in London when they adopted it afterwards. Public transport also works quite well in many places around the world. It also used to work really well in North America in the past. A past when the continent was much poorer. (I'm mostly talking about USA plus Canada here.) Public transport only works when after you step off the bus or train, you can get to your destination on foot. Density is outlawed in much of the USA and Canada. https://www.youtube.com/watch?v=MnyeRlMsTgI&t=416s https://www.youtube.com/watch?v=MnyeRlMsTgI&t=416s starts a good section about Fake London, Ontario. At great expense, they built a new train line. But approximately no one uses it, because you can't get anywhere when leaving the stations. The video shows an example of a station where the closest other building is about 150m away. And that's just a single building. The next ones are even further. Land use restrictions and minimum parking requirements are a major roadblock. And just throwing money at public transit directly won't solve those. Shared car ownership is an interesting idea. Uber can be seen as one implementation of this concept. It can be done profitably, but I'm not sure it has much impact on the shape of cities? In the grand scheme of things, there's not much money being put into self-driving cars so far. A quick Googling gives a Forbes article that suggests about 200 billion USD.

sigg3 4y ago

The technique can in principle be defeated today so it should not be employed as a single test, but rather another arrow in the quiver.

sean2 4y ago

One thing that I haven't seen mentioned is that many of the recent articles I've seen misuse the phrase "deep fake" and usually mean "face-swap algorithm" or "look-alike". The former, I believe has been able to defeat this test for 10 years at least and the latter has always been able defeat this trick.