15 ms·

How do you think zooming works?

Twisol 5y ago
If the pre-zoomed image was already downscaled (e.g. to fit a small display), then up to a point, zooming should simply downscale less, i.e. remove fewer pixels. Beyond that point, yes, a zoom would have to perform some kind of interpolation. In the CSI "zoom and enhance", anything beyond the simplest forms of interpolation would be part of "enhance", which is to say, not part of "zoom".
- ImprobableTruth 5y ago
  Even downscaling uses interpolation, but sure. The zoomed in part definitely wasn't shown at its native resolution though. And bilinear interpolation is as simple as you can get besides nearest neighbor, yet it still has the potential to introduce wrong information/cause artifacting. So even in the "CSI" sense, zooming can cause issues (unless you want to argue that only nearest neighbor is CSI 'zooming'? But then not even browsers do only CSI 'zooming').
_djo_ 5y ago
As Twisol said, zooming doesn't inherently require interpolation. But when it does require interpolation, it's usually an extremely simple and well-understood form like nearest neighbour interpolation that's not going to result in things like faces being inserted. It also only uses the available data in the image itself on which to work. It's fundamentally different from neural network-driven inference and interpolation that attempts to fill in pixels using evidence from other images that the network has been trained on.
- iainmerrick 5y ago
  How is a non-expert, and somebody whose job specifically requires them to be sceptical and to question unstated assumptions, supposed to be sure of that?
  - _djo_ 5y ago
    It's a simple enough thing to verify, or require the defence (in this case) to substantiate the claim without throwing out the method of showing evidence entirely. Courts have to deal with questions like this all the time and handle them well enough. Technology isn't magic, nor do we have to go down a rabbit hole on each point about what might be there rather than what is there.
    - A4ET8a8uTh0 5y ago
      It is not magic, but, as other posters alluded, sometimes it can be merely an educated guess. It is hard to accept a life changing verdict based on a guess.. even if it is an educated one. So no, each time a given technique is used, the way it works has to be explained so that a decision based on facts of the case is made.
      - _djo_ 5y ago
        Nothing wrong with that, but a common sense explanation by an expert witness ought to be enough. Does Apple need to be dragged into each and every case to testify as to how pinch to zoom works, as was demanded here? Courts have established mechanisms for handling technical questions around evidence. It's not like any of this is new.
        jtbayly 5y ago
        All that was required was an expert witness, right? So what’s the problem?
        _djo_ 5y ago
        The judge did not give the prosecution time to find an expert witness, did not ask his clerks to search for similar usage of iPad zooming in previous cases, and did not require the defence to provide expert testimony of their own to substantiate their claim. In the end the iPad was not used. Instead a 4K TV was used, which ironically probably upscaled the image too.
- vidarh 5y ago
  It's only fundamentally different until some image manipulation program chooses to label AI driven interpolation as zoom. Language is malleable, and so while it may be paranoid to question a feature labelled as zoom now, it's an open question how long that will last.
  - _djo_ 5y ago
    It was a claim made by the defence, which the judge insisted that the prosecution prove the absence of, rather than requiring the defence to substantiate. That's not how technical claims of that nature are normally handled.
    - ImprobableTruth 5y ago
      The prosecution presented modified evidence, claiming that the modification doesn't change it, which the defense objected to. Why would the burden of proof not fall on the prosecution?
      - _djo_ 5y ago
        Zooming into a video using the built-in feature on a device is modified evidence now? We'd have to rerun a bunch of trials in that case. The fantastical claim was from the defence. Onus should be on them to prove it, rather than making the prosecution prove a negative.
        core-utility 5y ago
        What does "built-in" have to do with anything? The entire point of the issue was that it wasn't just zoom in, it was enlargement (think upscaling). The software has to actively interpolate and add pixels where it thinks they're necessary in order to do so, and that changes the image. In most cases, it's probably not a big deal. But when it's from a nighttime camera of a person several hundred feet away that's only a few pixels in the original frame, there's a lot to go wrong there.
- ImprobableTruth 5y ago
  This is just pivoting. In this case zooming uses interpolation which "has the potential to introduce false information" exactly as you mentioned. It using only the available data doesn't mean that interpolation can't go wrong. So what if it's not AI? And yes, it's so well understood that the prosecution had no idea of what's going on, claiming that it's "common sense" that it works just like a magnifying glass, which is simply wrong. Using bilinear (or bicubic) interpolation makes the edges of an image blurrier, especially when you're zooming in as much as they did on low-quality footage.
  - _djo_ 5y ago
    Come now, you're reaching. The defence attorney specifically claimed that "iPads, which are made by Apple, have artificial intelligence in them that allow things to be viewed through three dimensions and logarithms." and "[The iPad] uses artificial intelligence or their logarithms to create what they believe is happening. So this isn't actually enhanced video; this is Apple's iPad programming creating what it thinks is there, not what necessarily is there." Aside from the confusion about logarithms, where I presume he meant algorithms, and the false claim about 3D, the defence was making a claim about artificial intelligence being used to upscale the footage. Which is inaccurate, and a specific claim being made that's unsupported by any evidence. Funny enough, the same sort of question arose with regard to a crime scene image that was enlarged by the state's forensic lab, but a simple explanation of the mechanism used was deemed sufficient.
    - ImprobableTruth 5y ago
      I'm reaching? Like, why would the defense's wrong argument be relevant? You're not arguing with the defense here. Your claim was that the judge's call is wrong because zooming doesn't actually do interpolation and has thus no potential to introduce wrong information. I've told you that zooming in this case does interpolate and that interpolation has the potential to go wrong.
      - jschwartzi 5y ago
        When interpolation goes wrong it’s predictably wrong, which is not true of more complex algorithms using AI. If we can predict how it goes wrong we can extract the meaningful information from it.

If the pre-zoomed image was already downscaled (e.g. to fit a small display), then up to a point, zooming should simply downscale less, i.e. remove fewer pixels. Beyond that point, yes, a zoom would have to perform some kind of interpolation. In the CSI "zoom and enhance", anything beyond the simplest forms of interpolation would be part of "enhance", which is to say, not part of "zoom".

ImprobableTruth 5y ago
Even downscaling uses interpolation, but sure. The zoomed in part definitely wasn't shown at its native resolution though. And bilinear interpolation is as simple as you can get besides nearest neighbor, yet it still has the potential to introduce wrong information/cause artifacting. So even in the "CSI" sense, zooming can cause issues (unless you want to argue that only nearest neighbor is CSI 'zooming'? But then not even browsers do only CSI 'zooming').

_djo_ 5y ago

As Twisol said, zooming doesn't inherently require interpolation. But when it does require interpolation, it's usually an extremely simple and well-understood form like nearest neighbour interpolation that's not going to result in things like faces being inserted. It also only uses the available data in the image itself on which to work. It's fundamentally different from neural network-driven inference and interpolation that attempts to fill in pixels using evidence from other images that the network has been trained on.

iainmerrick 5y ago
How is a non-expert, and somebody whose job specifically requires them to be sceptical and to question unstated assumptions, supposed to be sure of that?
- _djo_ 5y ago
  It's a simple enough thing to verify, or require the defence (in this case) to substantiate the claim without throwing out the method of showing evidence entirely. Courts have to deal with questions like this all the time and handle them well enough. Technology isn't magic, nor do we have to go down a rabbit hole on each point about what might be there rather than what is there.
  - A4ET8a8uTh0 5y ago
    It is not magic, but, as other posters alluded, sometimes it can be merely an educated guess. It is hard to accept a life changing verdict based on a guess.. even if it is an educated one. So no, each time a given technique is used, the way it works has to be explained so that a decision based on facts of the case is made.
    - _djo_ 5y ago
      Nothing wrong with that, but a common sense explanation by an expert witness ought to be enough. Does Apple need to be dragged into each and every case to testify as to how pinch to zoom works, as was demanded here? Courts have established mechanisms for handling technical questions around evidence. It's not like any of this is new.
      - jtbayly 5y ago
        All that was required was an expert witness, right? So what’s the problem?
        _djo_ 5y ago
        The judge did not give the prosecution time to find an expert witness, did not ask his clerks to search for similar usage of iPad zooming in previous cases, and did not require the defence to provide expert testimony of their own to substantiate their claim. In the end the iPad was not used. Instead a 4K TV was used, which ironically probably upscaled the image too.
vidarh 5y ago
It's only fundamentally different until some image manipulation program chooses to label AI driven interpolation as zoom. Language is malleable, and so while it may be paranoid to question a feature labelled as zoom now, it's an open question how long that will last.
- _djo_ 5y ago
  It was a claim made by the defence, which the judge insisted that the prosecution prove the absence of, rather than requiring the defence to substantiate. That's not how technical claims of that nature are normally handled.
  - ImprobableTruth 5y ago
    The prosecution presented modified evidence, claiming that the modification doesn't change it, which the defense objected to. Why would the burden of proof not fall on the prosecution?
    - _djo_ 5y ago
      Zooming into a video using the built-in feature on a device is modified evidence now? We'd have to rerun a bunch of trials in that case. The fantastical claim was from the defence. Onus should be on them to prove it, rather than making the prosecution prove a negative.
      - core-utility 5y ago
        What does "built-in" have to do with anything? The entire point of the issue was that it wasn't just zoom in, it was enlargement (think upscaling). The software has to actively interpolate and add pixels where it thinks they're necessary in order to do so, and that changes the image. In most cases, it's probably not a big deal. But when it's from a nighttime camera of a person several hundred feet away that's only a few pixels in the original frame, there's a lot to go wrong there.
ImprobableTruth 5y ago
This is just pivoting. In this case zooming uses interpolation which "has the potential to introduce false information" exactly as you mentioned. It using only the available data doesn't mean that interpolation can't go wrong. So what if it's not AI? And yes, it's so well understood that the prosecution had no idea of what's going on, claiming that it's "common sense" that it works just like a magnifying glass, which is simply wrong. Using bilinear (or bicubic) interpolation makes the edges of an image blurrier, especially when you're zooming in as much as they did on low-quality footage.
- _djo_ 5y ago
  Come now, you're reaching. The defence attorney specifically claimed that "iPads, which are made by Apple, have artificial intelligence in them that allow things to be viewed through three dimensions and logarithms." and "[The iPad] uses artificial intelligence or their logarithms to create what they believe is happening. So this isn't actually enhanced video; this is Apple's iPad programming creating what it thinks is there, not what necessarily is there." Aside from the confusion about logarithms, where I presume he meant algorithms, and the false claim about 3D, the defence was making a claim about artificial intelligence being used to upscale the footage. Which is inaccurate, and a specific claim being made that's unsupported by any evidence. Funny enough, the same sort of question arose with regard to a crime scene image that was enlarged by the state's forensic lab, but a simple explanation of the mechanism used was deemed sufficient.
  - ImprobableTruth 5y ago
    I'm reaching? Like, why would the defense's wrong argument be relevant? You're not arguing with the defense here. Your claim was that the judge's call is wrong because zooming doesn't actually do interpolation and has thus no potential to introduce wrong information. I've told you that zooming in this case does interpolate and that interpolation has the potential to go wrong.
    - jschwartzi 5y ago
      When interpolation goes wrong it’s predictably wrong, which is not true of more complex algorithms using AI. If we can predict how it goes wrong we can extract the meaningful information from it.