8 ms·

>an example of a judge taking a hyper-skeptic position on technology - namely, distrusting the zoom feature because it might add in extra information not presen

by jasode 5y ago

>an example of a judge taking a hyper-skeptic position on technology - namely, distrusting the zoom feature because it might add in extra information not present in the original image. Some mocked the judge but here's an example where an AI upscaling algorithm added a human face that was never there: https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-added-ryan-goslings-face-to-this-photo/ https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-a... For terminology trivia, that algorithm seems to be doing a human version of pareidolia :https://en.wikipedia.org/wiki/Pareidolia https://en.wikipedia.org/wiki/Pareidolia EDIT: based on some replies I see, my cite isn't to claim that Apple iOS is using AI enhancement during zoom to add fake pixels. The point is to consider why the judge (who is not a computer programmer) is skeptical. Therefore, his request for expert testimony to rule out that the zoom function doesn't add false information is not that unreasonable. In other words, what's obvious to a HN tech audience doesn't always apply to judges who don't have the same technical knowledge of how zoom algorithms actually work. So considering the judge's state of mind, the "fact" that iOS zoom doesn't add false incriminating pixels isn't a fact to his level of satisfaction unless the prosecutors bring in an expert to confirm it.

croes 5y ago
Does the iPad zoom function any upscaling?
- ImprobableTruth 5y ago
  You can't zoom without upscaling in some form. If you zoom in on an image in your browser, the image gets upscaled. It's going to be using a simple form of upscaling (default is bilinear), but it's still upscaling.
  - _djo_ 5y ago
    AI-driven upscaling which fills in pixels using data from other images is quite different to algorithmic interpolation that only uses the available information in the image being worked on.
    - ImprobableTruth 5y ago
      Both can introduce wrong information or cause artifacts. "It only uses the available information" is a red herring.
      - gifnamething 5y ago
        Then this is a problem inherent with digital photography. Any image you see is interpolated from Bayer data. There are multiple algorithms with different results, the colour needs to be adjusted, and the camera distortion needs to be reversed. All of this happens before you see anything on your screen.
      - _djo_ 5y ago
        A zooming or scaling algorithm that only uses available image data is deterministic, it should get the same result every time. One that relies on a neural network trained on other images is not, it might produce a subtly different image each time it's run and introduce artefacts from other images.
        andrepd 5y ago
        What stops a neural network zoom to be just labelled as "zoom"? I think its a very pertinent question from the judge.
        _djo_ 5y ago
        Asking a question is fine, accepting the defence's claim without further substantiation and the rest of the follow up is less so.
        jtbayly 5y ago
        And how do you resolve conflicting technological claims from the defense and prosecution? By expert witness. Which is what the judge did. So what’s the problem?
  - masklinn 5y ago
    > You can't zoom without upscaling in some form. Of course you can, by having the image originally be displayed downscaled for instance. The back camera of your average phone has a much higher resolution than its display, or most computer displays for that matter.
_djo_ 5y ago
Anyone would be supportive of judges being cautious over upscaling, which we know has the potential to introduce false information because it's doing inference and interpolation. Zooming is another issue entirely.
- ImprobableTruth 5y ago
  How do you think zooming works?
  - Twisol 5y ago
    If the pre-zoomed image was already downscaled (e.g. to fit a small display), then up to a point, zooming should simply downscale less, i.e. remove fewer pixels. Beyond that point, yes, a zoom would have to perform some kind of interpolation. In the CSI "zoom and enhance", anything beyond the simplest forms of interpolation would be part of "enhance", which is to say, not part of "zoom".
    - ImprobableTruth 5y ago
      Even downscaling uses interpolation, but sure. The zoomed in part definitely wasn't shown at its native resolution though. And bilinear interpolation is as simple as you can get besides nearest neighbor, yet it still has the potential to introduce wrong information/cause artifacting. So even in the "CSI" sense, zooming can cause issues (unless you want to argue that only nearest neighbor is CSI 'zooming'? But then not even browsers do only CSI 'zooming').
  - _djo_ 5y ago
    As Twisol said, zooming doesn't inherently require interpolation. But when it does require interpolation, it's usually an extremely simple and well-understood form like nearest neighbour interpolation that's not going to result in things like faces being inserted. It also only uses the available data in the image itself on which to work. It's fundamentally different from neural network-driven inference and interpolation that attempts to fill in pixels using evidence from other images that the network has been trained on.
londons_explore 5y ago
While as far as I know no pinch-to-zoom software does this today, I can totally imagine that in just a few years it might be common for pinch-zoom in browsers and photo gallery apps to do AI upscaling. Remember, for most users, a photo only has to be as accurate as their own memories. If extra detail is added to make the photo look better, it doesn't matter as long as it isn't detail the user remembers differently.
- ben-schaaf 5y ago
  Apple already uses AI to enhance photos with their Deep Fusion and Google makes even more use of AI in their Pixel 6. Added detail where it doesn't exist seems like a very reasonable concern.
cm2187 5y ago
And in a trial you only need to slightly modify a facial expression to affect the outcome. Imagine if some defendant appears to smile while a victim describes its ordeal.
- IshKebab 5y ago
  Yes I point that out on this very recent submission where it actually does modify the expression of a tiger: https://news.ycombinator.com/item?id=29202899#29213771 https://news.ycombinator.com/item?id=29202899#29213771
willis936 5y ago
The issue is the expert who was brought it in was not an expert. They failed to answer even the simplest of questions related to upscaling and did nothing to inform the court of the nature of the upscaling used. https://www.usnews.com/news/technology/articles/2021-11-11/blow-up-at-rittenhouse-trial-over-enlarging-photos-and-video https://www.usnews.com/news/technology/articles/2021-11-11/b...

by jasode 5y ago

croes 5y ago

Does the iPad zoom function any upscaling?

ImprobableTruth 5y ago
You can't zoom without upscaling in some form. If you zoom in on an image in your browser, the image gets upscaled. It's going to be using a simple form of upscaling (default is bilinear), but it's still upscaling.
- _djo_ 5y ago
  AI-driven upscaling which fills in pixels using data from other images is quite different to algorithmic interpolation that only uses the available information in the image being worked on.
  - ImprobableTruth 5y ago
    Both can introduce wrong information or cause artifacts. "It only uses the available information" is a red herring.
    - gifnamething 5y ago
      Then this is a problem inherent with digital photography. Any image you see is interpolated from Bayer data. There are multiple algorithms with different results, the colour needs to be adjusted, and the camera distortion needs to be reversed. All of this happens before you see anything on your screen.
    - _djo_ 5y ago
      A zooming or scaling algorithm that only uses available image data is deterministic, it should get the same result every time. One that relies on a neural network trained on other images is not, it might produce a subtly different image each time it's run and introduce artefacts from other images.
      - andrepd 5y ago
        What stops a neural network zoom to be just labelled as "zoom"? I think its a very pertinent question from the judge.
        _djo_ 5y ago
        Asking a question is fine, accepting the defence's claim without further substantiation and the rest of the follow up is less so.
        jtbayly 5y ago
        And how do you resolve conflicting technological claims from the defense and prosecution? By expert witness. Which is what the judge did. So what’s the problem?
- masklinn 5y ago
  > You can't zoom without upscaling in some form. Of course you can, by having the image originally be displayed downscaled for instance. The back camera of your average phone has a much higher resolution than its display, or most computer displays for that matter.

_djo_ 5y ago

Anyone would be supportive of judges being cautious over upscaling, which we know has the potential to introduce false information because it's doing inference and interpolation. Zooming is another issue entirely.

ImprobableTruth 5y ago
How do you think zooming works?
- Twisol 5y ago
  If the pre-zoomed image was already downscaled (e.g. to fit a small display), then up to a point, zooming should simply downscale less, i.e. remove fewer pixels. Beyond that point, yes, a zoom would have to perform some kind of interpolation. In the CSI "zoom and enhance", anything beyond the simplest forms of interpolation would be part of "enhance", which is to say, not part of "zoom".
  - ImprobableTruth 5y ago
    Even downscaling uses interpolation, but sure. The zoomed in part definitely wasn't shown at its native resolution though. And bilinear interpolation is as simple as you can get besides nearest neighbor, yet it still has the potential to introduce wrong information/cause artifacting. So even in the "CSI" sense, zooming can cause issues (unless you want to argue that only nearest neighbor is CSI 'zooming'? But then not even browsers do only CSI 'zooming').
- _djo_ 5y ago
  As Twisol said, zooming doesn't inherently require interpolation. But when it does require interpolation, it's usually an extremely simple and well-understood form like nearest neighbour interpolation that's not going to result in things like faces being inserted. It also only uses the available data in the image itself on which to work. It's fundamentally different from neural network-driven inference and interpolation that attempts to fill in pixels using evidence from other images that the network has been trained on.

londons_explore 5y ago

While as far as I know no pinch-to-zoom software does this today, I can totally imagine that in just a few years it might be common for pinch-zoom in browsers and photo gallery apps to do AI upscaling. Remember, for most users, a photo only has to be as accurate as their own memories. If extra detail is added to make the photo look better, it doesn't matter as long as it isn't detail the user remembers differently.

ben-schaaf 5y ago
Apple already uses AI to enhance photos with their Deep Fusion and Google makes even more use of AI in their Pixel 6. Added detail where it doesn't exist seems like a very reasonable concern.

cm2187 5y ago

And in a trial you only need to slightly modify a facial expression to affect the outcome. Imagine if some defendant appears to smile while a victim describes its ordeal.

IshKebab 5y ago
Yes I point that out on this very recent submission where it actually does modify the expression of a tiger: https://news.ycombinator.com/item?id=29202899#29213771 https://news.ycombinator.com/item?id=29202899#29213771

willis936 5y ago

The issue is the expert who was brought it in was not an expert. They failed to answer even the simplest of questions related to upscaling and did nothing to inform the court of the nature of the upscaling used. https://www.usnews.com/news/technology/articles/2021-11-11/blow-up-at-rittenhouse-trial-over-enlarging-photos-and-video https://www.usnews.com/news/technology/articles/2021-11-11/b...