7 ms·

Seems like a very complicated way of saying replace blurry moon image with high resolution moon image stored in the cloud.

mattnewton 4y ago
I think that’s not really how super resolution works, it’s closer to how diffusion models hallucinate details inspired by their training set. I roughly think about it being like you have a lot of high res moons lossy compressed into model weights until they are basically distilled into some abstract sense of “moon image”-ness “indexed” by blurry image patches. Running the network then “unzips” some of that detail over matching low resolution patches. (Using quotes to denote terms I am heavily abusing in this loose analogy) Edit: Importantly though I think this isn’t that different from what you are describing in terms of whether this is potentially misleading customers about the optics on their device, because it is inserting detail not actually captured by the camera sensor.
- whatshisface 4y ago
  The "it finds an image in its database" understanding of diffusion models doesn't work for SD because it can interpolate between prompts, but since there is only one moon, and it doesn't rotate, and their system doesn't work when the moon is partially obscured, there is really no need to describe it as anything more complex than that.
- Sporktacular 4y ago
  Super Resolution is only part of the process. It says they then apply "Scene Optimizer’s deep-learning-based AI detail enhancement engine". If the model and its weights contain detail not in the photo being taken, then it's tantamount to having high res images of the moon stored on camera and composited into the image. And if it doesn't then it's not the moon being displayed. Not that it's necessarily bad, but it could be if it fools someone into thinking they're buying superior electro/optics. Enough such that it warranted the line "Samsung continues to improve Scene Optimizer to reduce any potential confusion that may occur between the act of taking a picture of the real moon and an image of the moon".
  - mattnewton 4y ago
    > If the model and its weights contain detail not in the photo being taken, then it's tantamount to having high res images of the moon stored on camera and composited into the image. This is what is happening. I agree it’s tricking people into thinking it’s all optics and that’s kinda bad.
  - blincoln 4y ago
    I think it's actually worse than compositing in a high-resolution moon image. With AI enhancement, the details will look believable but may be completely inaccurate.
gincent 4y ago
It's not in the cloud, this is all done on-device. They have a moon detection AI model that they pass the image through, the output of which is then used to set up and then run the moon enhancement model, which fills in the detail from moon images it's been trained upon. These are part of a larger package of super-resolution models and processing pipeline that Samsung licenses from ArcSoft, a computational photography software company that specialises in mobile devices.
- jaclaz 4y ago
  >... Samsung licenses from ArcSoft ... Interesting, though there seems to be no reference to this kind of function in Arcsoft site: https://www.arcsoft.com/product/single-camera-solutions-on-smartphone.html https://www.arcsoft.com/product/single-camera-solutions-on-s... https://www.arcsoft.com/product/dual-camera-solutions-on-smart-phone.html https://www.arcsoft.com/product/dual-camera-solutions-on-sma... What they mention seems more "logical" enhancements, though the selfie feature of gender and age detection : https://www.arcsoft.com/product/single-camera-solutions-on-smartphone/selfie.html https://www.arcsoft.com/product/single-camera-solutions-on-s... is somehow creepy. Anyway, if they can recognize gender and age and what do you have in your refrigerator ("This offers users a smoother and smarter refrigerator experience.") : https://www.arcsoft.com/product/smart-refrigerator-solutions.html https://www.arcsoft.com/product/smart-refrigerator-solutions... surely they can recognize the moon. Now that they made me think about it, my refrigerator experience, while usually smooth enough, is pretty dumb and repetitive.
  - gincent 4y ago
    On Samsung devices that support this, it's implemented in the file libsuperresolution_raw.arcsoft.so in /system/lib64, if you're curious to have a look at how it works. Some strings from that file, relating to the moon detection and enhancement process: AHDR_SetMoonMask ai_moon ArcSoft_Moon_Detection_2.1.125231001.2 ArcSoft_MoonEnhancement_1.1.12021.257 ARC_SRR_DisableDaytimeMoonScene ARC_SRR_GetMoonSceneType ARC_SRR_SetMoonWeight bMoonMode bMoonProtect MFSR Moon Detect moon count is %d MoonDetect MoonDetect_Init %s_%02d_AIE_Output_ISO_%d_DRC_%0.2f_HDRMoon_%d_%dx%d %s_%02d_CropMoonMask_Idx_%d_%dx%d %s_%02d_DaynightAIMoon_Output_Idx_%d_%dx%d %s_%02d_DaytimeAIMoon_Output_Idx_%d_%dx%d %s_%02d_MoonDetection_Input_%dx%d %s_%02d_MoonHDR_Output_ISO_%d_DRC_%0.2f_HDR_%d_Moon_%d_%dx%d %s_%02d_MoonMask_Idx_%d_%dx%d %s_%02d_ResizedMoonMask_Idx_%d_%dx%d /sdcard/download/arcsr/%s_moon_mask_%dx%d.gray %s/FusionRun_pMoonmask_%dx%d.gray void mfsr_moon_detect(gk_mfsr *)
ilamont 4y ago
No no, it's Samsung's amazing deep-learning-based AI detail enhancement engine and OIS and VDIS technologies. /s What I'm wondering is what else is being swapped out.
- zuminator 4y ago
  What do you mean, what else? I'm genuinely curious what kind of nefariousness you imagine is going on? Like,they're swapping images of Fords with Hyundais? Swapping images of catenaries with nigh indistinguishable parabolas?
  - DougN7 4y ago
    What if they swap some details of an eye or a nose and make one person look just enough like another that an innocent person is convicted?
    - dmix 4y ago
      You think there aren't processes in place for this sort of thing? A due process even. IT experts are brought into cases all the time for stuff like this. This isn't even the standard camera "lying", it's explicitly a special tab for taking super-zoomed in AI optimized photos.
      - mrguyorama 4y ago
        And most of the "Experts" brought into courtrooms will argue whatever you pay them to argue. It's an industry. Lawyers aren't picking random AI engineers out of a mom and pop tech company, they have dedicated people who as a job sell their professional title to lawyers to launder a narrative through "an expert"
    - rootusrootus 4y ago
      Given that people still get convicted based on eyewitness testimony that has been repeatedly shown to be extremely inaccurate, I hold out little hope that the justice system will figure out how unreliable modern photography is anytime soon.
seabass-labrax 4y ago
If you want a picture of the Moon in cloud, a nice cold frontal weather system should suffice!

mattnewton 4y ago

I think that’s not really how super resolution works, it’s closer to how diffusion models hallucinate details inspired by their training set. I roughly think about it being like you have a lot of high res moons lossy compressed into model weights until they are basically distilled into some abstract sense of “moon image”-ness “indexed” by blurry image patches. Running the network then “unzips” some of that detail over matching low resolution patches. (Using quotes to denote terms I am heavily abusing in this loose analogy) Edit: Importantly though I think this isn’t that different from what you are describing in terms of whether this is potentially misleading customers about the optics on their device, because it is inserting detail not actually captured by the camera sensor.

whatshisface 4y ago
The "it finds an image in its database" understanding of diffusion models doesn't work for SD because it can interpolate between prompts, but since there is only one moon, and it doesn't rotate, and their system doesn't work when the moon is partially obscured, there is really no need to describe it as anything more complex than that.
Sporktacular 4y ago
Super Resolution is only part of the process. It says they then apply "Scene Optimizer’s deep-learning-based AI detail enhancement engine". If the model and its weights contain detail not in the photo being taken, then it's tantamount to having high res images of the moon stored on camera and composited into the image. And if it doesn't then it's not the moon being displayed. Not that it's necessarily bad, but it could be if it fools someone into thinking they're buying superior electro/optics. Enough such that it warranted the line "Samsung continues to improve Scene Optimizer to reduce any potential confusion that may occur between the act of taking a picture of the real moon and an image of the moon".
- mattnewton 4y ago
  > If the model and its weights contain detail not in the photo being taken, then it's tantamount to having high res images of the moon stored on camera and composited into the image. This is what is happening. I agree it’s tricking people into thinking it’s all optics and that’s kinda bad.
- blincoln 4y ago
  I think it's actually worse than compositing in a high-resolution moon image. With AI enhancement, the details will look believable but may be completely inaccurate.

gincent 4y ago

It's not in the cloud, this is all done on-device. They have a moon detection AI model that they pass the image through, the output of which is then used to set up and then run the moon enhancement model, which fills in the detail from moon images it's been trained upon. These are part of a larger package of super-resolution models and processing pipeline that Samsung licenses from ArcSoft, a computational photography software company that specialises in mobile devices.

jaclaz 4y ago
>... Samsung licenses from ArcSoft ... Interesting, though there seems to be no reference to this kind of function in Arcsoft site: https://www.arcsoft.com/product/single-camera-solutions-on-smartphone.html https://www.arcsoft.com/product/single-camera-solutions-on-s... https://www.arcsoft.com/product/dual-camera-solutions-on-smart-phone.html https://www.arcsoft.com/product/dual-camera-solutions-on-sma... What they mention seems more "logical" enhancements, though the selfie feature of gender and age detection : https://www.arcsoft.com/product/single-camera-solutions-on-smartphone/selfie.html https://www.arcsoft.com/product/single-camera-solutions-on-s... is somehow creepy. Anyway, if they can recognize gender and age and what do you have in your refrigerator ("This offers users a smoother and smarter refrigerator experience.") : https://www.arcsoft.com/product/smart-refrigerator-solutions.html https://www.arcsoft.com/product/smart-refrigerator-solutions... surely they can recognize the moon. Now that they made me think about it, my refrigerator experience, while usually smooth enough, is pretty dumb and repetitive.
- gincent 4y ago
  On Samsung devices that support this, it's implemented in the file libsuperresolution_raw.arcsoft.so in /system/lib64, if you're curious to have a look at how it works. Some strings from that file, relating to the moon detection and enhancement process: AHDR_SetMoonMask ai_moon ArcSoft_Moon_Detection_2.1.125231001.2 ArcSoft_MoonEnhancement_1.1.12021.257 ARC_SRR_DisableDaytimeMoonScene ARC_SRR_GetMoonSceneType ARC_SRR_SetMoonWeight bMoonMode bMoonProtect MFSR Moon Detect moon count is %d MoonDetect MoonDetect_Init %s_%02d_AIE_Output_ISO_%d_DRC_%0.2f_HDRMoon_%d_%dx%d %s_%02d_CropMoonMask_Idx_%d_%dx%d %s_%02d_DaynightAIMoon_Output_Idx_%d_%dx%d %s_%02d_DaytimeAIMoon_Output_Idx_%d_%dx%d %s_%02d_MoonDetection_Input_%dx%d %s_%02d_MoonHDR_Output_ISO_%d_DRC_%0.2f_HDR_%d_Moon_%d_%dx%d %s_%02d_MoonMask_Idx_%d_%dx%d %s_%02d_ResizedMoonMask_Idx_%d_%dx%d /sdcard/download/arcsr/%s_moon_mask_%dx%d.gray %s/FusionRun_pMoonmask_%dx%d.gray void mfsr_moon_detect(gk_mfsr *)

ilamont 4y ago

No no, it's Samsung's amazing deep-learning-based AI detail enhancement engine and OIS and VDIS technologies. /s What I'm wondering is what else is being swapped out.

zuminator 4y ago
What do you mean, what else? I'm genuinely curious what kind of nefariousness you imagine is going on? Like,they're swapping images of Fords with Hyundais? Swapping images of catenaries with nigh indistinguishable parabolas?
- DougN7 4y ago
  What if they swap some details of an eye or a nose and make one person look just enough like another that an innocent person is convicted?
  - dmix 4y ago
    You think there aren't processes in place for this sort of thing? A due process even. IT experts are brought into cases all the time for stuff like this. This isn't even the standard camera "lying", it's explicitly a special tab for taking super-zoomed in AI optimized photos.
    - mrguyorama 4y ago
      And most of the "Experts" brought into courtrooms will argue whatever you pay them to argue. It's an industry. Lawyers aren't picking random AI engineers out of a mom and pop tech company, they have dedicated people who as a job sell their professional title to lawyers to launder a narrative through "an expert"
  - rootusrootus 4y ago
    Given that people still get convicted based on eyewitness testimony that has been repeatedly shown to be extremely inaccurate, I hold out little hope that the justice system will figure out how unreliable modern photography is anytime soon.

seabass-labrax 4y ago

If you want a picture of the Moon in cloud, a nice cold frontal weather system should suffice!