5 ms·

I guess this is "astrophotography" for the sake of making pretty images, because I couldn't imagine the "computational AI" features he mentions towards the end

by ris 3y ago

I guess this is "astrophotography" for the sake of making pretty images, because I couldn't imagine the "computational AI" features he mentions towards the end of the article doing anything but horrify even an amateur astronomer.

Scea91 3y ago
What do you mean? Statistics and ML is used in astrophotography regularly. For example, the famous black hole “photo” [1]. https://www.nasa.gov/mission_pages/chandra/news/black-hole-image-makes-history https://www.nasa.gov/mission_pages/chandra/news/black-hole-i...
- etrautmann 3y ago
  Yes, but the computational pipeline is designed and implemented by the end user to achieve a scientific result. Here, it’s a black box where the resulting image cannot be treated as raw data.
  - chongli 3y ago
    Yes exactly. The training set used for the famous black hole image was a database of images created using black hole simulations based on our best available models. This is pretty far away from these consumer ML models trained on publicly available images which may as well be consider garbage from a scientific perspective.
    - astrange 3y ago
      Nothing about phone cameras involves "ML models trained on publicly available images".
      - chongli 3y ago
        You haven't been following the story, then, because Samsung phone cameras have been accused of doing exactly that: using a training set of publicly available images to "enhance" people's images by filling in details with a ML model.
        astrange 3y ago
        Those people don't understand how anything works though so I don't care what they said. And how would they know if it used "publicly available images"? It's the moon, of course it's publicly available. It's in the sky.
        chongli 3y ago
        You're focusing on the "publicly available" part but it's irrelevant to my point. Samsung's camera software is literally doctoring people's photos with fake data from its ML models. From the perspective of photographic integrity, it does not matter whether they scraped their training set off public photography sites or if they hired a team of photographers to build a dataset from scratch. To the user the effect is the same: it is not really an authentic photograph that is being created, it's a hybrid derived from (to the user) unknown sources.
        astrange 3y ago
        All photos work like this - cameras don't know what color things are, the color you see is made up by the camera. You've never taken an authentic photo on a digital camera. But… they don't use ML models made from publicly available images.
        chongli 3y ago
        There's a difference between ICC colour profiling (which is embedded in the RAW file and can be changed by the photographer without any degradation in image quality) and what Samsung is doing. The former can only affect the image colours globally (for example, allowing adjustment of white balance) whereas Samsung's changes are local, affecting image data in a profound way that can add hallucinated details which were not present in the original scene. One rather disturbing example is that the tool added teeth to baby photos!
        astrange 3y ago
        White balance adjustments to camera raws don't generally use ICC profiles, they do whatever the combination of camera format and raw processor want. Which can involve local adjustments, since you may want to process people differently from skies. So object recognition/segmentation models can definitely be involved, or what's essentially an upscaling model for better demosaicing. (That wouldn't be trained on public images though, unless they were the right format of camera raw.)
- empyrrhicist 3y ago
  That is completely unrelated to astrophotography as a hobby, taking pictures of space objects from the surface of the earth. Like, totally and completely not-comparable in any way.
birdman3131 3y ago
As somebody who has a Pixel 7 I completely agree. The AI makes my photos unusable. The worst part is I can see a usable image before it finishes processing. But then it turns it into pixilated garbage. I am often taking pictures of small text that is borderline unreadable to get part numbers off stuff. The AI processing is absolutely a bad idea.
- dzikimarian 3y ago
  You can disable lot of AI in settings, you can also enable saving RAWs.
- londons_explore 3y ago
  You can install an alternate camera app if you just want raw rixels with no computetional stuff. Having said that, I find the computational stuff turns a fairly poor sensor and lens into kinda acceptable photos, as long as I don't want to zoom in (I can see far further with my eyes than the camera can when it comes to taking photos of tiny far away things)
  - eek2121 3y ago
    on the iPhone you can enable raw image output. No need for an external app. Just open settings.
    - kalleboo 3y ago
      ProRAW in the system app is still heavily processed, it just saves more of the intermediate data to let you tweak more of the processing stages. You still need a third party camera app to capture true RAW photos. E.g. with ProRAW you can't remove most of the noise reduction, and you can't rescue images with large white balance issues (you get fringing in one of the processing steps since the white balance is baked into several of the intermediate images)
      - londons_explore 3y ago
        A 48 Mpixel 12 bit sensor with 128 frame stacking is 9 gigabytes of collected data per image. The flash memory isn't fast enough to store all of that in the ~2 seconds it is collected, nor is the RAM large enough to store it all. The iPhone is literally incapable of taking true RAW photos (at least not if you want to collect all the same info that the built in camera does)
vrtnis 3y ago
On that front, there is also one called the vaonis hyperia that seems totally automatic, but perhaps we miss out on the challenge of amateur astronomy..
ceejayoz 3y ago
I've found it very useful for showing curious people (especially kids) that there's a whole universe up there mostly unseen; you can whip it out and a minute or two later they can see the Milky Way in at least a little of its splendor. For those who show interest, I follow up with our 6" backyard telescope to see the Moon's mountains, Jupiter's moons, Saturn's rings, and Mars's ice caps. There's something different about experiencing it in person, even if the photos from a professional or a space telescope are empirically better.
orbital-decay 3y ago
If you read further, at first they mostly talk about stacking, even though it has nothing to do with "AI", but then they suddenly mix it with synthetic ML techniques, which defeats the point. Confusing. Nevertheless, you can do some proper astrophoto with smartphones without any AI, just like you would with any other camera. It won't be that good for obvious reasons, but it's possible and it will be better than using automated astrophoto modes in any app. Calibrate your lens and sensor, make a RAW stack with fixed exposure which is not too long, and some dark frames; use something like Sequator (or more sophisticated software) to process the stack.

by ris 3y ago

Scea91 3y ago

What do you mean? Statistics and ML is used in astrophotography regularly. For example, the famous black hole “photo” [1]. https://www.nasa.gov/mission_pages/chandra/news/black-hole-image-makes-history https://www.nasa.gov/mission_pages/chandra/news/black-hole-i...

etrautmann 3y ago
Yes, but the computational pipeline is designed and implemented by the end user to achieve a scientific result. Here, it’s a black box where the resulting image cannot be treated as raw data.
- chongli 3y ago
  Yes exactly. The training set used for the famous black hole image was a database of images created using black hole simulations based on our best available models. This is pretty far away from these consumer ML models trained on publicly available images which may as well be consider garbage from a scientific perspective.
  - astrange 3y ago
    Nothing about phone cameras involves "ML models trained on publicly available images".
    - chongli 3y ago
      You haven't been following the story, then, because Samsung phone cameras have been accused of doing exactly that: using a training set of publicly available images to "enhance" people's images by filling in details with a ML model.
      - astrange 3y ago
        Those people don't understand how anything works though so I don't care what they said. And how would they know if it used "publicly available images"? It's the moon, of course it's publicly available. It's in the sky.
        chongli 3y ago
        You're focusing on the "publicly available" part but it's irrelevant to my point. Samsung's camera software is literally doctoring people's photos with fake data from its ML models. From the perspective of photographic integrity, it does not matter whether they scraped their training set off public photography sites or if they hired a team of photographers to build a dataset from scratch. To the user the effect is the same: it is not really an authentic photograph that is being created, it's a hybrid derived from (to the user) unknown sources.
        astrange 3y ago
        All photos work like this - cameras don't know what color things are, the color you see is made up by the camera. You've never taken an authentic photo on a digital camera. But… they don't use ML models made from publicly available images.
        chongli 3y ago
        There's a difference between ICC colour profiling (which is embedded in the RAW file and can be changed by the photographer without any degradation in image quality) and what Samsung is doing. The former can only affect the image colours globally (for example, allowing adjustment of white balance) whereas Samsung's changes are local, affecting image data in a profound way that can add hallucinated details which were not present in the original scene. One rather disturbing example is that the tool added teeth to baby photos!
        astrange 3y ago
        White balance adjustments to camera raws don't generally use ICC profiles, they do whatever the combination of camera format and raw processor want. Which can involve local adjustments, since you may want to process people differently from skies. So object recognition/segmentation models can definitely be involved, or what's essentially an upscaling model for better demosaicing. (That wouldn't be trained on public images though, unless they were the right format of camera raw.)
empyrrhicist 3y ago
That is completely unrelated to astrophotography as a hobby, taking pictures of space objects from the surface of the earth. Like, totally and completely not-comparable in any way.

birdman3131 3y ago

As somebody who has a Pixel 7 I completely agree. The AI makes my photos unusable. The worst part is I can see a usable image before it finishes processing. But then it turns it into pixilated garbage. I am often taking pictures of small text that is borderline unreadable to get part numbers off stuff. The AI processing is absolutely a bad idea.

dzikimarian 3y ago
You can disable lot of AI in settings, you can also enable saving RAWs.
londons_explore 3y ago
You can install an alternate camera app if you just want raw rixels with no computetional stuff. Having said that, I find the computational stuff turns a fairly poor sensor and lens into kinda acceptable photos, as long as I don't want to zoom in (I can see far further with my eyes than the camera can when it comes to taking photos of tiny far away things)
- eek2121 3y ago
  on the iPhone you can enable raw image output. No need for an external app. Just open settings.
  - kalleboo 3y ago
    ProRAW in the system app is still heavily processed, it just saves more of the intermediate data to let you tweak more of the processing stages. You still need a third party camera app to capture true RAW photos. E.g. with ProRAW you can't remove most of the noise reduction, and you can't rescue images with large white balance issues (you get fringing in one of the processing steps since the white balance is baked into several of the intermediate images)
    - londons_explore 3y ago
      A 48 Mpixel 12 bit sensor with 128 frame stacking is 9 gigabytes of collected data per image. The flash memory isn't fast enough to store all of that in the ~2 seconds it is collected, nor is the RAM large enough to store it all. The iPhone is literally incapable of taking true RAW photos (at least not if you want to collect all the same info that the built in camera does)

vrtnis 3y ago

On that front, there is also one called the vaonis hyperia that seems totally automatic, but perhaps we miss out on the challenge of amateur astronomy..

ceejayoz 3y ago

I've found it very useful for showing curious people (especially kids) that there's a whole universe up there mostly unseen; you can whip it out and a minute or two later they can see the Milky Way in at least a little of its splendor. For those who show interest, I follow up with our 6" backyard telescope to see the Moon's mountains, Jupiter's moons, Saturn's rings, and Mars's ice caps. There's something different about experiencing it in person, even if the photos from a professional or a space telescope are empirically better.

orbital-decay 3y ago

If you read further, at first they mostly talk about stacking, even though it has nothing to do with "AI", but then they suddenly mix it with synthetic ML techniques, which defeats the point. Confusing. Nevertheless, you can do some proper astrophoto with smartphones without any AI, just like you would with any other camera. It won't be that good for obvious reasons, but it's possible and it will be better than using automated astrophoto modes in any app. Calibrate your lens and sensor, make a RAW stack with fixed exposure which is not too long, and some dark frames; use something like Sequator (or more sophisticated software) to process the stack.