5 ms·

It's funny how every time I read an astronomy headline mention the word "visible", my brain immediately interprets it as a conventional photograph and is slight

It's funny how every time I read an astronomy headline mention the word "visible", my brain immediately interprets it as a conventional photograph and is slightly disappointed when there isn't one.

mulmen 6y ago
Hah, thank you for summarizing this so well. I never really connected the dots on “visible” = “can be observed”.
- redis_mlc 6y ago
  Virtually no distant astronomical image is a normal photo that corresponds to what the human eye would see. They're multi-layer stacks of CCD observations filtered by frequency range, with false colors then assigned, not three RGB layers. IIRC, typically there's 7 - 11 ranges, depending on the instrument. So one of the layers included in the depiction might be infrared, then false-colored into something the eye can handle. Source: I worked in remote sensing.
  - sillysaurusx 6y ago
    Which file formats did you use? One of my frustrations with ML is that almost no one seems to be training on HDR data. It's all 8-bit RGB. There is a big opportunity here to train GANs on raw sensor data. But that raises the question of which file formats to focus on. Since you have lots of image processing experience, I'm very interested in what the data formats were like at your job. Was it proprietary formats? If so, were there any advantages over e.g. the RAW file format? Is RAW sufficient for standardizing "everything necessary to turn photon readings into a photograph that appeals to the human eye"? E.g. if I gather thousands of RAW photos, do I need any other information not present in the RAW file? Mainly I am interested in any insights you gained on the job: you might know of some unique advantage that certain data formats have over others. I'm extremely interested in training ML models to synthesize what comes into a camera (photon readings), as opposed to what the world looks like to the human eye (RGB values). (I apologize for not phrasing my questions more carefully. It's late. The essence is that right now, most ML training operate in terms of RGB, yet there seems to be many advantages to operating in "photon space". The trouble is, it's effortless to gather millions of RGB photos -- but it seems very difficult to gather a million RAW photos. I'm hoping someone knows of a massive trove of "some kind of raw data that can be turned into RGB values" – maybe that's RAW, maybe it's something else. Whatever it is, once I have it, I can feed it through an ML model.)
    - rwmj 6y ago
      (Not astronomy) When I worked in particle physics it was all TIFF, and often forks of libtiff with non-standard extensions, with nothing under version control. In hindsight it was a miracle the images were readable from one month to the next.
    - imhoguy 6y ago
      What problem you try to solve with GANs? Your RAW HDR photos are really taken by a sensor with RGB mask filtering out whatever is not visible spectrum. Scientific or indistrial imaging mostly use monochromatic information which is later colorized for human eyes. Even if you get millions of RAW photos they will come from different sensors with different properties (noise, sensitivity, mosaic, orientation...) and you need to to normalize them all. sRGB is most common normalization target for which all prosumer cameras are optimized for. The format is not really important here as it is usually array of pixels. You need more bits and resolution? Fine, but you also get more useless noise.
      - formerly_proven 6y ago
        Color and dynamic range of most sensors extends greatly beyond sRGB, it's the camera's JPEG processing engine which maps this larger range to the smaller sRGB (or AdobeRGB) range.
        imhoguy 6y ago
        You can attach custom ICC profile to JPEG, also to TIFF or PNG. But still you need to normalize data from different kind of sensors before you feed any ML engine with it. sRGB or AdobeRGB are most common color space profiles you can get with varied content.
    - formerly_proven 6y ago
      RAW from most cameras isn't "real RAW". Firstly it generally uses lossy compression secondly a lot of cameras apply some minor corrections to "RAW" data as well (e.g. noise reduction). Some would argue that most cameras can't give you "real" RAW since most sensors perform noise reduction one way or the other internally (e.g. most Sony sensors use correlated double sampling). Most sensors use analog PGAs to amplify the pixel signals before A/D conversion (that's the ISO setting)[2], which changes the characteristics of the sensor quite drastically (not just dynamic range and noise, a lot of older sensors had severe issues with tinting and banding at high gains). RAW formats are generally linear (those used for moving pictures are sometimes log-encoded, which makes more sense [1], but isn't "raw"). A library like libraw reads most RAW files and can give you an appropiately (for the sensor used) debayered RGB image. These images aren't in a defined color space; they're specific to every camera/sensor. In the moving pictures space OpenEXR is a common format for "arbitrary" images, since it allows any number of channels each with different formats (e.g. you can have a file with 27 channels per pixel, some of which are float and some of which are integer). [1] E.g. 12 bit log encoding gives even resolution across more of the dynamic range; a linear encoding gives half of each range to the brighter stop. [2] Sensors that don't have PGAs simply have a fixed sensitivity. If the camera offers an ISO setting, it is emulated in the digital domain by pulling the image.
  - Someone 6y ago
    That, and the magnification. Sagittarius A* is 26,000 light years away (https://en.m.wikipedia.org/wiki/Sagittarius_A* https://en.m.wikipedia.org/wiki/Sagittarius_A*). That’s one and a half billion astronomical units. The star in question is twice the sun’s mass, and, I guess, not too dissimilar in size. If so, if you had a camera that could take a 1,000,000,000 by 1,000,000,000 pixel photo of our sun from the earth and pointed it at this star, that star would fill a single pixel (Likely a few more more due to imaging imperfections, but those would mean the light gets smeared out) It typically takes multiple ‘photos’ to even get a single of those CD observations in a frequency range (https://en.wikipedia.org/wiki/S2_(star)#S0–102 https://en.wikipedia.org/wiki/S2_(star)#S0–102: “At one-sixteenth the brightness of S0–2, S0–102 was not initially recognized because it required many more years of observations to distinguish it from its local infrared background”
  - Wowfunhappy 6y ago
    But if you were to use a teleporter to see these objects with your own eyes, they would match what is in the “photo”, right? It’s not as though conventional cameras operate on the exact same principles as our eyes, either. And that’s without even getting into the amount of “computational photography” performed on phones.
    - cgriswald 6y ago
      Many objects are too dim to see with the naked eye even if you were close enough to see them. Even amongst those which are clearly visible, they’d appear quite a bit dimmer than they do in a long exposure photograph. The Orion Nebula is a good target for grokking the difference. A long exposure photo with a DLSR (which will be blurry if you don’t have tracking) you’ll see lots of color. Through a pair of binoculars, you won’t although you will see the nebula.
    - chmod775 6y ago
      > But if you were to use a teleporter to see these objects with your own eyes, they would match what is in the “photo”, right? No. Very likely not. The photos may be of wavelengths of light that the human eye can't see, so scientists shift (and stretch/shrink) those into the visible spectrum. That means the photos could contain something you can't see with the naked eye. If you were to teleport there, you would (probably) see something, but the colors likely wouldn't be like in the photo. This video has a pretty good explanation of what's going on: https://www.youtube.com/watch?v=WSG0MnmUsEY https://www.youtube.com/watch?v=WSG0MnmUsEY
- hankchinaski 6y ago
  indeed, every time i see such articles i remember my high school teachers’ metaphor: in science when we “observe” a phenomena doesn’t necessarily mean that we see it literally, like when we drop a pen to the floor and we hear the sound, we know a pen has fallen to the floor although we haven’t actually seen it with our eyes
  - lisper 6y ago
    There is actually an inverse correlation between the directness of an observation and the quality of the data. The human brain and sensory system are remarkable, but also limited and easily fooled. It is optimized for detecting food and predators, and reading the facial expressions of other humans, not making scientific observations. This is why, when accuracy matters, we build machines to do the work of observing for us.
    - lioeters 6y ago
      Thank you for pointing out the distinction between "direct" observation through human senses, and machine-mediated observation for accurate scientific measurements. It clarifies the motivation for extension/augmentation of human perception through technology, from microscopes to telescopes - and spectacles/glasses for that matter. It also implies a continuation of the trend, where more and more of human perception will be mediated by technology, to benefit from advantages like accuracy, long-term memory/storage for data collection and statistics. I guess augmented/virtual reality is part of it, where we can now perceive and experience a fully technology-mediated world, both real and imaginary. I picture sci-fi possibilities, like being able to "zoom in" to see things at magnifications beyond human biological limits - all controlled by thoughts.
- kazinator 6y ago
  This is hacker news; have you never encountered the visibility of identifiers in a scope? Or the visibility of side effects of a body of code? Visibility of a host on a network? ...
deleted 6y ago
[deleted]

It's funny how every time I read an astronomy headline mention the word "visible", my brain immediately interprets it as a conventional photograph and is slightly disappointed when there isn't one.

mulmen 6y ago

Hah, thank you for summarizing this so well. I never really connected the dots on “visible” = “can be observed”.

redis_mlc 6y ago
Virtually no distant astronomical image is a normal photo that corresponds to what the human eye would see. They're multi-layer stacks of CCD observations filtered by frequency range, with false colors then assigned, not three RGB layers. IIRC, typically there's 7 - 11 ranges, depending on the instrument. So one of the layers included in the depiction might be infrared, then false-colored into something the eye can handle. Source: I worked in remote sensing.
- sillysaurusx 6y ago
  Which file formats did you use? One of my frustrations with ML is that almost no one seems to be training on HDR data. It's all 8-bit RGB. There is a big opportunity here to train GANs on raw sensor data. But that raises the question of which file formats to focus on. Since you have lots of image processing experience, I'm very interested in what the data formats were like at your job. Was it proprietary formats? If so, were there any advantages over e.g. the RAW file format? Is RAW sufficient for standardizing "everything necessary to turn photon readings into a photograph that appeals to the human eye"? E.g. if I gather thousands of RAW photos, do I need any other information not present in the RAW file? Mainly I am interested in any insights you gained on the job: you might know of some unique advantage that certain data formats have over others. I'm extremely interested in training ML models to synthesize what comes into a camera (photon readings), as opposed to what the world looks like to the human eye (RGB values). (I apologize for not phrasing my questions more carefully. It's late. The essence is that right now, most ML training operate in terms of RGB, yet there seems to be many advantages to operating in "photon space". The trouble is, it's effortless to gather millions of RGB photos -- but it seems very difficult to gather a million RAW photos. I'm hoping someone knows of a massive trove of "some kind of raw data that can be turned into RGB values" – maybe that's RAW, maybe it's something else. Whatever it is, once I have it, I can feed it through an ML model.)
  - rwmj 6y ago
    (Not astronomy) When I worked in particle physics it was all TIFF, and often forks of libtiff with non-standard extensions, with nothing under version control. In hindsight it was a miracle the images were readable from one month to the next.
  - imhoguy 6y ago
    What problem you try to solve with GANs? Your RAW HDR photos are really taken by a sensor with RGB mask filtering out whatever is not visible spectrum. Scientific or indistrial imaging mostly use monochromatic information which is later colorized for human eyes. Even if you get millions of RAW photos they will come from different sensors with different properties (noise, sensitivity, mosaic, orientation...) and you need to to normalize them all. sRGB is most common normalization target for which all prosumer cameras are optimized for. The format is not really important here as it is usually array of pixels. You need more bits and resolution? Fine, but you also get more useless noise.
    - formerly_proven 6y ago
      Color and dynamic range of most sensors extends greatly beyond sRGB, it's the camera's JPEG processing engine which maps this larger range to the smaller sRGB (or AdobeRGB) range.
      - imhoguy 6y ago
        You can attach custom ICC profile to JPEG, also to TIFF or PNG. But still you need to normalize data from different kind of sensors before you feed any ML engine with it. sRGB or AdobeRGB are most common color space profiles you can get with varied content.
  - formerly_proven 6y ago
    RAW from most cameras isn't "real RAW". Firstly it generally uses lossy compression secondly a lot of cameras apply some minor corrections to "RAW" data as well (e.g. noise reduction). Some would argue that most cameras can't give you "real" RAW since most sensors perform noise reduction one way or the other internally (e.g. most Sony sensors use correlated double sampling). Most sensors use analog PGAs to amplify the pixel signals before A/D conversion (that's the ISO setting)[2], which changes the characteristics of the sensor quite drastically (not just dynamic range and noise, a lot of older sensors had severe issues with tinting and banding at high gains). RAW formats are generally linear (those used for moving pictures are sometimes log-encoded, which makes more sense [1], but isn't "raw"). A library like libraw reads most RAW files and can give you an appropiately (for the sensor used) debayered RGB image. These images aren't in a defined color space; they're specific to every camera/sensor. In the moving pictures space OpenEXR is a common format for "arbitrary" images, since it allows any number of channels each with different formats (e.g. you can have a file with 27 channels per pixel, some of which are float and some of which are integer). [1] E.g. 12 bit log encoding gives even resolution across more of the dynamic range; a linear encoding gives half of each range to the brighter stop. [2] Sensors that don't have PGAs simply have a fixed sensitivity. If the camera offers an ISO setting, it is emulated in the digital domain by pulling the image.
- Someone 6y ago
  That, and the magnification. Sagittarius A* is 26,000 light years away (https://en.m.wikipedia.org/wiki/Sagittarius_A* https://en.m.wikipedia.org/wiki/Sagittarius_A*). That’s one and a half billion astronomical units. The star in question is twice the sun’s mass, and, I guess, not too dissimilar in size. If so, if you had a camera that could take a 1,000,000,000 by 1,000,000,000 pixel photo of our sun from the earth and pointed it at this star, that star would fill a single pixel (Likely a few more more due to imaging imperfections, but those would mean the light gets smeared out) It typically takes multiple ‘photos’ to even get a single of those CD observations in a frequency range (https://en.wikipedia.org/wiki/S2_(star)#S0–102 https://en.wikipedia.org/wiki/S2_(star)#S0–102: “At one-sixteenth the brightness of S0–2, S0–102 was not initially recognized because it required many more years of observations to distinguish it from its local infrared background”
- Wowfunhappy 6y ago
  But if you were to use a teleporter to see these objects with your own eyes, they would match what is in the “photo”, right? It’s not as though conventional cameras operate on the exact same principles as our eyes, either. And that’s without even getting into the amount of “computational photography” performed on phones.
  - cgriswald 6y ago
    Many objects are too dim to see with the naked eye even if you were close enough to see them. Even amongst those which are clearly visible, they’d appear quite a bit dimmer than they do in a long exposure photograph. The Orion Nebula is a good target for grokking the difference. A long exposure photo with a DLSR (which will be blurry if you don’t have tracking) you’ll see lots of color. Through a pair of binoculars, you won’t although you will see the nebula.
  - chmod775 6y ago
    > But if you were to use a teleporter to see these objects with your own eyes, they would match what is in the “photo”, right? No. Very likely not. The photos may be of wavelengths of light that the human eye can't see, so scientists shift (and stretch/shrink) those into the visible spectrum. That means the photos could contain something you can't see with the naked eye. If you were to teleport there, you would (probably) see something, but the colors likely wouldn't be like in the photo. This video has a pretty good explanation of what's going on: https://www.youtube.com/watch?v=WSG0MnmUsEY https://www.youtube.com/watch?v=WSG0MnmUsEY
hankchinaski 6y ago
indeed, every time i see such articles i remember my high school teachers’ metaphor: in science when we “observe” a phenomena doesn’t necessarily mean that we see it literally, like when we drop a pen to the floor and we hear the sound, we know a pen has fallen to the floor although we haven’t actually seen it with our eyes
- lisper 6y ago
  There is actually an inverse correlation between the directness of an observation and the quality of the data. The human brain and sensory system are remarkable, but also limited and easily fooled. It is optimized for detecting food and predators, and reading the facial expressions of other humans, not making scientific observations. This is why, when accuracy matters, we build machines to do the work of observing for us.
  - lioeters 6y ago
    Thank you for pointing out the distinction between "direct" observation through human senses, and machine-mediated observation for accurate scientific measurements. It clarifies the motivation for extension/augmentation of human perception through technology, from microscopes to telescopes - and spectacles/glasses for that matter. It also implies a continuation of the trend, where more and more of human perception will be mediated by technology, to benefit from advantages like accuracy, long-term memory/storage for data collection and statistics. I guess augmented/virtual reality is part of it, where we can now perceive and experience a fully technology-mediated world, both real and imaginary. I picture sci-fi possibilities, like being able to "zoom in" to see things at magnifications beyond human biological limits - all controlled by thoughts.
kazinator 6y ago
This is hacker news; have you never encountered the visibility of identifiers in a scope? Or the visibility of side effects of a body of code? Visibility of a host on a network? ...

deleted 6y ago

[deleted]