6 ms·

Disclaimer: this is based on watching the talk and some basic machine learning knowledge. Im no expert. They have a sparse set of data that is part of an imag

Disclaimer: this is based on watching the talk and some basic machine learning knowledge. Im no expert. They have a sparse set of data that is part of an image. They have trained a model to look at the sparse set and make an educated guess about what the full image looks like. They do this by feeding it full images. The full images you feed into the model thus have an effect on the final image generated. In order to see how large that effect is, they trained different versions of the model with different sets of complete images. Some were images of what we thought a black hole looked like. There is potential that this heavily influences the model and ensures that the output looks like what we expect it to, even if that isnt actually true. They also trained the model with non-blackhole images. Since the output of the model was approximately the same, this indicates that the resulting output picture doesnt look like what we think a black hole looks like just because it was trained with black hole images. It likely really looks like that. The model doesn't need to be told what a black hole looks like. The sparse measurements combined with knowledge of how sparse data can be combined to form a generic image is enough. The model learned that the sparse data is not likely pure noise, instead there are shapes and lines and gradients that relate the sparse data points to each other. Her analogy of sketch artists is good. If you have a functionally complete description and give it to 3 sketch artists from different cultures who are used to different looking people, they will still draw the same person. However if your description isnt actually detailed enough, their sketches will significantly differ as they use their existing knowledge and bias to fill in the gaps with what they think is likely.

jamestenglish 7y ago
Not OP, but I too am confused. I understood the sketch artist analogy but that didn't seem related to this point: >They also trained the model with non-blackhole images. Since the output of the model was approximately the same, this indicates that the resulting output picture doesnt look like what we think a black hole looks like just because it was trained with black hole images. It likely really looks like that. If you are feeding non-blackhole images in and getting blackhole results out, wouldn't that be indicative of an over-trained model? Her other analogy was we can't rule out that there is an elephant at the center of the galaxy, but it sounds like if you feed a picture of an elephant in you'll get a picture of a blackhole out?
- pricecomstock 7y ago
  From what I understand, the training input images are just to establish the relationship between sparse data points and full image, regardless of subject matter. Since they were getting the black hole picture out of the trained model regardless of how it was trained, it's likely that the model was producing accurate results of what the "camera" was pointed at. If they had pointed it at an elephant, the model would have produced a picture of something elephant-like because it was somewhat accurately reconstructing a full image from sparse data points.
  - dennisgorelik 7y ago
    > getting the black hole picture out of the trained model regardless of how it was trained Did they try to feed random noise into their trained image builder? I suspect that the output of that trained image builder is always the same "black hole", even with random noise as an input.
    - QuotedForTruth 7y ago
      Probably not with random noise. With random noise there is literally no connection between pixels. With any actual picture there are connections. Like for instance a pixel is more likely to be the value of its neighbor or nearly so than any random value. This follows from the fact that the pictures are of actual objects with physical properties that determine the value of the pixel that maps to them. Most of the image can be characterized by continuous gradients with occasional edges. I think if you trained with random noise you would get random noise output.
- jdmichal 7y ago
  They're not just training the model to make pictures from nothing. They're training the model to make pictures from an input. So I assume they're simulating what an input would look like of, say, a planet or astroid or elephant or whatever, given that it was viewed through the relevant type of sensor system. Then when they feed in the black hole sensor data, they get pictures that look like the black holes we imagined. Even if we never told the model what a black hole looks like.
stilley2 7y ago
Hmm I skimmed the paper on the algorithm this morning and didn't get the impression they trained the model on other images. I thought they jointly estimated patches that make up the image and penalized deviations from these patches (i.e. estimated a sparse basis). I haven't watched the Ted talk yet though
- QuotedForTruth 7y ago
  They may have changed approaches since this two year old TED talk was made.
dennisgorelik 7y ago
> that effect is, they trained different versions of the model with different sets of complete images What does training mean? I thought that the training means to adjust Neural Network until it learns to convert our input into expected output of "complete image". But if thaining means to teach the model to produce expected "complete image", then how is it possible that "the output of the model was approximately the same" [for different training "complete image"s]?
- QuotedForTruth 7y ago
  Training is to feed in thousands of sets of {sparse sample of actual image, actual image}. The model is adjusted until the total difference between the output image and the actual image is minimized across all training images. The output images are approximately the same because the model is "looking" at training images at a lower level that we do. The talk says they chop the images up into small pieces. So the model never "sees" the full shapes that are in the full images. It only sees small local features. I guess it turns out that these smaller pieces are pretty generic in that they are common between images of black holes and everything else. The curve of an elephant trunk looks similar to the curve of an event horizon if you cut it out in a small enough piece. Perhaps if they didnt do this step, then the model would be more sensitive to the images its trained on.
  - dennisgorelik 7y ago
    Kathie said [1] "What you can do is to use methods where you [have] do not need any calibration whatsoever and you can still can get pretty good results. So here on the bottom at the top is the truth image, and this is simulated data, as we are increasing the amount of amplitude error and you can see here ... it's hard to see ... but it breaks down once you add too much gain here. But if we use just closure quantities - we are invariant to that. So that really, actually, been a really huge step for the project, because we had such bad gains. " [1] https://youtu.be/UGL_OL3OrCE?t=1177 https://youtu.be/UGL_OL3OrCE?t=1177

jamestenglish 7y ago

Not OP, but I too am confused. I understood the sketch artist analogy but that didn't seem related to this point: >They also trained the model with non-blackhole images. Since the output of the model was approximately the same, this indicates that the resulting output picture doesnt look like what we think a black hole looks like just because it was trained with black hole images. It likely really looks like that. If you are feeding non-blackhole images in and getting blackhole results out, wouldn't that be indicative of an over-trained model? Her other analogy was we can't rule out that there is an elephant at the center of the galaxy, but it sounds like if you feed a picture of an elephant in you'll get a picture of a blackhole out?

pricecomstock 7y ago
From what I understand, the training input images are just to establish the relationship between sparse data points and full image, regardless of subject matter. Since they were getting the black hole picture out of the trained model regardless of how it was trained, it's likely that the model was producing accurate results of what the "camera" was pointed at. If they had pointed it at an elephant, the model would have produced a picture of something elephant-like because it was somewhat accurately reconstructing a full image from sparse data points.
- dennisgorelik 7y ago
  > getting the black hole picture out of the trained model regardless of how it was trained Did they try to feed random noise into their trained image builder? I suspect that the output of that trained image builder is always the same "black hole", even with random noise as an input.
  - QuotedForTruth 7y ago
    Probably not with random noise. With random noise there is literally no connection between pixels. With any actual picture there are connections. Like for instance a pixel is more likely to be the value of its neighbor or nearly so than any random value. This follows from the fact that the pictures are of actual objects with physical properties that determine the value of the pixel that maps to them. Most of the image can be characterized by continuous gradients with occasional edges. I think if you trained with random noise you would get random noise output.
jdmichal 7y ago
They're not just training the model to make pictures from nothing. They're training the model to make pictures from an input. So I assume they're simulating what an input would look like of, say, a planet or astroid or elephant or whatever, given that it was viewed through the relevant type of sensor system. Then when they feed in the black hole sensor data, they get pictures that look like the black holes we imagined. Even if we never told the model what a black hole looks like.

stilley2 7y ago

Hmm I skimmed the paper on the algorithm this morning and didn't get the impression they trained the model on other images. I thought they jointly estimated patches that make up the image and penalized deviations from these patches (i.e. estimated a sparse basis). I haven't watched the Ted talk yet though

QuotedForTruth 7y ago
They may have changed approaches since this two year old TED talk was made.

dennisgorelik 7y ago

> that effect is, they trained different versions of the model with different sets of complete images What does training mean? I thought that the training means to adjust Neural Network until it learns to convert our input into expected output of "complete image". But if thaining means to teach the model to produce expected "complete image", then how is it possible that "the output of the model was approximately the same" [for different training "complete image"s]?

QuotedForTruth 7y ago
Training is to feed in thousands of sets of {sparse sample of actual image, actual image}. The model is adjusted until the total difference between the output image and the actual image is minimized across all training images. The output images are approximately the same because the model is "looking" at training images at a lower level that we do. The talk says they chop the images up into small pieces. So the model never "sees" the full shapes that are in the full images. It only sees small local features. I guess it turns out that these smaller pieces are pretty generic in that they are common between images of black holes and everything else. The curve of an elephant trunk looks similar to the curve of an event horizon if you cut it out in a small enough piece. Perhaps if they didnt do this step, then the model would be more sensitive to the images its trained on.
- dennisgorelik 7y ago
  Kathie said [1] "What you can do is to use methods where you [have] do not need any calibration whatsoever and you can still can get pretty good results. So here on the bottom at the top is the truth image, and this is simulated data, as we are increasing the amount of amplitude error and you can see here ... it's hard to see ... but it breaks down once you add too much gain here. But if we use just closure quantities - we are invariant to that. So that really, actually, been a really huge step for the project, because we had such bad gains. " [1] https://youtu.be/UGL_OL3OrCE?t=1177 https://youtu.be/UGL_OL3OrCE?t=1177