6 ms·

Too bad the reconstructed faces don't look anything like the presented faces and I'm sure these two example are some of the best results. I suspect the algorit

by spikels 13y ago

Too bad the reconstructed faces don't look anything like the presented faces and I'm sure these two example are some of the best results. I suspect the algorithm always outputs some face generated by a paramaterized face model (neutral net based?). Therefore even random output would generate a face. Then with some "tuning" and a little wishful thinking you might convince yourself this works. Am I being too skeptical?

friendcomputer 13y ago
From TFA: "Working with funding from the Yale Provost’s office, Cowen and post doctoral researcher Brice Kuhl, now an assistant professor at New York University, showed six subjects 300 different “training” faces while undergoing fMRI scans. They used the data to create a sort of statistical library of how those brains responded to individual faces. They then showed the six subjects new sets of faces while they were undergoing scans. Taking that fMRI data alone, researchers used their statistical library to reconstruct the faces their subjects were viewing." So yes, it will always output something like a face. It's more like they are using the FMRI to select among preset options. It's still potentially a great result, but we need more detail than this article provides.
JackFr 13y ago
I would say not skeptical enough. Generating a result which looks like some combination of the inputs, has a little bit of 'Wow' factor if you just look at the pictures and don't think too hard about it. But ultimately its not an exercise in which they can be wrong, and since the image returned is always going to be a face, they'll always kind of be right. An impressive result would be if they trained their system on the the 300 pictures, and then we're reliably able identify which picture the subject was looking at. That would be quantifiable and testable, and I presume the result would expose that this is all nonsense.
- 001sky 13y ago
  "Since the image returned is always going to be a face, they'll always kind of be right" So, non-falsifiable...?
Houshalter 13y ago
Yes they do. http://imgur.com/xzNwUTL http://imgur.com/xzNwUTL taken from icegreentea's comment, which was taken from the paper.
- spikels 13y ago
  Which of the outputs do you think look like the original face? Note: the second column is not output. Note the cherry picking of the output for the bottom picture in the Yale press release (forth row in this image).
  - Houshalter 13y ago
    They share many features with the original face. No they are not great, but it's pretty remarkable and far better than random.
    - JackFr 13y ago
      By many features do you mean two eyes, nose, mouth?
      - Houshalter 13y ago
        Just at a glance it got skin color, gender, hair color, and the smile.
timbre 13y ago
You are right that they assume the subject is looking at a face, and so will always produce "face-like" output. The paper includes a quantitative evaluation, in which they take a set of 30 distractor face images not used elsewhere in the study, and for each of them determine whether the reconstructed face is more similar to it, or to the original face the subject was looking at. On average, the reconstructed face is closer to the correct face than the distractor 62.5% of the time. So it's better than random, and I think it's pretty cool work, but the quality of the reconstructions is pretty terrible, considering that a randomly chosen distractor will usually be very different from the test face (~half the time opposite sex, frequently different race, different age, etc). For comparison, it would be interesting to evaluate some simple, obviously terrible reconstructions by the same metric. For example, we could "reconstruct" the face as a image that is a single, solid color, the average RGB value of the pixels in the original face. Another "reconstruction" that it seems would very likely do better under this evaluation metric is something like the "race-gender-age" of perp descriptions in the news ("white male in his thirties").

by spikels 13y ago

friendcomputer 13y ago

From TFA: "Working with funding from the Yale Provost’s office, Cowen and post doctoral researcher Brice Kuhl, now an assistant professor at New York University, showed six subjects 300 different “training” faces while undergoing fMRI scans. They used the data to create a sort of statistical library of how those brains responded to individual faces. They then showed the six subjects new sets of faces while they were undergoing scans. Taking that fMRI data alone, researchers used their statistical library to reconstruct the faces their subjects were viewing." So yes, it will always output something like a face. It's more like they are using the FMRI to select among preset options. It's still potentially a great result, but we need more detail than this article provides.

JackFr 13y ago

I would say not skeptical enough. Generating a result which looks like some combination of the inputs, has a little bit of 'Wow' factor if you just look at the pictures and don't think too hard about it. But ultimately its not an exercise in which they can be wrong, and since the image returned is always going to be a face, they'll always kind of be right. An impressive result would be if they trained their system on the the 300 pictures, and then we're reliably able identify which picture the subject was looking at. That would be quantifiable and testable, and I presume the result would expose that this is all nonsense.

001sky 13y ago
"Since the image returned is always going to be a face, they'll always kind of be right" So, non-falsifiable...?

Houshalter 13y ago

Yes they do. http://imgur.com/xzNwUTL http://imgur.com/xzNwUTL taken from icegreentea's comment, which was taken from the paper.

spikels 13y ago
Which of the outputs do you think look like the original face? Note: the second column is not output. Note the cherry picking of the output for the bottom picture in the Yale press release (forth row in this image).
- Houshalter 13y ago
  They share many features with the original face. No they are not great, but it's pretty remarkable and far better than random.
  - JackFr 13y ago
    By many features do you mean two eyes, nose, mouth?
    - Houshalter 13y ago
      Just at a glance it got skin color, gender, hair color, and the smile.

timbre 13y ago

You are right that they assume the subject is looking at a face, and so will always produce "face-like" output. The paper includes a quantitative evaluation, in which they take a set of 30 distractor face images not used elsewhere in the study, and for each of them determine whether the reconstructed face is more similar to it, or to the original face the subject was looking at. On average, the reconstructed face is closer to the correct face than the distractor 62.5% of the time. So it's better than random, and I think it's pretty cool work, but the quality of the reconstructions is pretty terrible, considering that a randomly chosen distractor will usually be very different from the test face (~half the time opposite sex, frequently different race, different age, etc). For comparison, it would be interesting to evaluate some simple, obviously terrible reconstructions by the same metric. For example, we could "reconstruct" the face as a image that is a single, solid color, the average RGB value of the pixels in the original face. Another "reconstruction" that it seems would very likely do better under this evaluation metric is something like the "race-gender-age" of perp descriptions in the news ("white male in his thirties").