4 ms·

I am just worried that they are picking up part of information from camera mic. Maybe camera mic output is not totally independent from the video sensor, and is

I am just worried that they are picking up part of information from camera mic. Maybe camera mic output is not totally independent from the video sensor, and is encoded in the final video.

jrockway 12y ago
They put the camera behind soundproof glass for one of their tests.
- MichailP 12y ago
  I know, but I still wonder about level of soundproof/mechanical vibration and the level of mic and camera sensor isolation VS level of optical vibration. In addition sound can travel many different paths since it is mechanical vibration, and traveling through sound proof glass is not the only path. Anw cool project.
  - aidos 12y ago
    I'm sure it's such a tiny amount that it isn't really relevant. Easy enough to test. Remove the bag of chips and just film the floor. See how much signal you can derive from that. Not exactly perfect science but gives a quick indication of "signal" that's reaching the system via other means.
    - jrockway 12y ago
      Actually, what I think would be even better would be to record the bag of chips through the window, but play the sound outside where the camera is.
prutschman 12y ago
The subtitle of the article is "Algorithm recovers speech from the vibrations of a potato-chip bag filmed through soundproof glass."
abedavis 12y ago
The camera used in most of these experiments (a Phantom high speed camera) doesn't even have a microphone - so that would be quite impossible.
- MichailP 12y ago
  Thanks. However, there is still a possibility that mechanical vibration is coming to the video sensor through some path (floor+camera tripod), and affecting the video. After all mechanical vibration is affecting the potato chips. Why is it so impossible to affect video sensor? Especially in the sensitive high speed camera? Let the downvotes begin :)
  - underyx 12y ago
    I fail to see how that would make this any less impressive.
    - MichailP 12y ago
      If my evil :) speculation is true, the camera sensor is just picking up mechanical vibrations from environment (coming through air, tripod etc.) and encoding them into video. There is no proof that the bag of chips is vibrating (they even say that the vibrations are not visible on footage). They are extracting something from video, but that something may just be spuriuos pickups by equipment, and not related image/video of bag of chips. Thus it would be just a silly way to measure spurious mechanical vibrations.
  - anigbrowl 12y ago
    Well, the chip bag weighs a few grams at most. The camera and gear weigh something like 50kg. The higher the video frame rate, the greater the damping you need to prevent unwanted vibration. Notice the tests were performed with a speaker playing back inside the room rather than a person, in order to minimize mechanical vibrations. The sensors are very firmly mounted inside the camera, trust me.

I am just worried that they are picking up part of information from camera mic. Maybe camera mic output is not totally independent from the video sensor, and is encoded in the final video.

jrockway 12y ago

They put the camera behind soundproof glass for one of their tests.

MichailP 12y ago
I know, but I still wonder about level of soundproof/mechanical vibration and the level of mic and camera sensor isolation VS level of optical vibration. In addition sound can travel many different paths since it is mechanical vibration, and traveling through sound proof glass is not the only path. Anw cool project.
- aidos 12y ago
  I'm sure it's such a tiny amount that it isn't really relevant. Easy enough to test. Remove the bag of chips and just film the floor. See how much signal you can derive from that. Not exactly perfect science but gives a quick indication of "signal" that's reaching the system via other means.
  - jrockway 12y ago
    Actually, what I think would be even better would be to record the bag of chips through the window, but play the sound outside where the camera is.

prutschman 12y ago

The subtitle of the article is "Algorithm recovers speech from the vibrations of a potato-chip bag filmed through soundproof glass."

abedavis 12y ago

The camera used in most of these experiments (a Phantom high speed camera) doesn't even have a microphone - so that would be quite impossible.

MichailP 12y ago
Thanks. However, there is still a possibility that mechanical vibration is coming to the video sensor through some path (floor+camera tripod), and affecting the video. After all mechanical vibration is affecting the potato chips. Why is it so impossible to affect video sensor? Especially in the sensitive high speed camera? Let the downvotes begin :)
- underyx 12y ago
  I fail to see how that would make this any less impressive.
  - MichailP 12y ago
    If my evil :) speculation is true, the camera sensor is just picking up mechanical vibrations from environment (coming through air, tripod etc.) and encoding them into video. There is no proof that the bag of chips is vibrating (they even say that the vibrations are not visible on footage). They are extracting something from video, but that something may just be spuriuos pickups by equipment, and not related image/video of bag of chips. Thus it would be just a silly way to measure spurious mechanical vibrations.
- anigbrowl 12y ago
  Well, the chip bag weighs a few grams at most. The camera and gear weigh something like 50kg. The higher the video frame rate, the greater the damping you need to prevent unwanted vibration. Notice the tests were performed with a speaker playing back inside the room rather than a person, in order to minimize mechanical vibrations. The sensors are very firmly mounted inside the camera, trust me.