2 ms·
According to the abstract: Here, we present a neural decoding approach in which machine learning models predict the contents of visual imagery during the sleep
by creamyhorror 14y ago
According to the abstract:
Here, we present a neural decoding approach in which machine learning models predict the contents of visual imagery during the sleep onset period given measured brain activity, by discovering links between human fMRI patterns and verbal reports with the assistance of lexical and image databases. Decoding models trained on stimulus-induced brain activity in visual cortical areas showed accurate classification, detection, and identification of contents.
So it sounds like the fMRI patterns are associated with words via verbal reports from humans, and the video is simply generated from image databases by using those words. In other words, they classify the fMRI patterns based on how well they correspond to various words (measured while the subjects are awake and giving verbal descriptions of images they're looking at), and then the program measures the subjects' fMRI patterns while they're dreaming, determines the strongest 'word signals' in the patterns, and selects images from a photo database corresponding to those words and assembles them into a video based on how strong the similarity score for each word is at each moment in time.
It's interesting, but the video seems like more of a "reenactment" of a dream than a prediction. It doesn't look like you can infer actions and occurrences within a dream from this. But this is all just my interpretation.
edit: This is a fundamentally different approach from the one used in other research (e.g. http://www.youtube.com/watch?v=nsjDnYxJ0bo http://www.youtube.com/watch?v=nsjDnYxJ0bo mentioned elsewhere) in which image data is statistically regressed on fMRI patterns generated while the subjects watch those images. That approach allows for a much more precise reenactment/prediction of the actual images being dreamed.