6 ms·

Guys Figure 1 is not real results, it's an illustration of the "goal" of the paper. The real results are in Table 3. And are much worse.

deleted 3y ago
[deleted]
explaininjs 3y ago
Interesting ploy. Present far-better-than-achieved results right on the front page with no text to explain their origin^, but make them poor enough quality to make it seem as if they might be real. ^ "Overall illustration of translate EEG waves into text through quantised encoding." doesn't count.
- mike_hearn 3y ago
  Urgh. And it gets worse from there. The bugs list on the repo has a closed and locked bug report from someone claiming that their code is using teacher forcing! https://github.com/duanyiqun/DeWave/issues/1 https://github.com/duanyiqun/DeWave/issues/1 In a normal recurrent neural network, the model predicts token-at-a-time. It predicts a token, and that token is appended to the total prediction so far which is then fed back into the model to generate the next token. In other words, the network generates all the predictions itself based off its own previous outputs and the other inputs (brainwaves in this case), meaning that a bad prediction can send the entire thing off track. In teacher forcing that isn't the case. All the tokens up to the point where it's predicting are taken from the correct inputs. That means the model is never exposed to its own previous errors. But of course in a real system you don't have access to the correct inputs, so this is not feasible to do in reality. The other repo says: "We have written a corrected version to use model.generate to evaluate the model, the result is not so good" but they don't give examples. This problem completely invalidates the paper's results. It is awful that they have effectively hidden and locked the thread in which the issue was reported. It's also kind of nonsensical that people doing such advanced ML work are claiming they accidentally didn't know the difference between model.forward() and model.generate(). I mean I'm not an ML researcher and might have mangled the description of teacher forcing, but even I know these aren't the same thing at all.
  - chpatrick 3y ago
    So instead of generating the next token from its own previous predictions (which is what it would do in real life), the code they used for the evaluation actually predicts from the ground truth?
    - ghayes 3y ago
      Which would basically turn the model into a plainly normal LLM without any need for utilizing the brainwave inputs, right?
  - AndrewKemendo 3y ago
    This is a super important point and I think warrants a letter to the editor
  - iamleppert 3y ago
    You’d be shocked how common this is in academia. Most of the time it goes undetected because the people writing the checks can’t be bothered to understand.
  - oeeker 3y ago
    how could such thing get published?
    - heyoni 3y ago
      My guess is repeatability is hard when it comes to AI
- godelski 3y ago
  What's interesting to me is that apparently a lot of people see nothing wrong with this[0]. That whole thread is wild and I'm just showing a small portion. Also, @dang, can we ban links to iflscience? They're a trash publication that entirely relies on clickbaity misrepresentations of research works. There is __always__ a better source that can be used. [0] https://news.ycombinator.com/item?id=38565424 https://news.ycombinator.com/item?id=38565424
oldesthacker 3y ago
The results of Table 3 are not really exciting. Could this change with 100 times more data? The key novelty in the specific context of this particular application is the quantized variational encoder used "to derive discrete codex encoding and align it with pre-trained language models."
seydor 3y ago
Why is it such a "pattern" in these brain-computer papers that the authors keep making wild clickbait claims. Last year it was the DishBrain paper, which caused a lot of reactions, as it referred to the tiny system as "sentient" (https://hal.science/hal-04012408 https://hal.science/hal-04012408) This year it is the "Brainoware" which is claimed to do speech recognition , and now this.

deleted 3y ago

[deleted]

explaininjs 3y ago

Interesting ploy. Present far-better-than-achieved results right on the front page with no text to explain their origin^, but make them poor enough quality to make it seem as if they might be real. ^ "Overall illustration of translate EEG waves into text through quantised encoding." doesn't count.

mike_hearn 3y ago
Urgh. And it gets worse from there. The bugs list on the repo has a closed and locked bug report from someone claiming that their code is using teacher forcing! https://github.com/duanyiqun/DeWave/issues/1 https://github.com/duanyiqun/DeWave/issues/1 In a normal recurrent neural network, the model predicts token-at-a-time. It predicts a token, and that token is appended to the total prediction so far which is then fed back into the model to generate the next token. In other words, the network generates all the predictions itself based off its own previous outputs and the other inputs (brainwaves in this case), meaning that a bad prediction can send the entire thing off track. In teacher forcing that isn't the case. All the tokens up to the point where it's predicting are taken from the correct inputs. That means the model is never exposed to its own previous errors. But of course in a real system you don't have access to the correct inputs, so this is not feasible to do in reality. The other repo says: "We have written a corrected version to use model.generate to evaluate the model, the result is not so good" but they don't give examples. This problem completely invalidates the paper's results. It is awful that they have effectively hidden and locked the thread in which the issue was reported. It's also kind of nonsensical that people doing such advanced ML work are claiming they accidentally didn't know the difference between model.forward() and model.generate(). I mean I'm not an ML researcher and might have mangled the description of teacher forcing, but even I know these aren't the same thing at all.
- chpatrick 3y ago
  So instead of generating the next token from its own previous predictions (which is what it would do in real life), the code they used for the evaluation actually predicts from the ground truth?
  - ghayes 3y ago
    Which would basically turn the model into a plainly normal LLM without any need for utilizing the brainwave inputs, right?
- AndrewKemendo 3y ago
  This is a super important point and I think warrants a letter to the editor
- iamleppert 3y ago
  You’d be shocked how common this is in academia. Most of the time it goes undetected because the people writing the checks can’t be bothered to understand.
- oeeker 3y ago
  how could such thing get published?
  - heyoni 3y ago
    My guess is repeatability is hard when it comes to AI
godelski 3y ago
What's interesting to me is that apparently a lot of people see nothing wrong with this[0]. That whole thread is wild and I'm just showing a small portion. Also, @dang, can we ban links to iflscience? They're a trash publication that entirely relies on clickbaity misrepresentations of research works. There is __always__ a better source that can be used. [0] https://news.ycombinator.com/item?id=38565424 https://news.ycombinator.com/item?id=38565424

oldesthacker 3y ago

The results of Table 3 are not really exciting. Could this change with 100 times more data? The key novelty in the specific context of this particular application is the quantized variational encoder used "to derive discrete codex encoding and align it with pre-trained language models."

seydor 3y ago

Why is it such a "pattern" in these brain-computer papers that the authors keep making wild clickbait claims. Last year it was the DishBrain paper, which caused a lot of reactions, as it referred to the tiny system as "sentient" (https://hal.science/hal-04012408 https://hal.science/hal-04012408) This year it is the "Brainoware" which is claimed to do speech recognition , and now this.