2 ms·
There seems to be a logical fallacy in the post. "We didn't pick up on xLSTM because our data was small and not discriminative enough. So xLSTM is bad on large
by theihrbdbhdue 3y ago
There seems to be a logical fallacy in the post.
"We didn't pick up on xLSTM because our data was small and not discriminative enough. So xLSTM is bad on larger data as well (even though we likely can't / couldn't pick out self-attention either)."
Regardless, the conclusion in itself is also easily invalidated, since the xLSTM paper itself has benchmarks against GPT-3 on a 15B token dataset.
Digression: I'm increasingly noticing just how bad the scientific community - across all STEM fields - is when it comes to epistemology.
This is infinitely worse in fields like humanities, since they can't even test their bs theories unless they are violent "revolutionaries".
Hopefully, with all the AI hype-bubble, good philosophy may finally be funded (and that "Ph.D" might actually not sound like an ironic joke).