3 ms·
In "Adversarial Examples for Evaluating Reading Comprehension Systems" https://arxiv.org/abs/1707.07328 https://arxiv.org/abs/1707.07328, it was found that addi
by sanxiyn 9y ago
In "Adversarial Examples for Evaluating Reading Comprehension Systems" https://arxiv.org/abs/1707.07328 https://arxiv.org/abs/1707.07328, it was found that adding a single distracting sentence can lower F1 score of BiDAF (which is used in demo here) from 75.5% to 34.3% on SQuAD. In comparison, human performance goes from 92.6% to 89.2%.
- andrew3726 9y agoThere's a blog post (the morning paper) about this: https://blog.acolyer.org/2017/09/13/adversarial-examples-for-evaluating-reading-comprehension-systems/ https://blog.acolyer.org/2017/09/13/adversarial-examples-for...