The thing that jumped out at me in this article was the use of Lime to explain models - I hadn't heard of it before. https://github.com/marcotcr/lime https://g

The thing that jumped out at me in this article was the use of Lime to explain models - I hadn't heard of it before. https://github.com/marcotcr/lime https://github.com/marcotcr/lime For NLP tasks, it looks like what it does is selectively delete words from the input and check the classifier output. This way it determines which words have the biggest effect on the output without needing to know anything about how your model works.