In the conclusion the authors state: "It would be particularly interesting to see whether such a design works in NLP or other domains." I think it's quite comm

In the conclusion the authors state: "It would be particularly interesting to see whether such a design works in NLP or other domains." I think it's quite common to use quasi "MLP mixtures" in natural language processing (NLP), since your input layer is typically a word embedding, which itself is a shallow MLP. There are also architectures such as deep averaging networks (DAN) that are effectively MLP "all the way through", see https://people.cs.umass.edu/~miyyer/pubs/2015_acl_dan.pdf https://people.cs.umass.edu/~miyyer/pubs/2015_acl_dan.pdf (NOTE: PDF document)