3 ms·
Thank you Turian, your ACL 2010 paper [1] was what got me somewhat early into word representations when it appeared during the first year of my PhD and ultimate
by ninjin 4y ago
Thank you Turian, your ACL 2010 paper [1] was what got me somewhat early into word representations when it appeared during the first year of my PhD and ultimately gave me a head start on “deep learning thinking” that I suspect helped me greatly on my path to a faculty position many years later. My own lab at the time was not really a hardcore machine learning lab, but we read “Natural Language Processing (Almost) from Scratch” as well. However, it really was your paper that became the gateway to representation learning (and ultimately deep learning) for myself and several other members of the group.
[1]: https://aclanthology.org/P10-1040 https://aclanthology.org/P10-1040
I clearly remember Christopher Manning explicitly mentioning your paper at the first ever (to the best of my knowledge) deep learning tutorial at a natural language processing conference (EMNLP 2012), at the very end before we left the room, with something along the lines of: “If there is only one thing you take away from this tutorial: Read Turian’s paper and add word representations as features to your classifiers for whatever task you may have – it works!”
For those less familiar with the history of things, this was prior to Mikolov’s word2vec that arrived in 2013, which apart from popularising neural vector representations (“king - man + woman ~= queen”) above all made fast training of these embeddings feasible – Wikipedia overnight really, compared to the days, weeks, or even months of training for both the cluster-based and neural-based alternatives that were used back then.
Outstanding work and very much ahead of its time. If you are ever around London, my group runs one of the largest public natural language processing talk series in the UK [2] and we would be more than happy to host you if you would be up for example to give a talk on a retrospective on the early days of things up until now.
[2]: https://www.meetup.com/UCL-Natural-Language-Processing-Meetup https://www.meetup.com/UCL-Natural-Language-Processing-Meetu...