4 ms·
That access form was... refreshing. Here's why this matters to me, an independent researcher who wants to start publishing again. In 2008, Ronan Collobert and
by bravura 4y ago
That access form was... refreshing.
Here's why this matters to me, an independent researcher who wants to start publishing again.
In 2008, Ronan Collobert and Jason Weston had published some work that made neural network training of word vector representations really fast. But only ML people read that paper. Yoshua Bengio and Lev-Arie Ratinov and I plugged old-school cluster based as well as fancy-but-uncool-and-icky neural network word representations into a variety of NLP models. It worked awesome. Before "transformers go brrrrrr" our paper told the NLP community, basically, self-supervised learning and neural networks go "brrrrrrr". People finally started paying attention in the language world, ML stopped being treated with suspicion and the field moved rapidly, our paper racked up 2700 cites and an ACL 10 Year "Test Of Time" award, and here we are.
I don't work in a big research lab but I still publish. I pay for my GPUs the old fashioned way. You know, out of pocket.
It took me ages to get access to GPT-3. Ilya was a colleague of mine, so I messaged him fb, but no dice. Why? I know I could pull other strings through my network but, like, really? Is this where we are right now?
All I'm saying is: It's nice to fill out a form asking my intended use and my previously related publications, as a means of gatekeeping. The access process feels more transparent and principled. Or maybe I'm just being grouchy.
- O__________O 4y agoLink to the ACL 10 Year "Test Of Time" award paper mentioned above: Word Representations: A Simple and General Method for Semi-Supervised Learning https://aclanthology.org/P10-1040/ https://aclanthology.org/P10-1040/ (PDF link) https://aclanthology.org/P10-1040.pdf https://aclanthology.org/P10-1040.pdf
- ninjin 4y agoThank you Turian, your ACL 2010 paper [1] was what got me somewhat early into word representations when it appeared during the first year of my PhD and ultimately gave me a head start on “deep learning thinking” that I suspect helped me greatly on my path to a faculty position many years later. My own lab at the time was not really a hardcore machine learning lab, but we read “Natural Language Processing (Almost) from Scratch” as well. However, it really was your paper that became the gateway to representation learning (and ultimately deep learning) for myself and several other members of the group. [1]: https://aclanthology.org/P10-1040 https://aclanthology.org/P10-1040 I clearly remember Christopher Manning explicitly mentioning your paper at the first ever (to the best of my knowledge) deep learning tutorial at a natural language processing conference (EMNLP 2012), at the very end before we left the room, with something along the lines of: “If there is only one thing you take away from this tutorial: Read Turian’s paper and add word representations as features to your classifiers for whatever task you may have – it works!” For those less familiar with the history of things, this was prior to Mikolov’s word2vec that arrived in 2013, which apart from popularising neural vector representations (“king - man + woman ~= queen”) above all made fast training of these embeddings feasible – Wikipedia overnight really, compared to the days, weeks, or even months of training for both the cluster-based and neural-based alternatives that were used back then. Outstanding work and very much ahead of its time. If you are ever around London, my group runs one of the largest public natural language processing talk series in the UK [2] and we would be more than happy to host you if you would be up for example to give a talk on a retrospective on the early days of things up until now. [2]: https://www.meetup.com/UCL-Natural-Language-Processing-Meetup https://www.meetup.com/UCL-Natural-Language-Processing-Meetu...