4 ms·

Why are people being so critical about this work? Sure, the blog post provides a simplified picture about what the system is actually capable of, but it's still

Why are people being so critical about this work? Sure, the blog post provides a simplified picture about what the system is actually capable of, but it's still helpful for a non-ML audience to get a better understanding of the high-level motivation behind the work. The OpenAI folks are trying to educate the broader public as well, not just ML/AI researchers. Imagine if this discovery were made by some undergraduate student who had little experience in the traditions of how ML benchmark experiments are done, or was just starting out her ML career. Would we be just as critical? As a researcher, I like seeing shorter communications like these, as it illuminates the thinking process of the researcher. Read ML papers for the ideas, not the results :) I personally don't mind blog posts that have a bit of hyped-up publicity. It's thanks to groups like DeepMind and OpenAI that have captured public imagination on the subject and accelerated such interest in prospective students in studying ML + AI + robotics. If the hype is indeed unjustified, then it'll become irrelevant in the long-term. One caveat is that researchers should be very careful to not mislead reporters who are looking for the next "killer robots" story. But that doesn't really apply here.

chrisfosterelli 10y ago
I personally think they did great. They targeted the blog post at a more general audience so most people can follow and get an overhead view of the idea, and then put two giant buttons for "View code" and "Read paper" right at the top of the blog post for those who want more technical writing and working code.
- nitrogen 10y ago
  Agreeed. As a non-ML developer, I think this is the paragraph that sells the work, even if it may be an oversimplification: We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment. We believe the phenomenon is not specific to our model, but is instead a general property of certain large neural networks that are trained to predict the next step or dimension in their inputs I think it says something very interesting about human language and information processing in general.
virmundi 10y ago
From my little experience with the AI community, I think people in it love to obfuscate things. Any attempt to make a topic approachable, even if some of the details are lost, get smacked around. I face this every day in my Masters. If you don't already come with a knowledge of AI + Stats, you're on your own. The community, including the teachers, don't want to teach the mundane.
- throwaway91111 10y ago
  What does "knowledge of AI" mean to you?
  - virmundi 10y ago
    I'm in the GA Tech program. Chose AI since I've never touched it. Every course has been terrible in this track. The lectures are awful. The readings take hours upon hours. The TAs pathologically refuse to answer guestions without channeling Confucius. Then I found old GA Tech videos and MIT open courseware. It's starting to make sense. I say all of this because intro to ai is highly rated. It can only get such a rating if it's coming from people that already know the material.
    - forrestbrazeal 10y ago
      Are you in the OMSCS or on-campus program? I'm currently enrolled in CS 6601 (Artificial Intelligence) via OMSCS, and while it's extremely difficult material that requires a lot of personal investment to master, I've found both TAs and other students to be as helpful as one could reasonably expect.
      - virmundi 10y ago
        OMSCS. The TAs refuse to answer questions because it could help cheating. They won't review homework because it could help cheating. They've created an environment so full of fear that it's impossible to learn from mistakes. I'm surprised the slack channel is allowed.
backpropaganda 10y ago
> next "killer robots" story ... doesn't really apply here. Are you sure about that? We're talking about a model/robot which understand sentiments, and can generate fake reviews to boost fake products. I can easily see this being picked up by the AI hype journalists. In fact, this model could even be used for nefarious purposes.
- Danylon 10y ago
  AI hype journalists will find something to write about, regardless of the industry making their research accessible to the wider public. Markov-chain generators have been around for a while, and have been used to throw off spam detectors. This should not stop research, but instead grow more research into adversarial usage of machine learning models.
- viksit 10y ago
  > I can easily see this being picked up by the AI hype journalists. Surely you mean AI journalist bots that can generate articles about how AI review bots generate fake reviews?
eanzenberg 10y ago
Is it wrong to be critical of research? Back in my previous life of doing basic research I scrutinized papers left and right. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/ towards the end has similar methodology and is 1.5 years old. Hype is an interesting thing especially when it comes from laymen.
- fiter 10y ago
  If they're getting better results than the previous state of the art, I think the most important point of this research is its critique of previous papers! The previous state of the art research needs to consider such old, basic techniques to improve their own results.
- laingc 10y ago
  As someone familiar with the field, you likely know this already, but the similarities between the Karpathy post from 2015 and this work from OpenAI is likely because Karpathy is a founder and lead researcher at OpenAI.
  - eanzenberg 10y ago
    Ya but he's surprisingly absent from being a paper author.
    - deleted 10y ago
      [deleted]
- nebabyte 10y ago
  > Hype is an interesting thing especially when it comes from laymen. Agreed, as can attributing value to said hype. An argument can generally be made for many things for why they "can be useful, in X situation" (like 'a layperson understand ML'), doesn't mean it has value in every context. (Or even that it's a particularly good example for its contrived purpose - just that it could/might suffice if nothing better exists.)
ValleyOfTheMtns 10y ago
>Why are people being so critical about this work? Some people can't stand to miss an opportunity to remind everyone about how smart they are. However... one criticism I have of the article is that their first graph doesn't start the y-axis at zero, giving a false impression of how much their method improves on others.

chrisfosterelli 10y ago

I personally think they did great. They targeted the blog post at a more general audience so most people can follow and get an overhead view of the idea, and then put two giant buttons for "View code" and "Read paper" right at the top of the blog post for those who want more technical writing and working code.

nitrogen 10y ago
Agreeed. As a non-ML developer, I think this is the paragraph that sells the work, even if it may be an oversimplification: We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment. We believe the phenomenon is not specific to our model, but is instead a general property of certain large neural networks that are trained to predict the next step or dimension in their inputs I think it says something very interesting about human language and information processing in general.

virmundi 10y ago

From my little experience with the AI community, I think people in it love to obfuscate things. Any attempt to make a topic approachable, even if some of the details are lost, get smacked around. I face this every day in my Masters. If you don't already come with a knowledge of AI + Stats, you're on your own. The community, including the teachers, don't want to teach the mundane.

throwaway91111 10y ago
What does "knowledge of AI" mean to you?
- virmundi 10y ago
  I'm in the GA Tech program. Chose AI since I've never touched it. Every course has been terrible in this track. The lectures are awful. The readings take hours upon hours. The TAs pathologically refuse to answer guestions without channeling Confucius. Then I found old GA Tech videos and MIT open courseware. It's starting to make sense. I say all of this because intro to ai is highly rated. It can only get such a rating if it's coming from people that already know the material.
  - forrestbrazeal 10y ago
    Are you in the OMSCS or on-campus program? I'm currently enrolled in CS 6601 (Artificial Intelligence) via OMSCS, and while it's extremely difficult material that requires a lot of personal investment to master, I've found both TAs and other students to be as helpful as one could reasonably expect.
    - virmundi 10y ago
      OMSCS. The TAs refuse to answer questions because it could help cheating. They won't review homework because it could help cheating. They've created an environment so full of fear that it's impossible to learn from mistakes. I'm surprised the slack channel is allowed.

backpropaganda 10y ago

> next "killer robots" story ... doesn't really apply here. Are you sure about that? We're talking about a model/robot which understand sentiments, and can generate fake reviews to boost fake products. I can easily see this being picked up by the AI hype journalists. In fact, this model could even be used for nefarious purposes.

Danylon 10y ago
AI hype journalists will find something to write about, regardless of the industry making their research accessible to the wider public. Markov-chain generators have been around for a while, and have been used to throw off spam detectors. This should not stop research, but instead grow more research into adversarial usage of machine learning models.
viksit 10y ago
> I can easily see this being picked up by the AI hype journalists. Surely you mean AI journalist bots that can generate articles about how AI review bots generate fake reviews?

eanzenberg 10y ago

Is it wrong to be critical of research? Back in my previous life of doing basic research I scrutinized papers left and right. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/ towards the end has similar methodology and is 1.5 years old. Hype is an interesting thing especially when it comes from laymen.

fiter 10y ago
If they're getting better results than the previous state of the art, I think the most important point of this research is its critique of previous papers! The previous state of the art research needs to consider such old, basic techniques to improve their own results.
laingc 10y ago
As someone familiar with the field, you likely know this already, but the similarities between the Karpathy post from 2015 and this work from OpenAI is likely because Karpathy is a founder and lead researcher at OpenAI.
- eanzenberg 10y ago
  Ya but he's surprisingly absent from being a paper author.
  - deleted 10y ago
    [deleted]
nebabyte 10y ago
> Hype is an interesting thing especially when it comes from laymen. Agreed, as can attributing value to said hype. An argument can generally be made for many things for why they "can be useful, in X situation" (like 'a layperson understand ML'), doesn't mean it has value in every context. (Or even that it's a particularly good example for its contrived purpose - just that it could/might suffice if nothing better exists.)

ValleyOfTheMtns 10y ago

>Why are people being so critical about this work? Some people can't stand to miss an opportunity to remind everyone about how smart they are. However... one criticism I have of the article is that their first graph doesn't start the y-axis at zero, giving a false impression of how much their method improves on others.