3 ms·
I'm not sure. I rarely have to do that explicitly in my head. Well I can't prove it but I strongly believe that our brains use part of speech too, unconsciously
by The_rationalist 7y ago
I'm not sure. I rarely have to do that explicitly in my head.
Well I can't prove it but I strongly believe that our brains use part of speech too, unconsciously.
Perhaps a model should learn to infer/guess them implicitly, from data.
That's exactly what deep learning POS tagger do, they are far better than hard coded algorithms. SOTA has 97.96% of accuracy.
In my opinion, having a world model (for common sense) and situational awareness (e.g. through sensor fusion, or from prior conversational history, or using some externally supplied conditioning) would be far more important.
Haha you basically want a general intelligence (AGI), I want it too! And not enough persons works on "architecting" such a thing. Opencog may interest you a lot then.
But the reality is many other "simpler" tasks are needed to make this happen.
having a world model (for common sense) is an NLP task
There are some interesting results https://github.com/sebastianruder/NLP-progress/blob/master/english/common_sense.md https://github.com/sebastianruder/NLP-progress/blob/master/e...
OpenAI does not work on this task sadly, at least for now.
You think adding POS tags would help it generate meaningful sentences?
I would be clearly insufficient yet necessary.
I believe they already use internally a POS tagger and a dependency parser.
- p1esk 7y agothey already use internally a POS tagger and a dependency parser. Interesting. Where did you see that?
- The_rationalist 7y agoWell it was just a belief. I may be wrong. I asked them by curiosity https://github.com/openai/gpt-2/issues/168 https://github.com/openai/gpt-2/issues/168 So we will know.
- p1esk 7y agoHow do you think it could be used there? A separate model just for providing tags, or the same model but trained to predict tags as well?
- The_rationalist 7y agoI was imagining using a separate model just for providing tags as they are very accurate. It would theoretically give gpt-2 useful data. GPT-2 has not (yet) been trained to predict POS tags to my knowledge, nor BERT, or ernie 2 or xlnet has, but I think they have great potential to improve POS accuracy.