2 ms·
This would be true if the entire field of NLP ML was simply a process where training data is used to fine tune a model into providing better scores at specific
by quocanh 5y ago
This would be true if the entire field of NLP ML was simply a process where training data is used to fine tune a model into providing better scores at specific tasks. It's easy to see why you would be fooled into thinking that this is the case: the most impressive applications that have been developed so far are simply this.
But I would argue that NLP is more than this. As we discover new architectures and techniques that score higher on accuracy, the mechanisms through which we achieve higher scores are clues to the underlying abstractions that make up our thoughts. For example, the success of Transformers (basically a vector search engine) is strong evidence that concepts can be represented as vectors, and even more, we can even measure the "closeness" of two concepts using a dot product between their respective corresponding vectors. In this sense, the big Transformer architectures like GPT-3 are compressing natural language sentences into alternative, mathematical representations through its encoding step, but also decompressing back into natural language sentences in the decoding step.
Natural Language Processing at this point is actually a big misnomer - it's more than talking robots; it's about translating human reality into mathematical representations. We are still far from true understanding, but so what? Infants crawl before they walk. NLPs' representations of reality are crude and our tools for manipulating those representations are even cruder. But every advance we make is a step towards understanding knowledge itself.