4 ms·
I fail to see how word-based models are character-based models with manual feature extraction. Word boundaries are read directly from deterministically tokenize
by kylebgorman 11y ago
I fail to see how word-based models are character-based models with manual feature extraction. Word boundaries are read directly from deterministically tokenized inputs.
And, as I mentioned upthread, it has been known for about ten years, long before the current neural net revival, that high-order character-based models are competitive with word-based models (at least in terms of perplexity).