In my experience it's usually best to start with all the words. If you use a decent implementation that supports sparse vectors it's no problem, certainly not f

In my experience it's usually best to start with all the words. If you use a decent implementation that supports sparse vectors it's no problem, certainly not for these sorts of data sizes. Usually you'll end up with a frequency threshold, but it's usually best to trim at the very low end --- like, words occurring 5 or fewer times. Further over-fitting can be controlled with regularization and parameter averaging.