I've had memory exhaustion (on a 4GB system) after I think about 600GB in a single crawl. Splitting it into multiple crawls is of course better. That was a sit

I've had memory exhaustion (on a 4GB system) after I think about 600GB in a single crawl. Splitting it into multiple crawls is of course better. That was a site specifically set up to deal with large collections of files though.