3 ms·
> The description should be more upfront that you are getting inaccurate answers. You can't really expect a data structure using a finite amount of memory to c
by gizmo 9y ago
> The description should be more upfront that you are getting inaccurate answers.
You can't really expect a data structure using a finite amount of memory to contain an infinite amount of data. Otherwise it wouldn't be advertised as a "counter", but as a magic bag of holding.
- tedmiston 9y agoContaining an infinite amount of data and giving accurate answers are saying different things though. It can have finite memory and still give 100% accurate answers by batching from disk. Chunking / iterative processing like this is an approach people commonly use with pandas which otherwise isn't great for data sets too big for RAM. http://pandas-docs.github.io/pandas-docs-travis/io.html#iterating-through-files-chunk-by-chunk http://pandas-docs.github.io/pandas-docs-travis/io.html#iter...