3 ms·
Has anyone tried firing up Pandas or something to load a multi-TB table? Would be interested to see if you run into some hidden snags.
by civilized 4y ago
Has anyone tried firing up Pandas or something to load a multi-TB table? Would be interested to see if you run into some hidden snags.
- jdeaton 4y agoI've done this though the data in the table was split across DataFrames in many concurrent processes. https://stackoverflow.com/questions/49438954/python-shared-memory-dictionary-for-mapping-big-data https://stackoverflow.com/questions/49438954/python-shared-m...