This is not a major problem for data science. We usually use numpy arrays (or scipy sparse arrays), which are memory efficient. There are also some other effici

This is not a major problem for data science. We usually use numpy arrays (or scipy sparse arrays), which are memory efficient. There are also some other efficient data structures for python: http://kmike.ru/python-data-structures/ http://kmike.ru/python-data-structures/ For preprocessing you can usually work on a line-by-line basis (e.g. don't keep the entire dataset in memory). This is a bigger problem for web apps, imo. But there you usually just throw money (e.g. more servers) at the problem.