Been meaning to try this, good write up. An alternative between keeping all data in-memory and going full relational database that's worked well for me is Pyth

Been meaning to try this, good write up. An alternative between keeping all data in-memory and going full relational database that's worked well for me is Python + H5PY. Write a simple class where your data getters / setters interact directly with the H5 file and don't keep it in memory. Super fast (data handling and setup), easy on memory, and doesn't lock you into the schema of a relational database.