I might be ignorant here ... but why not have disc backed datastructures and have some sort of ECS style interface that is like "normal" programming? In pseudo

I might be ignorant here ... but why not have disc backed datastructures and have some sort of ECS style interface that is like "normal" programming? In pseudocode: newtype id = int names = dict<id,string>() balances = dict<id,int>() credit_scores = dict<id,float>() function broke_customers(): return balances.filter(balance => balance < 0).keys() function exploitable_customers(): bs = balances.filter(balance => balance < 0) cs = credit_score.filter(score => score > 100) return bs.intersect(cs).keys() In the end, most queries are "just" set theory ... and having a very thin disc io layer allows to use the host language to process queried data on the fly. It's very basic ... and does not address performant views, clustering, migration, etc ... but it's simple ... and does work well as demonstrated by the ECS systems in game development (which are an application of the concept). (Sidenote: that's some messed up pseudocode ... I've been working with C#, Python and Haskell lately :-) )