4 ms·
I have a different view: it’s not the collection that’s the problem, it’s the firehose attached to the database. For the applications you mention, make aggregat
by lovecg 6y ago
I have a different view: it’s not the collection that’s the problem, it’s the firehose attached to the database. For the applications you mention, make aggregation over all records prohibitively expensive by design.
- edanm 6y agoA good idea, but again, there are plenty of cases where that fails (depending on what you mean by "aggregate"). - "How much money is currently owed in taxes to the government?" - "Can't tell you that, we're not allowed to aggregate data".
- lovecg 6y agoI still think there might be something here. You can allow certain aggregations (like “sum of the tax column”), but they have to be explicitly permitted; otherwise shuffle and hash everything enough times to make a single lookup sort of cheap, while a scan very expensive (plus distribute over enough physical servers and make the network between them low bandwidth to thwart lower level attacks). With enough regulatory or legal pressure on companies to lock down their data, paying this premium might start to look attractive; one could even found a startup peddling the World’s Slowest Database™! Edit: what I was thinking originally was that in the world of paper-only archives, these massive leaks were all but impossible, yet business could still be done. It should be possible to combine this slowness with the convenience of computers.
- edanm 6y agoI mean, that slowness is the reason we moved to computers. That said this is certainly interesting, I wonder if there has already been an exploration of this topic. Could definitely make an interesting startup idea :)