4 ms·
Can you point me at some? Because I've tried a few immutable data stores and been disappointed every time. Given about 10 GB of JSON structures, I keep finding
by rspeer 10y ago
Can you point me at some? Because I've tried a few immutable data stores and been disappointed every time. Given about 10 GB of JSON structures, I keep finding things that can't outperform the boring combo of:
* Convert the versioned data to tab-separated values
* COPY it into Postgres every time
* Hope Postgres can act immutable enough even though it wasn't designed to be
The closest I've come to improving this situation was Kyoto Cabinet (unusable license) and rolling my own damn hashtable (it worked okay but adding new kinds of indexes was just unmaintainable, there's a reason databases should be made by experts).
- ykler 10y agoI was curious about Camlistore and Tahoe-LAFS at one point. I didn't investigate them very thoroughly, however. How do they compare?
- deleted 10y ago[deleted]
- sjezewski 10y ago(Disclaimer - I work at pachyderm) http://pachyderm.io http://pachyderm.io Pachyderm is git for data. We work hard to make sure we can store data of different types (binary, text, json) efficiently. We also work hard to give you good mechanisms to read the data in a distributed way. I'd be curious how this suits your purposes.
- rspeer 10y agoJust started looking at Pachyderm. While I can see how a git-based filesystem can help with some use cases, does it do any kind of indexing at all? I see that the FAQ recommends exporting the data from Pachyderm into PostgreSQL, which leaves me where I am now.