4 ms·
This looks really interesting. How are you writing the data to disk? I don't see exactly what you're doing in the article. Are you just writing the data to fil
by azdle 12y ago
This looks really interesting.
How are you writing the data to disk? I don't see exactly what you're doing in the article. Are you just writing the data to files directly or is there some sort of DB in there? From what I've seen, the storage is the most difficult part.
- skazka16 12y agoJust shared a link with HN community. You may try to ask author on an article's page.
- azdle 12y agoActually, I just looked at the source and it looks like they're using ETS the built-in database in Erlang.
- tormeh 12y agoOTP includes a distributed database. Maybe that's what's used?
- oinksoft 12y agoData appears to be persisted to disk via Basho's bitcask. You mentioned in another comment that ETS is being used, which it is, but that's not on disk (DETS is the disk-backed version); ETS is used here to communicate between processes via public, named tables.
- lambdadmitry 12y agoYou are completely right, thank you.
- azdle 12y agoThanks for correcting me. I'm still just dipping my toe into erlang and forgot that there were two different versions. (And thanks to lambdadmitry for the additional insight.)
- lambdadmitry 12y agoThe data that we had (disclaimer: I started the project and left a couple of years ago) tolerated occasional loss of a small part of it, so there were nothing fancy there: "dirty" NIF-based circular buffers (courtesy of ecirca library) are dumped to binary and wrote on disk using basho's bitcask from time to time. Dumb and pretty efficient solution given that constraints.