4 ms·

I was working with Erlang for about 2 years in a multi-language environment, and we based our networking messaging infrastructure on top of the Erlang protocol.

I was working with Erlang for about 2 years in a multi-language environment, and we based our networking messaging infrastructure on top of the Erlang protocol. It's still there, but we're migrating from it. One major thing that turned me off of the language for our particular project was that there was no way to have a large read-only data structure in RAM and sic a bunch of parallel threads to analyze or search it. It is hackable if you mash your data structure into a large byte binary, since large bins live on a shared heap, but it's a pain to deal with non-native representation for your data. It's very obvious that Erlang is optimized for a particular usage model, and even when you're dealing with a number of functional parallel processes, it might still clash with what you're doing.

sargun 11y ago
I mean, you could always keep your large data structure in ETS?
- white-flame 11y ago
  That's not very fast in comparison.
- areed 11y ago
  ETS: http://www.erlang.org/doc/man/ets.html http://www.erlang.org/doc/man/ets.html
rudiger 11y ago
If you need a multi-gigabyte, read-only, in-memory data structure with support for ad-hoc querying and concurrent reads... you probably should just use a SQL database.
vtuulos 11y ago
DiscoDB does exactly what you described: It memory maps a large read-only data structure in RAM. It has an Erlang binding based on NIFs. http://discodb.readthedocs.org/en/latest/ http://discodb.readthedocs.org/en/latest/ https://github.com/discoproject/discodb https://github.com/discoproject/discodb
- white-flame 11y ago
  That's not the type of interface you'd want to build algorithms on which are constantly and deeply traversing data object links, if any type of performance is required. Same thing as with ETS or SQL. Implementing traversal-heavy algorithms across a large in-RAM data network does not require external tools, and is not helped either in performance or simplicity by decoupling direct pointer-based linkages into hash keys, and speaking to external interfaces. At least not for the work we were doing.

sargun 11y ago

I mean, you could always keep your large data structure in ETS?

white-flame 11y ago
That's not very fast in comparison.
areed 11y ago
ETS: http://www.erlang.org/doc/man/ets.html http://www.erlang.org/doc/man/ets.html

rudiger 11y ago

If you need a multi-gigabyte, read-only, in-memory data structure with support for ad-hoc querying and concurrent reads... you probably should just use a SQL database.

vtuulos 11y ago

DiscoDB does exactly what you described: It memory maps a large read-only data structure in RAM. It has an Erlang binding based on NIFs. http://discodb.readthedocs.org/en/latest/ http://discodb.readthedocs.org/en/latest/ https://github.com/discoproject/discodb https://github.com/discoproject/discodb

white-flame 11y ago
That's not the type of interface you'd want to build algorithms on which are constantly and deeply traversing data object links, if any type of performance is required. Same thing as with ETS or SQL. Implementing traversal-heavy algorithms across a large in-RAM data network does not require external tools, and is not helped either in performance or simplicity by decoupling direct pointer-based linkages into hash keys, and speaking to external interfaces. At least not for the work we were doing.