3 ms·

The BEAM even includes a database - Mnesia... https://www.erlang.org/doc/apps/mnesia/mnesia.html https://www.erlang.org/doc/apps/mnesia/mnesia.html

by _mlbt 1y ago

throwawaymaths 1y ago
Honestly I think mnesia is one of those "don't use it unless you know what you're doing" things. Just use postgres.
jerf 1y ago
Mnesia is not a database by any modern definition of the term and it should generally be avoided. It is at least 4 if not 5 orders of magnitude away from "being able to run Twitch". That is, yes, I'm serious, if you tried to run a Twitch clone "but 10,000x smaller" I would still expect Mnesia to completely fall over.
- toast0 1y ago
  Mnesia worked well enough for us at WhatsApp while I was there; although we didn't use it to store messages; long term message storage is on the end points (generally sqlite), messages in transit (offline) were stored in a file per user with the import/export written in C IIRC. We did add redundant in memory storage of messages in transit; but I don't remember the storage there; may have just been ets. We mostly used mnesia as a replicated key-value store, but we got a lot of value from having the data and the business logic colocated. Other nodes would send logical operations to processes on the mnesia node and those processes could run each operation one at a time on the data. Any concurrent logical requests for a given piece of data were implicitly serialized by the process mailbox. But almost all of our data was easy to shard, no high volume operations needed to address multiple tables. We heard a lot of things about mnesia scalability limits that just didn't match up with our experience; so I don't know what other people were doing, but you can get a glimpse of what we were doing in the Rick Reed talks at Erlang Factory. We certainly had some scalability challenges, but many (most?) are discussed in those talks; and my general recollection is that most of them were more like we were the only people running mnesia with tables of enormous size, so we had to make things work; but that's kind of how OTP is. The trickiest one to find, IMHO, was that IIRC mnesia_frag and ets (and our request sharding) all use(d?) the same hashing function, so adding more fragments would make distribution of keys per ets slot worse, ets wanted power of two slots, and would split based on average keys per slot, but we would have lots of keys on some slots and no keys on most slots. Changing the hash seed for ets was a 2 line code change that drastically improved performance on all of our sharded mnesia systems. Another fun one is that if you use mnesia to store data for long periods, you have to be very careful with the binaries you store; it's easy to end up with refc binaries that have extra space for append operations; storing them in mnesia means that append space is allocated but unusable; you might also store a sub binary that's a small part of a refc binary, the underlying binary can't be disposed of until the sub binary is. For both of those cases, cleaning the binaries before storing them with binary:copy/1 can really reduce your memory use. There's probably some cases where you do want to store a sub binary though? Mnesia doesn't (or didn't) include a good way to handle when two mnesia nodes sharing a schema disconnect and reconnect. We mostly solved that by ensuring our network was stable enough that that rarely happened. If your network is not stable, you will have a bad time with distributed Erlang in general, and Mnesia in specific. If I were building Twitch but smaller, and on the BEAM, I would absolutely put account databases in Mnesia; but messages and media would probably live as files. I wouldn't tend to put those into a SQL database on a server either though.
- victorbjorklund 1y ago
  That is not true. I dont use Mnesia because it is so different and thus harder for me than sql. But it is used. Klarna uses it. They are a pretty big company. Would I recommend it for most? No, but I also would not recommend cassandra and a bunch of other databases for most apps
schultzer 1y ago
mnesia is great, and you can get very far before you would make the jump to anything else. And it can be way faster then any other database. For obvious reasons. Although it would be great if it spoke SQL, maybe one day it will: https://github.com/elixir-dbvisor/sql https://github.com/elixir-dbvisor/sql since we can already pass it and get the AST.
- troupo 1y ago
  And when you do have to jump off, you'll be screwed. Because the jump off is usually from in-memory Mnesia with guaranteed sub-millisecond responses to a proper database with at least a magnitude higher latency. And you realize that your app is completely dependent on sub-millisecond responses :)
  - schultzer 1y ago
    Low latency is addictive. And by the time when you think you might need to change you’re probably become such a skilled engineer that you realize that everybody that shit on mnesia does not know what they talk about. :)

The BEAM even includes a database - Mnesia... https://www.erlang.org/doc/apps/mnesia/mnesia.html https://www.erlang.org/doc/apps/mnesia/mnesia.html

by _mlbt 1y ago

The BEAM even includes a database - Mnesia... https://www.erlang.org/doc/apps/mnesia/mnesia.html https://www.erlang.org/doc/apps/mnesia/mnesia.html

throwawaymaths 1y ago

Honestly I think mnesia is one of those "don't use it unless you know what you're doing" things. Just use postgres.

jerf 1y ago

Mnesia is not a database by any modern definition of the term and it should generally be avoided. It is at least 4 if not 5 orders of magnitude away from "being able to run Twitch". That is, yes, I'm serious, if you tried to run a Twitch clone "but 10,000x smaller" I would still expect Mnesia to completely fall over.

toast0 1y ago
Mnesia worked well enough for us at WhatsApp while I was there; although we didn't use it to store messages; long term message storage is on the end points (generally sqlite), messages in transit (offline) were stored in a file per user with the import/export written in C IIRC. We did add redundant in memory storage of messages in transit; but I don't remember the storage there; may have just been ets. We mostly used mnesia as a replicated key-value store, but we got a lot of value from having the data and the business logic colocated. Other nodes would send logical operations to processes on the mnesia node and those processes could run each operation one at a time on the data. Any concurrent logical requests for a given piece of data were implicitly serialized by the process mailbox. But almost all of our data was easy to shard, no high volume operations needed to address multiple tables. We heard a lot of things about mnesia scalability limits that just didn't match up with our experience; so I don't know what other people were doing, but you can get a glimpse of what we were doing in the Rick Reed talks at Erlang Factory. We certainly had some scalability challenges, but many (most?) are discussed in those talks; and my general recollection is that most of them were more like we were the only people running mnesia with tables of enormous size, so we had to make things work; but that's kind of how OTP is. The trickiest one to find, IMHO, was that IIRC mnesia_frag and ets (and our request sharding) all use(d?) the same hashing function, so adding more fragments would make distribution of keys per ets slot worse, ets wanted power of two slots, and would split based on average keys per slot, but we would have lots of keys on some slots and no keys on most slots. Changing the hash seed for ets was a 2 line code change that drastically improved performance on all of our sharded mnesia systems. Another fun one is that if you use mnesia to store data for long periods, you have to be very careful with the binaries you store; it's easy to end up with refc binaries that have extra space for append operations; storing them in mnesia means that append space is allocated but unusable; you might also store a sub binary that's a small part of a refc binary, the underlying binary can't be disposed of until the sub binary is. For both of those cases, cleaning the binaries before storing them with binary:copy/1 can really reduce your memory use. There's probably some cases where you do want to store a sub binary though? Mnesia doesn't (or didn't) include a good way to handle when two mnesia nodes sharing a schema disconnect and reconnect. We mostly solved that by ensuring our network was stable enough that that rarely happened. If your network is not stable, you will have a bad time with distributed Erlang in general, and Mnesia in specific. If I were building Twitch but smaller, and on the BEAM, I would absolutely put account databases in Mnesia; but messages and media would probably live as files. I wouldn't tend to put those into a SQL database on a server either though.
victorbjorklund 1y ago
That is not true. I dont use Mnesia because it is so different and thus harder for me than sql. But it is used. Klarna uses it. They are a pretty big company. Would I recommend it for most? No, but I also would not recommend cassandra and a bunch of other databases for most apps

schultzer 1y ago

mnesia is great, and you can get very far before you would make the jump to anything else. And it can be way faster then any other database. For obvious reasons. Although it would be great if it spoke SQL, maybe one day it will: https://github.com/elixir-dbvisor/sql https://github.com/elixir-dbvisor/sql since we can already pass it and get the AST.

troupo 1y ago
And when you do have to jump off, you'll be screwed. Because the jump off is usually from in-memory Mnesia with guaranteed sub-millisecond responses to a proper database with at least a magnitude higher latency. And you realize that your app is completely dependent on sub-millisecond responses :)
- schultzer 1y ago
  Low latency is addictive. And by the time when you think you might need to change you’re probably become such a skilled engineer that you realize that everybody that shit on mnesia does not know what they talk about. :)