3 ms·
Wow, lots of goodies. Erlang team has been unstoppable in the last few years. Some changes: * New state machine module: gen_statem (ssh module uses it as a fi
by rdtsc 11y ago
Wow, lots of goodies. Erlang team has been unstoppable in the last few years.
Some changes:
* New state machine module: gen_statem (ssh module uses it as a first example).
* 3-5x times faster open_port(spawn, ...)
* Must faster and better tracing (also lttng support)
* Keeping messages off heap for processes
* mnesia-ext : pluggable storage for Mnesia. This removes the 2GB per shard limit.
* Allow limiting process's max heap usage
* Observer tool allows configuration update frequency and length of graph windows
* crypto uses EVP calls so should be accelerated by hardware were supported
* Parallel/faster module code loading
- dmix 11y ago> pluggable storage for Mnesia. This removes the 2GB per shard limit. This is a pretty big deal isn't it? I've never used Mnesia but that 2GB limit was a common warning I came across in the docs and blog posts, despite noting it was a great DB to work with.
- rdtsc 11y agoYap pretty big. Allegedly this has already been running at Klarna in production for a while. LevelDB backend from Basho (eleveldb, it has extra optimizations and improvements besides just Erlang wrapper) has been used. Granted 15-20 some years ago perhaps 2GB was BigData(TM) ;-) For a while I understand you could set up sharding, so each shard is 2GB and spread shards across nodes.
- davidw 11y ago> Allow limiting process's max heap usage What happens when it hits that limit? Crash, or try to GC, or... ?
- rdtsc 11y agoHaven't looked into details. But I think it is configurable. Killing is an option or just sending a message to the error logger. There was also something about it sending a special gc trace event when it hits max.
- Jtsummers 11y agohttp://erlang.org/documentation/doc-8.0-rc1/erts-8.0/doc/html/erlang.html#process_flag-2 http://erlang.org/documentation/doc-8.0-rc1/erts-8.0/doc/htm... Configurable, but can kill the process in question and/or log when it occurs. Their suggestion is to log it first and tune it from there since heap size isn't easily predictable. EDIT: Scroll down a little bit to get this particular flag documentation.
- nodivbyzero 11y agoKeep doing it, Erlang team !!!
- jlouis 11y agoHah, Mnesia doesn't have a 2GB limit per shard now. But having access to a leveldb backing store means you can avoid keeping the dataset resident in memory. This gives you a nice memory/latency/disk tradeoff.
- rdtsc 11y agoOh right. That's for disk_only shards. Regular shards use the disk log and those don't have a limit if I remember correctly.
- vvanders 11y agoLots of good stuff, also excited for: * dirty schedulers improved which means much better scheduling of long running NIFs from what I understand: https://medium.com/@jlouis666/erlang-dirty-scheduler-overhead-6e1219dcc7#.n088oldco https://medium.com/@jlouis666/erlang-dirty-scheduler-overhea...