2 ms·
This is about system programming where system must be understood as a composition of several networked systems. It's different from a single-node system like p
by amirouche 13y ago
This is about system programming where system must be understood as a composition of several networked systems.
It's different from a single-node system like pure kernel software e.g. hdd drivers, among other things it's higher level.
For instance you have to deal with the Two Generals' Problem. They are "expert systems" which relates more to kernel dev and where kernel knowledge is required, like databases. Expert systems even distributed expert systems are meant to solve well identified problems, they are the CPU, GPU and HDD of the global system.
Usually the higher in the stack you go, the more specification of the logic that must be implemented is loose:
- less standardized
- less identified
Everywhere Logic is a moving target, like so called "business logic", the way you express the logic is very significant. Otherwise, said "code design" and code maintainability is a significant "business asset" as it allows to react quickly to change.
All the thing that are described in the slides are the features of the Scala language and platform that twitter think allows to write code "that is well designed upon which you can react quickly".
Mind the fact there is nothing specific in what was describe only to Scala as language or platform except maybe that it's build upon the JVM and that it's static typed.
Code design, code look'n'feel especially in context that involves concurrency is not a new or specific to distributed systems.
The most important point in the 11th slide http://monkey.org/~marius/talks/twittersystems/#11 http://monkey.org/~marius/talks/twittersystems/#11 that introduce what is following (try to guess) is the second bullet: "Very expressive". Also "The point is not the language. The ideas and techniques matter. Scala happens to be our language.".
Other people seems a bit worried about the lake of interest in system programming as it is understood before "horizontal scaling" era. Nothing could be more wrong, "single node systems" are an active area of research and engineering:
- AOT & JIT compilation
- Erlang VM (aka. BEAM) style: Termite VM, stackless
- OS: Plan 9
- languages: Scala, Julia...
Specifically in (high frequency) finance and research they can deep dive in "single node systems" to optimize things out because of higher reliability, latency needs, everything is bigger. Examples of such things:
- monetdb, the first column based database
- YAMI4, dubbed messaging solution for distributed systems with a realtime bias - developped at CERN.
Such softwares are not so famous or easy to know about outside of the field, nonetheless they exists.
Don't hesitate to complete or correct that list or anything else I've written.
edit: added a note about finance, scientific research & realtime bias.
- amirouche 13y agoalso have a look at The C10M problem http://c10m.robertgraham.com/p/manifesto.html http://c10m.robertgraham.com/p/manifesto.html