9 ms·

I don't understand how this is about systems programming. Would someone care to clarify?

by readme 13y ago

dbrower 13y ago
I take it to imply the 'system' they are programming is the distributed/concurrent complex they are running. To other people, 'system' is below that, and this is application programming, or "middleware" It seems like at any level, you write the system, and applications are above. -dB
- seanmcdirmid 13y ago
  What else does the systems community do these days besides concurrent distributed systems?
  - weland 13y ago
    Power management and flash/SSD-aware filesystems are also quite the rage.
    - seanmcdirmid 13y ago
      Ya: distributed computing, storage, and power are hot. Where did all the kernel research go?
      - weland 13y ago
        Much of it happens with different objectives, or at different levels, in the context of computing, storage and power management. See Akaros for example: http://akaros.cs.berkeley.edu/akaros-web/overview.php http://akaros.cs.berkeley.edu/akaros-web/overview.php .
snarfy 13y ago
I'm also confused. I have a book on my bookshelf titled "Systems Programming", and it's about writing kernels and runtimes in C. Can someone explain to me when the definition went from kernel level to server duct tape? I'm guessing it's about the same time the term 'full stack developer' started showing up.
amirouche 13y ago
This is about system programming where system must be understood as a composition of several networked systems. It's different from a single-node system like pure kernel software e.g. hdd drivers, among other things it's higher level. For instance you have to deal with the Two Generals' Problem. They are "expert systems" which relates more to kernel dev and where kernel knowledge is required, like databases. Expert systems even distributed expert systems are meant to solve well identified problems, they are the CPU, GPU and HDD of the global system. Usually the higher in the stack you go, the more specification of the logic that must be implemented is loose: - less standardized - less identified Everywhere Logic is a moving target, like so called "business logic", the way you express the logic is very significant. Otherwise, said "code design" and code maintainability is a significant "business asset" as it allows to react quickly to change. All the thing that are described in the slides are the features of the Scala language and platform that twitter think allows to write code "that is well designed upon which you can react quickly". Mind the fact there is nothing specific in what was describe only to Scala as language or platform except maybe that it's build upon the JVM and that it's static typed. Code design, code look'n'feel especially in context that involves concurrency is not a new or specific to distributed systems. The most important point in the 11th slide http://monkey.org/~marius/talks/twittersystems/#11 http://monkey.org/~marius/talks/twittersystems/#11 that introduce what is following (try to guess) is the second bullet: "Very expressive". Also "The point is not the language. The ideas and techniques matter. Scala happens to be our language.". Other people seems a bit worried about the lake of interest in system programming as it is understood before "horizontal scaling" era. Nothing could be more wrong, "single node systems" are an active area of research and engineering: - AOT & JIT compilation - Erlang VM (aka. BEAM) style: Termite VM, stackless - OS: Plan 9 - languages: Scala, Julia... Specifically in (high frequency) finance and research they can deep dive in "single node systems" to optimize things out because of higher reliability, latency needs, everything is bigger. Examples of such things: - monetdb, the first column based database - YAMI4, dubbed messaging solution for distributed systems with a realtime bias - developped at CERN. Such softwares are not so famous or easy to know about outside of the field, nonetheless they exists. Don't hesitate to complete or correct that list or anything else I've written. edit: added a note about finance, scientific research & realtime bias.
- 13y ago

I don't understand how this is about systems programming. Would someone care to clarify?

by readme 13y ago

I don't understand how this is about systems programming. Would someone care to clarify?

I take it to imply the 'system' they are programming is the distributed/concurrent complex they are running. To other people, 'system' is below that, and this is application programming, or "middleware" It seems like at any level, you write the system, and applications are above. -dB

seanmcdirmid 13y ago
What else does the systems community do these days besides concurrent distributed systems?
- weland 13y ago
  Power management and flash/SSD-aware filesystems are also quite the rage.
  - seanmcdirmid 13y ago
    Ya: distributed computing, storage, and power are hot. Where did all the kernel research go?
    - weland 13y ago
      Much of it happens with different objectives, or at different levels, in the context of computing, storage and power management. See Akaros for example: http://akaros.cs.berkeley.edu/akaros-web/overview.php http://akaros.cs.berkeley.edu/akaros-web/overview.php .

snarfy 13y ago

I'm also confused. I have a book on my bookshelf titled "Systems Programming", and it's about writing kernels and runtimes in C. Can someone explain to me when the definition went from kernel level to server duct tape? I'm guessing it's about the same time the term 'full stack developer' started showing up.

amirouche 13y ago

This is about system programming where system must be understood as a composition of several networked systems. It's different from a single-node system like pure kernel software e.g. hdd drivers, among other things it's higher level. For instance you have to deal with the Two Generals' Problem. They are "expert systems" which relates more to kernel dev and where kernel knowledge is required, like databases. Expert systems even distributed expert systems are meant to solve well identified problems, they are the CPU, GPU and HDD of the global system. Usually the higher in the stack you go, the more specification of the logic that must be implemented is loose: - less standardized - less identified Everywhere Logic is a moving target, like so called "business logic", the way you express the logic is very significant. Otherwise, said "code design" and code maintainability is a significant "business asset" as it allows to react quickly to change. All the thing that are described in the slides are the features of the Scala language and platform that twitter think allows to write code "that is well designed upon which you can react quickly". Mind the fact there is nothing specific in what was describe only to Scala as language or platform except maybe that it's build upon the JVM and that it's static typed. Code design, code look'n'feel especially in context that involves concurrency is not a new or specific to distributed systems. The most important point in the 11th slide http://monkey.org/~marius/talks/twittersystems/#11 http://monkey.org/~marius/talks/twittersystems/#11 that introduce what is following (try to guess) is the second bullet: "Very expressive". Also "The point is not the language. The ideas and techniques matter. Scala happens to be our language.". Other people seems a bit worried about the lake of interest in system programming as it is understood before "horizontal scaling" era. Nothing could be more wrong, "single node systems" are an active area of research and engineering: - AOT & JIT compilation - Erlang VM (aka. BEAM) style: Termite VM, stackless - OS: Plan 9 - languages: Scala, Julia... Specifically in (high frequency) finance and research they can deep dive in "single node systems" to optimize things out because of higher reliability, latency needs, everything is bigger. Examples of such things: - monetdb, the first column based database - YAMI4, dubbed messaging solution for distributed systems with a realtime bias - developped at CERN. Such softwares are not so famous or easy to know about outside of the field, nonetheless they exists. Don't hesitate to complete or correct that list or anything else I've written. edit: added a note about finance, scientific research & realtime bias.

13y ago