5 ms·

Systems programming in "scripting" languages

14 points by adamsmith 17y ago

jey 17y ago
Yup, and now with things like Cython you hardly have to make any distinction. It lets you write in Python by default, but you can drop down to a special Python dialect that compiles to C whenever needed. http://www.cython.org http://www.cython.org
timwiseman 17y ago
Good point. There are certainly cases where performance is paramount and it is hard to beat C in areas of pure performance, but those areas are becoming ever more rare. In fact, in some cases you can get better performance out of a language built for multithreading like Haskell or Lisp than out of anything but the most highly optimized, multithreaded C code.
- gloob 17y ago
  Perhaps I'm misunderstanding your definition of "built for multithreading", but in my (admittedly limited) experience, Lisp isn't it. To my knowledge, McCarthy's Recursive Functions paper, the Scheme standard, and the Common Lisp standard all fail to provide for concurrency (although something like it could be implemented with coroutines, I don't have the impression that's what you're referring to). That said, it's entirely possible I'm mistaken. I can't comment on Haskell, having no knowledge at all of it. Erlang is rather more in the region of what your comment seems to be getting at, from my perspective.
  - fpgeek 17y ago
    Haskell's purely functional core and ability to localize computational side effects via monads make it a natural fit for multithreaded programming. Pure (i.e. side-effect free) code can be semi-automatically parallelized (with the programmer giving guidance about where profitable parallelization points are, but just that). Shared-state concurrent programming can be made easier (and, most importantly, more modular) with Haskell's Software Transactional Memory support.
    - signa11 17y ago
      obligatory "Software Transactional Memory: Why Is It Only a Research Toy?" link: http://queue.acm.org/detail.cfm?id=1454466 http://queue.acm.org/detail.cfm?id=1454466.
      - fpgeek 17y ago
        Since I work for a startup ( http://www.bluespec.com http://www.bluespec.com ) that provides tools for developing hardware transactionally, let's just say that I think the story of hardware support for STM hasn't been written yet.
  - timwiseman 17y ago
    fpgeek explained what I was talking about in Haskell far better than I can. Perhaps I mispoke, being the very earliest of beginners in Lisp. I had thought that in realms of compiler controlled concurrency for certain types of functional programs it was very similar to Haskell in that respect. I apologize if I erred.
    - deleted 17y ago
      [deleted]
      - fpgeek 17y ago
        Well, to be completely fair, unlike some other languages, Lisp code can be reasonably written in a purely functional style (or a mostly-pure style that a compiler discovers and takes advantage of with something similar to soft-typing). However, what I hear anecdotally is that Lisp code isn't typically written that way. As you say, Clojure is the closest with its immutable local variables and its controlled mutation / concurrency models (of which STM is one).
kmavm 17y ago
This is an appealing train of thought, but it turns out to be very hard to make this pay off, at least in all the concrete situations I've been able to consider. The reason is subtle: "I/O bound" workloads often only become so when they started out "memory bound." If you've got more memory, you can often buffer or cache your way out of being I/O bound. And squeezing the last few percent out of machine memory is a weakness of the productivity languages' runtimes, because it runs counter to the natural goals of a garbage collector. Consider, say, a database. In its equilibrium state, serving a read-biased workload, the DBMS's ability to use every cotton-picking byte of RAM to cache precious blocks will powerfully determine performance. The workload only really becomes "I/O bound" once you run out of RAM, and a runtime that exacts an X% tax on RAM will become I/O bound that much more quickly. This seems to be an inherent trade-off in the "automatic memory management" paradigm; even Sun's JVM is not immune to this. It's also the right trade-off for a huge majority of code. But it's one of the reasons systems programs in minority languages are driving into a stiff wind.
- jey 17y ago
  That's a very small slice of programs. Most systems software doesn't have a DBMS-like workload, e.g. network servers are I/O bound but not memory bound.

14 points by adamsmith 17y ago

jey 17y ago

Yup, and now with things like Cython you hardly have to make any distinction. It lets you write in Python by default, but you can drop down to a special Python dialect that compiles to C whenever needed. http://www.cython.org http://www.cython.org

timwiseman 17y ago

Good point. There are certainly cases where performance is paramount and it is hard to beat C in areas of pure performance, but those areas are becoming ever more rare. In fact, in some cases you can get better performance out of a language built for multithreading like Haskell or Lisp than out of anything but the most highly optimized, multithreaded C code.

gloob 17y ago
Perhaps I'm misunderstanding your definition of "built for multithreading", but in my (admittedly limited) experience, Lisp isn't it. To my knowledge, McCarthy's Recursive Functions paper, the Scheme standard, and the Common Lisp standard all fail to provide for concurrency (although something like it could be implemented with coroutines, I don't have the impression that's what you're referring to). That said, it's entirely possible I'm mistaken. I can't comment on Haskell, having no knowledge at all of it. Erlang is rather more in the region of what your comment seems to be getting at, from my perspective.
- fpgeek 17y ago
  Haskell's purely functional core and ability to localize computational side effects via monads make it a natural fit for multithreaded programming. Pure (i.e. side-effect free) code can be semi-automatically parallelized (with the programmer giving guidance about where profitable parallelization points are, but just that). Shared-state concurrent programming can be made easier (and, most importantly, more modular) with Haskell's Software Transactional Memory support.
  - signa11 17y ago
    obligatory "Software Transactional Memory: Why Is It Only a Research Toy?" link: http://queue.acm.org/detail.cfm?id=1454466 http://queue.acm.org/detail.cfm?id=1454466.
    - fpgeek 17y ago
      Since I work for a startup ( http://www.bluespec.com http://www.bluespec.com ) that provides tools for developing hardware transactionally, let's just say that I think the story of hardware support for STM hasn't been written yet.
- timwiseman 17y ago
  fpgeek explained what I was talking about in Haskell far better than I can. Perhaps I mispoke, being the very earliest of beginners in Lisp. I had thought that in realms of compiler controlled concurrency for certain types of functional programs it was very similar to Haskell in that respect. I apologize if I erred.
  - deleted 17y ago
    [deleted]
    - fpgeek 17y ago
      Well, to be completely fair, unlike some other languages, Lisp code can be reasonably written in a purely functional style (or a mostly-pure style that a compiler discovers and takes advantage of with something similar to soft-typing). However, what I hear anecdotally is that Lisp code isn't typically written that way. As you say, Clojure is the closest with its immutable local variables and its controlled mutation / concurrency models (of which STM is one).

kmavm 17y ago

This is an appealing train of thought, but it turns out to be very hard to make this pay off, at least in all the concrete situations I've been able to consider. The reason is subtle: "I/O bound" workloads often only become so when they started out "memory bound." If you've got more memory, you can often buffer or cache your way out of being I/O bound. And squeezing the last few percent out of machine memory is a weakness of the productivity languages' runtimes, because it runs counter to the natural goals of a garbage collector. Consider, say, a database. In its equilibrium state, serving a read-biased workload, the DBMS's ability to use every cotton-picking byte of RAM to cache precious blocks will powerfully determine performance. The workload only really becomes "I/O bound" once you run out of RAM, and a runtime that exacts an X% tax on RAM will become I/O bound that much more quickly. This seems to be an inherent trade-off in the "automatic memory management" paradigm; even Sun's JVM is not immune to this. It's also the right trade-off for a huge majority of code. But it's one of the reasons systems programs in minority languages are driving into a stiff wind.

jey 17y ago
That's a very small slice of programs. Most systems software doesn't have a DBMS-like workload, e.g. network servers are I/O bound but not memory bound.