5 ms·

The Erlang Challenge

21 points by tb 19y ago

mechanical_fish 19y ago
I'm guessing that very few people here on news.yc are qualified to compose the Cobol Challenge: the problem for which Cobol is uniquely qualified to provide the elegant solution. Of course, the budget and the legacy code will have to be a very explicit part of that problem. Meanwhile, this Erlang Challenge only tends to emphasize my impression -- which is just a stereotype, really -- that Erlang is certainly not the hundred-year language: it's a special-purpose language that is brilliant at a narrow class of problem. I'm convinced that, if I have some process that I want to spread across half a million processor cores, I should ask my company's Erlang guru to work on it (because, come on, if you can afford half a million processor cores you can afford at least one Erlang guru). What I'm not yet convinced of is that a world with N programmers who work in general-purpose languages needs more than (0.005)N Erlang gurus. The challenge of the Erlang advocate is not to convince me, over and over, that Erlang wins its class; the challenge is to convince me that Erlang's class of problem is so important to my life that I should study Erlang rather than vascular surgery, or television repair, or other obscure technical skills that I don't know that much about.
- lg 19y ago
  Well the PS3 has 9 cores, or something like that. Don't you think that says something about the computers we'll all be using, and by extension, the parallel-processing capabilities we'll have to take advantage of in the future? Personally, I'd rather not learn Erlang either, but I hope that this sort of easy parallel processing becomes one of those popular features that every language will implement sooner or later. Hm, I wonder if it should be part of Arc's core...
  - zach 19y ago
    The "eventually everything will be massively parallel" argument is not very convincing. The Be guys made this argument 15 years ago and we're only now getting to where they were back then (2/4 processor cores). Consumers aren't that excited by more cores, and developers still don't know what to do with them. So there is massive investment in advancing the status quo. Sony thought they could impose the PS3's multiprocessing regime on game developers through market force. But developers found the programming overhead unmanageable, which led to a less-than-compelling lineup, which failed to interest consumers, which means that now developers just work on the Xbox 360 version and port to PS3. I was offered a job porting an existing game engine to the PS3 under the gun (the game had to ship in six months) and I would not touch it with a ten-foot pole.
    - albertcardona 19y ago
      "and developers still don't know what to do with them." Excuse me, at least in the imaging processing community we know very well what to do with all those extra cores. Particularly when processing 1 Gb images, and/or a couple hundred thousand 2048x2048 16-bit images.
- iamwil 19y ago
  The follow-up question then would be how many existing problems can be converted to the type that Erlang can solve well? And how many problems previously impractical are now practical to solve? So if it ends up to be a bigger class than you thought, you may well be limiting the number problems that you can solve easily and practically out of the ones that will be important in the future. I think it's important because it allows a program to easier to scale out rather than scale up. We'd want speedups in this way because CPUs are becoming multi-cores, and to take advantage of them one will have to write some type of parallel program, since it's been difficult to fully automate parallelizing serial programs. In addition, parallelized systems (not just erlang) can be more fault tolerant and can fail more gracefully (or hobble along, if you'd like to call it that). Sensor networks are one example. Instead of a single radar to detect the environment, you many sensors out there, and they network themselves and report what they see/hear. If some of them gets destroyed that's ok, because the system's still functioning with less sensors. So what concrete examples of problems fall into this class Erlang is good at? The obvious ones are the embarrassingly parallelized algorithms, like genetic algorithms, neural networks, 3d rendering, and if I'm not mistaken, FFTs. Indexing web pages is another. That said, there are problems that are inherently serial. It takes 9 months to begat a baby, no matter how many pregnant women you have. Ruby and Haxe language writers are both implementing the actor model like Erlang, if that's any indication of how important they think it is. While I don't think Erlang will be the 100 year language, the ideas by which it's a poster child will reverberate in the descendant languages for a long time to come.
  - randomhack 19y ago
    well I have never understood the "use erlang because CPUs are going multicore" argument. See if I write a program in C (which for some mysterious reason doesnt use threads) and run it on a single core and it runs in 1 sec. If an equivalent erlang program runs in 10 seconds on 1 core, then I will need at least 10 cores before it beats C. That is assuming 100% of my application/algorithm is parallel. While erlang is a nice language, saying that it runs faster on multicores is just a lame excuse.
    - iamwil 19y ago
- yariv 19y ago
  Just because Erlang is very good at solving a certain class of problems, doesn't mean it's inferior at solving general computing problems. The Erlang challenge highlights what Erlang is good at, but it's not meant to convince you that Erlang is good at everything. I've never claimed that this challenge proves you should use Erlang to solve any problem that comes your way.
  - lg 19y ago
    You said it. I checked out the Erlang tutorial just now, and apparently you can't use user-defined functions in the test-expression of a conditional, to prevent people from putting side-effecty functions in there? Now maybe there's some good reason for that, that has to do with the mysteries of handling concurrency or something, but it gives me the prima-facie feeling of java for the fp crowd.
    - tb 19y ago
      Mostly the reason for that is to allow the compiler to optimise the order of evaluation of the conditions. If you could put functions with side-effects in there then it would be impossible to do that since the compiler would have to guarantee some kind of consistent evaluation order. There have been proposals for some way to mark user-defined functions as side-effect free and thus allow them to be used in guards. None has had acceptance from the core Erlang developers yet. In any case this is not usually too much of a problem. The Erlang idiom is to use 'case' instead of 'if' in most cases (no pun intended) and where a calculated value is needed then just assign it to a variable beforehand - not quite as pretty I agree but no big deal.
      - lg 19y ago
        So my small problem sounds like a bigger problem. Why won't Erlang let me specify the order of evaluation? You know, like, left to right?
        bayareaguy 19y ago
        Dumb question: Erlang computations are supposed to be side-effect free right? If they are, why does order of evaluation matter?
yariv 19y ago
I must say that I am taken aback by the title you used to link to my article. I did not mean it as a snide response to the Arc challenge. I thought the Arc challenge was pretty cool, and I replied to it with an Erlang implementation -- you can see it here http://arclanguage.org/item?id=722 http://arclanguage.org/item?id=722. If it come across as snide, then I didn't communicate it as I had intended. The reason I said it's in the "spirit" of the Arc challenge is that the Arc challenge highlights what Arc is good at, and the Erlang challenge highlights what Erlang is good at.
- neilk 19y ago
  Interesting. I thought you were making a sly joke too. I guess this just proves people read their biases into whatever they encounter. Like many, I thought the Arc challenge amounted to a test of how much a language was like Arc (guess what, Arc wins). So I assumed your post had to be satire. And -- maybe the requirement of spawning a million threads doesn't seem like a cruel joke to you, but it is to anyone who has to work in a language other than Erlang.
  - yariv 19y ago
    Apparently, I should have been more sensitive to the way people would interpret this challenge and picked my words differently. A few other languages, including Scala and Stackless Python, have lightweight concurrency capabilities. AFAIK those languages do allow you to spawn millions of processes (or actors), but they don't handle garbage collection, scheduling, IO, distributed programming and native code interfacing as well as Erlang does.
    - deleted 19y ago
      [deleted]
    - neilk 19y ago
      I don't know if there's anything different you should have done. If you discuss two different computer languages in any piece of writing on the internet, a large majority of the readers will assume you are advocating one over the other. I've seen this so many times already. I'm just sad that I fell into the same trap.

21 points by tb 19y ago

I'm guessing that very few people here on news.yc are qualified to compose the Cobol Challenge: the problem for which Cobol is uniquely qualified to provide the elegant solution. Of course, the budget and the legacy code will have to be a very explicit part of that problem. Meanwhile, this Erlang Challenge only tends to emphasize my impression -- which is just a stereotype, really -- that Erlang is certainly not the hundred-year language: it's a special-purpose language that is brilliant at a narrow class of problem. I'm convinced that, if I have some process that I want to spread across half a million processor cores, I should ask my company's Erlang guru to work on it (because, come on, if you can afford half a million processor cores you can afford at least one Erlang guru). What I'm not yet convinced of is that a world with N programmers who work in general-purpose languages needs more than (0.005)N Erlang gurus. The challenge of the Erlang advocate is not to convince me, over and over, that Erlang wins its class; the challenge is to convince me that Erlang's class of problem is so important to my life that I should study Erlang rather than vascular surgery, or television repair, or other obscure technical skills that I don't know that much about.

lg 19y ago
Well the PS3 has 9 cores, or something like that. Don't you think that says something about the computers we'll all be using, and by extension, the parallel-processing capabilities we'll have to take advantage of in the future? Personally, I'd rather not learn Erlang either, but I hope that this sort of easy parallel processing becomes one of those popular features that every language will implement sooner or later. Hm, I wonder if it should be part of Arc's core...
- zach 19y ago
  The "eventually everything will be massively parallel" argument is not very convincing. The Be guys made this argument 15 years ago and we're only now getting to where they were back then (2/4 processor cores). Consumers aren't that excited by more cores, and developers still don't know what to do with them. So there is massive investment in advancing the status quo. Sony thought they could impose the PS3's multiprocessing regime on game developers through market force. But developers found the programming overhead unmanageable, which led to a less-than-compelling lineup, which failed to interest consumers, which means that now developers just work on the Xbox 360 version and port to PS3. I was offered a job porting an existing game engine to the PS3 under the gun (the game had to ship in six months) and I would not touch it with a ten-foot pole.
  - albertcardona 19y ago
    "and developers still don't know what to do with them." Excuse me, at least in the imaging processing community we know very well what to do with all those extra cores. Particularly when processing 1 Gb images, and/or a couple hundred thousand 2048x2048 16-bit images.
iamwil 19y ago
The follow-up question then would be how many existing problems can be converted to the type that Erlang can solve well? And how many problems previously impractical are now practical to solve? So if it ends up to be a bigger class than you thought, you may well be limiting the number problems that you can solve easily and practically out of the ones that will be important in the future. I think it's important because it allows a program to easier to scale out rather than scale up. We'd want speedups in this way because CPUs are becoming multi-cores, and to take advantage of them one will have to write some type of parallel program, since it's been difficult to fully automate parallelizing serial programs. In addition, parallelized systems (not just erlang) can be more fault tolerant and can fail more gracefully (or hobble along, if you'd like to call it that). Sensor networks are one example. Instead of a single radar to detect the environment, you many sensors out there, and they network themselves and report what they see/hear. If some of them gets destroyed that's ok, because the system's still functioning with less sensors. So what concrete examples of problems fall into this class Erlang is good at? The obvious ones are the embarrassingly parallelized algorithms, like genetic algorithms, neural networks, 3d rendering, and if I'm not mistaken, FFTs. Indexing web pages is another. That said, there are problems that are inherently serial. It takes 9 months to begat a baby, no matter how many pregnant women you have. Ruby and Haxe language writers are both implementing the actor model like Erlang, if that's any indication of how important they think it is. While I don't think Erlang will be the 100 year language, the ideas by which it's a poster child will reverberate in the descendant languages for a long time to come.
- randomhack 19y ago
  well I have never understood the "use erlang because CPUs are going multicore" argument. See if I write a program in C (which for some mysterious reason doesnt use threads) and run it on a single core and it runs in 1 sec. If an equivalent erlang program runs in 10 seconds on 1 core, then I will need at least 10 cores before it beats C. That is assuming 100% of my application/algorithm is parallel. While erlang is a nice language, saying that it runs faster on multicores is just a lame excuse.
  - iamwil 19y ago
yariv 19y ago
Just because Erlang is very good at solving a certain class of problems, doesn't mean it's inferior at solving general computing problems. The Erlang challenge highlights what Erlang is good at, but it's not meant to convince you that Erlang is good at everything. I've never claimed that this challenge proves you should use Erlang to solve any problem that comes your way.
- lg 19y ago
  You said it. I checked out the Erlang tutorial just now, and apparently you can't use user-defined functions in the test-expression of a conditional, to prevent people from putting side-effecty functions in there? Now maybe there's some good reason for that, that has to do with the mysteries of handling concurrency or something, but it gives me the prima-facie feeling of java for the fp crowd.
  - tb 19y ago
    Mostly the reason for that is to allow the compiler to optimise the order of evaluation of the conditions. If you could put functions with side-effects in there then it would be impossible to do that since the compiler would have to guarantee some kind of consistent evaluation order. There have been proposals for some way to mark user-defined functions as side-effect free and thus allow them to be used in guards. None has had acceptance from the core Erlang developers yet. In any case this is not usually too much of a problem. The Erlang idiom is to use 'case' instead of 'if' in most cases (no pun intended) and where a calculated value is needed then just assign it to a variable beforehand - not quite as pretty I agree but no big deal.
    - lg 19y ago
      So my small problem sounds like a bigger problem. Why won't Erlang let me specify the order of evaluation? You know, like, left to right?
      - bayareaguy 19y ago
        Dumb question: Erlang computations are supposed to be side-effect free right? If they are, why does order of evaluation matter?

yariv 19y ago

I must say that I am taken aback by the title you used to link to my article. I did not mean it as a snide response to the Arc challenge. I thought the Arc challenge was pretty cool, and I replied to it with an Erlang implementation -- you can see it here http://arclanguage.org/item?id=722 http://arclanguage.org/item?id=722. If it come across as snide, then I didn't communicate it as I had intended. The reason I said it's in the "spirit" of the Arc challenge is that the Arc challenge highlights what Arc is good at, and the Erlang challenge highlights what Erlang is good at.

neilk 19y ago
Interesting. I thought you were making a sly joke too. I guess this just proves people read their biases into whatever they encounter. Like many, I thought the Arc challenge amounted to a test of how much a language was like Arc (guess what, Arc wins). So I assumed your post had to be satire. And -- maybe the requirement of spawning a million threads doesn't seem like a cruel joke to you, but it is to anyone who has to work in a language other than Erlang.
- yariv 19y ago
  Apparently, I should have been more sensitive to the way people would interpret this challenge and picked my words differently. A few other languages, including Scala and Stackless Python, have lightweight concurrency capabilities. AFAIK those languages do allow you to spawn millions of processes (or actors), but they don't handle garbage collection, scheduling, IO, distributed programming and native code interfacing as well as Erlang does.
  - deleted 19y ago
    [deleted]
  - neilk 19y ago
    I don't know if there's anything different you should have done. If you discuss two different computer languages in any piece of writing on the internet, a large majority of the readers will assume you are advocating one over the other. I've seen this so many times already. I'm just sad that I fell into the same trap.