7 ms·

How and Why We Switched from Erlang to Python (2011)

135 points by zerr 13y ago

zerr 13y ago
The main dealbreaker for me is 8 bytes (16 bytes on x64) per character in strings - which is a list of integers. Each element contains two integers - one for the character and one for the next pointer. Why Erlang community couldn't came up with something like Haskell's ByteString? I mean, there is a bit-syntax in Erlang, but with a very limited functionality of what you would expect for string handling.
- lame 13y ago
  You can use binaries. In Elixir strings are by default binaries containing UTF-8 codepoints, which are already nicely handled by binary matching and construction.
- ibotty 13y ago
  do you need to do much "rich" string processing in erlang? it certainly is not designed for that usecase. you _can_ do quiet a bit of string processing in erlang's bytestrings (note: not unicode-aware), see e.g. cowboy (which has nice to read sourcecode). as to why erlang does not have a haskell-like ByteString, i guess, it's because haskell has very fast (but potentially unsafe) c-bindings. erlang's ports are too slow to use them for something that small-grained. i'm not sure, why no BIFs were introduced to bring somthing similar, honestly.
  - lobster_johnson 13y ago
    > it certainly is not designed for that usecase. That's a misleading statement, I think; it's like saying Python was not designed for web application development (which is true but also misleading). It's more correct to say that Erlang was designed without much thought given to string processing. Erlang was designed as a fairly general-purpose language, only one with non-encoding-aware strings; the real problem is that it hasn't caught up, even though Unicode has existed since 1991 and has long been incorporated into most languages and most software by now. Instead of adapting, Erlang seems to be stagnating in one area that users are frequently complaining about. In this day and age, I would argue that string processing is quite important for the things that Erlang can, or should, be used for. Anecdotally, when I first tried Erlang I tried to create a naive, parallel log processor which read lines and spawned off lines to a pool of workers. As a newbie I was quickly stumped because Erlang's file I/O is abysmally designed. I eventually gave up the project, and later I found that Tim Bray, also an Erlang noob, had struggled with the exact same issue [1][2]. You would think that this is something Erlang would excel at, but apparently it's not. I have been disappointed with Haskell's string support, too -- it's all over the place (String vs. Text types, the weird Data.Encoding module, horrible regexp library, etc.) -- but at least it's fully Unicode-aware and it has a fast ByteString type. [1] https://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang https://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang [2] https://www.tbray.org/ongoing/When/200x/2007/09/21/Erlang https://www.tbray.org/ongoing/When/200x/2007/09/21/Erlang
    - davidw 13y ago
      I like Erlang a lot, and am currently working on a project that utilizes Chicago Boss, but there's a lot of truth in what you write. Erlang is somewhat strange for a language in that it got put to use in some very important critical systems before it got big. Things like Ruby, Tcl and Python slowly got popular and changed along the way. Doing an Erlang 2.0 that improves on some of the warts is probably not so easy...
      - lobster_johnson 13y ago
        That's a good observation. I don't know much about Erlang's history beyond what is on Wikipedia, but I suspect that the number of people that used Erlang before it was open source was relatively small and limited to the technical community within Ericsson. That small, focused user group explains the language's weird and weirdly antiquated style and feature set, but it doesn't quite explain why the language has not been evolved to become more modern. (Of course, the fact that something as ingenious could come out of such a small community is also very impressive and wonderful.)
      - lobster_johnson 13y ago
        If you are interested in the history of Erlang, this is fascinating: https://docs.google.com/viewer?url=http%3A%2F%2Fwebcem01.cem.itesm.mx%3A8005%2Ferlang%2Fcd%2Fdownloads%2Fhopl_erlang.pdf https://docs.google.com/viewer?url=http%3A%2F%2Fwebcem01.cem...
    - jlouis 13y ago
      Unicode is a big thing in R16 and R17 where more of the system will use the unicode support we do have. Erlang modules will use utf-8 encoding and so on. As for a built-in bstring() type, it is being discussed, but there are other things more important to get into the language, maps for instance. Erlang is rather conservative in the speed with which we add stuff to the language. Erlang has some of the fastest I/O in a runtime. But it is not easy to use correctly. The same goes for string processing. Erlang can be blinding fast at that, but you must understand how to make it fast.
- peerst 13y ago
  Well Erlang does have binaries for quite a while and many use them as high performance strings. So you can also use lists for strings, but why do you if you don't like the characteristics of lists?
- jlouis 13y ago
  We have the equivalent of Haskells (lazy) ByteString in the types binary() and iolist(). We have had that for at least 6 years. Their space efficiency is also on par with Haskells. If you process big amounts of data as a string(), then you are probably doing something wrong in your architecture.
samwilliams 13y ago
Old discussion here[0]. This is the third time this has been submitted. [0] https://news.ycombinator.com/item?id=2852415 https://news.ycombinator.com/item?id=2852415
jonahx 13y ago
Not trying to be mean, but I was honestly confused by a couple things: 1. The server for the company's core product is written in Erlang, but no one at the company is an Erlang expert? 2. The company decides to rewrite the core of its product in python, and gives the job to an intern? I feel like I must be missing something....
- djhworld 13y ago
  Usually in cases like this, the people who wrote the original implementation probably left the company.
- josh2600 13y ago
  This happened on Facebook's chat system too. For some reason, and I know no other language where this is true to quite this degree (except maybe lisp), but Erlang has a tendency for people to believe they understand how to build an app using it when in reality they can do nothing of the sort (at least in Erlang. It's common for folks to 'do it in Erlang' because its seen as different and somewhat elitist. People think of Erlang engineers as 'hardcore'; I shudder to attribute skill to a language rather than a person, but I digress...). In the case of FB, as told to me, a bunch of folks in the valley said "of course we know Erlang!" and built the app but when push came to shove the only hardcore Erlang guys were in Poland and that made working on chat difficult. I'm not sure how much of that is truth, but its a recurring theme in Erlang projects, one that I've personally heard time and again. For some reason, Erlang inspires confidence, perhaps even foolhardiness. Side note: I work at an Erlang shop and our lead engineers speak at Erlang Factory every year. It's a great language and quite powerful but it can be difficult to understand.
  - davidw 13y ago
    Erlang has gotten some hype about how "amazingly scalable" it is. The reality is that, as with anything, it takes work to make it scale, and once things get Really Big, it gets difficult. My sneaking suspicion is that, in some cases, people pick it because they read that it's "web scale" or something like that without really knowing the details, and when things fall apart, they revert to something that they know better. BTW, reference for the FB chat thing being rewritten?
pyvpx 13y ago
they didn't switch from Erlang because of some language or implementation shortfall, but because after two years no one on staff was proficient in the language. all that tells us is there are more persons familiar with python than erlang. film @ 11...
- peerst 13y ago
  If I run a Erlang server and am no professional Erlang developer after 2 years what does that say about me?
  - buster 13y ago
    He said that python is the actual language of choice in the company. That's why they don't have erlang professionals, i guess. Maybe good enough to throw together this API server but not good enough to make is scale and reliable?
  - angersock 13y ago
    That it's time to learn Rails? :D
- alberich 13y ago
  > but because after two years no one on staff was proficient in the language Hm, I wonder if they tried to train someone or if they waited to see if someone would study Erlang at home for free.
melling 13y ago
He just ran Python benchmarks and said that it was good enough. It's rather disappointing not to see performance compared to Erlang.
- Ramone 13y ago
  I consider that to be a bright spot in this post. What would a performance comparison for this very specific app prove? The way I read it, the only goal of the rewrite was a more easily maintainable code-base with performance that was "good enough". It sounds like he was successful in meeting that goal. With that said, comparing Python and Erlang on the basis of performance is a bit like looking for the world's tallest midget isn't it? Performance isn't the point at all, within a certain threshold.
  - mturmon 13y ago
    Yes. The OP was careful and modest in its rationale for why they reimplemented this module.
minimedj 13y ago
At first a was like "Whooot???" Then I read: >>No one on our team is an Erlang expert, and we have had trouble debugging downtime and performance problems.<< Now it is clean. As always: You must know your stuff amd you better be an expert than not.
ksec 13y ago
Why is this getting upvoted? I read the title, ( sounds familiar ), look at domain mixpanel.com, didn't they did these a while ago? Turns out to be same post from 2011. Come on! Stop posting the same thing over and over again.
- jckt 13y ago
  I realised that on HN it's quite acceptable to do this (although normally posts from the past would be prepended/appended with a (YEAR). I believe the reasoning is that not everyone has seen these articles, and as they are more than just news, many of them are worthy of occasional re-discussion over time. For example, this blogpost (https://news.ycombinator.com/item?id=5204324 https://news.ycombinator.com/item?id=5204324) about Julia-lang was posted in 2012 and then in 2013, and a comparison of HN's discussion over time is quite interesting (to me, at least).
- yen223 13y ago
  That's because people like me haven't read the post.
xuming 13y ago
how cheap
rjzzleep 13y ago
This struck me when I was watching a talk about some web company talking about their scaling issues with their erlang stack. You could have replaced erlang with django or rails and it would have made zero difference. People are essentially making the very same mistakes the first startup I worked at 6 years ago doing php was doing. That being said, I really like Erlang, and I can think of a few problem domains where Erlang would indeed be a really good fit. But serving HTML in my opinion is so trivial that all the fancy stuff that Erlang offers matters little in that regard.
arocks 13y ago
Another two months and we should see the headline, "How and Why We Switched from Python to Go."
- freyrs3 13y ago
  Followed by "How and Why We Switched from Go to Erlang" completing the cycle.
  - codygman 13y ago
    LOL, this would be hilarious.
  - voidlogic 13y ago
    That transition (Go->Erlang) doesn't seem likely. Why? My impression this company's primary motivations switching to Python were 1. Maintainability (how maintainable Erlang was for them). and 2. Performance. 1. Maintainability: Go is ALGOL/C-family-esk and it will be much easier for a Java/C#/Python/<insert very common lang> programmer to ramp up on it than Erlang. Go code is sometimes called boring because it tends to be very straight forward and pragmatic- this makes for very maintainable large systems. 2. Performance: Go performs much better than Python or Erlang[1-3]. Computer Language Benchmarks Game: 1: Go vs Python: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=go&lang2=python3&data=u64q http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t... 2: Go vs Erlang: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=go&lang2=erlang&data=u64q http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t... TechEmpower: Web Framework Benchmarks: 3: http://www.techempower.com/benchmarks/ http://www.techempower.com/benchmarks/
    - freyrs3 13y ago
      > Go performs much better than Python or Erlang[1-3]. I'm going to go out on a limb and say that performance is more subtle than can be reduced to a single "is better than" about throughput in HTTP requests or computing mandelbrot fractals in the shootout. All of the languages have a specific area where they shine. Erlang in largely concurrent systems. Python also has a sizable mindshare in the HPC world.

135 points by zerr 13y ago

zerr 13y ago

The main dealbreaker for me is 8 bytes (16 bytes on x64) per character in strings - which is a list of integers. Each element contains two integers - one for the character and one for the next pointer. Why Erlang community couldn't came up with something like Haskell's ByteString? I mean, there is a bit-syntax in Erlang, but with a very limited functionality of what you would expect for string handling.

lame 13y ago
You can use binaries. In Elixir strings are by default binaries containing UTF-8 codepoints, which are already nicely handled by binary matching and construction.
ibotty 13y ago
do you need to do much "rich" string processing in erlang? it certainly is not designed for that usecase. you _can_ do quiet a bit of string processing in erlang's bytestrings (note: not unicode-aware), see e.g. cowboy (which has nice to read sourcecode). as to why erlang does not have a haskell-like ByteString, i guess, it's because haskell has very fast (but potentially unsafe) c-bindings. erlang's ports are too slow to use them for something that small-grained. i'm not sure, why no BIFs were introduced to bring somthing similar, honestly.
- lobster_johnson 13y ago
  > it certainly is not designed for that usecase. That's a misleading statement, I think; it's like saying Python was not designed for web application development (which is true but also misleading). It's more correct to say that Erlang was designed without much thought given to string processing. Erlang was designed as a fairly general-purpose language, only one with non-encoding-aware strings; the real problem is that it hasn't caught up, even though Unicode has existed since 1991 and has long been incorporated into most languages and most software by now. Instead of adapting, Erlang seems to be stagnating in one area that users are frequently complaining about. In this day and age, I would argue that string processing is quite important for the things that Erlang can, or should, be used for. Anecdotally, when I first tried Erlang I tried to create a naive, parallel log processor which read lines and spawned off lines to a pool of workers. As a newbie I was quickly stumped because Erlang's file I/O is abysmally designed. I eventually gave up the project, and later I found that Tim Bray, also an Erlang noob, had struggled with the exact same issue [1][2]. You would think that this is something Erlang would excel at, but apparently it's not. I have been disappointed with Haskell's string support, too -- it's all over the place (String vs. Text types, the weird Data.Encoding module, horrible regexp library, etc.) -- but at least it's fully Unicode-aware and it has a fast ByteString type. [1] https://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang https://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang [2] https://www.tbray.org/ongoing/When/200x/2007/09/21/Erlang https://www.tbray.org/ongoing/When/200x/2007/09/21/Erlang
  - davidw 13y ago
    I like Erlang a lot, and am currently working on a project that utilizes Chicago Boss, but there's a lot of truth in what you write. Erlang is somewhat strange for a language in that it got put to use in some very important critical systems before it got big. Things like Ruby, Tcl and Python slowly got popular and changed along the way. Doing an Erlang 2.0 that improves on some of the warts is probably not so easy...
    - lobster_johnson 13y ago
      That's a good observation. I don't know much about Erlang's history beyond what is on Wikipedia, but I suspect that the number of people that used Erlang before it was open source was relatively small and limited to the technical community within Ericsson. That small, focused user group explains the language's weird and weirdly antiquated style and feature set, but it doesn't quite explain why the language has not been evolved to become more modern. (Of course, the fact that something as ingenious could come out of such a small community is also very impressive and wonderful.)
    - lobster_johnson 13y ago
      If you are interested in the history of Erlang, this is fascinating: https://docs.google.com/viewer?url=http%3A%2F%2Fwebcem01.cem.itesm.mx%3A8005%2Ferlang%2Fcd%2Fdownloads%2Fhopl_erlang.pdf https://docs.google.com/viewer?url=http%3A%2F%2Fwebcem01.cem...
  - jlouis 13y ago
    Unicode is a big thing in R16 and R17 where more of the system will use the unicode support we do have. Erlang modules will use utf-8 encoding and so on. As for a built-in bstring() type, it is being discussed, but there are other things more important to get into the language, maps for instance. Erlang is rather conservative in the speed with which we add stuff to the language. Erlang has some of the fastest I/O in a runtime. But it is not easy to use correctly. The same goes for string processing. Erlang can be blinding fast at that, but you must understand how to make it fast.
peerst 13y ago
Well Erlang does have binaries for quite a while and many use them as high performance strings. So you can also use lists for strings, but why do you if you don't like the characteristics of lists?
jlouis 13y ago
We have the equivalent of Haskells (lazy) ByteString in the types binary() and iolist(). We have had that for at least 6 years. Their space efficiency is also on par with Haskells. If you process big amounts of data as a string(), then you are probably doing something wrong in your architecture.

samwilliams 13y ago

Old discussion here[0]. This is the third time this has been submitted. [0] https://news.ycombinator.com/item?id=2852415 https://news.ycombinator.com/item?id=2852415

jonahx 13y ago

Not trying to be mean, but I was honestly confused by a couple things: 1. The server for the company's core product is written in Erlang, but no one at the company is an Erlang expert? 2. The company decides to rewrite the core of its product in python, and gives the job to an intern? I feel like I must be missing something....

djhworld 13y ago
Usually in cases like this, the people who wrote the original implementation probably left the company.
josh2600 13y ago
This happened on Facebook's chat system too. For some reason, and I know no other language where this is true to quite this degree (except maybe lisp), but Erlang has a tendency for people to believe they understand how to build an app using it when in reality they can do nothing of the sort (at least in Erlang. It's common for folks to 'do it in Erlang' because its seen as different and somewhat elitist. People think of Erlang engineers as 'hardcore'; I shudder to attribute skill to a language rather than a person, but I digress...). In the case of FB, as told to me, a bunch of folks in the valley said "of course we know Erlang!" and built the app but when push came to shove the only hardcore Erlang guys were in Poland and that made working on chat difficult. I'm not sure how much of that is truth, but its a recurring theme in Erlang projects, one that I've personally heard time and again. For some reason, Erlang inspires confidence, perhaps even foolhardiness. Side note: I work at an Erlang shop and our lead engineers speak at Erlang Factory every year. It's a great language and quite powerful but it can be difficult to understand.
- davidw 13y ago
  Erlang has gotten some hype about how "amazingly scalable" it is. The reality is that, as with anything, it takes work to make it scale, and once things get Really Big, it gets difficult. My sneaking suspicion is that, in some cases, people pick it because they read that it's "web scale" or something like that without really knowing the details, and when things fall apart, they revert to something that they know better. BTW, reference for the FB chat thing being rewritten?

pyvpx 13y ago

they didn't switch from Erlang because of some language or implementation shortfall, but because after two years no one on staff was proficient in the language. all that tells us is there are more persons familiar with python than erlang. film @ 11...

peerst 13y ago
If I run a Erlang server and am no professional Erlang developer after 2 years what does that say about me?
- buster 13y ago
  He said that python is the actual language of choice in the company. That's why they don't have erlang professionals, i guess. Maybe good enough to throw together this API server but not good enough to make is scale and reliable?
- angersock 13y ago
  That it's time to learn Rails? :D
alberich 13y ago
> but because after two years no one on staff was proficient in the language Hm, I wonder if they tried to train someone or if they waited to see if someone would study Erlang at home for free.

melling 13y ago

He just ran Python benchmarks and said that it was good enough. It's rather disappointing not to see performance compared to Erlang.

Ramone 13y ago
I consider that to be a bright spot in this post. What would a performance comparison for this very specific app prove? The way I read it, the only goal of the rewrite was a more easily maintainable code-base with performance that was "good enough". It sounds like he was successful in meeting that goal. With that said, comparing Python and Erlang on the basis of performance is a bit like looking for the world's tallest midget isn't it? Performance isn't the point at all, within a certain threshold.
- mturmon 13y ago
  Yes. The OP was careful and modest in its rationale for why they reimplemented this module.

minimedj 13y ago

At first a was like "Whooot???" Then I read: >>No one on our team is an Erlang expert, and we have had trouble debugging downtime and performance problems.<< Now it is clean. As always: You must know your stuff amd you better be an expert than not.

ksec 13y ago

Why is this getting upvoted? I read the title, ( sounds familiar ), look at domain mixpanel.com, didn't they did these a while ago? Turns out to be same post from 2011. Come on! Stop posting the same thing over and over again.

jckt 13y ago
I realised that on HN it's quite acceptable to do this (although normally posts from the past would be prepended/appended with a (YEAR). I believe the reasoning is that not everyone has seen these articles, and as they are more than just news, many of them are worthy of occasional re-discussion over time. For example, this blogpost (https://news.ycombinator.com/item?id=5204324 https://news.ycombinator.com/item?id=5204324) about Julia-lang was posted in 2012 and then in 2013, and a comparison of HN's discussion over time is quite interesting (to me, at least).
yen223 13y ago
That's because people like me haven't read the post.

xuming 13y ago

how cheap

rjzzleep 13y ago

This struck me when I was watching a talk about some web company talking about their scaling issues with their erlang stack. You could have replaced erlang with django or rails and it would have made zero difference. People are essentially making the very same mistakes the first startup I worked at 6 years ago doing php was doing. That being said, I really like Erlang, and I can think of a few problem domains where Erlang would indeed be a really good fit. But serving HTML in my opinion is so trivial that all the fancy stuff that Erlang offers matters little in that regard.

arocks 13y ago

Another two months and we should see the headline, "How and Why We Switched from Python to Go."

freyrs3 13y ago
Followed by "How and Why We Switched from Go to Erlang" completing the cycle.
- codygman 13y ago
  LOL, this would be hilarious.
- voidlogic 13y ago
  That transition (Go->Erlang) doesn't seem likely. Why? My impression this company's primary motivations switching to Python were 1. Maintainability (how maintainable Erlang was for them). and 2. Performance. 1. Maintainability: Go is ALGOL/C-family-esk and it will be much easier for a Java/C#/Python/<insert very common lang> programmer to ramp up on it than Erlang. Go code is sometimes called boring because it tends to be very straight forward and pragmatic- this makes for very maintainable large systems. 2. Performance: Go performs much better than Python or Erlang[1-3]. Computer Language Benchmarks Game: 1: Go vs Python: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=go&lang2=python3&data=u64q http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t... 2: Go vs Erlang: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=go&lang2=erlang&data=u64q http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t... TechEmpower: Web Framework Benchmarks: 3: http://www.techempower.com/benchmarks/ http://www.techempower.com/benchmarks/
  - freyrs3 13y ago
    > Go performs much better than Python or Erlang[1-3]. I'm going to go out on a limb and say that performance is more subtle than can be reduced to a single "is better than" about throughput in HTTP requests or computing mandelbrot fractals in the shootout. All of the languages have a specific area where they shine. Erlang in largely concurrent systems. Python also has a sizable mindshare in the HPC world.