5 ms·

Erlang isn't dying but not many people care for it in capital market. Looks at the traits of Erlang, it's designed for telecom environment which is almost oppos

by aynyc 4y ago

Erlang isn't dying but not many people care for it in capital market. Looks at the traits of Erlang, it's designed for telecom environment which is almost opposite of capital market. 1. Fault tolerance is almost all hardware driven. Capital market participants spent billions of dollar building reliable private networks. 2. Distributed in trading doesn't mean what FAANG are doing. Trading systems for example usually run 2 hot-warm systems with real-time replication commits, that's about it. They don't run horizontally scaled systems because network time would kill their performance. In my experience, I haven't heard anyone asking for hot swap code and high availability (outside of trading hours). Are there shops out there that run Erlang? Sure, but most companies don't care about Erlang simply because its value isn't better than Java/C/C++.

jonhohle 4y ago
Funny anecdote about one of the FAANGs: before the idea of service meshes and Consul and Envoy a similar idea was being rolled out to transparently proxy, encrypt, and authenticate traffic between services. The initial implementation was on erlang and didn’t cause any operational issues for our team, from what I remember. The project was moved to another team who decided for version 2 to rewrite in Java (high made sense from an integration perspective). Unfortunately, they hadn’t tested at scale along with whatever it was servicing and as it was being rolled out it quickly ballooned in memory causing systems to swap and fall over. They eventually got it working, but the rollout was less than ideal and unfortunately an erlang project that affected a significant amount of traffic without notice was deprecated.
- weatherlight 4y ago
  Everyone always talks/jokes about, "Rewrite it in Rust," think about all the business applications over the years that had been rewritten to Java, just because. Usually under the guise of, "Kids coming out of college will have an easier time wrapping their heads around Java," than some other lang. In my time as an engineer, I've seen quite a few Scala and Clojure apps, get rewritten in this manner. This is all antidotal, of course.
  - EdwardDiego 4y ago
    Porting Clojure and Scala to Java can also have performance advantages, but yeah, Java was an industrial language before Golang made it cool.
  - newswasboring 4y ago
    I sometimes wonder if this logic makes sense. I'm currently leading a fully inexperienced team where my main challenge isn't delivering products but qualified people. I have to make them learn so many company specific and codebase specific things that feels like learning a new language would be a least of it. This is of course colored by my language learning loving lens, so I can't be sure about this.
    - somekyle2 4y ago
      I've been on teams responsible for components written in more niche, powerful languages by previous employees who were deep in that particular language community. The author was really smart, the code was good. It was a big headache. First off, different build setup, different runtime, couldn't use most of our standard libs, so it was an island. It being in a language most people didn't know well meant it mostly only got updated as necessary; we certainly had people who could read the code and make tweaks with some comfort, but it was mostly just tweaks because nobody had the depth to really engage with the high level design idioms. It being a niche lang/community, it moved fast, so after a year or so our tool was apparently using deprecated practices; in our supported languages, teams did large-scale fixup across repos for new versions, but this being a special lang/toolchain, it was on us. Eventually, I believe it got rewritten, somewhat simpler, in one of our supported languages, and it was fine. Maybe this is a success story: someone made a useful tool in a weird language and we used it successfully for at least a year. Maybe it wouldn't have been made or would've been worse or more expensive if they'd been forced to write it in a supported language. Maybe it would've needed to be rewritten anyway. My view is a simple one; as one of the leads responsible for it, it was a special case bit of code, something people tried to avoid touching. Nothing about it required a niche language technically, so the added difficulty of working around that was friction and risk. It wasn't that people couldn't learn the language; they could, we had lang PhDs. It's just better if you can keep people where they have existing expertise, and spend your learning points on solving problems users care about.
- hinkley 4y ago
  I’m running into this concern at work. If you have one main app or service constellation, then you can’t make any big changes like these because they have to scale from test benchtop to production with few steps in between and there’s just no accurate way to know what you’re going to see until you try. Traffic shaping only does so much. I’ve been moving our tools and sidecars and lesser services to do more reuse, not for the code and effort duplication reasons, but so these technologies get more burnin time before they have to do it live. We have one service that is particularly good for this because it doesn’t see a lot of traffic on average, but it bursts up to production requests levels during a certain workflow that is less time sensitive than live traffic but still gets scrutiny because it blocks a run book until it’s done. Parts of that run book can be run as often as you like if you stop before the steps that alter production state, so you can run it a few times to take the median time and see if you’ve improved or regressed. We turn things on there and if they haven’t caught on fire we feel a lot better about flipping it on for user facing traffic.
innocentoldguy 4y ago
The company I currently work for switched from Java to Elixir/Erlang because developer productivity was so much higher. Being able to turn around features and fixes in a fraction of the time has been quite valuable for us.
- freedomben 4y ago
  I would love to hear more about this if you're willing! Java shops (In my experience) have been the most unlikely to switch shops there are. Some questions: What pain were you feeling? Who came up with the idea/proposal to switch and how did they deliver it? Was it hard to get people on board? How difficult was it for people to learn Elixir? Did people take to it quickly and/or were there people who had a hard time? How long did the transition take to complete?
  - innocentoldguy 4y ago
    Hello, freedomben. I'm back from my business trip, so here are my answers to your questions, as promised. > What pain were you feeling? The turn-around time for refactoring, adding new features, changing API endpoints, etc. was taking too long in Java. > Who came up with the idea/proposal to switch and how did they deliver it? The engineering team, staff engineers, and the VP of Engineering recognized the productivity issues and decided to experiment with other languages to see if a switch would result in quicker turn-around times. First, we came up with a long list of objective metrics we could use to test various languages against each other. Next, we made a list of languages we wanted to test (e.g., Java, Elixir, Go, Clojure, etc.). Then, we defined a test program, including a REST API, that we would build to compare the languages. Finally, we broke up into small teams, assigned each team a language to test, and spent about a week writing our test program in a variety of languages. Once we were done, we had a retrospective and scored our experiences with each language we tested using the list of objective metrics we had created. In the end, Elixir won. > Was it hard to get people on board? It wasn't hard to convince the team we should switch to Elixir. The team's own test data revealed that. > How difficult was it for people to learn Elixir? It wasn't too bad. Most of our engineers grasped it quickly. I think two factors made the switch successful: 1. Two of us already had professional Elixir experience and could mentor other team members. 2. The company purchased team licenses for the Pragmatic Studio's Elixir courses, which were invaluable. > Did people take to it quickly and/or were there people who had a hard time? Most of our engineers were able to pick up Elixir quickly. Some of the backend data engineers took longer though, just because they didn't have to work with it every day. > How long did the transition take to complete? We completed the rewrites of all our products in less than a year. I hope this was useful. Let me know if you have any other questions.
toast0 4y ago
> 2. Distributed in trading doesn't mean what FAANG are doing. Trading systems for example usually run 2 hot-warm systems with real-time replication commits, that's about it. They don't run horizontally scaled systems because network time would kill their performance. That's pretty close to Ericsson's use case for dist; they built it to run redundant computers in the same chassis for telephone switches. It just also happens to work fairly well at much larger cluster sizes too.

by aynyc 4y ago

jonhohle 4y ago

Funny anecdote about one of the FAANGs: before the idea of service meshes and Consul and Envoy a similar idea was being rolled out to transparently proxy, encrypt, and authenticate traffic between services. The initial implementation was on erlang and didn’t cause any operational issues for our team, from what I remember. The project was moved to another team who decided for version 2 to rewrite in Java (high made sense from an integration perspective). Unfortunately, they hadn’t tested at scale along with whatever it was servicing and as it was being rolled out it quickly ballooned in memory causing systems to swap and fall over. They eventually got it working, but the rollout was less than ideal and unfortunately an erlang project that affected a significant amount of traffic without notice was deprecated.

weatherlight 4y ago
Everyone always talks/jokes about, "Rewrite it in Rust," think about all the business applications over the years that had been rewritten to Java, just because. Usually under the guise of, "Kids coming out of college will have an easier time wrapping their heads around Java," than some other lang. In my time as an engineer, I've seen quite a few Scala and Clojure apps, get rewritten in this manner. This is all antidotal, of course.
- EdwardDiego 4y ago
  Porting Clojure and Scala to Java can also have performance advantages, but yeah, Java was an industrial language before Golang made it cool.
- newswasboring 4y ago
  I sometimes wonder if this logic makes sense. I'm currently leading a fully inexperienced team where my main challenge isn't delivering products but qualified people. I have to make them learn so many company specific and codebase specific things that feels like learning a new language would be a least of it. This is of course colored by my language learning loving lens, so I can't be sure about this.
  - somekyle2 4y ago
    I've been on teams responsible for components written in more niche, powerful languages by previous employees who were deep in that particular language community. The author was really smart, the code was good. It was a big headache. First off, different build setup, different runtime, couldn't use most of our standard libs, so it was an island. It being in a language most people didn't know well meant it mostly only got updated as necessary; we certainly had people who could read the code and make tweaks with some comfort, but it was mostly just tweaks because nobody had the depth to really engage with the high level design idioms. It being a niche lang/community, it moved fast, so after a year or so our tool was apparently using deprecated practices; in our supported languages, teams did large-scale fixup across repos for new versions, but this being a special lang/toolchain, it was on us. Eventually, I believe it got rewritten, somewhat simpler, in one of our supported languages, and it was fine. Maybe this is a success story: someone made a useful tool in a weird language and we used it successfully for at least a year. Maybe it wouldn't have been made or would've been worse or more expensive if they'd been forced to write it in a supported language. Maybe it would've needed to be rewritten anyway. My view is a simple one; as one of the leads responsible for it, it was a special case bit of code, something people tried to avoid touching. Nothing about it required a niche language technically, so the added difficulty of working around that was friction and risk. It wasn't that people couldn't learn the language; they could, we had lang PhDs. It's just better if you can keep people where they have existing expertise, and spend your learning points on solving problems users care about.
hinkley 4y ago
I’m running into this concern at work. If you have one main app or service constellation, then you can’t make any big changes like these because they have to scale from test benchtop to production with few steps in between and there’s just no accurate way to know what you’re going to see until you try. Traffic shaping only does so much. I’ve been moving our tools and sidecars and lesser services to do more reuse, not for the code and effort duplication reasons, but so these technologies get more burnin time before they have to do it live. We have one service that is particularly good for this because it doesn’t see a lot of traffic on average, but it bursts up to production requests levels during a certain workflow that is less time sensitive than live traffic but still gets scrutiny because it blocks a run book until it’s done. Parts of that run book can be run as often as you like if you stop before the steps that alter production state, so you can run it a few times to take the median time and see if you’ve improved or regressed. We turn things on there and if they haven’t caught on fire we feel a lot better about flipping it on for user facing traffic.

innocentoldguy 4y ago

The company I currently work for switched from Java to Elixir/Erlang because developer productivity was so much higher. Being able to turn around features and fixes in a fraction of the time has been quite valuable for us.

freedomben 4y ago
I would love to hear more about this if you're willing! Java shops (In my experience) have been the most unlikely to switch shops there are. Some questions: What pain were you feeling? Who came up with the idea/proposal to switch and how did they deliver it? Was it hard to get people on board? How difficult was it for people to learn Elixir? Did people take to it quickly and/or were there people who had a hard time? How long did the transition take to complete?
- innocentoldguy 4y ago
  Hello, freedomben. I'm back from my business trip, so here are my answers to your questions, as promised. > What pain were you feeling? The turn-around time for refactoring, adding new features, changing API endpoints, etc. was taking too long in Java. > Who came up with the idea/proposal to switch and how did they deliver it? The engineering team, staff engineers, and the VP of Engineering recognized the productivity issues and decided to experiment with other languages to see if a switch would result in quicker turn-around times. First, we came up with a long list of objective metrics we could use to test various languages against each other. Next, we made a list of languages we wanted to test (e.g., Java, Elixir, Go, Clojure, etc.). Then, we defined a test program, including a REST API, that we would build to compare the languages. Finally, we broke up into small teams, assigned each team a language to test, and spent about a week writing our test program in a variety of languages. Once we were done, we had a retrospective and scored our experiences with each language we tested using the list of objective metrics we had created. In the end, Elixir won. > Was it hard to get people on board? It wasn't hard to convince the team we should switch to Elixir. The team's own test data revealed that. > How difficult was it for people to learn Elixir? It wasn't too bad. Most of our engineers grasped it quickly. I think two factors made the switch successful: 1. Two of us already had professional Elixir experience and could mentor other team members. 2. The company purchased team licenses for the Pragmatic Studio's Elixir courses, which were invaluable. > Did people take to it quickly and/or were there people who had a hard time? Most of our engineers were able to pick up Elixir quickly. Some of the backend data engineers took longer though, just because they didn't have to work with it every day. > How long did the transition take to complete? We completed the rewrites of all our products in less than a year. I hope this was useful. Let me know if you have any other questions.

toast0 4y ago

> 2. Distributed in trading doesn't mean what FAANG are doing. Trading systems for example usually run 2 hot-warm systems with real-time replication commits, that's about it. They don't run horizontally scaled systems because network time would kill their performance. That's pretty close to Ericsson's use case for dist; they built it to run redundant computers in the same chassis for telephone switches. It just also happens to work fairly well at much larger cluster sizes too.