5 ms·

I know a company that has built their entire technology stack on some Erlang VM based language. There was really no reason to do so at all - it's an ordinary c

I know a company that has built their entire technology stack on some Erlang VM based language. There was really no reason to do so at all - it's an ordinary company building an ordinary application. But some CTO or dev lead years ago thought it important to use whatever the fancy tech of the day was to build a fairly ordinary business application. But now they have trouble hiring people because there's a tiny pool of people who know that language. And it is years down the track and all their technology is built with whatever that language/framework is - too expensive to easily bail out, but probably necessary anyway. In 2023 you've got to have really good reasons for not using C#, or Java, or Python, or TypeScript for building your web project. Maybe Golang - it's getting some good traction in the DevOps space. If you make decisions about technologies and this latest thing appeals to you - maybe use it for your personal projects. The CEO has employed you to make responsible decisions not play toys.

siwatanejo 3y ago
I hire based on ability/intelligence, not knowledge.
- andrewstuart 3y ago
  >> I hire based on ability/intelligence, not knowledge. Well, yeah that's really your only choice if the experience you need is not easily available. Don't tell me, you are the guy who puts in the not mainstream technology and needs to defend that with justifications such as "when we use (non mainstream technology X) we get super motivated job applicants because anyone with an interest in non mainstream technologies is inherently passionate and interested". That's not a sound hiring strategy.
  - inopinatus 3y ago
    And you are the guy who left Visual Basic, COBOL, Perl, and FORTRAN off their "obviously safe choices for hiring in the future" list, despite them all having once been perceived as such. These inconvenient counterexamples blow up the entire thesis. In practice, the "safe choice" for longevity is only evident in hindsight. If anyone is hiring for TypeScript in 2050 I'll eat my hat
    - h0l0cube 3y ago
      I'm in sympathy with your main argument, but given that people still, in 2023, hire for COBOL, a language made in 1959, makes me think you'll be eating your hat.
      - inopinatus 3y ago
        Notwithstanding that I'm effectively forecasting the decline & fall of TypeScript as javascript-flavour-of-the-month, note that I shrewdly also granted myself the better part of three decades to launch an edible headwear startup
        claytongulick 3y ago
        Most of the perceived value that Typescript brings will probably fade once js gets type annotations [1], though Typescript will always have more advanced syntax and capabilities. There are interesting discussions about Typescript becoming more of a run-time type checker, which would be opt-in and have significant performance penalties, but would give more guarantees of type safety. [1] https://github.com/tc39/proposal-type-annotations https://github.com/tc39/proposal-type-annotations
- calvinmorrison 3y ago
  This sounds fun until your entire stack is on Perl in 2023 (which, is fine-ish when your CTO was the perl project lead). But then you have a bajillion dependencies that aren't really updated anymore, you end up owning most of the Mail::Toolkit libraries, etc. and nobody really knows perl that well. I mean, a lot of us can jimmy around in Perl, but of the real Perl Gurus in the world, you will end up hiring 50% just to keep your company going
- ttymck 3y ago
  I think I'm rather able and intelligent, but there are just so many footguns in golang that my team doesn't have the knowledge to avoid.
  - drzaiusx11 3y ago
    Just curious, what are some of those footguns your team is encountering? Only thing that comes to mind is the somewhat tedious error handling imposed by the language. Which language would you say had _less_ footguns than go? Or was this sarcasm and it went straight over my head?
    - ttymck 3y ago
      I guess I am just bad at golang. Loop variable pointer and nil struct is not nil interface are the two that continue to hound us.
      - ttymck 3y ago
        Also: type MyError {} var _ error = (&MyError)(nil) errors.As(err, &MyError{}) --- Panicking because it needs to be &&MyError{} instead is a similar gotcha.
- drzaiusx11 3y ago
  "Ability" by itself is a bit vague here, but I will say that having a proven trackrecord in their ability to learn is something I specifically look for in a candidate. Not only learning new tech, but also learning from mistakes and experiences over time. However, I don't think raw "intelligence" is the right metric for hiring. Unless you specifically mean emotional intelligence, which can curtainly contribute to improving communication skills on the team. I've also found that when hiring someone with mostly expertise outside your current stack it's critical to provide good feedback loops and mentoring early on so they can "get up to speed" and contributing valuable, idiomatic code as fast as possible without feeling as though your entire onboarding experience is trial by fire. This mentoring takes up extra resources and is a net drain on your team in the short term, but pays off in the long term.
girvo 3y ago
It’s very easy to teach a good developer a new language. We use Nim for embedded firmware development at my work, and have been expanding our team. It’s been simple: hire good C/C++ devs who are interested in learning something new but related, and teaching them. Takes about two weeks before they’re comfy, sometimes less (python experience seems to help a little due to some surface level syntactic similarities)
- ttymck 3y ago
  What if you can't find an expert to teach the other developers?
  - rad_gruchalski 3y ago
    You are hiring too late.
    - ttymck 3y ago
      So if you end up joining one of these companies, run for the hills, no?
  - chefandy 3y ago
    Where did the rest of the code come from, or who made the decision to start that way? They should at least be able to spearhead it. Obviously it's different for tiny organizations, but generally, if someone inexpertly made architectural decisions without building the institutional knowledge to continue without them, the language they chose will just be one of many big problems.
    - troupo 3y ago
      People who build things are not necessarily able to teach things. These are two different skill sets.
      - chefandy 3y ago
sergiotapia 3y ago
>There was really no reason to do so at all Please - I can do a ton of stuff in Elixir without having to reach for some weird/paid third party service. https://twitter.com/ryanrwinchester/status/1580652324405776386/photo/1 https://twitter.com/ryanrwinchester/status/15806523244057763... Now with Liveview I can even ditch the grotesque React toolchain entirely. Also, in my experience as defacto CTO of a unicorn YC company (Papa), hiring was not insanely difficult. I hired about ~45 engineers Great Ruby/Go devs translated pretty easily into Elixir after minimal training. The language is beautiful and simple.
- mattbaker 3y ago
  Agreed, it’s been so easy to onboard people onto Elixir we’ve just looked for raw talent knowing they’d be up on Elixir quickly. As long as you have one experienced elixir engineer to guide the people learning I think it’s easy to move fast.
inopinatus 3y ago
You're posting on a website built in its own Lisp variant and inhabited by people who think it's fun to write compilers.
throwawaymaths 3y ago
This is completely insane because at the same time go on any elixir forum and you will find lots of highly talented developers looking for jobs. As usual the problem is that leadership has no fucking clue how to hire.
- andrewstuart 3y ago
  >> go on any elixir forum and you will find lots of highly talented developers looking for jobs Sure, in a downturn after a year of waves of layoffs, and maybe in San Francisco, but the non-mainstream tech will live longer than a few economic employment ups and downs.
  - sarchertech 3y ago
    Last 2 companies I’ve worked at have been elixir shops. Both of them had no trouble at all hiring elixir devs. At my previous company we hired close to 30 in a 6 month period in 2021. At my current bigco job we had more elixir devs pass the interview process than we have open slots (and we had a ton of slots). I’ll also add that over many years of doing this, the elixir devs I’ve interviewed tend to be far above average, which makes up for the lower number of candidates. If you want to hire thousands of fresh grads each year and don’t want to spend any time on training, sure go for something they teach in school.
  - throwawaymaths 3y ago
    No it's been pretty much that way for years.
deleted 3y ago
[deleted]
zem 3y ago
i've shared this story on here before, but years ago i worked for a startup where we had a janky homegrown distributed system cobbled together from a mix of python, rabbitmq and state stored in postgres tables. four of us, two senior and two junior engineers, decided to see if we could learn erlang and build a more coherent and well-engineered replacement for it. it took us four months, starting from never having used the language, to learn our way around it and set up a proof of concept that was both more performant and more stable than the existing system. and this was all while doing our regular work as well. erlang is an extremely simple language to learn and become productive with, especially if the task you're trying to do fits the grain of the language and beam environment. (ironically, the erlang solution never went anywhere because management decided they did not want to take a chance on a lesser-used language. but from a getting employees standpoint it's perfectly feasible to hire people who are simply good engineers in whatever language and give them a month to learn erlang.)
sbrother 3y ago
Are you talking about WhatsApp? Discord?
- toast0 3y ago
  I can't speak for Discord, but at WhatsApp, we chose Erlang for a very good reason: ejabberd is a very clear base to start with if you want to run a chat server, and ejabberd runs on Erlang. At least Facebook, WhatsApp, Riot Games, and Discord have done it, and I think there's plenty more. While I was there, I was the third most Erlang knowledgeable server hire, because I rememebered Ericsson open sourcing it decades ago, but hadn't used it before. We hired two people I can recall who were experienced with Erlang (well, and a client developer who used Erlang in school but didn't use it for WhatsApp). This wasn't a big deal, Erlang is a small language, and smart developers pick up enough to be useful pretty quickly. There's a learning curve for distributed systems challenges, but IMHO, message passing concurrency makes a lot of things a lot simpler than shared memory concurrency, and you can get a lot of good work done where the new person does the bulk of the work and a mentor helps with distributed systems bits. Anyway, Facebook chat decided to abandon Erlang for hiring reasons, and transitioned to a C++ server, and I don't think it was that hard; Erlang wasn't some boat anchor holding them back when they decided to switch, like the GP suggested. Even if it was, that's kind of an amusing complaint: this thing is so awful, and we can't even replace it because ??? it's too good?
  - Capricorn2481 3y ago
    > Even if it was, that's kind of an amusing complaint: this thing is so awful, and we can't even replace it because ??? it's too good? Obviously they were suggesting that rewrites/changing your tech stack for a project is expensive and not everyone has the luxury to do it. Facebook does.
kgeist 3y ago
>But now they have trouble hiring people because there's a tiny pool of people who know that language >In 2023 you've got to have really good reasons for not using C#, or Java, or Python, or TypeScript for building your web project. Maybe Golang In my country, very few people know Go, so we hire devs without requiring them to know Go - we teach them the language, and after a few weeks, they already feel pretty confident with it. In fact, most devs are actually interested in trying something new, some new language. Maybe your employer's problem is that they are only looking for CVs where the language is mentioned as a keyword, and don't consider anything else?
crabbone 3y ago
There's always a good reason not to use C#. That's actually the number one language that is crossed out of the candidate lists due to the company behind it and very "uneven" platform support. As for using a unpopular language. My (mostly untested) conviction is that if this unpopular language is still popular among an albeit small group of programmers, it's a blessing. While hiring is difficult, you get much higher quality human resources. You get to enjoy working with smart people. I've known a company whose business was to track real estate values. They had a Web site and some backend that did somewhat complicated statistics trying to predict prices or rent based on all kinds of factors they aggregated. A lot of the functionality of the backend was in scraping local news, city authority documents etc. They wrote everything in Clojure (including front-end, which was Clojure-script). Great people, created a great, useful and successful business. I've worked in companies early adopters of Go and Rust. It was a great experience. I've learned a lot during that time from my coworkers. I also briefly worked for a company whose main product was written in D. It was another great place to work in terms of quality of human resources and the quality of code I had to work with.

siwatanejo 3y ago

I hire based on ability/intelligence, not knowledge.

andrewstuart 3y ago
>> I hire based on ability/intelligence, not knowledge. Well, yeah that's really your only choice if the experience you need is not easily available. Don't tell me, you are the guy who puts in the not mainstream technology and needs to defend that with justifications such as "when we use (non mainstream technology X) we get super motivated job applicants because anyone with an interest in non mainstream technologies is inherently passionate and interested". That's not a sound hiring strategy.
- inopinatus 3y ago
  And you are the guy who left Visual Basic, COBOL, Perl, and FORTRAN off their "obviously safe choices for hiring in the future" list, despite them all having once been perceived as such. These inconvenient counterexamples blow up the entire thesis. In practice, the "safe choice" for longevity is only evident in hindsight. If anyone is hiring for TypeScript in 2050 I'll eat my hat
  - h0l0cube 3y ago
    I'm in sympathy with your main argument, but given that people still, in 2023, hire for COBOL, a language made in 1959, makes me think you'll be eating your hat.
    - inopinatus 3y ago
      Notwithstanding that I'm effectively forecasting the decline & fall of TypeScript as javascript-flavour-of-the-month, note that I shrewdly also granted myself the better part of three decades to launch an edible headwear startup
      - claytongulick 3y ago
        Most of the perceived value that Typescript brings will probably fade once js gets type annotations [1], though Typescript will always have more advanced syntax and capabilities. There are interesting discussions about Typescript becoming more of a run-time type checker, which would be opt-in and have significant performance penalties, but would give more guarantees of type safety. [1] https://github.com/tc39/proposal-type-annotations https://github.com/tc39/proposal-type-annotations
calvinmorrison 3y ago
This sounds fun until your entire stack is on Perl in 2023 (which, is fine-ish when your CTO was the perl project lead). But then you have a bajillion dependencies that aren't really updated anymore, you end up owning most of the Mail::Toolkit libraries, etc. and nobody really knows perl that well. I mean, a lot of us can jimmy around in Perl, but of the real Perl Gurus in the world, you will end up hiring 50% just to keep your company going
ttymck 3y ago
I think I'm rather able and intelligent, but there are just so many footguns in golang that my team doesn't have the knowledge to avoid.
- drzaiusx11 3y ago
  Just curious, what are some of those footguns your team is encountering? Only thing that comes to mind is the somewhat tedious error handling imposed by the language. Which language would you say had _less_ footguns than go? Or was this sarcasm and it went straight over my head?
  - ttymck 3y ago
    I guess I am just bad at golang. Loop variable pointer and nil struct is not nil interface are the two that continue to hound us.
    - ttymck 3y ago
      Also: type MyError {} var _ error = (&MyError)(nil) errors.As(err, &MyError{}) --- Panicking because it needs to be &&MyError{} instead is a similar gotcha.
drzaiusx11 3y ago
"Ability" by itself is a bit vague here, but I will say that having a proven trackrecord in their ability to learn is something I specifically look for in a candidate. Not only learning new tech, but also learning from mistakes and experiences over time. However, I don't think raw "intelligence" is the right metric for hiring. Unless you specifically mean emotional intelligence, which can curtainly contribute to improving communication skills on the team. I've also found that when hiring someone with mostly expertise outside your current stack it's critical to provide good feedback loops and mentoring early on so they can "get up to speed" and contributing valuable, idiomatic code as fast as possible without feeling as though your entire onboarding experience is trial by fire. This mentoring takes up extra resources and is a net drain on your team in the short term, but pays off in the long term.

girvo 3y ago

It’s very easy to teach a good developer a new language. We use Nim for embedded firmware development at my work, and have been expanding our team. It’s been simple: hire good C/C++ devs who are interested in learning something new but related, and teaching them. Takes about two weeks before they’re comfy, sometimes less (python experience seems to help a little due to some surface level syntactic similarities)

ttymck 3y ago
What if you can't find an expert to teach the other developers?
- rad_gruchalski 3y ago
  You are hiring too late.
  - ttymck 3y ago
    So if you end up joining one of these companies, run for the hills, no?
- chefandy 3y ago
  Where did the rest of the code come from, or who made the decision to start that way? They should at least be able to spearhead it. Obviously it's different for tiny organizations, but generally, if someone inexpertly made architectural decisions without building the institutional knowledge to continue without them, the language they chose will just be one of many big problems.
  - troupo 3y ago
    People who build things are not necessarily able to teach things. These are two different skill sets.
    - chefandy 3y ago

sergiotapia 3y ago

>There was really no reason to do so at all Please - I can do a ton of stuff in Elixir without having to reach for some weird/paid third party service. https://twitter.com/ryanrwinchester/status/1580652324405776386/photo/1 https://twitter.com/ryanrwinchester/status/15806523244057763... Now with Liveview I can even ditch the grotesque React toolchain entirely. Also, in my experience as defacto CTO of a unicorn YC company (Papa), hiring was not insanely difficult. I hired about ~45 engineers Great Ruby/Go devs translated pretty easily into Elixir after minimal training. The language is beautiful and simple.

mattbaker 3y ago
Agreed, it’s been so easy to onboard people onto Elixir we’ve just looked for raw talent knowing they’d be up on Elixir quickly. As long as you have one experienced elixir engineer to guide the people learning I think it’s easy to move fast.

inopinatus 3y ago

You're posting on a website built in its own Lisp variant and inhabited by people who think it's fun to write compilers.

throwawaymaths 3y ago

This is completely insane because at the same time go on any elixir forum and you will find lots of highly talented developers looking for jobs. As usual the problem is that leadership has no fucking clue how to hire.

andrewstuart 3y ago
>> go on any elixir forum and you will find lots of highly talented developers looking for jobs Sure, in a downturn after a year of waves of layoffs, and maybe in San Francisco, but the non-mainstream tech will live longer than a few economic employment ups and downs.
- sarchertech 3y ago
  Last 2 companies I’ve worked at have been elixir shops. Both of them had no trouble at all hiring elixir devs. At my previous company we hired close to 30 in a 6 month period in 2021. At my current bigco job we had more elixir devs pass the interview process than we have open slots (and we had a ton of slots). I’ll also add that over many years of doing this, the elixir devs I’ve interviewed tend to be far above average, which makes up for the lower number of candidates. If you want to hire thousands of fresh grads each year and don’t want to spend any time on training, sure go for something they teach in school.
- throwawaymaths 3y ago
  No it's been pretty much that way for years.

deleted 3y ago

[deleted]

zem 3y ago

i've shared this story on here before, but years ago i worked for a startup where we had a janky homegrown distributed system cobbled together from a mix of python, rabbitmq and state stored in postgres tables. four of us, two senior and two junior engineers, decided to see if we could learn erlang and build a more coherent and well-engineered replacement for it. it took us four months, starting from never having used the language, to learn our way around it and set up a proof of concept that was both more performant and more stable than the existing system. and this was all while doing our regular work as well. erlang is an extremely simple language to learn and become productive with, especially if the task you're trying to do fits the grain of the language and beam environment. (ironically, the erlang solution never went anywhere because management decided they did not want to take a chance on a lesser-used language. but from a getting employees standpoint it's perfectly feasible to hire people who are simply good engineers in whatever language and give them a month to learn erlang.)

sbrother 3y ago

Are you talking about WhatsApp? Discord?

toast0 3y ago
I can't speak for Discord, but at WhatsApp, we chose Erlang for a very good reason: ejabberd is a very clear base to start with if you want to run a chat server, and ejabberd runs on Erlang. At least Facebook, WhatsApp, Riot Games, and Discord have done it, and I think there's plenty more. While I was there, I was the third most Erlang knowledgeable server hire, because I rememebered Ericsson open sourcing it decades ago, but hadn't used it before. We hired two people I can recall who were experienced with Erlang (well, and a client developer who used Erlang in school but didn't use it for WhatsApp). This wasn't a big deal, Erlang is a small language, and smart developers pick up enough to be useful pretty quickly. There's a learning curve for distributed systems challenges, but IMHO, message passing concurrency makes a lot of things a lot simpler than shared memory concurrency, and you can get a lot of good work done where the new person does the bulk of the work and a mentor helps with distributed systems bits. Anyway, Facebook chat decided to abandon Erlang for hiring reasons, and transitioned to a C++ server, and I don't think it was that hard; Erlang wasn't some boat anchor holding them back when they decided to switch, like the GP suggested. Even if it was, that's kind of an amusing complaint: this thing is so awful, and we can't even replace it because ??? it's too good?
- Capricorn2481 3y ago
  > Even if it was, that's kind of an amusing complaint: this thing is so awful, and we can't even replace it because ??? it's too good? Obviously they were suggesting that rewrites/changing your tech stack for a project is expensive and not everyone has the luxury to do it. Facebook does.

kgeist 3y ago

>But now they have trouble hiring people because there's a tiny pool of people who know that language >In 2023 you've got to have really good reasons for not using C#, or Java, or Python, or TypeScript for building your web project. Maybe Golang In my country, very few people know Go, so we hire devs without requiring them to know Go - we teach them the language, and after a few weeks, they already feel pretty confident with it. In fact, most devs are actually interested in trying something new, some new language. Maybe your employer's problem is that they are only looking for CVs where the language is mentioned as a keyword, and don't consider anything else?

crabbone 3y ago

There's always a good reason not to use C#. That's actually the number one language that is crossed out of the candidate lists due to the company behind it and very "uneven" platform support. As for using a unpopular language. My (mostly untested) conviction is that if this unpopular language is still popular among an albeit small group of programmers, it's a blessing. While hiring is difficult, you get much higher quality human resources. You get to enjoy working with smart people. I've known a company whose business was to track real estate values. They had a Web site and some backend that did somewhat complicated statistics trying to predict prices or rent based on all kinds of factors they aggregated. A lot of the functionality of the backend was in scraping local news, city authority documents etc. They wrote everything in Clojure (including front-end, which was Clojure-script). Great people, created a great, useful and successful business. I've worked in companies early adopters of Go and Rust. It was a great experience. I've learned a lot during that time from my coworkers. I also briefly worked for a company whose main product was written in D. It was another great place to work in terms of quality of human resources and the quality of code I had to work with.