13 ms·
TL;DNR, use a language your company can support. It doesn't matter how suited to the job a language is, if it's single Engineer or small team, what happens whe
by as-j 6y ago
TL;DNR, use a language your company can support. It doesn't matter how suited to the job a language is, if it's single Engineer or small team, what happens when they move on? How do you support it? Who's on call?
Not Elixir, but a cautionary tale from our Erlang project. ~8 years ago a our IoT backend was written in Erlang, this was the early days of IoT, so sure it made sense as a technology, could scale well, handle all the symmetric sessions, etc. It was a good tool for the job and in theory could scale well.
But, there's always a but. We're a Python shop on the backend and C on embedded devices. Engineers move on, there some lean times, and after a few years there was no one left who could support an Erlang system. So we hired for the position, and it's key infrastructure, but not really a full time project, so we hired a Python+Erland person.
But! Now they're on call 24/7 since the regular on call roster are python people and when the Erlang service goes wrong, it's call the one engineer. So, do you hire 2 or 3 people to support the service? No, you design it out. At the time is was 2018, and IoT services were a plenty, so we could use an existing service and python.
Another way of looking at it, let's say it's a critical service and you need at least 3 people to support it/be on call. If each engineer costs $200k/year, that's $600k/year. Does this new language save you that much over using a more generic and widly known language in the org?
- nicoburns 6y agoWhy not get your existing engineers to learn erlang if you have such a need?
- eshyong 6y agoProbably cost of training. You'd have to hire consultants to teach best practices and that could cost significantly more than hiring a new engineer. Plus there are other risks to maintaining two codebases in different languages.
- ajkjk 6y agoYou have to hire consultants to teach best practices?
- RSZC 6y agoHonestly it's not a bad idea. Erlang is not a language that is easy for an experienced engineer to pick up and quickly create maintainable code.
- DevKoala 6y agoI had the same reaction. However, I work for a small team with driven developers. It is expected from our engineers to learn “best practices” on their own time when we introduce new technologies. We are also compensated extremely well according to these expectations. If you have a team of engineers with a different mindset, you will need consultants.
- eshyong 6y agoNot saying this applies to every company, but I've worked at two different companies that have done something similar: * Healthcare startup that basically outsourced its entire infra/security team from a consulting company. The consultants rewrote everything and took over engineering leadership, basically acting as gatekeepers for anything SRE-related. * Worked at an older DNS company that hired a bunch of consultants to help engineers migrate their web framework from Play to Spring. Said engineers used Play because it was shiny but didn't have the Scala expertise to keep maintaining it, so they moved to pure Java.
- maxwindiff 6y agoWhether you hire an outside consultant or not doesn’t matter. You have to pay the cost one way or another, either by your employees spending time learning the language + experimenting + making mistakes, or by going through some training and hopefully making less mistakes (not guaranteed).
- jasonlotito 6y agoBecause "learning" Erlang is like learning any language. Sure, you can learn the language in short period of time, but really being able to professionally support a production level app to the level needed will take a long time and a lot of practice. And that practice isn't a week long course where you suddenly learned Erlang and then that's it. You aren't working with Erlang any more. You don't need someone "learning" Erlang. You need someone who works with it.
- shreddit 6y agoOne does not simply learn erlang
- PragmaticPulp 6y agoBecause engineers don't take kindly to companies changing the direction of their career development. If you're an up-and-coming Golang developer, how happy are you going to be when your company takes you off of a Golang project and makes you the new Erlang person? Now how happy are you going to be when you're informed that you're learning Erlang to maintain an old, flakey codebase written by 2 guys who left the company and are now unreachable? Oh, and you're going to need to be on call for this because no one else knows how to fix it.
- brazzy 6y agoSo how about don't be stupid about it and instead make three engineers devote 1/3 of their time to this? That way it's a broadening rather than a redirection of their career, they can help each other, the on call duties are shared, and the business isn't dependant on a single person.
- mrlatinos 6y agoDedicating a third of my time to a language I had no previous intention of learning is going to be a hard sell, even if you label it as "broadening." If I'm a python developer, I'm now spending less time on my core competency, giving other python developers an edge. Seems like you're actually narrowing my career, unless I wish to apply for jobs seeking mediocre Erlang programmers...
- bryanrasmussen 6y ago>If I'm a python developer, I'm now spending less time on my core competency, giving other python developers an edge. so this argues against the theory that learning other languages and other programming paradigms makes you a better programmer in the languages you already know.
- Leherenn 6y agoOr at least, that it is less valuable than spending the same time learning your core language. This might be true based on your specific situation (relatively new to the language, but already know some other varied languages); I don't know.
- aethant 6y agoAll well and good if the engineers want to learn/work with that language. But not everyone does. I’ve left jobs over forced decisions like this.
- chx 6y ago> Who's on call? Side note: Pagerduty comes with many hilarious sounds, including a very realistic meow. Thus we practice meow driven development: I don't want my phone to meow desperately.
- PragmaticPulp 6y agoYour comment was so spot-on that I suspected we worked at the same company for a minute. Erlang / Elixir are languages that are easy to begin learning, but difficult to master. In our case, the team was able to get the basics up and running quickly, but the service became increasingly difficult to manage as our product requirements expanded beyond the strengths of the Erlang ecosystem's core competencies. In our case, the original team of developers was excited to write a greenfield service in Erlang, but much less excited to maintain and scale it long-term. Some of them used their experience to speak at Erlang conferences, which led to them being recruited away by companies who were desperate for Erlang developers to maintain their legacy code after the core team left. Ironic. At first we tried to train existing engineers on Erlang. Doesn't work. Not because Erlang is bad, but because people don't want their companies changing the direction of their career development just to support someone else's legacy project that requires an on-call team to support. So your two options are: Hire at least 2, preferably 3 Erlang engineers for the reasons discussed above. Harder than it sounds, because AFAICT there are a lot of companies with 2010-era Erlang backends that made sense at the time but now require dedicated maintenance engineers. Or you can rewrite it in a more common language, which is the route we chose. Rewrites are a difficult decision for well-known reasons, but ultimately it was the correct choice. Being able to drop 10+ engineers into a project as needed is a huge accelerator compared to funneling 100% of your work through the 2 Erlang gurus in the corner who maintain a codebase that no one else wants to touch.
- rixed 6y ago> We're a Python shop Who decided that your company's raison d'être was to write some Python?
- doteka 6y agoThat's generally decided by the experience of the existing team members.
- rixed 6y ago... whom an inexperienced hiring department entirely trust when they assure that the technologies they are already familiar with are the best fit for the company, let's hire only people who think the same?
- rlander 6y agoTo be fair, if you’re rewriting an Erlang system in Python/JS and you don’t feel like you’re losing much, Erlang was the wrong choice from the beginning. The cases were Erlang shines are very hard to replicate in these languages.
- nell 6y ago> The cases were Erlang shines are very hard to replicate in these languages. I'm just beginning to get interested in the Erlang ecosystem. Could you please add more color to this and give some examples where Erlang shines? IIRC, Whatsapp backend used to be on Erlang.
- niels_bom 6y agoAfaik it still is.
- adrr 6y agoThat would make sense that whatsapp backend is in Erlang. Erlang was developed for telecommunication systems where you need performance and the ability to hot swap code.
- sitkack 6y agoErlang's hotswap and Python's hotswap are of comparable power, ie, both are crappy. Being able to do something isn't the same as doing it well. You can hot swap all languages.
- pritambaral 6y agoIf you're claiming that Erlang is no better at hot-swapping code than "all languages", you haven't seen Erlang's hot-swapping abilities. "All languages" provide you the ability to run arbitrary code, and thus theoretically support hot-swapping, but if you try to put it into practice you'll soon run into real obstacles you hadn't considered that either bite you or at the least make your job a lot harder. Erlang provides those practical solutions that make hot-swapping easy (in other words, practically possible). TL;DR: On the question of buy vs. build when it comes to hot-swapping abilities, Erlang falls on the left while "all languages" start at the right.
- dathinab 6y agoYes that. But also only use paradigms which the company can support. For example good static typing is not just about adding type declarations but about taking advantage of the type system to the right degree. (Like don't ever overdo it, it's killing productivity most times). But what degree falls into the productivity boosting spot and from which point on it will hinder productivity depends a lot on the team. The problem often starts when the team changes and e.g. a pure scala/rust/C++/Ocml team get people cross entering from languages without any usefull static typing. What had been reasonable and useful might now be a hindrance to the team! I ran into a situation where this caused massive delay. (Well through the actual root cause was miscommunications, people not complaining about problems and instead trying to find ways to fix them behind the back of other people force pushing nice looking simple solutions which just happen to fail many edge cases, killed the write, refactor, test cycle and increase maintenance cost longtime :( ) So well 1st never overuse type systems. 2nd communicate problems, especially if you have problems understanding anything even if it just takes longer then you think it should.
- kiliancs 6y agoOur adoption of TypeScript led to a significant reduction in defects, and it definitely isn't killing productivity. Most of the time we take advantage of existing types so if anything it speeds things up in the long run.
- dathinab 6y agoYou are misunderstanding what I mean. Through I haven't be to clear I think. What I mean is code which not only uses types and takes advantage of the type system but overdoes it. In many languages with a powerful type system you can encode much more then just basic types and this is useful to reduce possible error cases. But if you overdo it the type system part becomes too complex, type error messages too unreadable and it hinders productivity and on-boarding. I'm a big fan of static typing and believe that more encoding more constraints in the type system is _generally_ good. But if the resulting types become to complex it often not worth it.
- 6y ago
- craigsmansion 6y ago> there was no one left who could support an Erlang system. If you have $200k "python engineers" on the payroll who wouldn't jump at the opportunity to do some additional Erlang, maybe it's time to reconsider your hiring practices and that is the real cautionary tale.
- maxwindiff 6y agoI want to downvote this comment so much.
- cosmotic 6y agoWhy?
- maxwindiff 6y agoThis kind of sentiment come up every so often on HN and I'm just sick of it. Sure, programmers should strive to improve their craft, just like professionals in every industry. But there are plenty of ways to improve yourself and learning a new language is hardly the most effective choice (and in fact could be counterproductive).
- scull7 6y agoHow does learning a new way to think about a problem become counterproductive? Each language you learn gives you a new angle from which you can approach a problem. Is it counter productive to learn a second spoken language? I haven’t come across an argument against that, even learning a dead language like Latin or Ancient Greek has benefits.
- maxwindiff 6y agoPerhaps counterproductive is too strong a word, but there could definitely be problems in the short term. Programmers love to apply things they learned in their work, and taking a concept or pattern from language A and applying them to language B is a natural reaction. The most common culprits are monads and actors. Ergonomics (readability, cognitive cost, debugging, IDE support, etc) became afterthoughts in the midst of excitement, and the long term health of the codebase (and teammates) suffers. I have made mistakes like that (introducing monads/future and making the team learn guava) and I still regret it. Its not to say futures are useless - there were a lot of async logic in the codebase and futures solve a real problem, but its not clear if the benefits outweigh the cost.
- opportune 6y agoYour "python engineers" couldn't do anything even touching Erlang? What
- deleted 6y ago[deleted]
- sghill 6y ago> Does this new language save you that much over using a more generic and widly known language in the org? This is a great question to ask. It’s worth extending beyond language to frameworks and libraries as well. It’s important to optimize for the development experience across all the apps in a team. Especially once the excitement of using a new language is gone and there are bugs to fix. I‘ve rewritten a few projects in a common language for the org. Each time the biggest benefit was how straightforward it became to work on. Problems you’ve found somewhere else become easy to spot (and search for). Writing another test is the same as the last project you had open. Anyone in the org can contribute with a minimal learning curve. This can be the difference in getting bugs fixed in a day or a month.
- _8ljf 6y ago“after a few years” Key phrase right there. Because it doesn’t matter what language a system is written in: if management doesn’t bother to hire replacements until after everyone who understands that system and why it works the way it does has already left, of course it’s going to crater shortly thereafter. That’s not a language problem, that’s a business continuity problem. And it’s lamentably common as dirt. Any monkey can learn to churn code. Learning the business domain, what its problems are, and how to solve them; that’s the bit that actually matters. PEBKAC, at every level.
- a-saleh 6y agoI think one approach when choosing more niche language might be, first look for a book like https://www.erlang-in-anger.com/ https://www.erlang-in-anger.com/, read through it and consider, if this is the sort of stuff my colleagues will want/be capable to deal with. Then consider how hard it would be to get more people that have used such language/ecosystem in anger.