6 ms·

A lot of these techniques are things that take a bit more up-front effort but pay off over the long (or even medium) term. I think that's why it might make peop

by tikhonj 10y ago

A lot of these techniques are things that take a bit more up-front effort but pay off over the long (or even medium) term. I think that's why it might make people seem less productive—although of course a lot depends on your perspective and experience. Measuring productivity is an open problem in software engineering and, in my experience, people's intuitions about it are all over the board, which means that nobody's any good at it. DSLs are a great example of this long-term dynamic. Compared to throwing together an ad-hoc library, a DSL approach takes a lot more design effort but, ultimately results in a system that's more coherent, elegant and expressive. I've worked on both kinds of projects and the difference is palpable: with the first style, productivity is constant at best—getting things together in the first place is a struggle, and then adding features or fixing bugs continues to be a struggle. On the other hand, the second style of project is even more of a struggle at first, but once it works it's like magic: new features are easier to add than you'd expect, and I've had way more things work after my first attempt than anyone has a right to expect. I know which style I prefer, and it's definitely not because I "fetishize challenging knowledge for challenge sake"—it's because I'm willing to put effort up front for a long-term reward. And it's not even that long-term—I've found these things pay off over weeks or months, not years, so I'd take the same deliberate approach unless my deadlines were literally days away.

alexandercrohde 10y ago
Well, I can't speak to your career experience, but I can speak to mine. My experience has been (31, SF) that companies are fundamentally disorganized, frequently reinvent things (a behavior seen at all levels), are skewed by politics, and ultimately are rarely successful due to the code. The fact is, that for most startups it's not the code that's make-or-break (twitter, snapchat, facebook, airbnb, uber), it's the business execution. It's been my career experience that the only way to be a 10x engineer is to prevent management from engaging in unnecessary projects (Bob wants to rewrite X in node, Joe wants to make a service that only has the responsibility of CRUD to 1 table, Sally wants to move it all to noSql). It's been my career experience that soft skills give the best ROI. So when I see a list like this, I find it hard to imagine how "Profunctor Optics" is what Zynga (or any company I've worked at) needed to be successful.
- runT1ME 10y ago
  >So when I see a list like this, I find it hard to imagine how "Profunctor Optics" is what Zynga (or any company I've worked at) needed to be successful. I mean, by that criteria, why learn anything? The world is not just startups btw, I work on products that have very very defined requirements. Hell, sometimes I'm implementing an RFC. FP is a huge win for us, and yes i've used profunctor optics in a high performance network application.
  - a_imho 10y ago
    >I mean, by that criteria, why learn anything? This might be tongue in cheek, but I agree with the parent. Engineering excellence will not further your career much, companies tend to oversell meritocracy. Salaries flat out quickly, and there is a huge supply of good enough developers, if you step out of the line too much.
    - runT1ME 10y ago
      > Salaries flat out quickly What other professions besides lawyers or doctors regularly have a 3 or 4x range for salaries of people more or less with the same job description?
      - a_imho 10y ago
        In addition to those? Actors, musicians, sport people. In the office? Managers. Hence the advice, focus on soft skills. I would wager the top 1% on the developer track earn nowhere near to the top 10% on the managerial track. And your best bet is to regularly change jobs otherwise good luck getting anything more than a 5-10% increase, while managerial salaries and bonuses scale with the size of the organization. Also, there is a lot to be said about the difficulty of the work.
  - deleted 10y ago
    [deleted]
- sitkack 10y ago
  I have had a number of projects that were absolutely successful from a technical standpoint because of FP methods. No one knew except team members that were unfamiliar with the techniques. While they were not full converts, many "saw the light" and reduced mutation and wrote purer functions after that. The business itself was oblivious to why the project was successful. Or how we were able to extend it so quickly and retain stability. Even on that project, 90% of the difficulty was social/political. Just as Java allowed average programmers to write above average code, the same thing can be said of FP.
pron 10y ago
> with the first style, productivity is constant at best—getting things together in the first place is a struggle, and then adding features or fixing bugs continues to be a struggle. On the other hand, the second style of project is even more of a struggle at first, but once it works it's like magic: new features are easier to add than you'd expect, and I've had way more things work after my first attempt than anyone has a right to expect. I would like to challenge that by pointing out that not a single piece of large software (except maybe a compiler or two) has ever been written using the skills/concepts of the higher levels of this chart. What you say may be true if generalized to better upfront design etc., but there's also no evidence that this is better done using the approaches discussed. It's not even about measuring productivity; it's about arguing over the properties of an empty set. When it comes to small (<100KLOC) and non-large (<1MLOC) programs, then the difficulties are not that hard to begin with, and depend mostly on the essential complexity of the problem, rather than abstraction/code organization etc.. > it's because I'm willing to put effort up front for a long-term reward. You're equating upfront effort with a particular choice of technique, and one that has never been put to the test.
- loup-vaillant 10y ago
  > not a single piece of large software (except maybe a compiler or two) has ever been written using the skills/concepts of the higher levels of this chart. I wonder: to what extent this is because those higher level concept tend to shrink programs in the first place? I mean, big is bad, there's no debating this. Big programs can only arise from necessity or stupidity. And if those fancy features are any use, they must be reducing the need for big. I would ask myself a slightly different question: what kind of program still have to be big, even if you have all these fine concepts at your disposal? And of course, big is risky, and that tend to make us chose more conservative options. Why use Haskell where C++ has shown in the past it could do that kind of job? Not to mention the network effects: even if Haskell as a language was better than C++ as some specific job, you may still choose C++ because of libraries, commercial support, and developer availability. > When it comes to small (<100KLOC) and non-large (<1MLOC) programs Oh, so that's how you're calibrated… My, to me, small means <10KLOC. 100KLOC is already big, and 1MLOC is gigantic. Besides, I've seen a couple multi-million lines programs, and I don't believe for a second they had to exceed 100KLOC. Such programs are more about piling historical accident on top of historical accident than about encoding a complex problem domain. A small compiler takes a couple thousand lines of code. With the proper tools, it can often be squeezed into 2KLOC (Source: the STEPS project from http://vpri.org http://vpri.org). It won't optimize like GCC, but you rarely have to anyway. Now you have a DSL for writing an executable specification. What kind of specification is so complex that it cannot even fit in 50 books? Of course you have to keenly understand the problem domain to pull that of, and that often means solving it the crappy way the first time around. Which means you're never going to rewrite it the proper way: it would be way too risky. So of course it has seldom been put to the test. One does not just hinge an entire business on original research. > You're equating upfront effort with a particular choice of technique, and one that has never been put to the test. No, not never. I have at least one example: one of my uncles once had to write a number of database transactions. It was one of his first job. He had 1 year to do it. Seeing how tedious it would be, he first though about the problem, then devised a DSL in which he could write the damn transactions. Since he wasn't exactly senior, and had crappy tools (he used B), the DSL took him about 6-7 months to perfect. (During which management were on the verge of panic, because no transaction has been done yet.) At the 8 month mark, all transactions were done, tested, and had a surprisingly low bug count. The client was very pleased and the contract was renewed for another batch of transactions. Same size, but to be done in 8 months this time. That was given to a co-worker, who used the my uncle's DSL to perform 8 months worth of contracted work in 1 month. DSLs sometimes work.

by tikhonj 10y ago

alexandercrohde 10y ago

Well, I can't speak to your career experience, but I can speak to mine. My experience has been (31, SF) that companies are fundamentally disorganized, frequently reinvent things (a behavior seen at all levels), are skewed by politics, and ultimately are rarely successful due to the code. The fact is, that for most startups it's not the code that's make-or-break (twitter, snapchat, facebook, airbnb, uber), it's the business execution. It's been my career experience that the only way to be a 10x engineer is to prevent management from engaging in unnecessary projects (Bob wants to rewrite X in node, Joe wants to make a service that only has the responsibility of CRUD to 1 table, Sally wants to move it all to noSql). It's been my career experience that soft skills give the best ROI. So when I see a list like this, I find it hard to imagine how "Profunctor Optics" is what Zynga (or any company I've worked at) needed to be successful.

runT1ME 10y ago
>So when I see a list like this, I find it hard to imagine how "Profunctor Optics" is what Zynga (or any company I've worked at) needed to be successful. I mean, by that criteria, why learn anything? The world is not just startups btw, I work on products that have very very defined requirements. Hell, sometimes I'm implementing an RFC. FP is a huge win for us, and yes i've used profunctor optics in a high performance network application.
- a_imho 10y ago
  >I mean, by that criteria, why learn anything? This might be tongue in cheek, but I agree with the parent. Engineering excellence will not further your career much, companies tend to oversell meritocracy. Salaries flat out quickly, and there is a huge supply of good enough developers, if you step out of the line too much.
  - runT1ME 10y ago
    > Salaries flat out quickly What other professions besides lawyers or doctors regularly have a 3 or 4x range for salaries of people more or less with the same job description?
    - a_imho 10y ago
      In addition to those? Actors, musicians, sport people. In the office? Managers. Hence the advice, focus on soft skills. I would wager the top 1% on the developer track earn nowhere near to the top 10% on the managerial track. And your best bet is to regularly change jobs otherwise good luck getting anything more than a 5-10% increase, while managerial salaries and bonuses scale with the size of the organization. Also, there is a lot to be said about the difficulty of the work.
- deleted 10y ago
  [deleted]
sitkack 10y ago
I have had a number of projects that were absolutely successful from a technical standpoint because of FP methods. No one knew except team members that were unfamiliar with the techniques. While they were not full converts, many "saw the light" and reduced mutation and wrote purer functions after that. The business itself was oblivious to why the project was successful. Or how we were able to extend it so quickly and retain stability. Even on that project, 90% of the difficulty was social/political. Just as Java allowed average programmers to write above average code, the same thing can be said of FP.

pron 10y ago

> with the first style, productivity is constant at best—getting things together in the first place is a struggle, and then adding features or fixing bugs continues to be a struggle. On the other hand, the second style of project is even more of a struggle at first, but once it works it's like magic: new features are easier to add than you'd expect, and I've had way more things work after my first attempt than anyone has a right to expect. I would like to challenge that by pointing out that not a single piece of large software (except maybe a compiler or two) has ever been written using the skills/concepts of the higher levels of this chart. What you say may be true if generalized to better upfront design etc., but there's also no evidence that this is better done using the approaches discussed. It's not even about measuring productivity; it's about arguing over the properties of an empty set. When it comes to small (<100KLOC) and non-large (<1MLOC) programs, then the difficulties are not that hard to begin with, and depend mostly on the essential complexity of the problem, rather than abstraction/code organization etc.. > it's because I'm willing to put effort up front for a long-term reward. You're equating upfront effort with a particular choice of technique, and one that has never been put to the test.

loup-vaillant 10y ago
> not a single piece of large software (except maybe a compiler or two) has ever been written using the skills/concepts of the higher levels of this chart. I wonder: to what extent this is because those higher level concept tend to shrink programs in the first place? I mean, big is bad, there's no debating this. Big programs can only arise from necessity or stupidity. And if those fancy features are any use, they must be reducing the need for big. I would ask myself a slightly different question: what kind of program still have to be big, even if you have all these fine concepts at your disposal? And of course, big is risky, and that tend to make us chose more conservative options. Why use Haskell where C++ has shown in the past it could do that kind of job? Not to mention the network effects: even if Haskell as a language was better than C++ as some specific job, you may still choose C++ because of libraries, commercial support, and developer availability. > When it comes to small (<100KLOC) and non-large (<1MLOC) programs Oh, so that's how you're calibrated… My, to me, small means <10KLOC. 100KLOC is already big, and 1MLOC is gigantic. Besides, I've seen a couple multi-million lines programs, and I don't believe for a second they had to exceed 100KLOC. Such programs are more about piling historical accident on top of historical accident than about encoding a complex problem domain. A small compiler takes a couple thousand lines of code. With the proper tools, it can often be squeezed into 2KLOC (Source: the STEPS project from http://vpri.org http://vpri.org). It won't optimize like GCC, but you rarely have to anyway. Now you have a DSL for writing an executable specification. What kind of specification is so complex that it cannot even fit in 50 books? Of course you have to keenly understand the problem domain to pull that of, and that often means solving it the crappy way the first time around. Which means you're never going to rewrite it the proper way: it would be way too risky. So of course it has seldom been put to the test. One does not just hinge an entire business on original research. > You're equating upfront effort with a particular choice of technique, and one that has never been put to the test. No, not never. I have at least one example: one of my uncles once had to write a number of database transactions. It was one of his first job. He had 1 year to do it. Seeing how tedious it would be, he first though about the problem, then devised a DSL in which he could write the damn transactions. Since he wasn't exactly senior, and had crappy tools (he used B), the DSL took him about 6-7 months to perfect. (During which management were on the verge of panic, because no transaction has been done yet.) At the 8 month mark, all transactions were done, tested, and had a surprisingly low bug count. The client was very pleased and the contract was renewed for another batch of transactions. Same size, but to be done in 8 months this time. That was given to a co-worker, who used the my uncle's DSL to perform 8 months worth of contracted work in 1 month. DSLs sometimes work.