5 ms·

My experiences with the encroachment of FP into otherwise imperative or OO languages has mostly been: 1. Write a bunch of functional code. 2. Marvel at how cl

My experiences with the encroachment of FP into otherwise imperative or OO languages has mostly been: 1. Write a bunch of functional code. 2. Marvel at how clean and neat things are. 3. Get customer reports of performance problems. 4. Spend a lot of time ripping out FP components and replacing them with imperative code.

peheje 4y ago
What languages? Did you performance profile before to see what was slow?
- kstenerud 4y ago
  C++, Python, Kotlin, some others I can't remember anymore. The most common issue is hidden O(n) situations, which don't show up except in production. So what we end up with is a mix of various paradigms worthy of Dr Frankenstein, and a debugging nightmare. I've been in engineering long enough to see most trends repeat at least twice (Javascript UIs are ALMOST back to the point of the 90s for example). After awhile you learn to distrust anyone who evangelizses a "new" way that was actually invented in the 60s (and is likely being introduced as a panacea instead of with the original caveats). Now that I'm a greybeard myself, I finally understand why greybeards were always so grumpy.
  - yobbo 4y ago
    > hidden O(n) situations How are these faster in imperative code?
    - occamrazor 4y ago
      They are usually not hidden, so they are at least easier to identify.
      - kstenerud 4y ago
        This is the crux of the matter. Every paradigm is argued about using small program examples, but the real-world issues come as the codebases become large (1M LOC or so). Then the cracks become worriesome. Imperative non-OO codebases become swamps of maybe (hopefully?) layered code, where each layer contains its own sinkholes and completely bizarre structure (or lack thereof). It's basically a bunch of mini-codebases that meet in an ad-hoc manner, making it very difficult to understand what the hell is going on. OO codebases encapsulate a LOT better, but working around its straitjacket design becomes harder and harder as you go through rituals like dependency injection, interfaces, adapters, facades, class clusters, and byzantine inheritance hierarchies. Now, rather than a swamp of despair, you have pebbles of functionality. And while these were great when your codebase was 50k LOC, it's now a gravel pile where you can't figure out where anything starts or ends. FP codebases like to push the details out of the way, making it easier to see at a high level what's going on. But as the codebase grows, you now have multiple iceberg problems, where the devils in the details begin to derail your code in production, and suddenly you find yourself needing to delve down rabbit hole after rabbit hole, chasing issues that weave and bob in utterly chaotic ways due to the lazy evaluation. Reasoning about what it will actually do in what time becomes more and more difficult (putting it in vulgar terms: You're basically exchanging topical complexity for temporal complexity). TANSTAAFL
        Capricorn2481 4y ago
        Do you pick the lesser of 3 evils, or are there solutions to these cracks?
        peheje 4y ago
        I haven't worked with a big FP codebase, but I actually thought about this when doing even small toy programs in FP. Sometimes I'm like... do they merge the multiple-sequential-maps that's possibly being done here? Does the code re-use the elements of map that doesn't change? Whereas was it C# or Go, I could just read it. OTOH isn't that just - you need to know what the function you are calling is doing and how, and read the doc. Whereas in imperative you would implement it yourself, with the pitfalls of that. If you are coding in Python and you are using a lot of numpy you need to know how to feed data in, how to get data out, etc. so maybe FP programming should treat the STD-lib more like an external library because the building blocks are a bit more high-level?
throwaway234232 4y ago
I've ran into this before, but to be fair I was using Monad Transformers.
ithrow 4y ago
That's the common theme even when using clojure too.
deleted 4y ago
[deleted]
ncmncm 4y ago
The big mistake is in believing there is any value in this- or that-oriented programming, or in a this- or that-oriented language. An engineer uses what is available to produce an economically balanced solution. Done well, that may involve a few OO-ish bits, some functional bits, data-oriented bits, reactive bits, plus hybrid combinations of those and others. The problem dictates the form of the solution. Every big-enough problem decomposes into a collection of very different subproblems, each of which so decomposes further. The right approach for the top level rarely matches that for most of its parts, nor each for the others, and likewise down the line. Unless you want to use a different language for each level and subproblem, you will want support for all "orientations" in your language. Thus, a "functional" or "OO" language is an absurdity; likewise, a "functional", "OO", or what-have-you program. They make as much sense as a saw-oriented carpenter's toolbox or table, or a screwdriver-oriented machinist's toolkit or engine. Programs should be like water, fluidly matching the shape of the problem solved. If that is 20% functional, 10% O-O, and the rest imperative, so be it. Specialization is for insects. Everything is Turing complete: you can code a solution in any restricted vocabulary. But that means succeeding proves nothing and benefits nobody, but wasted your time and others'. Use a language that does it all, and use it all. Not necessarily all on every program; but all styles get consideration for each part.
- lvass 4y ago
  "Subproblem" is likely an exaggeration here, otherwise I vehemently disagree. Take Erlang for example, it's theoretically a general purpose language but highly specialized. Would WhatsApp reach a billion users with 50 engineers without this specialization? I doubt it. It's absolutely fine for things to be heavily specialized, as it is fine for our entire civilization to rely division of labor. As the number of developers increase, so should the number of specializations. While a non-turing-complete language doesn't make much sense, web servers, linear algebra and GUIs for example are sufficiently different so that using the same language is unwise.
  - ncmncm 4y ago
    There are very successful insects.
kaba0 4y ago
Would you mind describing what sort of platform have you experienced such? Was it actually measured that the FP part was responsible for the slow downs? Let’s be honest, most typical software written by the industry is not the like which has a single hot loop. They do a bunch of different things, usually with small number of entries. I don’t see how FP would be a hindrance to this typical workload.
ParetoOptimal 4y ago
That's because step 4 is really: > 4. Spend a lot of time ripping out FP components i'm less familiar with improving the performance of and replacing them with imperative code I have years of experience improving the performance of. The cause of mist if these kinds of things will be dominated by familiarity for a long time.
Akronymus 4y ago
> Spend a lot of time ripping out FP components and replacing them with imperative code. Or use the functional code as a form of tests?

peheje 4y ago

What languages? Did you performance profile before to see what was slow?

kstenerud 4y ago
C++, Python, Kotlin, some others I can't remember anymore. The most common issue is hidden O(n) situations, which don't show up except in production. So what we end up with is a mix of various paradigms worthy of Dr Frankenstein, and a debugging nightmare. I've been in engineering long enough to see most trends repeat at least twice (Javascript UIs are ALMOST back to the point of the 90s for example). After awhile you learn to distrust anyone who evangelizses a "new" way that was actually invented in the 60s (and is likely being introduced as a panacea instead of with the original caveats). Now that I'm a greybeard myself, I finally understand why greybeards were always so grumpy.
- yobbo 4y ago
  > hidden O(n) situations How are these faster in imperative code?
  - occamrazor 4y ago
    They are usually not hidden, so they are at least easier to identify.
    - kstenerud 4y ago
      This is the crux of the matter. Every paradigm is argued about using small program examples, but the real-world issues come as the codebases become large (1M LOC or so). Then the cracks become worriesome. Imperative non-OO codebases become swamps of maybe (hopefully?) layered code, where each layer contains its own sinkholes and completely bizarre structure (or lack thereof). It's basically a bunch of mini-codebases that meet in an ad-hoc manner, making it very difficult to understand what the hell is going on. OO codebases encapsulate a LOT better, but working around its straitjacket design becomes harder and harder as you go through rituals like dependency injection, interfaces, adapters, facades, class clusters, and byzantine inheritance hierarchies. Now, rather than a swamp of despair, you have pebbles of functionality. And while these were great when your codebase was 50k LOC, it's now a gravel pile where you can't figure out where anything starts or ends. FP codebases like to push the details out of the way, making it easier to see at a high level what's going on. But as the codebase grows, you now have multiple iceberg problems, where the devils in the details begin to derail your code in production, and suddenly you find yourself needing to delve down rabbit hole after rabbit hole, chasing issues that weave and bob in utterly chaotic ways due to the lazy evaluation. Reasoning about what it will actually do in what time becomes more and more difficult (putting it in vulgar terms: You're basically exchanging topical complexity for temporal complexity). TANSTAAFL
      - Capricorn2481 4y ago
        Do you pick the lesser of 3 evils, or are there solutions to these cracks?
      - peheje 4y ago
        I haven't worked with a big FP codebase, but I actually thought about this when doing even small toy programs in FP. Sometimes I'm like... do they merge the multiple-sequential-maps that's possibly being done here? Does the code re-use the elements of map that doesn't change? Whereas was it C# or Go, I could just read it. OTOH isn't that just - you need to know what the function you are calling is doing and how, and read the doc. Whereas in imperative you would implement it yourself, with the pitfalls of that. If you are coding in Python and you are using a lot of numpy you need to know how to feed data in, how to get data out, etc. so maybe FP programming should treat the STD-lib more like an external library because the building blocks are a bit more high-level?

throwaway234232 4y ago

I've ran into this before, but to be fair I was using Monad Transformers.

ithrow 4y ago

That's the common theme even when using clojure too.

deleted 4y ago

[deleted]

ncmncm 4y ago

The big mistake is in believing there is any value in this- or that-oriented programming, or in a this- or that-oriented language. An engineer uses what is available to produce an economically balanced solution. Done well, that may involve a few OO-ish bits, some functional bits, data-oriented bits, reactive bits, plus hybrid combinations of those and others. The problem dictates the form of the solution. Every big-enough problem decomposes into a collection of very different subproblems, each of which so decomposes further. The right approach for the top level rarely matches that for most of its parts, nor each for the others, and likewise down the line. Unless you want to use a different language for each level and subproblem, you will want support for all "orientations" in your language. Thus, a "functional" or "OO" language is an absurdity; likewise, a "functional", "OO", or what-have-you program. They make as much sense as a saw-oriented carpenter's toolbox or table, or a screwdriver-oriented machinist's toolkit or engine. Programs should be like water, fluidly matching the shape of the problem solved. If that is 20% functional, 10% O-O, and the rest imperative, so be it. Specialization is for insects. Everything is Turing complete: you can code a solution in any restricted vocabulary. But that means succeeding proves nothing and benefits nobody, but wasted your time and others'. Use a language that does it all, and use it all. Not necessarily all on every program; but all styles get consideration for each part.

lvass 4y ago
"Subproblem" is likely an exaggeration here, otherwise I vehemently disagree. Take Erlang for example, it's theoretically a general purpose language but highly specialized. Would WhatsApp reach a billion users with 50 engineers without this specialization? I doubt it. It's absolutely fine for things to be heavily specialized, as it is fine for our entire civilization to rely division of labor. As the number of developers increase, so should the number of specializations. While a non-turing-complete language doesn't make much sense, web servers, linear algebra and GUIs for example are sufficiently different so that using the same language is unwise.
- ncmncm 4y ago
  There are very successful insects.

kaba0 4y ago

Would you mind describing what sort of platform have you experienced such? Was it actually measured that the FP part was responsible for the slow downs? Let’s be honest, most typical software written by the industry is not the like which has a single hot loop. They do a bunch of different things, usually with small number of entries. I don’t see how FP would be a hindrance to this typical workload.

ParetoOptimal 4y ago

That's because step 4 is really: > 4. Spend a lot of time ripping out FP components i'm less familiar with improving the performance of and replacing them with imperative code I have years of experience improving the performance of. The cause of mist if these kinds of things will be dominated by familiarity for a long time.

Akronymus 4y ago

> Spend a lot of time ripping out FP components and replacing them with imperative code. Or use the functional code as a form of tests?