5 ms·

Except for all the research around functional programming? This is like when I hear people claim that physics has not advanced in the last 50-70 years.

by epgui 3y ago

charcircuit 3y ago
Functional programming is not based off how hardware is implemented. Serial execution of instructions and mutating chunks of memory at a time are all core parts of how the hardware works which aren't functional. Doing graph reduction and making tons of copies will be slow.
- epgui 3y ago
  Hardware was for a very long time a limiting factor in the practicality of FP. For most general applications, today this is a relative non-issue. FP is also particularly well suited for cloud computing and parallel computation.
  - charcircuit 3y ago
    The parent comment was talking about aligning programming languages with the hardware. I am not commenting about the viablity of those languages, but rather that if your goal is to write the most performant code by understanding the strengths and weaknesses of the hardware than using fp concepts is not the way to do it.
    - epgui 3y ago
      I feel like in both of your comments you've changed the topic slightly. I responded to the following comment, which I interpreted literally: > Very little innovation in programming languages has happened regarding new realities at the hardware level especially transition from serial to parallel execution
    - AnimalMuppet 3y ago
      Well... if code were pure (in the FP sense), then a "sufficiently smart compiler" could move it around to extract the maximum performance. But, as always, the sufficiently smart compiler never shows up. So we're left with the humans doing the tuning, and as you say, FP is kind of antithetical to that approach.
      - Chris_Newton 3y ago
        But how will language performance evolve as the nature of the hardware our programs run on evolves? IMHO this is not an easy question to answer right now. C compilers don’t produce 100% optimal assembly language in all cases, but typically the assumptions they make are light. The executable code they output is somewhat predictable and often close enough to hand-optimised assembly in efficiency that we ignore the difference. But this whole approach to programming was originally designed for single-threaded execution on a CPU with a chunk of RAM for storage. What happens if we never find a way to get a single core to run much faster but processors come with ever more cores and introduce other new features for parallel execution? What happens if we evolve towards ever more distributed systems, but farming out big jobs to a set of specialised components in the cloud at a much lower level than we do today? What happens if systems start coming with other kinds of memory that have different characteristics to RAM as standard, from content-addressable memory we already have today to who-knows-what as quantum technology evolves? If we change the rules then maybe a different style of programming will end up being more efficient. It’s true that today’s functional programming languages that control mutation and other side effects usually don’t compile down to machine code as efficiently as a well-written C program can. The heavier runtimes to manage responsibilities like garbage collection and the reliance on purely functional data structures that we don’t yet know how to convert to efficient imperative code under the hood are bottlenecks. But on the other hand, those languages can make much stronger assumptions than a lower-level language like C in other ways, and maybe those assumptions will allow compilers to safely allocate different behaviour to new hardware in ways that weren’t possible before, and maybe dividing a big job into 500 quantum foobar jobs that each run half as fast as a single-threaded foobaz job still ends up doing the job 200x faster overall.
        3y ago
  - sidlls 3y ago
    "FP" camps tend to come in two flavors: "I'm a mathematician writing a computer program, and all problems will be made to look like math problems even if it means the program becomes an inscrutable mess of types and dense syntax" and "functional-ish idioms are included". The latter is useful, sometimes, for cloud computing and parallel computation; the former tends to have too many problems (slow build, slow execution, poor jargon laden syntax, etc.) to be very useful outside of academia. I am (perhaps obviously) biased, here, but I tend to just roll my eyes whenever any of my colleagues suggests we should use functional programming to solve a problem. There are actually very few real-world use cases where it's objectively better.
    - wredue 3y ago
      Is “wanting evidence for constant silver bullet claims that never actually pan out” really “obviously biased”? It’s not just you. Functional Programming really does not adequately solve any of the problems that its advocates claim while refusing to provide any evidence. And it’s not “biased” to write these claims off.
      - epgui 3y ago
        I don't know what "silver bullet" claims you've heard, but I'm not sure how that is relevant to this thread. I don't think I've made any outlandish claims, or any claims that aren't substantiated by a preponderance of academic literature.
        roenxi 3y ago
        I find this whole thread fascinating because nobody seems to have identified what functional programming is defined as (programmers can just use functions to code and end up with functional programming, it isn't an obscure style), who the advocates are or what they claim. And yet, without any substance, the debate rages.
        epgui 3y ago
        > nobody seems to have identified what functional programming is defined as Nobody asked (until now, so thank you for asking!), and this is a fairly well discussed topic for anyone who cares to search! You will probably get a lot of slightly different answers depending on who you ask or where you look, but I think a very strong common thread is "referential transparency". Functional programming gives you that, and that is the property that makes FP particularly well suited for parallel computation. Referential transparency is related to the concept of "function purity" (in the sense that either one usually guarantees the other), which you will often hear people talk about as well. The two concepts are so intimately tied that sometimes I wonder if they're two different perspectives on the same thing. This, along with the fact that FP has been an active area of research (an important part of innovation) for a long time, is why I brought it up. https://en.wikipedia.org/wiki/Functional_programming https://en.wikipedia.org/wiki/Functional_programming https://en.wikipedia.org/wiki/Pure_function https://en.wikipedia.org/wiki/Pure_function https://en.wikipedia.org/wiki/Referential_transparency https://en.wikipedia.org/wiki/Referential_transparency https://softwareengineering.stackexchange.com/questions/293851/what-is-it-about-functional-programming-that-makes-it-inherently-adapted-to-para https://softwareengineering.stackexchange.com/questions/2938... --- > programmers can just use functions to code and end up with functional programming, it isn't an obscure style That's not how it works. Note that functional programming has nothing to do with merely "writing functions". --- > without any substance, the debate rages The substance is there and there is plenty of it, but learning requires work.
- markusde 3y ago
  You're mixing up functional programming with an execution model for functional languages. These are not the same.
  - charcircuit 3y ago
    Research around functional languages involves their execution models. Even ignoring execution models immutability is a staple of functional programming which is not good for performance.
- kaba0 3y ago
  Are we still doing this stupid ass reasoning around FP? Is the CPU really that serial, when it literally reschedules your instructions based on the graph of their interconnections? Also, just think about all the optimizations your "serial" programming language does -- are your yourself really write all those mutations? Or is that the compiler, that in many cases can do a much better job? Now what if the language's semantics allowed even more freedom for the compiler in exchange for more restrictions on the language? Sure, we still don't have "sufficiently advanced compilers" that would replace programmers, but FP languages absolutely trade blows with most imperative languages in many different kinds of problems. Very visibly when parallelism comes to the picture, as it turns out, a smart parallel-aware data structure will easily beat out their serial counterparts here.
  - ThrowawayR2 3y ago
    > "Is the CPU really that serial, when it literally reschedules your instructions based on the graph of their interconnections?" Yes. Yes, it is because all of that rescheduling and reordering is completely hidden at great effort and expense to make it seem like the instruction stream is executing in exactly in the order specified. If it weren't, lines of code would essentially execute in an indeterminate order and no programs would function.
nh2 3y ago
Futhark (https://futhark-lang.org https://futhark-lang.org) is an example of a functional language specifically designed for writing "normal" high-level functional code that compiles to parallel execution.
- epgui 3y ago
  Genuine question-- is there something about Futhark that makes it particularly well-suited for parallel execution compared to any other functional programming language (especially of the purely functional kind)? FP in general is inherently well suited for this application. As I understand it, Futhark aims to leverage GPUs in particular, and that approach seems to be what makes it unique within the category of FP languages?
  - Athas 3y ago
    Functional languages have a core that is well suited for parallelism, but all mainstream functional languages have various features that are unsuitable for parallel execution - particularly if you want to target limited machines like GPUs. The Futhark programming model is standard functional combinators: map, reduce, scan, etc. You can do that in any Functional language, and it is mostly trivial to rewrite a Futhark program in Haskell or SML or OCaml. Futhark removes a lot of things (like recursive data structures) to make efficient parallel execution easier (and even possible), and adds various minor conveniences. But you don't really need to add something for functional programming to be suitable for parallelism; you just have to avoid certain common things (like linked lists or laziness).

Except for all the research around functional programming? This is like when I hear people claim that physics has not advanced in the last 50-70 years.

by epgui 3y ago

Except for all the research around functional programming? This is like when I hear people claim that physics has not advanced in the last 50-70 years.

charcircuit 3y ago

Functional programming is not based off how hardware is implemented. Serial execution of instructions and mutating chunks of memory at a time are all core parts of how the hardware works which aren't functional. Doing graph reduction and making tons of copies will be slow.

epgui 3y ago
Hardware was for a very long time a limiting factor in the practicality of FP. For most general applications, today this is a relative non-issue. FP is also particularly well suited for cloud computing and parallel computation.
- charcircuit 3y ago
  The parent comment was talking about aligning programming languages with the hardware. I am not commenting about the viablity of those languages, but rather that if your goal is to write the most performant code by understanding the strengths and weaknesses of the hardware than using fp concepts is not the way to do it.
  - epgui 3y ago
    I feel like in both of your comments you've changed the topic slightly. I responded to the following comment, which I interpreted literally: > Very little innovation in programming languages has happened regarding new realities at the hardware level especially transition from serial to parallel execution
  - AnimalMuppet 3y ago
    Well... if code were pure (in the FP sense), then a "sufficiently smart compiler" could move it around to extract the maximum performance. But, as always, the sufficiently smart compiler never shows up. So we're left with the humans doing the tuning, and as you say, FP is kind of antithetical to that approach.
    - Chris_Newton 3y ago
      But how will language performance evolve as the nature of the hardware our programs run on evolves? IMHO this is not an easy question to answer right now. C compilers don’t produce 100% optimal assembly language in all cases, but typically the assumptions they make are light. The executable code they output is somewhat predictable and often close enough to hand-optimised assembly in efficiency that we ignore the difference. But this whole approach to programming was originally designed for single-threaded execution on a CPU with a chunk of RAM for storage. What happens if we never find a way to get a single core to run much faster but processors come with ever more cores and introduce other new features for parallel execution? What happens if we evolve towards ever more distributed systems, but farming out big jobs to a set of specialised components in the cloud at a much lower level than we do today? What happens if systems start coming with other kinds of memory that have different characteristics to RAM as standard, from content-addressable memory we already have today to who-knows-what as quantum technology evolves? If we change the rules then maybe a different style of programming will end up being more efficient. It’s true that today’s functional programming languages that control mutation and other side effects usually don’t compile down to machine code as efficiently as a well-written C program can. The heavier runtimes to manage responsibilities like garbage collection and the reliance on purely functional data structures that we don’t yet know how to convert to efficient imperative code under the hood are bottlenecks. But on the other hand, those languages can make much stronger assumptions than a lower-level language like C in other ways, and maybe those assumptions will allow compilers to safely allocate different behaviour to new hardware in ways that weren’t possible before, and maybe dividing a big job into 500 quantum foobar jobs that each run half as fast as a single-threaded foobaz job still ends up doing the job 200x faster overall.
      - 3y ago
- sidlls 3y ago
  "FP" camps tend to come in two flavors: "I'm a mathematician writing a computer program, and all problems will be made to look like math problems even if it means the program becomes an inscrutable mess of types and dense syntax" and "functional-ish idioms are included". The latter is useful, sometimes, for cloud computing and parallel computation; the former tends to have too many problems (slow build, slow execution, poor jargon laden syntax, etc.) to be very useful outside of academia. I am (perhaps obviously) biased, here, but I tend to just roll my eyes whenever any of my colleagues suggests we should use functional programming to solve a problem. There are actually very few real-world use cases where it's objectively better.
  - wredue 3y ago
    Is “wanting evidence for constant silver bullet claims that never actually pan out” really “obviously biased”? It’s not just you. Functional Programming really does not adequately solve any of the problems that its advocates claim while refusing to provide any evidence. And it’s not “biased” to write these claims off.
    - epgui 3y ago
      I don't know what "silver bullet" claims you've heard, but I'm not sure how that is relevant to this thread. I don't think I've made any outlandish claims, or any claims that aren't substantiated by a preponderance of academic literature.
      - roenxi 3y ago
        I find this whole thread fascinating because nobody seems to have identified what functional programming is defined as (programmers can just use functions to code and end up with functional programming, it isn't an obscure style), who the advocates are or what they claim. And yet, without any substance, the debate rages.
        epgui 3y ago
        > nobody seems to have identified what functional programming is defined as Nobody asked (until now, so thank you for asking!), and this is a fairly well discussed topic for anyone who cares to search! You will probably get a lot of slightly different answers depending on who you ask or where you look, but I think a very strong common thread is "referential transparency". Functional programming gives you that, and that is the property that makes FP particularly well suited for parallel computation. Referential transparency is related to the concept of "function purity" (in the sense that either one usually guarantees the other), which you will often hear people talk about as well. The two concepts are so intimately tied that sometimes I wonder if they're two different perspectives on the same thing. This, along with the fact that FP has been an active area of research (an important part of innovation) for a long time, is why I brought it up. https://en.wikipedia.org/wiki/Functional_programming https://en.wikipedia.org/wiki/Functional_programming https://en.wikipedia.org/wiki/Pure_function https://en.wikipedia.org/wiki/Pure_function https://en.wikipedia.org/wiki/Referential_transparency https://en.wikipedia.org/wiki/Referential_transparency https://softwareengineering.stackexchange.com/questions/293851/what-is-it-about-functional-programming-that-makes-it-inherently-adapted-to-para https://softwareengineering.stackexchange.com/questions/2938... --- > programmers can just use functions to code and end up with functional programming, it isn't an obscure style That's not how it works. Note that functional programming has nothing to do with merely "writing functions". --- > without any substance, the debate rages The substance is there and there is plenty of it, but learning requires work.
markusde 3y ago
You're mixing up functional programming with an execution model for functional languages. These are not the same.
- charcircuit 3y ago
  Research around functional languages involves their execution models. Even ignoring execution models immutability is a staple of functional programming which is not good for performance.
kaba0 3y ago
Are we still doing this stupid ass reasoning around FP? Is the CPU really that serial, when it literally reschedules your instructions based on the graph of their interconnections? Also, just think about all the optimizations your "serial" programming language does -- are your yourself really write all those mutations? Or is that the compiler, that in many cases can do a much better job? Now what if the language's semantics allowed even more freedom for the compiler in exchange for more restrictions on the language? Sure, we still don't have "sufficiently advanced compilers" that would replace programmers, but FP languages absolutely trade blows with most imperative languages in many different kinds of problems. Very visibly when parallelism comes to the picture, as it turns out, a smart parallel-aware data structure will easily beat out their serial counterparts here.
- ThrowawayR2 3y ago
  > "Is the CPU really that serial, when it literally reschedules your instructions based on the graph of their interconnections?" Yes. Yes, it is because all of that rescheduling and reordering is completely hidden at great effort and expense to make it seem like the instruction stream is executing in exactly in the order specified. If it weren't, lines of code would essentially execute in an indeterminate order and no programs would function.

nh2 3y ago

Futhark (https://futhark-lang.org https://futhark-lang.org) is an example of a functional language specifically designed for writing "normal" high-level functional code that compiles to parallel execution.

epgui 3y ago
Genuine question-- is there something about Futhark that makes it particularly well-suited for parallel execution compared to any other functional programming language (especially of the purely functional kind)? FP in general is inherently well suited for this application. As I understand it, Futhark aims to leverage GPUs in particular, and that approach seems to be what makes it unique within the category of FP languages?
- Athas 3y ago
  Functional languages have a core that is well suited for parallelism, but all mainstream functional languages have various features that are unsuitable for parallel execution - particularly if you want to target limited machines like GPUs. The Futhark programming model is standard functional combinators: map, reduce, scan, etc. You can do that in any Functional language, and it is mostly trivial to rewrite a Futhark program in Haskell or SML or OCaml. Futhark removes a lot of things (like recursive data structures) to make efficient parallel execution easier (and even possible), and adds various minor conveniences. But you don't really need to add something for functional programming to be suitable for parallelism; you just have to avoid certain common things (like linked lists or laziness).