5 ms·

I'm self-taught and while I can see the formal appeal of functional programming, I'm a low-level coder that implicitly reduces state to bare-metal registers and

by varelse 11y ago

I'm self-taught and while I can see the formal appeal of functional programming, I'm a low-level coder that implicitly reduces state to bare-metal registers and cache in GPUs and embedded systems. Since I'm already sweating bits over state down to the byte-level, I just don't see what FP would improve on this.

peterfirefly 11y ago
The big improvements often come from better algorithms and data structures. Using a function style or a higher-level language sometimes makes it easier to modify the program (and hence use a better algorithm). I remember people swearing by assembly language back in the 80's, thinking C would make their programs slow and bloated. In reality, it was usually the other way around because C code is much more malleable than assembly. You might want to give FP a second look.
- varelse 11y ago
  There is a very active attempt to make GPUs FP-friendly called C++AMP. And for streaming transformation tasks, it's a nice abstraction. https://en.wikipedia.org/wiki/C%2B%2B_AMP https://en.wikipedia.org/wiki/C%2B%2B_AMP However, my best algorithms are stylistically equivalent to a map-reduce with combiners entirely in GPU-space. The map tasks themselves carry all the scope I need. They are independent units of work executed by independent warps (synchronized groups of 32 GPU threads) whose results need to be reduced to floating point numbers. That reduction is done with 64-bit fixed point atomic ops (my combiners) because: 1) This insures the sum is deterministic 2) Ever since Kepler (GK104), it's much faster than reduction buffers and uses a fraction of the memory 3) It's possible because I can precompute the expected dynamic range and adjust the fixed point exponent accordingly Now given that, do you see a functional programming equivalent that significantly improves on this design? I haven't yet.
  - peterfirefly 11y ago
    map-reduce is already functional, isn't it? Who cares if you implement the combining with atomic ops or by sending data to a process that does the combining or whether you stuff the data in a buffer and then reduce it afterwards? (Of course you might care for performance reasons -- but conceptually it's the same thing.)
    - varelse 11y ago
      Yes, it is true conceptually, but: 1) "Who cares" is not the sort of thing you want to say to someone who cares about performance because: 2) Sending the data to buffers for subsequent reduction was the first implementation (2009). But it was a memory hog and a 5% or so slowdown to perform the reduction subsequently rather than concurrently with Atomic Ops and 64-bit Fixed Point (2012). But I guess what we're arriving at is that this is essentially a functional design using imperative code? I can live with that.
      - peterfirefly 11y ago
        > But I guess what we're arriving at is that this is essentially a functional design using imperative code? I can live with that. Yep. That's precisely what it is.
mlitchard 11y ago
Here's the relationship. How can you guarantee your low-level code is correct? Here's one way. You have a language like Idris create a DSL that targets the machine you want. You can model what the set of correct programs in your domain look like, and then produce correct code. That's why functional programming in general, and dependent-type programming in particular, is useful for low-level problem domains.
- varelse 11y ago
  That's not a problem. The application in question was a redesign of an MPI FORTRAN implementation that can produce the same outputs to within FPRE. To verify the low-level code, we just run in double-precision and make sure it matches to 1 part in 10e-9 so across 200+ daily conformance tests. Production mode runs in single-precision with 64-bit Fixed Point accumulation. And that's also been demonstrated to have numerical stability nearly equivalent to running in full double-precision.
  - mlitchard 11y ago
    It's not a problem for you, but it's a problem generally. That's why so many people are working to solve it.
its_me_ken_lai 11y ago
I don't think this applies to you. The main message of the article/blog is that 1) Humans can't keep track of states well and 2) Machines can't create temporary states efficiently and since you're already keeping track of states manually and working at the low enough level where you're maintaining your temporary states, you don't apply.

by varelse 11y ago

peterfirefly 11y ago

The big improvements often come from better algorithms and data structures. Using a function style or a higher-level language sometimes makes it easier to modify the program (and hence use a better algorithm). I remember people swearing by assembly language back in the 80's, thinking C would make their programs slow and bloated. In reality, it was usually the other way around because C code is much more malleable than assembly. You might want to give FP a second look.

varelse 11y ago
There is a very active attempt to make GPUs FP-friendly called C++AMP. And for streaming transformation tasks, it's a nice abstraction. https://en.wikipedia.org/wiki/C%2B%2B_AMP https://en.wikipedia.org/wiki/C%2B%2B_AMP However, my best algorithms are stylistically equivalent to a map-reduce with combiners entirely in GPU-space. The map tasks themselves carry all the scope I need. They are independent units of work executed by independent warps (synchronized groups of 32 GPU threads) whose results need to be reduced to floating point numbers. That reduction is done with 64-bit fixed point atomic ops (my combiners) because: 1) This insures the sum is deterministic 2) Ever since Kepler (GK104), it's much faster than reduction buffers and uses a fraction of the memory 3) It's possible because I can precompute the expected dynamic range and adjust the fixed point exponent accordingly Now given that, do you see a functional programming equivalent that significantly improves on this design? I haven't yet.
- peterfirefly 11y ago
  map-reduce is already functional, isn't it? Who cares if you implement the combining with atomic ops or by sending data to a process that does the combining or whether you stuff the data in a buffer and then reduce it afterwards? (Of course you might care for performance reasons -- but conceptually it's the same thing.)
  - varelse 11y ago
    Yes, it is true conceptually, but: 1) "Who cares" is not the sort of thing you want to say to someone who cares about performance because: 2) Sending the data to buffers for subsequent reduction was the first implementation (2009). But it was a memory hog and a 5% or so slowdown to perform the reduction subsequently rather than concurrently with Atomic Ops and 64-bit Fixed Point (2012). But I guess what we're arriving at is that this is essentially a functional design using imperative code? I can live with that.
    - peterfirefly 11y ago
      > But I guess what we're arriving at is that this is essentially a functional design using imperative code? I can live with that. Yep. That's precisely what it is.

mlitchard 11y ago

Here's the relationship. How can you guarantee your low-level code is correct? Here's one way. You have a language like Idris create a DSL that targets the machine you want. You can model what the set of correct programs in your domain look like, and then produce correct code. That's why functional programming in general, and dependent-type programming in particular, is useful for low-level problem domains.

varelse 11y ago
That's not a problem. The application in question was a redesign of an MPI FORTRAN implementation that can produce the same outputs to within FPRE. To verify the low-level code, we just run in double-precision and make sure it matches to 1 part in 10e-9 so across 200+ daily conformance tests. Production mode runs in single-precision with 64-bit Fixed Point accumulation. And that's also been demonstrated to have numerical stability nearly equivalent to running in full double-precision.
- mlitchard 11y ago
  It's not a problem for you, but it's a problem generally. That's why so many people are working to solve it.

its_me_ken_lai 11y ago

I don't think this applies to you. The main message of the article/blog is that 1) Humans can't keep track of states well and 2) Machines can't create temporary states efficiently and since you're already keeping track of states manually and working at the low enough level where you're maintaining your temporary states, you don't apply.