5 ms·

> create abstractions which would require 100s or 1000s of lines of code to duplicate. I would like to see an example of this, I just don't think streaming lib

by skohan 5y ago

> create abstractions which would require 100s or 1000s of lines of code to duplicate. I would like to see an example of this, I just don't think streaming libraries, or abstractions in general, can get rid of that much complexity. In most cases, abstractions can't really get rid of complexity, they just move it around. The issue I have with streaming abstractions is that the implementation tends to be overly complex. For instance, Rx libraries tend to be thousands of lines of code, and they're often written in ways that make it impossible to trace code or understand a call-stack. It makes some things easier, but when you run into a problem it becomes a nightmare to debug. I'm just not sure it's warranted a majority of the time. Those libraries tend to be huge because they have to cover every single use-case, and they work at a very high level of abstraction. When you're working with stream-like domains, it's often possible to implement the subset of what you would need of that streaming abstraction with normal code, and in a way which is much easier to understand and debug.

chii 5y ago
> you run into a problem it becomes a nightmare to debug. this happens sometimes due to both java being a PITA to do functional programming with, and also partly due to the initial programmer of the streaming logic not breaking down into composable functions. And with most people being more familiar with imperative style, it makes the functional style harder to maintain _for them_.
- skohan 5y ago
  I'm not just talking about Java - I've also seen this in mobile projects in Swift for example. > And with most people being more familiar with imperative style, it makes the functional style harder to maintain for them. It has nothing to do with functional style. It's perfectly feasible to write functional code with minimal abstraction which is easy to trace and understand. The problem is in handing over your flow of control to an overly complex black box of an abstraction layer.
whateveracct 5y ago
> In most cases, abstractions can't really get rid of complexity, they just move it around. People say this pithy truism all the time, and it's just not true. You're probably just talking about indirection and encapsulation. Real abstractions by their very nature reduce complexity. You could say "No true Scotsman" but I deal with actual abstractions in Haskell all the time and it only simplifies code. So maybe Java is just deficient in its abstraction capability, but parametric polymorphism and pure streams sound like a good start to me. If a steam makes you uncomfy because it's doing O(n) things "under the hood" and you "like to know what the computer is doing" that's probably just a personal comprehension issue.
- skohan 5y ago
  Who said I was talking about Java? There are some abstractions which objectively reduce complexity: i.e. the C programming language abstracts over assembly, and essentially formalizes a subset of assembly in a way which reduces cognitive load for the programmer. This is a good abstraction, but this is not what streaming libraries do. > If a steam makes you uncomfy because it's doing O(n) things "under the hood" and you "like to know what the computer is doing" that's probably just a personal comprehension issue. If you have to reach for "you can only not like this because you don't understand this", it means you don't have a real argument. I have implemented event-stream-like systems before, and I am perfectly aware what they do "under the hood". I can give you an example of how these libraries go wrong. I was working a few years ago on a project which made heavy use of event streams, and one of my colleagues opened a PR which passed all unit tests locally, but failed when the CI system ran the tests. After about half a day of trying to figure out the problem, we determined that it was because a different scheduler was being used when the tests were run during the CI pipeline, which resulted in some concurrency related issues. What's worse, it didn't fail reliably, but only intermittently, so this made it extremely difficult to track down. This is what happens when you hand over your flow of control to another system: it makes it very difficult to reason about. Abstractions are good when they reduce boilerplate and provide a shorthand for monotonous work. When abstractions try to be smart or magical and do work for you, it's almost always a bad thing.
  - 5y ago

by skohan 5y ago

chii 5y ago

> you run into a problem it becomes a nightmare to debug. this happens sometimes due to both java being a PITA to do functional programming with, and also partly due to the initial programmer of the streaming logic not breaking down into composable functions. And with most people being more familiar with imperative style, it makes the functional style harder to maintain _for them_.

skohan 5y ago
I'm not just talking about Java - I've also seen this in mobile projects in Swift for example. > And with most people being more familiar with imperative style, it makes the functional style harder to maintain for them. It has nothing to do with functional style. It's perfectly feasible to write functional code with minimal abstraction which is easy to trace and understand. The problem is in handing over your flow of control to an overly complex black box of an abstraction layer.

whateveracct 5y ago

> In most cases, abstractions can't really get rid of complexity, they just move it around. People say this pithy truism all the time, and it's just not true. You're probably just talking about indirection and encapsulation. Real abstractions by their very nature reduce complexity. You could say "No true Scotsman" but I deal with actual abstractions in Haskell all the time and it only simplifies code. So maybe Java is just deficient in its abstraction capability, but parametric polymorphism and pure streams sound like a good start to me. If a steam makes you uncomfy because it's doing O(n) things "under the hood" and you "like to know what the computer is doing" that's probably just a personal comprehension issue.

skohan 5y ago
Who said I was talking about Java? There are some abstractions which objectively reduce complexity: i.e. the C programming language abstracts over assembly, and essentially formalizes a subset of assembly in a way which reduces cognitive load for the programmer. This is a good abstraction, but this is not what streaming libraries do. > If a steam makes you uncomfy because it's doing O(n) things "under the hood" and you "like to know what the computer is doing" that's probably just a personal comprehension issue. If you have to reach for "you can only not like this because you don't understand this", it means you don't have a real argument. I have implemented event-stream-like systems before, and I am perfectly aware what they do "under the hood". I can give you an example of how these libraries go wrong. I was working a few years ago on a project which made heavy use of event streams, and one of my colleagues opened a PR which passed all unit tests locally, but failed when the CI system ran the tests. After about half a day of trying to figure out the problem, we determined that it was because a different scheduler was being used when the tests were run during the CI pipeline, which resulted in some concurrency related issues. What's worse, it didn't fail reliably, but only intermittently, so this made it extremely difficult to track down. This is what happens when you hand over your flow of control to another system: it makes it very difficult to reason about. Abstractions are good when they reduce boilerplate and provide a shorthand for monotonous work. When abstractions try to be smart or magical and do work for you, it's almost always a bad thing.
- 5y ago