4 ms·

Lenses are cool, but they make me wonder: how many different paths of a deep immutable data hierarchy are transformed in an application to make this abstraction

by pron 4y ago

Lenses are cool, but they make me wonder: how many different paths of a deep immutable data hierarchy are transformed in an application to make this abstraction worthwhile, as opposed to replicating the entire traversal at each location? If the number is small then, however cool, this abstraction may be more trouble than it's worth. Just because you can reify some concept in an elegant composable construct doesn't mean that you should in the sense that it saves you a significant amount of effort. One way the number of paths could be (arbitrarily) large is if the transformations originate in user input, and the user can choose to arbitrarily transform any node in the structure. But then that input isn't typechecked (because it's not part of the program code), and requires interpretation and validation, so you might as well do the process of interpretation, validation, and transformation in one go using reflection. You don't lose the typechecking that you don't have in the first place, and the result is even more general. Now if you don't have reflection, lenses could make the interpretation of an input path much simpler (a switch of one level at a time), but Java does have reflection. So lenses solve a problem, but the question is: how big of a problem is it (especially in a language with reflection)?

ivan_gammel 4y ago
I’ve been playing with this idea around the time generics arrived in Java and never found a good use case for it. Number of transitions an object may have can usually be counted by fingers on one hand, meaning that you can have a method for each one and retain encapsulation, which lenses break by design.
endgame 4y ago
The real power IMHO comes once you start using traversals, which are like lenses but focus 0..n elements instead of exactly 1, and compose well with themselves and with lenses. (It gets better still after that, as you add in Folds, Isos, Prisms, etc., but we'll leave those for now) Once you have a traversal that can pull out immediate children of the same type (e.g., "given a DOM node, traverse all of its children"), you can use a library of transformations like Haskell's Control.Lens.Plated module from package "lens" and write queries and transformations over arbitrary structures in a very compact way. I have used this a few times: some examples include walking complex documents to extract particular information from tables, or rewriting every "import" node in a syntax tree, but leaving the rest of the program untouched. To support Traversals in Java under that get/set form, you would probably need get/set members that were functions to/from T and Array<T>, and then you'd have to write separate compose operators for each composition: - lens + lens = lens - lens + traversal = traversal - traversal + lens = traversal - traversal + traversal = traversal This is one reason that Haskell's "lens" package has that funky type alias for Lens instead of a record-of-functions, and a mature lens library is one of the main reasons that Haskell is my favourite general-purpose programming language. You _can_ force such a take on optics into Java. Someone found an implementation of profunctor optics in Minecraft's DataFixerUpper: https://www.reddit.com/r/programming/comments/9lyplq/microsoft_open_sources_parts_of_minecraft_java/e7aids2/ https://www.reddit.com/r/programming/comments/9lyplq/microso... That subthread has some really good meaty comments in it, if you're interested in this sort of thing.
- pron 4y ago
  > (e.g., "given a DOM node, traverse all of its children") you can use a library of transformations ... and write queries and transformations over arbitrary structures in a very compact way. But in Java you'd do that with reflection. So the question remains, how many such different queries/transformations you actually have (in the source code, where type checking is available, rather than user input) to make the effort-saving worthwhile?
  - chowells 4y ago
    Maybe you would rather do that with reflection, but I'd rather do something that isn't fundamentally the same tools that create security vulnerabilities. Every time you can not-use reflection is a win.
    - ivan_gammel 4y ago
      I think there’s some misunderstanding between the two of you. Reflection is indeed the sane and convenient solution for working with structures when type is not known at compile time (e.g. general purpose library for DI, ORM, security or serialization). It’s definitely not the way to manipulate object trees when type is known in compile type. However the point about complexity is valid: is it really worth patching strongly coupled solution violating SRP and breaking encapsulation with some design pattern for an anti-pattern (setters)?
    - pron 4y ago
      Records were specifically designed for safe reflection (and safe serialization). What causes security vulnerabilities is so-called deep reflection, i.e. the use of setAccessible, and can then violate various invariants. That's not what I'm referring to here. Reflection ≠ deep reflection.
  - kqr 4y ago
    Part of the convenience is in having uniform syntax for data access and modification, no matter what the target is. Is it "worthwhile" to use this when you have it? No doubt in my mind. Is it worthwhile to add a build time dependency for it? Maybe depends on application.
lmm 4y ago
Remember not to think about it "statically". If you're comfortable with lenses, then the cost of introducing more precise types and modelling all your states goes down, so you do more of it. E.g. maybe it becomes worth using a separate type for pre- and post-validation versions of some datastructure where before it wasn't, because you need to be able to traverse both versions with the same code, but now you can.
- pron 4y ago
  True, but that only pushes the question of value down the line. I'm curious about lenses because Java did have a serious problem that required a solution: working with "simple" data correctly was difficult. The chosen solution was ADTs, so we did buy into that. But the approach being explored for transforming records (https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md https://github.com/openjdk/amber-docs/blob/master/eg-drafts/...) only works one level at a time rather than for an entire path. So I wonder how valuable it would be to have a solution for paths. If the answer is that it's mostly valuable for an approach we haven't bought into yet, then we might not need to consider it just yet.
ryloric 4y ago
> Just because you can reify some concept in an elegant composable construct doesn't mean that you should in the sense that it saves you a significant amount of effort. aka monads lmao

by pron 4y ago

ivan_gammel 4y ago

I’ve been playing with this idea around the time generics arrived in Java and never found a good use case for it. Number of transitions an object may have can usually be counted by fingers on one hand, meaning that you can have a method for each one and retain encapsulation, which lenses break by design.

endgame 4y ago

The real power IMHO comes once you start using traversals, which are like lenses but focus 0..n elements instead of exactly 1, and compose well with themselves and with lenses. (It gets better still after that, as you add in Folds, Isos, Prisms, etc., but we'll leave those for now) Once you have a traversal that can pull out immediate children of the same type (e.g., "given a DOM node, traverse all of its children"), you can use a library of transformations like Haskell's Control.Lens.Plated module from package "lens" and write queries and transformations over arbitrary structures in a very compact way. I have used this a few times: some examples include walking complex documents to extract particular information from tables, or rewriting every "import" node in a syntax tree, but leaving the rest of the program untouched. To support Traversals in Java under that get/set form, you would probably need get/set members that were functions to/from T and Array<T>, and then you'd have to write separate compose operators for each composition: - lens + lens = lens - lens + traversal = traversal - traversal + lens = traversal - traversal + traversal = traversal This is one reason that Haskell's "lens" package has that funky type alias for Lens instead of a record-of-functions, and a mature lens library is one of the main reasons that Haskell is my favourite general-purpose programming language. You _can_ force such a take on optics into Java. Someone found an implementation of profunctor optics in Minecraft's DataFixerUpper: https://www.reddit.com/r/programming/comments/9lyplq/microsoft_open_sources_parts_of_minecraft_java/e7aids2/ https://www.reddit.com/r/programming/comments/9lyplq/microso... That subthread has some really good meaty comments in it, if you're interested in this sort of thing.

pron 4y ago
> (e.g., "given a DOM node, traverse all of its children") you can use a library of transformations ... and write queries and transformations over arbitrary structures in a very compact way. But in Java you'd do that with reflection. So the question remains, how many such different queries/transformations you actually have (in the source code, where type checking is available, rather than user input) to make the effort-saving worthwhile?
- chowells 4y ago
  Maybe you would rather do that with reflection, but I'd rather do something that isn't fundamentally the same tools that create security vulnerabilities. Every time you can not-use reflection is a win.
  - ivan_gammel 4y ago
    I think there’s some misunderstanding between the two of you. Reflection is indeed the sane and convenient solution for working with structures when type is not known at compile time (e.g. general purpose library for DI, ORM, security or serialization). It’s definitely not the way to manipulate object trees when type is known in compile type. However the point about complexity is valid: is it really worth patching strongly coupled solution violating SRP and breaking encapsulation with some design pattern for an anti-pattern (setters)?
  - pron 4y ago
    Records were specifically designed for safe reflection (and safe serialization). What causes security vulnerabilities is so-called deep reflection, i.e. the use of setAccessible, and can then violate various invariants. That's not what I'm referring to here. Reflection ≠ deep reflection.
- kqr 4y ago
  Part of the convenience is in having uniform syntax for data access and modification, no matter what the target is. Is it "worthwhile" to use this when you have it? No doubt in my mind. Is it worthwhile to add a build time dependency for it? Maybe depends on application.

lmm 4y ago

Remember not to think about it "statically". If you're comfortable with lenses, then the cost of introducing more precise types and modelling all your states goes down, so you do more of it. E.g. maybe it becomes worth using a separate type for pre- and post-validation versions of some datastructure where before it wasn't, because you need to be able to traverse both versions with the same code, but now you can.

pron 4y ago
True, but that only pushes the question of value down the line. I'm curious about lenses because Java did have a serious problem that required a solution: working with "simple" data correctly was difficult. The chosen solution was ADTs, so we did buy into that. But the approach being explored for transforming records (https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md https://github.com/openjdk/amber-docs/blob/master/eg-drafts/...) only works one level at a time rather than for an entire path. So I wonder how valuable it would be to have a solution for paths. If the answer is that it's mostly valuable for an approach we haven't bought into yet, then we might not need to consider it just yet.

ryloric 4y ago

> Just because you can reify some concept in an elegant composable construct doesn't mean that you should in the sense that it saves you a significant amount of effort. aka monads lmao