4 ms·

Why are parallel collections an "albatross"? I can understand it if he plans to replace the functionality you get by writing "list.par.map(func).seq" with "list

by tormeh 12y ago

Why are parallel collections an "albatross"? I can understand it if he plans to replace the functionality you get by writing "list.par.map(func).seq" with "list.parmap(func)" or something, though. Parallel list functions are one of my favorite features in Scala, and I think it would be a serious loss if they disappeared. Also, I wonder what the two Scala forkers are thinking about Dotty (the next-gen Typesafe scala compiler) and how they imagine merging their ideas in.

extempore 12y ago
It's not a question of whether they should exist at all but of whether they should infiltrate the collections hierarchy to the extent that someone has to maintain code which looks like this. And there's plenty of it: this is only an example. trait ParIterableViewLike[+T, +Coll <: Parallel, +CollSeq, +This <: ParIterableView[T, Coll, CollSeq] with ParIterableViewLike[T, Coll, CollSeq, This, ThisSeq], +ThisSeq <: IterableView[T, CollSeq] with IterableViewLike[T, CollSeq, ThisSeq]] extends GenIterableView[T, Coll] with GenIterableViewLike[T, Coll, This] with ParIterable[T] with ParIterableLike[T, This, ThisSeq]
- vidarh 12y ago
  As a recovering former C++ programmer, that reminds me of the verbosity of C++ meta programming...
  - jkbyc 12y ago
    Which language have you moved to?
- tormeh 12y ago
  Well, that's a pretty good reason to fix something, I agree. But what are your thoughts about Dotty? I assume it's insufficient for you, but is Typesafe going in the right direction here, is it not relevant to what you are doing or will it make things harder?
  - extempore 12y ago
    Dotty is unlikely to attain relevance - for me or as a future version of scala - because there's no reason to expect it to be any less of an implementation nightmare than is scalac. It is being written by the same person, applying the same engineering standards and the same checks and balances, so starting with a less unsound type system isn't going to matter.
    - seanmcdirmid 12y ago
      I worked on scalac for two years, so I know most of the same warts and issues; I also have had my differences with Martin in the past. But you know what: he has been courageously innovative and has come up with code that works pretty well considering the risks taken for Scala. Most people would have thought what Scala was trying to do was impossible (combining FP and OO with type inference? But semi-unification is undecidable!). But he did it, and you know what, scalac isn't that bad either. Heck, I've done much worse, but the point is to just keep trying. You want everything to be a nice well understood package because its "easier that way?" But sometimes it isn't easy and the code isn't pretty; not because the programmer is horrible, but because the problems are hard and not very well understood. Does that mean we should shy away from the problems and just do something safe and conservative like C#? If everyone thought like this, we would make no progress. Kudos to Martin for being fearless, and let's hope he doesn't cave.
      - extempore 12y ago
        That's right, I'm the stodgy conservative one and martin is the fearless innovator. Where is this version of the movie playing? I'd enjoy seeing it. Your second paragraph lands somewhere between completely wrong and "not even wrong" if it is supposed to have some bearing on the matter. You don't appear to know the first thing about what I've said, what I've done, or what is really going on here. Reflexive loyalty of this kind, unhinged from any objective reality, is what has brought us today's scalac. You should reconsider whether your having worked on the scala 2.7 eclipse plugin nearly a decade ago qualifies you to take a position on this. Then to position martin as the scrappy underdog - "let's hope he doesn't cave" - just let us know when the novel is finished, and I hope it's better than the movie.
runT1ME 12y ago
The collections library itself is probably one of the worst things about scala. I've ranted on the mailing list before, but it's overuse of inheritance rather than a type class based approach kills reasoning and modularity. Subtype polymorphic collections libraries are unquestionably a failed experiment.
- eeperson 12y ago
  Why do you feel that it is a failed experiment? As a user of the collections, it seems to me to work out. Is it more of a problem with the implementation of the collections?
  - zak_mc_kracken 12y ago
    Agreed. There is a vocal minority of people (academics mostly) yelling at how subtype polymorphism is broken and unusable. It certainly has broken aspects (with respect to equality) but by and large, it's extremely usable, intuitive and it enables powerful abstractions. The corner cases where it falls apart seem to be irrelevant for most uses.
  - runT1ME 12y ago
    I should have added that despite me not liking the std lib collections library, it's still stands above all other collection libs I've used in other languages. The problem is that the std. lib collections tries to unify mutable/immutable and lazy/eager/finite/infinte datastructures into one class hierarchy. So you have methods like "size" or "add" on generic interfaces that mean different things depending on the implementation. So the entire abstraction ends up being leaky. Scalaz has a few common implementations of immutable, eager data structures and ends up using a type class approach to modularity and code reuse that ends up not being leaky, it's a joy to use. I don't think there's anything wrong with using the collections, just be aware that if you run into some edge cases, it's not the language, it's design choices made by the library writer and there may be better implementations.
- frowaway001 12y ago
  I like dynamic dispatch. Having code which suddenly changes the implementation based on type ascriptions is just not what I want in this case. Apart from that, I think collections can be improved considerably even while keeping dynamic dispatch.

by tormeh 12y ago

extempore 12y ago

It's not a question of whether they should exist at all but of whether they should infiltrate the collections hierarchy to the extent that someone has to maintain code which looks like this. And there's plenty of it: this is only an example. trait ParIterableViewLike[+T, +Coll <: Parallel, +CollSeq, +This <: ParIterableView[T, Coll, CollSeq] with ParIterableViewLike[T, Coll, CollSeq, This, ThisSeq], +ThisSeq <: IterableView[T, CollSeq] with IterableViewLike[T, CollSeq, ThisSeq]] extends GenIterableView[T, Coll] with GenIterableViewLike[T, Coll, This] with ParIterable[T] with ParIterableLike[T, This, ThisSeq]

vidarh 12y ago
As a recovering former C++ programmer, that reminds me of the verbosity of C++ meta programming...
- jkbyc 12y ago
  Which language have you moved to?
tormeh 12y ago
Well, that's a pretty good reason to fix something, I agree. But what are your thoughts about Dotty? I assume it's insufficient for you, but is Typesafe going in the right direction here, is it not relevant to what you are doing or will it make things harder?
- extempore 12y ago
  Dotty is unlikely to attain relevance - for me or as a future version of scala - because there's no reason to expect it to be any less of an implementation nightmare than is scalac. It is being written by the same person, applying the same engineering standards and the same checks and balances, so starting with a less unsound type system isn't going to matter.
  - seanmcdirmid 12y ago
    I worked on scalac for two years, so I know most of the same warts and issues; I also have had my differences with Martin in the past. But you know what: he has been courageously innovative and has come up with code that works pretty well considering the risks taken for Scala. Most people would have thought what Scala was trying to do was impossible (combining FP and OO with type inference? But semi-unification is undecidable!). But he did it, and you know what, scalac isn't that bad either. Heck, I've done much worse, but the point is to just keep trying. You want everything to be a nice well understood package because its "easier that way?" But sometimes it isn't easy and the code isn't pretty; not because the programmer is horrible, but because the problems are hard and not very well understood. Does that mean we should shy away from the problems and just do something safe and conservative like C#? If everyone thought like this, we would make no progress. Kudos to Martin for being fearless, and let's hope he doesn't cave.
    - extempore 12y ago
      That's right, I'm the stodgy conservative one and martin is the fearless innovator. Where is this version of the movie playing? I'd enjoy seeing it. Your second paragraph lands somewhere between completely wrong and "not even wrong" if it is supposed to have some bearing on the matter. You don't appear to know the first thing about what I've said, what I've done, or what is really going on here. Reflexive loyalty of this kind, unhinged from any objective reality, is what has brought us today's scalac. You should reconsider whether your having worked on the scala 2.7 eclipse plugin nearly a decade ago qualifies you to take a position on this. Then to position martin as the scrappy underdog - "let's hope he doesn't cave" - just let us know when the novel is finished, and I hope it's better than the movie.

runT1ME 12y ago

The collections library itself is probably one of the worst things about scala. I've ranted on the mailing list before, but it's overuse of inheritance rather than a type class based approach kills reasoning and modularity. Subtype polymorphic collections libraries are unquestionably a failed experiment.

eeperson 12y ago
Why do you feel that it is a failed experiment? As a user of the collections, it seems to me to work out. Is it more of a problem with the implementation of the collections?
- zak_mc_kracken 12y ago
  Agreed. There is a vocal minority of people (academics mostly) yelling at how subtype polymorphism is broken and unusable. It certainly has broken aspects (with respect to equality) but by and large, it's extremely usable, intuitive and it enables powerful abstractions. The corner cases where it falls apart seem to be irrelevant for most uses.
- runT1ME 12y ago
  I should have added that despite me not liking the std lib collections library, it's still stands above all other collection libs I've used in other languages. The problem is that the std. lib collections tries to unify mutable/immutable and lazy/eager/finite/infinte datastructures into one class hierarchy. So you have methods like "size" or "add" on generic interfaces that mean different things depending on the implementation. So the entire abstraction ends up being leaky. Scalaz has a few common implementations of immutable, eager data structures and ends up using a type class approach to modularity and code reuse that ends up not being leaky, it's a joy to use. I don't think there's anything wrong with using the collections, just be aware that if you run into some edge cases, it's not the language, it's design choices made by the library writer and there may be better implementations.
frowaway001 12y ago
I like dynamic dispatch. Having code which suddenly changes the implementation based on type ascriptions is just not what I want in this case. Apart from that, I think collections can be improved considerably even while keeping dynamic dispatch.