4 ms·

-- I'm not convinced that the overhead of copying immutable data doesn't outweigh the benefits of the ability to trivially parallelize the code. Let me introdu

by mark242 13y ago

-- I'm not convinced that the overhead of copying immutable data doesn't outweigh the benefits of the ability to trivially parallelize the code. Let me introduce you to my friend, Mister ConcurrentModificationException. He's a wily fellow-- he rarely pops up when you're doing local development, but he has this nasty habit of showing up in production whenever you start hitting a decent load. He's like Keyser Soze. You see him, then you try to debug the place he shows up, and poof he's gone, back into the shadows from whence he came, only to reappear when you try to iterate over that singleton java.util.ArrayList once more. Now, let me introduce you to my other friend, Miss scala.collection.immutable.List. She's a superheroine. You can run code using her on one thread, or ten threads, or ten thousand threads, she doesn't care, you're always going to get the same iterator back when you call List.foreach. For the very small price of a little bit of garbage (which Mister ParallelGC takes care of very readily), Miss immutable.List will return that filtered list the exact same way, every single time you run the filter method with those parameters you set. You can run code with her in development, staging, production, she's always ready to go, and you'll never run into Mister ConcurrentModificationException when you're dealing with Miss immutable.List. -- I'm not convinced that a project What do you mean? All I read was "I'm not convinced that a project". Someone else must have come by and edited your post before I was able to read it.

quacker 13y ago
singleton java.util.ArrayList So...don't use data structures that are not thread-safe in threaded environments? Wrap it with Collections.synchronizedCollection. Or make copies, if you prefer. For the very small price of a little bit of garbage... What small price? If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure (or at least some portion of it) when I want to modify anything inside of it?
- coopdog 13y ago
  It's actually all just pointers to data, so when you change something it makes a new node and copies a few new pointers so the new tree is still correct. All of the old data and pointers that aren't changed stay exactly where they are.
- mark242 13y ago
  Ah, synchronizedCollection, the... hang on... wait a sec... one more sec, someone's doing something... now? no, not yet, wait... how about... NOW! synchronizedCollection, the global lock for Java's mutable datatypes. Here's the fundamental difference between Java and Scala: if I say, in Scala, val people = List[String]("quacker", "mark242") then by default people is an immutable list. I don't have to do anything special. In Java, either I'm suddenly using the Guava libraries to get something similiar (but not the same), or I'm doing all kinds of funky dances around list iterators, or I'm using Arrays.copyOf, or what have you. In any case, you have all of this extraneous code, when it isn't necessary. Quick: give me a list comprehension method in Java that takes a list of Strings and returns that list, filtered, of Strings that are only five characters or longer. Make it null-safe and thread-safe. This isn't difficult-- you're thinking about 6-7 lines of Java code in your head, right? Null check, synchronized, new ArrayList, for(s in sList), that kind of thing, right? In Scala, it's this. Some would argue you don't even need a separate method. def getFiveCharacters(s: List[String]) = s.filter(_.length >= 5) If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure when I want to modify anything within? Dear god, man, what are you doing wrong in your Java code? This is the exact scenario where you want an immutable list, otherwise you'll be running synchronized code and precisely one core of your CPU will be glowing red like lava, while your other CPU cores sit idle. I just did this in the scala repl: (0 to 1000000).toList.par.filter(_ % 100 == 0) What that does is grab all of the integers from 0 to 1,000,000, convert them to an immutable List, then filter that List (yes-- an iterator! with a filtered copy!) by only taking numbers divisible by 100. The whole thing takes a couple hundred ms in the repl (which is fantastic, since it's compiling then executing the code) and almost no memory overhead. For code that is absolutely thread safe. And the ".par" makes this run in parallel on all my CPU cores.
  - anonymoushn 13y ago
    > Null check, synchronized, new ArrayList, for(s in sList), that kind of thing, right? > In Scala, it's this. Some would argue you don't even need a separate method. It seems a bit unfair to force null checks on the Java code and then present a Scala alternative that throws NPE in two different places.
- dragonwriter 13y ago
  > What small price? If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure (or at least some portion of it) when I want to modify anything inside of it? Depends on the nature of the modification; to choose the right immutable data structure, you want to choose the one that has the best behavior under your expected use. Pushing something on to the head of an immutable list doesn't require a copy, the new list has the new object as the head and the old list (not a copy) as the tail. Similarly, removing the head doesn't require a copy, it just returns the tail of the existing list. If you want to replace something in the middle of the list, you'll need to copy everything closer to the head than the modified item. Different immutable collections have different "cheap" access patterns.
seanmcdirmid 13y ago
>Let me introduce you to my friend, Mister ConcurrentModificationException. He's a wily fellow-- he rarely pops up when you're doing local development, but he has this nasty habit of showing up in production whenever you start hitting a decent load. You do realize that concurrency and parallelization are very different topics right? One is a necessary property, the other is one of oppurtunity. It makes sense that collections designed for concurrency would suck at parallelism.
- mark242 13y ago
  Here is the magic of Scala. A collection designed for concurrency: val users = List("seanmcdirmid", "mark242").reverse That same collection, designed for parallelism and concurrency: val users = List("seanmcdirmid", "mark242").par.reverse
  - seanmcdirmid 13y ago
    The APIs are equivalent and safe, but parallelism is a feature of opportunity. The overhead of using an immutable collection is often not overcome by the use of parallel threads, making the tradeoff not worth it. Even CUDA has brought mutability back to GPUs for performance reasons.
deleted 13y ago
[deleted]
ExpiredLink 13y ago
> Let me introduce you to my friend, Mister ConcurrentModificationException. Thanks for reminding me of him. I have never seen him during my last five years of programming in Java. I hope he's alive and well.

by mark242 13y ago

quacker 13y ago

singleton java.util.ArrayList So...don't use data structures that are not thread-safe in threaded environments? Wrap it with Collections.synchronizedCollection. Or make copies, if you prefer. For the very small price of a little bit of garbage... What small price? If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure (or at least some portion of it) when I want to modify anything inside of it?

coopdog 13y ago
It's actually all just pointers to data, so when you change something it makes a new node and copies a few new pointers so the new tree is still correct. All of the old data and pointers that aren't changed stay exactly where they are.
mark242 13y ago
Ah, synchronizedCollection, the... hang on... wait a sec... one more sec, someone's doing something... now? no, not yet, wait... how about... NOW! synchronizedCollection, the global lock for Java's mutable datatypes. Here's the fundamental difference between Java and Scala: if I say, in Scala, val people = List[String]("quacker", "mark242") then by default people is an immutable list. I don't have to do anything special. In Java, either I'm suddenly using the Guava libraries to get something similiar (but not the same), or I'm doing all kinds of funky dances around list iterators, or I'm using Arrays.copyOf, or what have you. In any case, you have all of this extraneous code, when it isn't necessary. Quick: give me a list comprehension method in Java that takes a list of Strings and returns that list, filtered, of Strings that are only five characters or longer. Make it null-safe and thread-safe. This isn't difficult-- you're thinking about 6-7 lines of Java code in your head, right? Null check, synchronized, new ArrayList, for(s in sList), that kind of thing, right? In Scala, it's this. Some would argue you don't even need a separate method. def getFiveCharacters(s: List[String]) = s.filter(_.length >= 5) If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure when I want to modify anything within? Dear god, man, what are you doing wrong in your Java code? This is the exact scenario where you want an immutable list, otherwise you'll be running synchronized code and precisely one core of your CPU will be glowing red like lava, while your other CPU cores sit idle. I just did this in the scala repl: (0 to 1000000).toList.par.filter(_ % 100 == 0) What that does is grab all of the integers from 0 to 1,000,000, convert them to an immutable List, then filter that List (yes-- an iterator! with a filtered copy!) by only taking numbers divisible by 100. The whole thing takes a couple hundred ms in the repl (which is fantastic, since it's compiling then executing the code) and almost no memory overhead. For code that is absolutely thread safe. And the ".par" makes this run in parallel on all my CPU cores.
- anonymoushn 13y ago
  > Null check, synchronized, new ArrayList, for(s in sList), that kind of thing, right? > In Scala, it's this. Some would argue you don't even need a separate method. It seems a bit unfair to force null checks on the Java code and then present a Scala alternative that throws NPE in two different places.
dragonwriter 13y ago
> What small price? If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure (or at least some portion of it) when I want to modify anything inside of it? Depends on the nature of the modification; to choose the right immutable data structure, you want to choose the one that has the best behavior under your expected use. Pushing something on to the head of an immutable list doesn't require a copy, the new list has the new object as the head and the old list (not a copy) as the tail. Similarly, removing the head doesn't require a copy, it just returns the tail of the existing list. If you want to replace something in the middle of the list, you'll need to copy everything closer to the head than the modified item. Different immutable collections have different "cheap" access patterns.

seanmcdirmid 13y ago

>Let me introduce you to my friend, Mister ConcurrentModificationException. He's a wily fellow-- he rarely pops up when you're doing local development, but he has this nasty habit of showing up in production whenever you start hitting a decent load. You do realize that concurrency and parallelization are very different topics right? One is a necessary property, the other is one of oppurtunity. It makes sense that collections designed for concurrency would suck at parallelism.

mark242 13y ago
Here is the magic of Scala. A collection designed for concurrency: val users = List("seanmcdirmid", "mark242").reverse That same collection, designed for parallelism and concurrency: val users = List("seanmcdirmid", "mark242").par.reverse
- seanmcdirmid 13y ago
  The APIs are equivalent and safe, but parallelism is a feature of opportunity. The overhead of using an immutable collection is often not overcome by the use of parallel threads, making the tradeoff not worth it. Even CUDA has brought mutability back to GPUs for performance reasons.

deleted 13y ago

[deleted]

ExpiredLink 13y ago

> Let me introduce you to my friend, Mister ConcurrentModificationException. Thanks for reminding me of him. I have never seen him during my last five years of programming in Java. I hope he's alive and well.