Yeah, since spark 1.6 (some optimizations in spark 1.5) the data being operated on is managed "off heap" which means it doesn't add to garbage collection times

Yeah, since spark 1.6 (some optimizations in spark 1.5) the data being operated on is managed "off heap" which means it doesn't add to garbage collection times and is stored in a more contiguous and cache friendly layout. It's definitely using Unsafe but can't speak to the exact implementation.