Seeing the talk today, I think the Spark team is very aware that CPU cycles haven't become more abundant, while I/O has. There was an entire slide to this effec

Seeing the talk today, I think the Spark team is very aware that CPU cycles haven't become more abundant, while I/O has. There was an entire slide to this effect. I don't know enough about the space to say more than that, but it seems it's definitely on their mind. One of the things I remember being mentioned is operating on data in a much more efficient binary representation than native Java memory model, allowing better use of cache and such.