3 ms·
SSDs as RAM + projects like Tachyon are the future of big data processing -- current workflows are still way too slow. I wonder how this will affect Spark.
by egillie 12y ago
SSDs as RAM + projects like Tachyon are the future of big data processing -- current workflows are still way too slow. I wonder how this will affect Spark.
- sixdimensional 12y agoI am not a silver bullet kind of person, but I think this kind of technology plus things like Spark are potentially the holy grail. Ok, let me rephrase - I've often believed that there was a sort of "ultimate architecture" with distributed cluster computing, distributed programming frameworks, high-speed interconnect, and flash based storage. I think we are getting closer and closer each day. I wish I worked in R&D at any of the companies doing this fundamental research now - I have a feeling they have some amazing things cooked up behind closed doors.
- progman 12y ago> SSDs as RAM Not yet. I think first we will have SSD as static RAM plus dynamic DIMM RAM as Cache. I wonder when SSD will compete with DIMM regarding maximum number of write operations.