3 ms·

History of Apache Arrow: Developing a standard for in-memory columnar processing

• kwillets 8y ago
  >In the Big Data world, we didn’t see much talk about columnar data until Google’s Dremel paper in 2010 Try 2005: http://db.csail.mit.edu/projects/cstore/#papers http://db.csail.mit.edu/projects/cstore/#papers We had a very large Vertica installation up in 2009. But as Abadi has noted it's an in-memory format, and it removes a lot of bottlenecks at the CPU-Memory interface. In that sense it's quite new; we've been wrestling with tricky in-memory formats for some time.