Protobufs is a great comparison point and exactly what we upgraded from. It's like protobuf but specialized for rich data tables and, for those living outside g

Protobufs is a great comparison point and exactly what we upgraded from. It's like protobuf but specialized for rich data tables and, for those living outside google's proprietary layers, modern sensibilities. This brings wins like self-describing (no .proto, it optionally includes a schema), streaming, columnar by default, etc. By standardizing the notion of a table (and chunk within it, a record batch), it plugs into compute/io frameworks that work on tables: spark, pandas, etc. Most compute/db framework that gives you lists/tables will likely have out of the box support for it, and even when not explicit, via tools like turbodc for SQL DBs. Arrow has compute libs, but we don't use them (yet). More as interop for compute engines that are fast (ex: rapids for gpu) or feature rich (ex: pandas for chunks). Likewise, for I/O, interop for good formats there: on-the-fly arrow -> persistent parquet|orc. It's enabling us to do cool stuff like fast interop between our cpu<>gpu code, and in the latest initiative, crossing even process & language runtime boundaries w/ zero-copy.