2 ms·
PyTorch 1.12: TorchArrow, Functional API for Modules and NvFuser
- jaskyle 4y agoTorchArrow looks pretty cool: TorchArrow is a machine learning preprocessing library over batch data, providing performant and Pandas-style easy-to-use API for model development. Currently it provides a Python DataFrame that allows extensible UDFs with Velox, with the following features: - Seamless handoff with PyTorch or other model authoring, such as Tensor collation and easily plugging into PyTorch DataLoader and DataPipes - Zero copy for external readers via Arrow in-memory columnar format - Multiple execution runtimes support: - High-performance C++ UDF support with vectorization