6 ms·
nice, is this something you've tried in WASM or WebGL/WebGPU? at some point it makes sense to simply recommend using WASM SQLite (or DuckDB) to load the data f
by leeoniya 3y ago
nice, is this something you've tried in WASM or WebGL/WebGPU?
at some point it makes sense to simply recommend using WASM SQLite (or DuckDB) to load the data for OLAP stuff. im not gonna encourage anyone to use JS on multi-GB files. even a few hundred MB becomes questionable ;)
- texodus 3y agoThe CSV parser for [Perspective](https://github.com/finos/perspective https://github.com/finos/perspective) uses the [Apache Arrow](https://github.com/apache/arrow https://github.com/apache/arrow) C++ CSV parser compiled to WASM. It's not currently multi-threaded but this is possible as well to my understanding.
- leeoniya 3y agowhat i learned from benchmarking Dekkai, is that simply "being in WASM" is a poor indicator of actual performance if you ever need to pull the values across that boundary into JS...especially strings. i'll take a look and see if i can incorporate Perspective's parser into the bench, thanks!