6 ms·

Thanks! Maple looks really cool - really interesting demo video, too! This is quite interesting - we've not explored really large files so far and being honest

Thanks! Maple looks really cool - really interesting demo video, too! This is quite interesting - we've not explored really large files so far and being honest, we haven't thought that far either. Didn't know about CBOR! I will have to look deeper into how this can save on memory. I was wondering though, since WASM memory is limited to 4GB, if I have sufficiently large memory on my compute device, at least one tab should be able to handle 1gb+ files too, correct?

hitradostava 3y ago
Thanks :-) I've not done much profiling on DuckDB and what the overhead is - i.e. after the data is parsed how much memory is used. Would be really interesting to push it to the limit - or to explore not loading the entire file in, but only reading the relevant parts, but again that probably requires a conversion first, e.g. to parquet or some other column based storage format.
- prasoonds 3y ago
  that sounds like a fun project :) maybe I'll take a stab at it!