3 ms·
crashed with big file https://www.papaparse.com/resources/files/big.csv https://www.papaparse.com/resources/files/big.csv i evaluated sqlite for my tool and wro
by pdyc 2y ago
crashed with big file https://www.papaparse.com/resources/files/big.csv https://www.papaparse.com/resources/files/big.csv
i evaluated sqlite for my tool and wrote about it here https://newbeelearn.com/blog/million-rows-csv-debug-story/ https://newbeelearn.com/blog/million-rows-csv-debug-story/
BTW i checked your sqlite.wasm to be quite compact, just 340kb are you doing custom compilation.
- freakynit 2y agoHey, thanks for trying out. I did check out with the csv you linked. It worked for me. https://ibb.co/cb3SFbR https://ibb.co/cb3SFbR. Before making it live, I did already test it with 100MB+ CSV's having like 10+ columns. I also checked my sentry integration, but couldn't find any errors to your issue. Seems to me like OS/browser level ram restriction. In case you have anything to help me with debug this, do share.. Im using regular sqlite-wasm. No custom stuff. Thanks..
- itsbrex 2y agoWorks on my iPhone with the ‘big.csv’ file as well
- freakynit 2y agoThanks a lot for trying it out and verifying :)
- pdyc 2y agoi tested on machine with 4gb ram, also i modified the query to remove the limit i.e. "select * from data" can you let me know from where you are picking sqlite wasm? i checked here https://sqlite.org/download.html https://sqlite.org/download.html and sqlite wasm shared here is approx 830kb Edit: never mind checked here https://github.com/sql-js/sql.js/releases/tag/v1.12.0 https://github.com/sql-js/sql.js/releases/tag/v1.12.0 and its indeed around 330kb.
- kfajdsl 2y agoLoading that CSV and running `select * from data` left me with 879MB of memory usage on the tab.