3 ms·
I tested 1 million rows * 12 columns. It works fine with pure client's web-workers and indexedDB. For larger datasets(like CSV file > 150MB), especially for the
by loa_observer 4y ago
I tested 1 million rows * 12 columns. It works fine with pure client's web-workers and indexedDB. For larger datasets(like CSV file > 150MB), especially for the "mega automation" feature, It turns to be slow and sometimes lead crash in browsers.
For now, I have some temporally solutions, setting a sample size is the easiest.
Another way is put some heavy computation tasks to scaleable OLAP services, like a clickhouse clusters.
I will continue to research more ways to make the client handle as much data as possible and fast enough so more people can deploy it by their own with few costs.