4 ms·

31 libraries, zero for "Large Data Set". That tells you everything you need to know about Javascript/browser graphics for data science. A. The DOM cannot handl

31 libraries, zero for "Large Data Set". That tells you everything you need to know about Javascript/browser graphics for data science. A. The DOM cannot handle as many nodes as are needed for large datasets and/or B. even with Canvas/webgl etc the browser platform itself chokes on dataset sizes much bigger than 1/2 gig (ie, "small" data). So Javascript is fantastic for explanatory graphics, but forget about exploration.

nawitus 11y ago
Humans typically can't comprehend 500 MiB of data in a chart well, so the large data set is mapped into a smaller data set for visualization.
- vegabook 11y ago
  Completely agreed. Problem is while you're exploring to find the correct dimensional reduction / chuckout strategy, you're typically rapidly scanning through lots of scatter plots / density plots / whatever in grids and my experience is that the browser and javascript don't cut it here for reasons of throughput. If it's not the DOM that's blocking things, it's the commmunication channel to the browser that is too slow, and you can't dump all your data into the browser to avoid the communication slowness, because it then chokes. Chrome will barely allow a JS object a mere gig in size. Firefox dies well before that. This is not to say that we're not moving in the right direction on javascript/browser-based exploration, just that for now it's premature. BTW I'm succesfully using Jupyter (ipython notebook) as a hybrid solution. Bit of a best of both worlds. But then we're no longer talking JS for the graphics.
lmeyerov 11y ago
Our JS lib (not listed here) outperforms best-of-class native. We basically connect your browser's GPU to a GPU cluster in the cloud, but the data analyst doesn't care. We focus on real-time interactions with big graphs (e.g., connecting to your spark cluster) and include intensive visual computations like real-time edge bundling. We previously applied the same techniques to big time series (horizon) and treemaps. Happy to chat more if relevant. You can see a bit at graphistry.com .

nawitus 11y ago

Humans typically can't comprehend 500 MiB of data in a chart well, so the large data set is mapped into a smaller data set for visualization.

vegabook 11y ago
Completely agreed. Problem is while you're exploring to find the correct dimensional reduction / chuckout strategy, you're typically rapidly scanning through lots of scatter plots / density plots / whatever in grids and my experience is that the browser and javascript don't cut it here for reasons of throughput. If it's not the DOM that's blocking things, it's the commmunication channel to the browser that is too slow, and you can't dump all your data into the browser to avoid the communication slowness, because it then chokes. Chrome will barely allow a JS object a mere gig in size. Firefox dies well before that. This is not to say that we're not moving in the right direction on javascript/browser-based exploration, just that for now it's premature. BTW I'm succesfully using Jupyter (ipython notebook) as a hybrid solution. Bit of a best of both worlds. But then we're no longer talking JS for the graphics.

lmeyerov 11y ago

Our JS lib (not listed here) outperforms best-of-class native. We basically connect your browser's GPU to a GPU cluster in the cloud, but the data analyst doesn't care. We focus on real-time interactions with big graphs (e.g., connecting to your spark cluster) and include intensive visual computations like real-time edge bundling. We previously applied the same techniques to big time series (horizon) and treemaps. Happy to chat more if relevant. You can see a bit at graphistry.com .