4 ms·

This is very cool. But the inherent limitation here is that it only works well for small datasets. If you have a million rows you don't want to download it all

by petilon 13y ago

This is very cool. But the inherent limitation here is that it only works well for small datasets. If you have a million rows you don't want to download it all into your web browser and summarize it in the browser. Instead you want to summarize it on the server and only download the summary to the web browser.

pc86 13y ago
Agreed; to be fair though, is it really appropriate to use a client-side pivot table library if your use case necessitates a server-side approach?
dspillett 13y ago
For a lot of data-sets the amount being passed around is going to be small enough though. I expect CPU power to be more of a limit than bandwidth: depending on what grouping options the user picks a large dataset could take a lot of time to draw making the interface feel very clunky, particularly if like us you have a chunk of your user-base stuck on classic IE instead of more modern browsers. We are working on something similar to enhance our app's existing custom reporting features. While we're erring in the side of "client side will be fine for our current requirements" if you don't mind the extra state storage server-side (or extra database hits to re-collect the data from the back-end each time) you could keep the data there and pull the results back via AJAX calls - with careful design you could even be share the slicing/dicing code between client & server and make the choice dynamically (if the data is small send it all to the client then you are as responsive as possible, if it is above a certain payload size keep it server-side to improve load time at the expense of interactive latency due to the AJAX calls as the user plays with the data). And/Or you could give the user the choice.
nicolaskruchten 13y ago
Emphatically agreed! We routinely use this tool to summarize datasets which are, at their most disaggregated, hundreds of millions of rows. We have a server-side process which rolls those up into hundreds of thousands of rows, which are then fed to the browser via a query interface which tends to send only a few thousand at a time, which is easily handled by this tool. Each step in this pipeline 'summarizes' to an extent, and the last mile is this interactive client-side tool, which would be less responsive if it were server-side.

by petilon 13y ago

pc86 13y ago

Agreed; to be fair though, is it really appropriate to use a client-side pivot table library if your use case necessitates a server-side approach?

dspillett 13y ago

For a lot of data-sets the amount being passed around is going to be small enough though. I expect CPU power to be more of a limit than bandwidth: depending on what grouping options the user picks a large dataset could take a lot of time to draw making the interface feel very clunky, particularly if like us you have a chunk of your user-base stuck on classic IE instead of more modern browsers. We are working on something similar to enhance our app's existing custom reporting features. While we're erring in the side of "client side will be fine for our current requirements" if you don't mind the extra state storage server-side (or extra database hits to re-collect the data from the back-end each time) you could keep the data there and pull the results back via AJAX calls - with careful design you could even be share the slicing/dicing code between client & server and make the choice dynamically (if the data is small send it all to the client then you are as responsive as possible, if it is above a certain payload size keep it server-side to improve load time at the expense of interactive latency due to the AJAX calls as the user plays with the data). And/Or you could give the user the choice.

nicolaskruchten 13y ago

Emphatically agreed! We routinely use this tool to summarize datasets which are, at their most disaggregated, hundreds of millions of rows. We have a server-side process which rolls those up into hundreds of thousands of rows, which are then fed to the browser via a query interface which tends to send only a few thousand at a time, which is easily handled by this tool. Each step in this pipeline 'summarizes' to an extent, and the last mile is this interactive client-side tool, which would be less responsive if it were server-side.