4 ms·

This is probably going to send this whole thread on a wild tangent, but here goes... I've posted this before, but check out this link: http://www.elevatesoft.

This is probably going to send this whole thread on a wild tangent, but here goes... I've posted this before, but check out this link: http://www.elevatesoft.com:8081/maxgridtest/maxgridtest.html http://www.elevatesoft.com:8081/maxgridtest/maxgridtest.html That app was created with our web development product (check the site if you want to know more), and has the following characteristics: Two files, one HTML loader and one monolithic JS app, so latency during loading is minimal. The HTML loader is ~189K and the JS app is ~462K, and includes the entire runtime and UI layer, and a lot of the control/component library. Both the HTML and JS are aggressively compressed/obfuscated by a compiler, and the coding is done in a statically-typed, OO/procedural language with RTTI and other nice things. The UI was designed using a WYSIWYG designer with two-way tools (code-behind). So, there are products/tools out there that will do something along the lines of what you want. And the existing JS engines are very good in terms of performance, so all that developers like us need to do is some quick compilation to JS and we're all set. However, I do agree with you on two points: 1) JS, by itself, just isn't structured enough for large-scale applications. 2) The push towards libraries and away from frameworks was misguided because JS, by itself, doesn't have the means to allow for this approach to be successful. Instead, what we have now is every single small library reproducing the same functionality over and over again. Case in point: I was looking at writing an external interface (tells our compiler how to type-check external JS code) to ChartJS this week (great little library), and started looking at the code. 80-90% of the "common" code in the library was code that was already present, in some form, in our UI/runtime layer, and that was around 70K right there. Multiply this by the number of small libraries, and you end up with a lot of duplication of functionality that is, essentially, dead weight. I don't know if it's 10MB of dead weight, but it's pretty significant.

dsp1234 11y ago
190KB of maxgridtest.html took 1.44 s to download 462KB of maxgridtest.js took 2.63 s to download 5.2MB of datasets?method=rows&dataset=IPCountry&Country=%27United%20States%27 took 21.22 s to load "39246 rows load in 40948 msecs" - 41secs total just to see something! This is not an awesome presentation, and misses some of the stuff the previous commenters mention about "windows 3.1" like "don't load an entire dataset in memory, but show a small slice of it at a time". At the time, memory was very limited, so developers were forced to be efficient, and it was good for users. Even on my old 486 DOS machine, I could load up a spreadsheet application, and navigate through it (with way more than 40k rows) at nearly instant speed because it was smart about what it was doing and what it's limitations were (memory, disk access speed, etc)
- TimJYoung 11y ago
  Our product is for developing applications, not consumer-facing web sites. It is expected that the first hit on the HTML/JS is going to take more time than normal (we offer progress options for this). After the initial hit, the browser will cache both, and the load is instantaneous until it is updated again. Also, the total size/download time that you're seeing isn't for just a combo box and a grid, it's for the entire UI layer and a lot of the component library. IOW, as you add more and more to the application, you're only going to see very minor increments in the total size of the application because most of the code is already baked in. Most extensive client-side JS UI frameworks are at least 300K or so, minified. In retrospect, it was probably a bad idea to post that particular example without an explanation of why that app was created. The whole point of that particular example is to show that you can handle large numbers of rows in JS apps without killing the browser if your UI framework does smart things like implement virtual grids. Many other frameworks don't handle things very well when the number of rows grow beyond 1000 or so, and memory consumption gets out of hand very quickly.
  - dsp1234 11y ago
    After the initial hit, the browser will cache both, and the load is instantaneous until it is updated again. Ok. That's pretty much the definition of normal HTTP caching policy though. Many other frameworks don't handle things very well when the number of rows grow beyond 1000 or so, and memory consumption gets out of hand very quickly. Fair enough. Obviously you've got the UI down as it's butter smooth once everything is loaded. But many of the popular JavaScript grid systems already have stuff like virtual grids, and have had them for a while (scroller[0][1] for datatables was released in 2012 for example). They may not be quite as smooth as yours, but the DataTables example[0] (timing from the moment each has started loading of the data until there is some visual data on the screen) is less than 300ms, whereas yours is still over 10 seconds (not a 100% comparison as yours is 5MB of data). Definitely rather have people working on actual product, rather than waiting 30 seconds to do something (and each time the dropdown is switch to another country and back it's another 5MB download and 30 second wait). we offer progress options for this Then please, please show this. Showing people a demo of good code doing smart things is a great first impression. For example, a demo of 5 million rows of data via server side processing with that butter smooth animation would be amazing. Then when people are hooked, and start asking about doing stuff like loading 100% of their giant files into browser memory, you can impress them more with how there is no slowdown other than the pain they cause themselves by not loading data piecemail. But ultimately, any demo that takes 40 seconds to load is just going to be unsavory without a lot of context. [0] - http://datatables.net/extensions/scroller/examples/initialisation/large_js_source.html http://datatables.net/extensions/scroller/examples/initialis... - client side scrolling through 50k records (generated inline rather than downloaded, making for an unfair comparison) [1] - http://datatables.net/extensions/scroller/examples/initialisation/server-side_processing.html http://datatables.net/extensions/scroller/examples/initialis... - infinite scroll on server side processing of 5M records
icebraining 11y ago
It might be great for the developers, but frankly, as a user, I find that demo very disagreeable. It breaks multiple useful UI conventions that work fine in native browser components, and it still took 5s to download a dropdown and an empty table (for comparison, the HN frontpage takes <1s here).
- TimJYoung 11y ago
  Please see my reply above about the loading time. Also, which UI conventions are you referring to, specifically ?
  - icebraining 11y ago
    Also, which UI conventions are you referring to, specifically ? I'm from Portugal, so I click on the dropdown and press P. The standard behaviour is to jump to the first result, but this one automatically selects it, closing the dropdown from under me and starting an update. There's no horizontal scrollbar even when the content doesn't fit, so if I reduce the window horizontally, it becomes unusable. When I click on a row of the table, nothing changes; I must move the cursor to see it has in fact selected it. And the text is not selectable.
    - TimJYoung 11y ago
      Thanks for the feedback, it is very much appreciated. The near-search is just how it was coded (it's responding to a selection by loading the rows). It is a bit off-putting, though, so I'll change that. The horizontal scrollbar is the same thing. You can specify how you want the surface (body element) to behave with respect to scrolling, and the I turned off the scrolling. As for the clicking, that's the "hot" state over "focus" state preference in the UI layer. We're also thinking of changing that: it used to be "focus" over "hot", but it was changed, and I'm not sure that it was a good decision because of what you describe.
    - TimJYoung 11y ago
      I've updated the example to fix the issues that you mentioned. Also, the text isn't selectable on purpose. There's an option for our grid control to put it into "row select" mode where it is only used for navigation/selection (similar to a listview control under Windows).

dsp1234 11y ago

190KB of maxgridtest.html took 1.44 s to download 462KB of maxgridtest.js took 2.63 s to download 5.2MB of datasets?method=rows&dataset=IPCountry&Country=%27United%20States%27 took 21.22 s to load "39246 rows load in 40948 msecs" - 41secs total just to see something! This is not an awesome presentation, and misses some of the stuff the previous commenters mention about "windows 3.1" like "don't load an entire dataset in memory, but show a small slice of it at a time". At the time, memory was very limited, so developers were forced to be efficient, and it was good for users. Even on my old 486 DOS machine, I could load up a spreadsheet application, and navigate through it (with way more than 40k rows) at nearly instant speed because it was smart about what it was doing and what it's limitations were (memory, disk access speed, etc)

TimJYoung 11y ago
Our product is for developing applications, not consumer-facing web sites. It is expected that the first hit on the HTML/JS is going to take more time than normal (we offer progress options for this). After the initial hit, the browser will cache both, and the load is instantaneous until it is updated again. Also, the total size/download time that you're seeing isn't for just a combo box and a grid, it's for the entire UI layer and a lot of the component library. IOW, as you add more and more to the application, you're only going to see very minor increments in the total size of the application because most of the code is already baked in. Most extensive client-side JS UI frameworks are at least 300K or so, minified. In retrospect, it was probably a bad idea to post that particular example without an explanation of why that app was created. The whole point of that particular example is to show that you can handle large numbers of rows in JS apps without killing the browser if your UI framework does smart things like implement virtual grids. Many other frameworks don't handle things very well when the number of rows grow beyond 1000 or so, and memory consumption gets out of hand very quickly.
- dsp1234 11y ago
  After the initial hit, the browser will cache both, and the load is instantaneous until it is updated again. Ok. That's pretty much the definition of normal HTTP caching policy though. Many other frameworks don't handle things very well when the number of rows grow beyond 1000 or so, and memory consumption gets out of hand very quickly. Fair enough. Obviously you've got the UI down as it's butter smooth once everything is loaded. But many of the popular JavaScript grid systems already have stuff like virtual grids, and have had them for a while (scroller[0][1] for datatables was released in 2012 for example). They may not be quite as smooth as yours, but the DataTables example[0] (timing from the moment each has started loading of the data until there is some visual data on the screen) is less than 300ms, whereas yours is still over 10 seconds (not a 100% comparison as yours is 5MB of data). Definitely rather have people working on actual product, rather than waiting 30 seconds to do something (and each time the dropdown is switch to another country and back it's another 5MB download and 30 second wait). we offer progress options for this Then please, please show this. Showing people a demo of good code doing smart things is a great first impression. For example, a demo of 5 million rows of data via server side processing with that butter smooth animation would be amazing. Then when people are hooked, and start asking about doing stuff like loading 100% of their giant files into browser memory, you can impress them more with how there is no slowdown other than the pain they cause themselves by not loading data piecemail. But ultimately, any demo that takes 40 seconds to load is just going to be unsavory without a lot of context. [0] - http://datatables.net/extensions/scroller/examples/initialisation/large_js_source.html http://datatables.net/extensions/scroller/examples/initialis... - client side scrolling through 50k records (generated inline rather than downloaded, making for an unfair comparison) [1] - http://datatables.net/extensions/scroller/examples/initialisation/server-side_processing.html http://datatables.net/extensions/scroller/examples/initialis... - infinite scroll on server side processing of 5M records

icebraining 11y ago

It might be great for the developers, but frankly, as a user, I find that demo very disagreeable. It breaks multiple useful UI conventions that work fine in native browser components, and it still took 5s to download a dropdown and an empty table (for comparison, the HN frontpage takes <1s here).

TimJYoung 11y ago
Please see my reply above about the loading time. Also, which UI conventions are you referring to, specifically ?
- icebraining 11y ago
  Also, which UI conventions are you referring to, specifically ? I'm from Portugal, so I click on the dropdown and press P. The standard behaviour is to jump to the first result, but this one automatically selects it, closing the dropdown from under me and starting an update. There's no horizontal scrollbar even when the content doesn't fit, so if I reduce the window horizontally, it becomes unusable. When I click on a row of the table, nothing changes; I must move the cursor to see it has in fact selected it. And the text is not selectable.
  - TimJYoung 11y ago
    Thanks for the feedback, it is very much appreciated. The near-search is just how it was coded (it's responding to a selection by loading the rows). It is a bit off-putting, though, so I'll change that. The horizontal scrollbar is the same thing. You can specify how you want the surface (body element) to behave with respect to scrolling, and the I turned off the scrolling. As for the clicking, that's the "hot" state over "focus" state preference in the UI layer. We're also thinking of changing that: it used to be "focus" over "hot", but it was changed, and I'm not sure that it was a good decision because of what you describe.
  - TimJYoung 11y ago
    I've updated the example to fix the issues that you mentioned. Also, the text isn't selectable on purpose. There's an option for our grid control to put it into "row select" mode where it is only used for navigation/selection (similar to a listview control under Windows).