4 ms·

Ten years ago the hash-map implementation in Internet Explorer was also O(n2). I had to inject javascript into some vendor code to avoid this after our product

Ten years ago the hash-map implementation in Internet Explorer was also O(n2). I had to inject javascript into some vendor code to avoid this after our production environment died. I ended up replacing the underlying hash-map into multiple smaller maps based on a hash of the items. So I'd have 50 maps of 1000 items instead of one map of 50000 items.

Etheryte 7y ago
I'm interested to know, what practical application did you load 50.000 items into the frontend for 10 years ago?
- masklinn 7y ago
  10 years ago is 2009. Large web applications had been breaking out of intranets (where they'd lived for quite some time at that point) for years at that point. GMail was launched in 2004, Google Maps in 2005 (that's also the year "AJAX" was coined) the JS library war was done and over with (jquery, prototype, mochikit, mootools, dojo, YUI, … were all released between 2005 and 2006). IE7 was 3 years old, Google Chrome was reaching its first year, IE8 was just released.
  - Etheryte 7y ago
    You seem to be missing the crux of the issue I was trying to address — even today, loading 50.000 items into a frontend application would be a very, very niche edge case. 10 years ago even more so. Tacking on arbitrary reference points from Wikipedia doesn't change any of that.
    - masklinn 7y ago
      > You seem to be missing the crux of the issue I was trying to address — even today, loading 50.000 items into a frontend application would be a very, very niche edge case. It's a medium-sized inventory, if you need fast / offline access then loading it to the client makes a lot of sense. I'm sure there are plenty other things of which you can easily reach 50k, and that you'd want to index or cross-reference somehow.
    - aflag 7y ago
      50k items seems actually quite small. Front loading data can make your app look a lot snappier at the cost of a slightly longer load time. It may be worth it in many situations.
    - mntmoss 7y ago
      I can think of plenty of tasks where a 2009 frontend will potentially be exposed to workloads in the 10k-100k elements range: * Editing objects in a 3D scene(e.g. a AAA open-world game or a CGI film) * Plotting events in a long-running log on a graph * Scraping and presenting data from web sources The common thread here is that you have most of your data in application memory but not necessarily in a formal database system, and so you shoulder the full burden of managing it properly. In most cases the solution is to define a real backend and filter it there, paginate or otherwise reduce the amount that gets presented because data at that scale won't be usable by humans to begin with. But sometimes you do have a reason to explicitly want a "big list of everything," and equally as often you end up with the big list of everything just by accident. It just comes with the territory of report generation tasks.
- laszlokorte 7y ago
  Almost 10 years ago (2010) I built a virtual table scroller in javascript for editing spreadsheet of 100 columns by 10.000 rows in the browser. I learned that just replacing the whole page's innerHTML by a concatinated string on each frame/mousewheel event is way faster then updating the cells content and position individually, and resuls in butter smooth scrolling.
- radicalbyte 7y ago
  I didn't, my vendor's developers did. It was a data visualization tool which let users make dashboards based on your data-warehouse. You could add filters to let users "slice-and-dice" the data in the dashboard. Needless to say one of the measures had a large number of parameters. The dashboard in question allowed users to see the change in measurements of [X] over time where [X] was one of 50k poisons that the Environment Agency were measuring. It a pretty reasonable question and - in browsers with O(n) hash lookup in their hashmaps - performed excellently.
- radicalbyte 7y ago
  For some context: at that time I was working on a Window Forms ERP tool whose planning module would regularly load and display several million items. It performed surprisingly well considering what it was doing. We also regularly had grid containing millions of items (or snapshots thereof).

Etheryte 7y ago

I'm interested to know, what practical application did you load 50.000 items into the frontend for 10 years ago?

masklinn 7y ago
10 years ago is 2009. Large web applications had been breaking out of intranets (where they'd lived for quite some time at that point) for years at that point. GMail was launched in 2004, Google Maps in 2005 (that's also the year "AJAX" was coined) the JS library war was done and over with (jquery, prototype, mochikit, mootools, dojo, YUI, … were all released between 2005 and 2006). IE7 was 3 years old, Google Chrome was reaching its first year, IE8 was just released.
- Etheryte 7y ago
  You seem to be missing the crux of the issue I was trying to address — even today, loading 50.000 items into a frontend application would be a very, very niche edge case. 10 years ago even more so. Tacking on arbitrary reference points from Wikipedia doesn't change any of that.
  - masklinn 7y ago
    > You seem to be missing the crux of the issue I was trying to address — even today, loading 50.000 items into a frontend application would be a very, very niche edge case. It's a medium-sized inventory, if you need fast / offline access then loading it to the client makes a lot of sense. I'm sure there are plenty other things of which you can easily reach 50k, and that you'd want to index or cross-reference somehow.
  - aflag 7y ago
    50k items seems actually quite small. Front loading data can make your app look a lot snappier at the cost of a slightly longer load time. It may be worth it in many situations.
  - mntmoss 7y ago
    I can think of plenty of tasks where a 2009 frontend will potentially be exposed to workloads in the 10k-100k elements range: * Editing objects in a 3D scene(e.g. a AAA open-world game or a CGI film) * Plotting events in a long-running log on a graph * Scraping and presenting data from web sources The common thread here is that you have most of your data in application memory but not necessarily in a formal database system, and so you shoulder the full burden of managing it properly. In most cases the solution is to define a real backend and filter it there, paginate or otherwise reduce the amount that gets presented because data at that scale won't be usable by humans to begin with. But sometimes you do have a reason to explicitly want a "big list of everything," and equally as often you end up with the big list of everything just by accident. It just comes with the territory of report generation tasks.
laszlokorte 7y ago
Almost 10 years ago (2010) I built a virtual table scroller in javascript for editing spreadsheet of 100 columns by 10.000 rows in the browser. I learned that just replacing the whole page's innerHTML by a concatinated string on each frame/mousewheel event is way faster then updating the cells content and position individually, and resuls in butter smooth scrolling.
radicalbyte 7y ago
I didn't, my vendor's developers did. It was a data visualization tool which let users make dashboards based on your data-warehouse. You could add filters to let users "slice-and-dice" the data in the dashboard. Needless to say one of the measures had a large number of parameters. The dashboard in question allowed users to see the change in measurements of [X] over time where [X] was one of 50k poisons that the Environment Agency were measuring. It a pretty reasonable question and - in browsers with O(n) hash lookup in their hashmaps - performed excellently.
radicalbyte 7y ago
For some context: at that time I was working on a Window Forms ERP tool whose planning module would regularly load and display several million items. It performed surprisingly well considering what it was doing. We also regularly had grid containing millions of items (or snapshots thereof).