4 ms·

At Discord we've actually spent a bit of time optimizing around frequently updating tables of data. On the extreme end of things, we have lists that contain se

by jhgg 10y ago

At Discord we've actually spent a bit of time optimizing around frequently updating tables of data. On the extreme end of things, we have lists that contain several thousand items that update a few elements that are potentially updating up to ten times a second. The biggest being the member list on a server. One of our servers has ~6500 online right now, with people coming online/offline/away every second, changing their current game, typing, etc... Few lessons we've learned: 1) Don't use ImmutableJS for small objects with a few keys containing scalar values, it's too slow. It's actually faster to just a) copy objects ({...oldValues, newKey: newValue}) or b) re-use objects and power shouldComponentUpdate in another fashion. In some cases, we use an "object version" that is incremented every time the object is actually updated. Then compare that in shouldComponentUpdate. 2) Only render what's visible. In our lists, each item has a fixed height, so we only render elements that are visible. This cuts down from having to render 6500 items in that list, to a good 20-30.

guscost 10y ago
Yep, the test is considering all the calls to get() and getIn() as part of the React render overhead but these methods are much slower than accessing properties directly. Running a comparison after removing the ImmutableJS state and adding version tags to the players (the game object can use its "clock" as a version), the production React implementation seems about twice as fast (1518 ms for React, 3377 ms for Vue, not at all scientific though). Here's the implementation without ImmutableJS: https://github.com/guscost/VueReactPerf https://github.com/guscost/VueReactPerf Note: Using version tags with shouldComponentUpdate is a great way to optimize for speed, but then you're responsible for updating all your versions manually. If you forget one, the component will "freeze" and you may spend some time tracking down the problem. Just a friendly reminder that immutable data and unoptimized components are both better in this respect.
- guscost 10y ago
  OK on a different computer I'm seeing 768 ms for React and 398 ms for Vue. Maybe I got the tests backwards before. Or maybe there is that much variability. I'll have to check again.
techterrier 10y ago
6500 items in a table..that sounds like a lot more than will fit on a screen?
- techterrier 10y ago
  ') Only render what's visible. In our lists, each item has a fixed height, so we only render elements that are visible. This cuts down from having to render 6500 items in that list, to a good 20-30.' silly me! Apologies!
leebyron 10y ago
Hey, author of Immutable.js here, You're definitely right to point out that Immutable.js is really not always what you want for small values that change all the time. Immutable.js collections are best used for just that: collections. If you have to copy an array or map with a ton of entries to make a change, it's going to be super slow - Persistent Immutable data structures can help with that. But I agree with your conclusion that any JS objects that are being used as small tuples are definitely better off staying plain JS objects. You also don't have to use all Immutable.js or nothing at all. Similar to how using an ES6 Map collection means you don't have to stop using JS objects. Use Immutable collections where they offer benefits, but spreading it everywhere is not a performance panacea, as you rightly discovered.
- jhgg 10y ago
  Love your work on Immutable.JS. When I was typing out the original reply, I meant to say "Don't use Immutable.JS for small objects" but ended up putting a period in the middle... which could have led to some confusion. I've edited it to make it more clear. We've also had some perf issues with using Immutable JS for even larger arrays. I think paying the cost of shallow copying the entire array when modifying it ended up winning in the long run as our render functions could loop over the array much faster.
morley 10y ago
> we only render elements that are visible. I'm curious how you keep track of visibility? I have a React app that does something similar to this by checking on scroll, but I kept getting killed by the browser having to check layout so many times on scroll, even if I throttled the event listener.
- jhgg 10y ago
  Since our elements have fixed height, we just do some simple maths in JS land to compute what should be visible, rather than measuring stuff using the DOM (which forces layouts).
dangoor 10y ago
I often point people to icepick which is a tiny library for working with immutable data in plain JS objects. It's a nice fit for when Immutable.js is too heavy https://github.com/aearly/icepick https://github.com/aearly/icepick
- acemarke 10y ago
  On that note, I've got a page devoted to immutable-data-related libs in my Redux addons page. Lists a couple purpose-built structure libs (Immutable and Mori), several data libs built around freezing, and a whole bunch of utility libs that provide abstractions around immutably updating plain objects: https://github.com/markerikson/redux-ecosystem-links/blob/master/immutable-data.md https://github.com/markerikson/redux-ecosystem-links/blob/ma... .

by jhgg 10y ago

guscost 10y ago

Yep, the test is considering all the calls to get() and getIn() as part of the React render overhead but these methods are much slower than accessing properties directly. Running a comparison after removing the ImmutableJS state and adding version tags to the players (the game object can use its "clock" as a version), the production React implementation seems about twice as fast (1518 ms for React, 3377 ms for Vue, not at all scientific though). Here's the implementation without ImmutableJS: https://github.com/guscost/VueReactPerf https://github.com/guscost/VueReactPerf Note: Using version tags with shouldComponentUpdate is a great way to optimize for speed, but then you're responsible for updating all your versions manually. If you forget one, the component will "freeze" and you may spend some time tracking down the problem. Just a friendly reminder that immutable data and unoptimized components are both better in this respect.

guscost 10y ago
OK on a different computer I'm seeing 768 ms for React and 398 ms for Vue. Maybe I got the tests backwards before. Or maybe there is that much variability. I'll have to check again.

techterrier 10y ago

6500 items in a table..that sounds like a lot more than will fit on a screen?

techterrier 10y ago
') Only render what's visible. In our lists, each item has a fixed height, so we only render elements that are visible. This cuts down from having to render 6500 items in that list, to a good 20-30.' silly me! Apologies!

leebyron 10y ago

Hey, author of Immutable.js here, You're definitely right to point out that Immutable.js is really not always what you want for small values that change all the time. Immutable.js collections are best used for just that: collections. If you have to copy an array or map with a ton of entries to make a change, it's going to be super slow - Persistent Immutable data structures can help with that. But I agree with your conclusion that any JS objects that are being used as small tuples are definitely better off staying plain JS objects. You also don't have to use all Immutable.js or nothing at all. Similar to how using an ES6 Map collection means you don't have to stop using JS objects. Use Immutable collections where they offer benefits, but spreading it everywhere is not a performance panacea, as you rightly discovered.

jhgg 10y ago
Love your work on Immutable.JS. When I was typing out the original reply, I meant to say "Don't use Immutable.JS for small objects" but ended up putting a period in the middle... which could have led to some confusion. I've edited it to make it more clear. We've also had some perf issues with using Immutable JS for even larger arrays. I think paying the cost of shallow copying the entire array when modifying it ended up winning in the long run as our render functions could loop over the array much faster.

morley 10y ago

> we only render elements that are visible. I'm curious how you keep track of visibility? I have a React app that does something similar to this by checking on scroll, but I kept getting killed by the browser having to check layout so many times on scroll, even if I throttled the event listener.

jhgg 10y ago
Since our elements have fixed height, we just do some simple maths in JS land to compute what should be visible, rather than measuring stuff using the DOM (which forces layouts).

dangoor 10y ago

I often point people to icepick which is a tiny library for working with immutable data in plain JS objects. It's a nice fit for when Immutable.js is too heavy https://github.com/aearly/icepick https://github.com/aearly/icepick

acemarke 10y ago
On that note, I've got a page devoted to immutable-data-related libs in my Redux addons page. Lists a couple purpose-built structure libs (Immutable and Mori), several data libs built around freezing, and a whole bunch of utility libs that provide abstractions around immutably updating plain objects: https://github.com/markerikson/redux-ecosystem-links/blob/master/immutable-data.md https://github.com/markerikson/redux-ecosystem-links/blob/ma... .