3 ms·
I suspect this rather obvious flaw (from the article: not taking deletion into account when calculating trends) is due to it being easier to create a data pipel
by telchar 5y ago
I suspect this rather obvious flaw (from the article: not taking deletion into account when calculating trends) is due to it being easier to create a data pipeline that is purely additive at the scale they're operating at. Not having to consider deletions and updates simplifies development. This is too much of a vulnerability not to fix, though. It amazes me Twitter somehow manages to remain relevant with how broken it is in various ways.
- zozbot234 5y agoDeletion is easy if you're using ML based on feature vectors to extract "trends", because you can just add one of the opposite vector and the system is still purely "additive" at scale.
- a1sabau 5y agoNot if you're using probabilistic data structures like Count-Min[0]. You don't care about the exact count. You just want an estimate with a certain probability. Such structures only support addition, not removal. [0] - https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch