4 ms·
The algorithms themselves aren’t expensive, no. Having extra entire complete copies of the relationship-plus-posts graph, denormalized in various ways (incl. i
by derefr 2y ago
The algorithms themselves aren’t expensive, no.
Having extra entire complete copies of the relationship-plus-posts graph, denormalized in various ways (incl. in ways that inherently prevent use of easily map-reducible algorithms, and so require heavy vertical scaling) such that you can run the algorithms, is what’s expensive.
And constantly feeding the data into those denormalized models, using specially-tuned realtime ETL technologies that themselves do distributed scaling to ensure no infinite queue backlogging from activity bursts, is also expensive.
- photonthug 2y agoIn general i would argue that it is bloat that makes it not feasible to fund as a nonprofit, although people may have different ideas about where / what that bloat is. Recommendation algorithms, or unnecessary product changes, etc. I don’t think it’s true that it’s intrinsically impossible for a public service to be self funding, and I think that not everything has to grow / change forever to remain relevant. We need to figure this kind of stuff out, I mean Wikipedia is nice and all but it’s really bad that humanity in general has to rely on megacorp for things as basic as maps while we say we’re living in the Information Age