4 ms·
It seems like this will use up a crapton of bandwidth with its 'push' style notifications. If someone with a million Facebook followers makes a new post, one en
by pwf 14y ago
It seems like this will use up a crapton of bandwidth with its 'push' style notifications. If someone with a million Facebook followers makes a new post, one entry is made in the database and then users pull it down as they visit their own pages.
If a million people decide to 'camp in my tent' (?), my server is suddenly pushing out gigs of data every time I make a post.
- danielsiders 14y agoWe anticipate follower counts similar to the levels seen on Twitter today. Managing 1 few hundred followers isn't too difficult, thousands can be managed by cranking up the dynos on heroku. But yeah, Justin Bieber is going to need a hosted solution to handle that kind of load (like he has now with Twitter, YouTube, etc). We also have a setting in subscriber settings that control which types of posts are pushed in their entirety to different followers vs just a notification being sent vs nothing at all. So blog entries and status updates might get pushed to all 1M, but you probably wouldn't push an HD video update to all of them without some more serious server architecture.
- Torgo 14y agoWhat about allow a publisher to delegate a subscription endpoint via something like pubsubhubub? Then publisher just has to push to the aggregator, everybody does their pulls from them.
- mappu 14y agoThat only moves the load problem, doesn't solve it - given a decentralised system you're likely to be responsible for load on the aggregator server as well.
- pwf 14y agoWhat arguments do you have against a 'pull' style for notifications? Something like a GET to /latest that passes in the last time the client checked for updates and only gets back the URI of the last 10 or so posts since then. An HD video post could contain simply a URI that points to the video, so the video isn't even downloaded until the 'app' within tent asks for it. This way you could have a path with all the videos on it that you could proxy off to a server with more bandwidth. It would get the user a lot more control over how their content is accessed. Edit: And what if my server is down when one of my friends makes a post? Will I never see it?
- Titanous 14y agoPull-style requests actually use up quite a bit more bandwidth than push notifications, because every client and server has to query every few seconds instead of just getting notified when there is a new post. Posts have 'views', so a video or photo post would be pushed out with the 'meta' view which would include a URL to the content instead of the content inline.
- pwf 14y agoI don't think so, considering that not all of my followers are going to be using the service all of the time. If the server just blindly downloaded everything always, then yes, it would be more bandwidth. But with proper caching strategies and servers only requesting new posts when clients are active, I think you could save a lot of bandwidth by not pushing everything.
- allertonm 14y agoWhile the example of Justin Bieber makes it seem to make sense to push the cost of following to the followed rather than the follower, I am not at all sure this makes sense in the general case. It puts the user at the mercy of spammers & bots and ISTM making the follower bear the cost of following would disincentivize this kind of bad behaviour.
- lucaspiller 14y agoEven worse are pictures and videos. Rather than including a URL to it, the 'push' includes the base64 encoded binary data. Hmm.
- danielsiders 14y agoPosts have different "views" one of which is meta which would just have a URL pointing to the photo instead of including it in the post. See: http://tent.io/docs/post-types http://tent.io/docs/post-types
- drivebyacct2 14y agoGood, I was going to say, after evaluating PuSH I would have assumed that some things can be heavy pushes and other things can be light pushes.
- lucaspiller 14y agoAh ha, I missed that. Good work!
- kevingranade 14y agoStatus.net/OStatus uses a similar PUSH model, basically it scales with number of followers, and yes, if you have thousands of followers the load will be significant. As a data point, I've been running a status.net instance since last December, during which time I've had 50-80* followers, have made an average of 12 posts/day, have followed 100-190* others, and my instance has transferred a grand total of 10,279MB * over that entire period. followers and followeds gradually increased over the duration. I have this number handy since the connection is reverse-proxied via pagekite.net, which is metered. Over the last few months, I've had all static content offloaded to another server, which reduced the bandwith used, I'm not sure by how much.
- pwf 14y agoYou definitely seem like what I would consider a light user then. My wife has 1000+ Facebook friends, and posts 15-20 times a day. If I round to 1GB/mo in your case, then she would probably use around 20GB/mo... which is actually not bad. Thank you for the numbers!
- jsilence 14y agoWell, in a decentralized social network you'd have to make sure the tentpole you are living under is able to hold the weight. This is decentralized. In my expectation this network is not about a few celebs having millions of followers. I'd love to see it become peer to peer on the human level. Friends who are actually real friends, or at least people you have met and have had a real human interaction with.