2 ms·

> Before this new surge in activity, the firehose would produce around 24 GB/day of traffic. The firehose is all public data going into the network right? Isn

> Before this new surge in activity, the firehose would produce around 24 GB/day of traffic. The firehose is all public data going into the network right? Isn't that pretty tiny for a worldwide social network? And the fact one country can cause a 10x surge in traffic also suggests its worldwide footprint must be tiny...

londons_explore 2y ago
24GB/day is very much a "we could host this on one server with a bit of read caching" scale. Every year or so, you plop in one more 10TB SSD.
- ericvolp12 2y ago
  Yes the actual record content on the network isn't huge at the moment but the firehose doesn't include blobs (images and videos) which take up significantly more space. Either way, yeah it's pretty lightweight. Total number of records on the network is around 2.5Bn in the ~1.5 years Bluesky has been around. I aggregated some stats when we hit 10M users here - https://bsky.app/profile/did:plc:q6gjnaw2blty4crticxkmujt/post/3l47puwmjnl2x https://bsky.app/profile/did:plc:q6gjnaw2blty4crticxkmujt/po...
ericvolp12 2y ago
The 10x surge in traffic was us gaining 3.5M new users over the course of a week (growing the entire userbase by >33%) and all these users have been incredibly active on a daily basis. Lots of these numbers are public and the impact of the surge can be seen here: https://bskycharts.edavis.dev/static/dynazoom.html?plugin_name=edavis.dev%2Fbskycharts.edavis.dev%2Fbsky_users_total&start_iso8601=2024-04-22T11%3A15%3A23-0700&stop_iso8601=2024-09-24T11%3A15%3A23-0700&start_epoch=1713809723&stop_epoch=1727201723&lower_limit=&upper_limit=&size_x=800&size_y=400&cgiurl_graph=%2Fmunin-cgi%2Fmunin-cgi-graph https://bskycharts.edavis.dev/static/dynazoom.html?plugin_na... Note the graphs in that link only show users that take a publicly visible action (i.e. post, like, follow, etc.) and won't show lurkers at all.
str4d 2y ago
> The firehose is all public data going into the network right? It's the "main subset" of the public data, being "events on the network": for the Bluesky app that's posts, reposts, replies, likes, etc. Most of those records are just metadata (e.g. a repost record references the post being reposted, rather than embedding it). Post / reply records include the post text (limited to 300 graphemes). In particular, the firehose traffic does _not_ include images or videos; it only includes references to "blob"s. Clients separately fetch those blobs from the PDSs (or from a CDN caching the data).
deleted 2y ago
[deleted]

londons_explore 2y ago

24GB/day is very much a "we could host this on one server with a bit of read caching" scale. Every year or so, you plop in one more 10TB SSD.

ericvolp12 2y ago
Yes the actual record content on the network isn't huge at the moment but the firehose doesn't include blobs (images and videos) which take up significantly more space. Either way, yeah it's pretty lightweight. Total number of records on the network is around 2.5Bn in the ~1.5 years Bluesky has been around. I aggregated some stats when we hit 10M users here - https://bsky.app/profile/did:plc:q6gjnaw2blty4crticxkmujt/post/3l47puwmjnl2x https://bsky.app/profile/did:plc:q6gjnaw2blty4crticxkmujt/po...

ericvolp12 2y ago

The 10x surge in traffic was us gaining 3.5M new users over the course of a week (growing the entire userbase by >33%) and all these users have been incredibly active on a daily basis. Lots of these numbers are public and the impact of the surge can be seen here: https://bskycharts.edavis.dev/static/dynazoom.html?plugin_name=edavis.dev%2Fbskycharts.edavis.dev%2Fbsky_users_total&start_iso8601=2024-04-22T11%3A15%3A23-0700&stop_iso8601=2024-09-24T11%3A15%3A23-0700&start_epoch=1713809723&stop_epoch=1727201723&lower_limit=&upper_limit=&size_x=800&size_y=400&cgiurl_graph=%2Fmunin-cgi%2Fmunin-cgi-graph https://bskycharts.edavis.dev/static/dynazoom.html?plugin_na... Note the graphs in that link only show users that take a publicly visible action (i.e. post, like, follow, etc.) and won't show lurkers at all.

str4d 2y ago

> The firehose is all public data going into the network right? It's the "main subset" of the public data, being "events on the network": for the Bluesky app that's posts, reposts, replies, likes, etc. Most of those records are just metadata (e.g. a repost record references the post being reposted, rather than embedding it). Post / reply records include the post text (limited to 300 graphemes). In particular, the firehose traffic does _not_ include images or videos; it only includes references to "blob"s. Clients separately fetch those blobs from the PDSs (or from a CDN caching the data).

deleted 2y ago

[deleted]