4 ms·

The "160 gigabytes" text was linked to the Johnny Mnemonic wikipedia page, so it it possible they're just exaggerating and meant "it was growing at a rate we di

by smcl 4y ago

The "160 gigabytes" text was linked to the Johnny Mnemonic wikipedia page, so it it possible they're just exaggerating and meant "it was growing at a rate we didn't expect or couldn't really sustain"? This got me wondering, I created a Mastodon user @ defcon.social and my first couple of posts were images, just to test things out and get started with my chosen client - Ivory by Tapbots (the creators of Tweetdeck, RIP) after tptacek spoke glowingly of it here. Then I realised that by posting those I'm basically taking up space on a community service without paying for it, and I wasn't sure exactly what the etiquette was. On a commercial, monetised service like Twitter it's different as there are advertisers (or at least there were...) but afaik "defcon.social" is just a non-profit and I'd really like not to be a big cost sink. Also I wondered if there's any overhead involved in following users across many different instances - if my @defcon.social user follows a bunch of users on various other instances does that place additional burden on their mastodon instance or just on my local client? Might be time for me to read the defcon.social fine print and T&Cs (I only glanced over it) or even about Mastodon/ActivityPub generally...

UncleEntity 4y ago
From what I can gather every follower gets an individual copy of every post sent to their server who either deduplicates it and stores it in their database or just stores the message per subscriber. So if 10,000 people on server A subscribe to someone on server B then server B will send 10,000 duplicate messages to server A. I hope I’m wrong but that’s what I’ve heard.
- stnmtn 4y ago
  Why are tech people so behind mastodon? If this is true this sounds incredibly worrying to scale; to the point where mastodon is limiting it's own growth potential
  - dylan604 4y ago
    If. Did you research if this was true before posting a knee jerk reaction to something that might not be true? If not, you are no better than the person that posted the misleading comment.
  - smcl 4y ago
    Remember there's a fair bit of speculation here, I just thoughtlessly blasted my question here without researching (not sure where to start, tbh) and UncleEntity didn't seem too sure about the duplicated/redundant requests. It's entirely possible that Mastodon doesn't have this problem
- maegul 4y ago
  Oh my! I’d heard ActivityPub was chatty but haven’t looked into the details. This, if true, sounds … not good. Anyone with expertise here who can speak to the scaling of ActivityPub?
  - mariusor 4y ago
    The size of an ActivityPub payload overhead is minimal. On each individual server they are ideally saved based on their unique identifier (which are RFC3987 IRIs) and I doubt that there is any software that will do it individually for each inbox instead of storing one copy to which each inbox links to. To minimize even this, each server can store just the IRI(which is mandated to be de-referenceable) and load the payload when prompted by UI. I doubt that anyone does this also.
    - maegul 4y ago
      Yes that makes sense! Thanks!
- dpkirchner 4y ago
  Reminds me of: > This program posts news to thousands of machines throughout the entire civilized world. You message will cost the net hundreds if not thousands of dollars to send everywhere. Please be sure you know what you are doing. From the rn usenet client, back in the day.
  - dylan604 4y ago
    Maybe if more people thought this, a lot less posts would be made. I'm guessing that would be a net positive for society in general.
    - cratermoon 4y ago
      > Maybe if more people thought this, a lot less posts would be made This is why we can't have nice things: https://en.wikipedia.org/wiki/Laurence_Canter_and_Martha_Siegel https://en.wikipedia.org/wiki/Laurence_Canter_and_Martha_Sie...
- librexpr 4y ago
  I wish people wouldn't spread rumors like this. ActivityPub does have shared inboxes[0] and Mastodon does use them[1], so no duplicate messages will be sent in your example. [0] https://www.w3.org/TR/activitypub/#x7-1-3-shared-inbox-delivery https://www.w3.org/TR/activitypub/#x7-1-3-shared-inbox-deliv... [1] https://github.com/mastodon/mastodon/blob/a5a00d7f7adff5e0afbd23ac1e1b16120137509a/app/models/account.rb#L385 https://github.com/mastodon/mastodon/blob/a5a00d7f7adff5e0af...
deleted 4y ago
[deleted]
jabroni_salad 4y ago
I mainly use twitter to keep tabs on some artists and none of them want to move. Reading this, maybe it is for the best. My follows would create a lot of overhead for wherever it is I park my account compared to someone who just participates in text.
KaiserPro 4y ago
I have my own instance, and it eats disk space, in a fun and sometimes difficult to recover way. It caches all media that you view, and I assume it might prefetch stuff to. It has a tool that prunes out old cached images and such. However that script isn't setup to run automatically (or if it is, its far too loosey-goosey) However the biggest killer is that it doesn't clear cached user page header images. Here is the output of my cache. Headers are not cleaned, and need to be deleted manually. Attachments: 1.43 GB (8.19 MB local) Custom emoji: 61.9 MB (0 Bytes local) Preview cards: 344 MB Avatars: 3.01 GB (71.7 KB local) Headers: 6.69 GB (172 KB local) Backups: 0 Bytes Imports: 0 Bytes Settings: 36.2 KB

by smcl 4y ago

UncleEntity 4y ago

From what I can gather every follower gets an individual copy of every post sent to their server who either deduplicates it and stores it in their database or just stores the message per subscriber. So if 10,000 people on server A subscribe to someone on server B then server B will send 10,000 duplicate messages to server A. I hope I’m wrong but that’s what I’ve heard.

stnmtn 4y ago
Why are tech people so behind mastodon? If this is true this sounds incredibly worrying to scale; to the point where mastodon is limiting it's own growth potential
- dylan604 4y ago
  If. Did you research if this was true before posting a knee jerk reaction to something that might not be true? If not, you are no better than the person that posted the misleading comment.
- smcl 4y ago
  Remember there's a fair bit of speculation here, I just thoughtlessly blasted my question here without researching (not sure where to start, tbh) and UncleEntity didn't seem too sure about the duplicated/redundant requests. It's entirely possible that Mastodon doesn't have this problem
maegul 4y ago
Oh my! I’d heard ActivityPub was chatty but haven’t looked into the details. This, if true, sounds … not good. Anyone with expertise here who can speak to the scaling of ActivityPub?
- mariusor 4y ago
  The size of an ActivityPub payload overhead is minimal. On each individual server they are ideally saved based on their unique identifier (which are RFC3987 IRIs) and I doubt that there is any software that will do it individually for each inbox instead of storing one copy to which each inbox links to. To minimize even this, each server can store just the IRI(which is mandated to be de-referenceable) and load the payload when prompted by UI. I doubt that anyone does this also.
  - maegul 4y ago
    Yes that makes sense! Thanks!
dpkirchner 4y ago
Reminds me of: > This program posts news to thousands of machines throughout the entire civilized world. You message will cost the net hundreds if not thousands of dollars to send everywhere. Please be sure you know what you are doing. From the rn usenet client, back in the day.
- dylan604 4y ago
  Maybe if more people thought this, a lot less posts would be made. I'm guessing that would be a net positive for society in general.
  - cratermoon 4y ago
    > Maybe if more people thought this, a lot less posts would be made This is why we can't have nice things: https://en.wikipedia.org/wiki/Laurence_Canter_and_Martha_Siegel https://en.wikipedia.org/wiki/Laurence_Canter_and_Martha_Sie...
librexpr 4y ago
I wish people wouldn't spread rumors like this. ActivityPub does have shared inboxes[0] and Mastodon does use them[1], so no duplicate messages will be sent in your example. [0] https://www.w3.org/TR/activitypub/#x7-1-3-shared-inbox-delivery https://www.w3.org/TR/activitypub/#x7-1-3-shared-inbox-deliv... [1] https://github.com/mastodon/mastodon/blob/a5a00d7f7adff5e0afbd23ac1e1b16120137509a/app/models/account.rb#L385 https://github.com/mastodon/mastodon/blob/a5a00d7f7adff5e0af...

deleted 4y ago

[deleted]

jabroni_salad 4y ago

I mainly use twitter to keep tabs on some artists and none of them want to move. Reading this, maybe it is for the best. My follows would create a lot of overhead for wherever it is I park my account compared to someone who just participates in text.

KaiserPro 4y ago

I have my own instance, and it eats disk space, in a fun and sometimes difficult to recover way. It caches all media that you view, and I assume it might prefetch stuff to. It has a tool that prunes out old cached images and such. However that script isn't setup to run automatically (or if it is, its far too loosey-goosey) However the biggest killer is that it doesn't clear cached user page header images. Here is the output of my cache. Headers are not cleaned, and need to be deleted manually. Attachments: 1.43 GB (8.19 MB local) Custom emoji: 61.9 MB (0 Bytes local) Preview cards: 344 MB Avatars: 3.01 GB (71.7 KB local) Headers: 6.69 GB (172 KB local) Backups: 0 Bytes Imports: 0 Bytes Settings: 36.2 KB