5 ms·

The tone of that whole thread in combination with posts like > Imagine if I had to store the 50k+ tweets I've made on Twitter on my device, and upload ALL of t

by tazjin 3y ago

The tone of that whole thread in combination with posts like > Imagine if I had to store the 50k+ tweets I've made on Twitter on my device, and upload ALL of them to a new server whenever a community server went down. doesn't make this person seem particularly competent. I'm getting a vague feeling that this is normal discourse on Mastodon though, and that not using social media much shifts your personal overton window for what is acceptable communication until you essentially don't overlap with the very online crowd anymore.

sundvor 3y ago
Everything is upper case and absolutes these days, so many appear to have lost the capacity for nuance; it's turned into a shouting match for our attention.
doctor_eval 3y ago
Yea I was like, 50K tweets converted to bytes is like… well it’s one second to upload on my home internet connection. After that, it just felt like rant.
- jeroenhd 3y ago
  For what it's worth, when a server does go down and thousands of people start uploading their 50k skeets that all need to be cryptographically verified, other major servers will have quite the scaling challenge. Cryptographic verification is intentionally compute heavy after all. 50k entries isn't a whole lot but as people hosting Mastodon servers have found out, things start slowing down when 1000 people transfer those 50k entries at the same time.
  - vidarh 3y ago
    Mastodon does not currently support importing posts from your old account. Just followers/following. It's a nuisance, and there's nothing preventing it per se, since the posts are signed, but that part could be better.
    - jeroenhd 3y ago
      I know, and although I understand why (ActivityPub IDs are generally URLs and changing them would probably cause duplicates, but there are URLs that don't change). It's quite annoying that it can't be done right now, but I'll admit that I also don't care enough to come up with a patch for it.
      - vidarh 3y ago
        There are two ways around this: 1. They are URI's, and while ActivityPub say they should be https URL's, they don't need to be, and could e.g. point at IPFS or similar. 2. JSON-LD signatures are used by Mastodon, and included in the export, and nothing stops another instance from validating those and serving them up with the original URIs in the id from new URLs, as a means of making it clear the server didn't originate them (there'd be a trust issue if the other servers is unable to get hold of the keys because the original server is gone, but no more so than if the new server had simply republished the content, so the "worst case" is to distrust the original id's). There are some corner cases there, around trusting the identity of the old and new account represents the same user, so I do think a recovery key type scheme would be nice to allow a user to prove the old and new id is the same (if changing id; I also think we could really use decoupling the expectation that a webfinger id is inherently tied to a Mastodon account - you can sort of do that today; nothing stops you from serving up a separate webfinger result and use it as an alias, but there are usability issues to solve there).
        jeroenhd 3y ago
        You're right, there's actually no reason why a second server couldn't serve up toots with the old IDs. The main problem would probably be keeping track of what server to fetch these messages from after a move (or even a second move) to a different server and keeping the metadata attached in sync.
    - mkr-hn 3y ago
      Calckey recently gained the ability to migrate posts. Even from Mastodon. Maybe that'll motivate Mastodon and the forks to prioritize it.
  - tazjin 3y ago
    > Cryptographic verification is intentionally compute heavy after all You're thinking of password hashing functions like argon2, there's no reason for normal signing and verification operations to be intentionally expensive as that's not where the security guarantees come from.
    - jeroenhd 3y ago
      Indeed, verifying a signature does not rely on mathematical complexity like hashing does. However, they do usually involve finite fields and large numbers that require a whole bunch of complex mathematical operations. Then there's also the fact that optimizations are often not available as they would allow for timing attacks. Compared to something like simple a CRC32 checksum to verify that the data was transmitted correctly, these operations will always be complex and more time intensive than you'd prefer them to be.
  - doctor_eval 3y ago
    That might be true - but how else can it work? The data has to get onto the server somehow, and the days of unsigned or unencrypted data ended with Snowden. The OP was going on about storing all the data on-device and uploading it, but regardless of where it’s stored, if a bunch of people have to move, the thundering herd problem, so to speak, will still exist. Also, signature verification is not slow. I don’t know what AT uses or how good this source from 2020 is [0] nor what machine it ran on, but ed25519 verification takes about 50us for a 32 byte signature verification. That suggests this guys 50k posts will be validated in 2.5s. If we assume just a single server then that’s about 34k users per day. Or put another way, 30 servers to onboard one million people in a day. None of this seems outrageous to me. [0] https://safenetforum.org/t/ed25519-vs-bls-performance/32613 https://safenetforum.org/t/ed25519-vs-bls-performance/32613
    - jeroenhd 3y ago
      BlueSky uses @noble/secp256k1 which performs this stuff in Javascript, with about 880* verifications per second on the Apple M2 (a chip with a relatively high IPC, likely higher than your average server). Verifying those messages will take about a minute of CPU time per user (assuming no impact from cache misses due to threads swapping in and out and processing new data). I think that's quite significant. * = https://github.com/paulmillr/noble-curves https://github.com/paulmillr/noble-curves
      - doctor_eval 3y ago
        Yes, OK I agree, that is pretty bad. That’s about 1400 users per day per server using napkin maths. Definitely not the kind of scale I would have expected. But is this the only possible implementation? I suppose for the OP to have a reasonable point (that it’s “a crock of shit”), this would need to be an intractable problem.
        jeroenhd 3y ago
        I assume a server written in native code would be faster, but I don't know by how much. The normal use case for this algorithm has to do with Bitcoin and the Bitcoin libraries aren't written for multithreaded social media servers so it's hard to say. Theoretically it could be fast, but you'll need to find someone crazy enough to implement thie protofol first.
- pseg134 3y ago
  Twitter has pictures so well, probably more than one second.
  - BlueTemplar 3y ago
    Hmm, Twitter has certainly drifted a lot from when it was SMS-compatible with an open API... https://www.theverge.com/2012/7/9/3135406/twitter-api-open-closed-facebook-walled-garden https://www.theverge.com/2012/7/9/3135406/twitter-api-open-c...
a13o 3y ago
On your last point, I think the key distinction is the number of people in the social circle. On Twitter, it was the entire world. And if you wanted to hold the conch shell, you did what the algorithm wanted which was OUTRAGE. On Mastodon, it's theoretically scoped down to just your server and a handful of deliberate federations. This exists. You can find these servers and have a BBS-like experience. Tooters give you the time of day and assholes are shown the door. The Very Online folks, however, use Mastodon differently. For them it's more like "I'll build my own Twitter!" And what they want is more OUTRAGE feed to tap into 20 times a day, but with them in control of the algorithm. So they federate freely, with very large servers, making no material progress on the thing that makes Twitter unhealthy. The fact that Mastodon can seemingly be used both ways is a good accomplishment. It's not the technology's fault the second use-case is socially toxic. But it does make it very hard to talk about Mastodon because you don't immediately know which way anyone is using it.

by tazjin 3y ago

sundvor 3y ago

Everything is upper case and absolutes these days, so many appear to have lost the capacity for nuance; it's turned into a shouting match for our attention.

doctor_eval 3y ago

Yea I was like, 50K tweets converted to bytes is like… well it’s one second to upload on my home internet connection. After that, it just felt like rant.

jeroenhd 3y ago
For what it's worth, when a server does go down and thousands of people start uploading their 50k skeets that all need to be cryptographically verified, other major servers will have quite the scaling challenge. Cryptographic verification is intentionally compute heavy after all. 50k entries isn't a whole lot but as people hosting Mastodon servers have found out, things start slowing down when 1000 people transfer those 50k entries at the same time.
- vidarh 3y ago
  Mastodon does not currently support importing posts from your old account. Just followers/following. It's a nuisance, and there's nothing preventing it per se, since the posts are signed, but that part could be better.
  - jeroenhd 3y ago
    I know, and although I understand why (ActivityPub IDs are generally URLs and changing them would probably cause duplicates, but there are URLs that don't change). It's quite annoying that it can't be done right now, but I'll admit that I also don't care enough to come up with a patch for it.
    - vidarh 3y ago
      There are two ways around this: 1. They are URI's, and while ActivityPub say they should be https URL's, they don't need to be, and could e.g. point at IPFS or similar. 2. JSON-LD signatures are used by Mastodon, and included in the export, and nothing stops another instance from validating those and serving them up with the original URIs in the id from new URLs, as a means of making it clear the server didn't originate them (there'd be a trust issue if the other servers is unable to get hold of the keys because the original server is gone, but no more so than if the new server had simply republished the content, so the "worst case" is to distrust the original id's). There are some corner cases there, around trusting the identity of the old and new account represents the same user, so I do think a recovery key type scheme would be nice to allow a user to prove the old and new id is the same (if changing id; I also think we could really use decoupling the expectation that a webfinger id is inherently tied to a Mastodon account - you can sort of do that today; nothing stops you from serving up a separate webfinger result and use it as an alias, but there are usability issues to solve there).
      - jeroenhd 3y ago
        You're right, there's actually no reason why a second server couldn't serve up toots with the old IDs. The main problem would probably be keeping track of what server to fetch these messages from after a move (or even a second move) to a different server and keeping the metadata attached in sync.
  - mkr-hn 3y ago
    Calckey recently gained the ability to migrate posts. Even from Mastodon. Maybe that'll motivate Mastodon and the forks to prioritize it.
- tazjin 3y ago
  > Cryptographic verification is intentionally compute heavy after all You're thinking of password hashing functions like argon2, there's no reason for normal signing and verification operations to be intentionally expensive as that's not where the security guarantees come from.
  - jeroenhd 3y ago
    Indeed, verifying a signature does not rely on mathematical complexity like hashing does. However, they do usually involve finite fields and large numbers that require a whole bunch of complex mathematical operations. Then there's also the fact that optimizations are often not available as they would allow for timing attacks. Compared to something like simple a CRC32 checksum to verify that the data was transmitted correctly, these operations will always be complex and more time intensive than you'd prefer them to be.
- doctor_eval 3y ago
  That might be true - but how else can it work? The data has to get onto the server somehow, and the days of unsigned or unencrypted data ended with Snowden. The OP was going on about storing all the data on-device and uploading it, but regardless of where it’s stored, if a bunch of people have to move, the thundering herd problem, so to speak, will still exist. Also, signature verification is not slow. I don’t know what AT uses or how good this source from 2020 is [0] nor what machine it ran on, but ed25519 verification takes about 50us for a 32 byte signature verification. That suggests this guys 50k posts will be validated in 2.5s. If we assume just a single server then that’s about 34k users per day. Or put another way, 30 servers to onboard one million people in a day. None of this seems outrageous to me. [0] https://safenetforum.org/t/ed25519-vs-bls-performance/32613 https://safenetforum.org/t/ed25519-vs-bls-performance/32613
  - jeroenhd 3y ago
    BlueSky uses @noble/secp256k1 which performs this stuff in Javascript, with about 880* verifications per second on the Apple M2 (a chip with a relatively high IPC, likely higher than your average server). Verifying those messages will take about a minute of CPU time per user (assuming no impact from cache misses due to threads swapping in and out and processing new data). I think that's quite significant. * = https://github.com/paulmillr/noble-curves https://github.com/paulmillr/noble-curves
    - doctor_eval 3y ago
      Yes, OK I agree, that is pretty bad. That’s about 1400 users per day per server using napkin maths. Definitely not the kind of scale I would have expected. But is this the only possible implementation? I suppose for the OP to have a reasonable point (that it’s “a crock of shit”), this would need to be an intractable problem.
      - jeroenhd 3y ago
        I assume a server written in native code would be faster, but I don't know by how much. The normal use case for this algorithm has to do with Bitcoin and the Bitcoin libraries aren't written for multithreaded social media servers so it's hard to say. Theoretically it could be fast, but you'll need to find someone crazy enough to implement thie protofol first.
pseg134 3y ago
Twitter has pictures so well, probably more than one second.
- BlueTemplar 3y ago
  Hmm, Twitter has certainly drifted a lot from when it was SMS-compatible with an open API... https://www.theverge.com/2012/7/9/3135406/twitter-api-open-closed-facebook-walled-garden https://www.theverge.com/2012/7/9/3135406/twitter-api-open-c...

a13o 3y ago

On your last point, I think the key distinction is the number of people in the social circle. On Twitter, it was the entire world. And if you wanted to hold the conch shell, you did what the algorithm wanted which was OUTRAGE. On Mastodon, it's theoretically scoped down to just your server and a handful of deliberate federations. This exists. You can find these servers and have a BBS-like experience. Tooters give you the time of day and assholes are shown the door. The Very Online folks, however, use Mastodon differently. For them it's more like "I'll build my own Twitter!" And what they want is more OUTRAGE feed to tap into 20 times a day, but with them in control of the algorithm. So they federate freely, with very large servers, making no material progress on the thing that makes Twitter unhealthy. The fact that Mastodon can seemingly be used both ways is a good accomplishment. It's not the technology's fault the second use-case is socially toxic. But it does make it very hard to talk about Mastodon because you don't immediately know which way anyone is using it.