3 ms·

Could you elaborate more about Matrix's metadata distribution? What exactly is being distributed and when? I'd like to see that if you have it as I wasn't aware

Could you elaborate more about Matrix's metadata distribution? What exactly is being distributed and when? I'd like to see that if you have it as I wasn't aware of this. PS: I'm not trying to contradict you here, on the contrary. I'd like to deep dive into it as it is indeed a big issue. If Matrix really takes off (which I hope it does) I would want to avoid dataminers just federating to it and grabbing data that way. Of course some metadata would need to be able to be shared for the federated network to function. I understand that too.

ezst 6y ago
Sure! The first "practical"* difference between Matrix and XMPP is that Matrix' chatrooms are "necessarily decentralized" (whereas in XMPP that's optional [1]). How it works is that users involved in a given room have their servers synchronize the room's state with each other (room state is messages, presence status, read receipts, "is composing" status, …) so when one server goes down/netsplits, messages, join/leave events, read markers, … can keep flowing among online users (and when the server goes up again, it might attempt to merge its state with that of the network while catching-up). This looks and behaves very much like a blockchain: you end-up with distributed "copies since origins" of the room + all its metadata all over the network (plus distributed inefficiency, but I digress). *: I put that quotes, because I don't believe it matters the slightest in practice. XMPP nodes can be easily distributed, which addresses the same reliability concern, but at a layer better suited to handle the complexity. WhatsApp runs on a fork of ejabberd (erlang), while Matrix homeservers, especially matrix.org are notorious for their terrible QoS/resources-hungriness while handling a fraction of the concurrent users. [1]: https://xmpp.org/extensions/xep-0289.html https://xmpp.org/extensions/xep-0289.html
- GekkePrutser 6y ago
  Thanks, I'll have to dig into the protocol, maybe do some sniffing on my homeserver :) I assume not every node gets the metadata for all the rooms in the world, just the ones its users participate in? Anyway looking forward to investigating this.
  - ezst 6y ago
    Indeed, that's just for the rooms where users are joined. But be aware that if you join a large room on matrix.org from your own homeserver, it might fill-up your disks with a lot of old state (and that might take large amounts of CPU+RAM for days, too).

ezst 6y ago

Sure! The first "practical"* difference between Matrix and XMPP is that Matrix' chatrooms are "necessarily decentralized" (whereas in XMPP that's optional [1]). How it works is that users involved in a given room have their servers synchronize the room's state with each other (room state is messages, presence status, read receipts, "is composing" status, …) so when one server goes down/netsplits, messages, join/leave events, read markers, … can keep flowing among online users (and when the server goes up again, it might attempt to merge its state with that of the network while catching-up). This looks and behaves very much like a blockchain: you end-up with distributed "copies since origins" of the room + all its metadata all over the network (plus distributed inefficiency, but I digress). *: I put that quotes, because I don't believe it matters the slightest in practice. XMPP nodes can be easily distributed, which addresses the same reliability concern, but at a layer better suited to handle the complexity. WhatsApp runs on a fork of ejabberd (erlang), while Matrix homeservers, especially matrix.org are notorious for their terrible QoS/resources-hungriness while handling a fraction of the concurrent users. [1]: https://xmpp.org/extensions/xep-0289.html https://xmpp.org/extensions/xep-0289.html

GekkePrutser 6y ago
Thanks, I'll have to dig into the protocol, maybe do some sniffing on my homeserver :) I assume not every node gets the metadata for all the rooms in the world, just the ones its users participate in? Anyway looking forward to investigating this.
- ezst 6y ago
  Indeed, that's just for the rooms where users are joined. But be aware that if you join a large room on matrix.org from your own homeserver, it might fill-up your disks with a lot of old state (and that might take large amounts of CPU+RAM for days, too).