3 ms·

Arathorn seems to always be on HN but chose to leave this question unanswered. I wonder why? I'll hazard a guess: New Vector LLC sees it is in their financial

Arathorn seems to always be on HN but chose to leave this question unanswered. I wonder why? I'll hazard a guess: New Vector LLC sees it is in their financial interest that they own the only functional implementations of the Matrix protocol and what it can interact with, so there's no interest in federation with the wider Fediverse and other protocols. Now Arathorn, come tell me why I'm wrong

Arathorn 6y ago
sigh The reason I skipped over this was because the author seemed to be pushing an agenda elsewhere in the thread ("Matrix is like XMPP but with a heavier server": https://news.ycombinator.com/item?id=26279816 https://news.ycombinator.com/item?id=26279816, and "Matrix has one reference client/server with really good features but hardware requirements that place it out of ordinary folks' reach.": https://news.ycombinator.com/item?id=26279780 https://news.ycombinator.com/item?id=26279780) which made me wonder whether answering all their posts was going to be a good use of time. In practice, the thread hasn't exploded into a XMPP v. Matrix holy war, so it looks like my concerns were ill-founded. To actually answer the question from the post in question: > Is there a working group talking about vocabulary or interop with other federated networks (ActivityPub/XMPP)? No, there isn't a formal WG - instead there are a set of informal conversations between the 3 projects. I lurk on xsf@muc.xmpp.org and standards@xmpp.org in case interop discussions come up; I also hang around various cross-project rooms where Gargron and Christopher Allen Lemmer and others are active on the AP side. As a result, cross-project initiatives sometimes pop up - for instance Matrix supported Mastodon in adding E2EE at https://github.com/tootsuite/mastodon/pull/13820 https://github.com/tootsuite/mastodon/pull/13820, and we've also tried to work with XMPP on using Olm for E2EE as an alternative to libsignalprotocol in OMEMO. In terms of content vocabulary specifically: we researched JSON-LD a bunch when working on MSC1767 (back when it was MSC1225) - the notes predate us using GFM for MSCs and are archived at https://docs.google.com/document/d/1m4VTRqclB3JEMZBjbr4t5cvIMQUNSGxy6rYN4YtopIk/edit#heading=h.uqsgmp37gqgn https://docs.google.com/document/d/1m4VTRqclB3JEMZBjbr4t5cvI... In practice, I'm not convinced there's a huge amount of value of standardising on the same vocabulary though - whether that's MIME, JSON-LD, RDF-flavoured JSON-LD, IPLD, rich XMPP stanzas, etc - as long as there's a way to map between them. Given you need to map the protocol layer in general when bridging protocols, why not map the content format too? That said, if folks have solved the problem elsewhere satisfactorily, then we'd have no problem with piggybacking on it on Matrix (just as others have piggybacked on Matrix's E2EE). In terms of the parent's comment: > I'll hazard a guess: New Vector LLC sees it is in their financial interest that they own the only functional implementations of the Matrix protocol and what it can interact with, so there's no interest in federation with the wider Fediverse and other protocols. No, my priority is trying to make the wider Matrix ecosystem as diverse and healthy as possible, including bridging nicely other protocols into Matrix. The more different projects building on Matrix (e.g. hummingbard) the more potential in the network and protocol for everyone involved. A rising tide lifts all ships, as it were. My dream for Matrix is to become the realtime communication layer of the open Web, and for there to be as many different parties building on it as do the open Web today. If we pull that off, then Element (formerly known as New Vector) would hopefully benefit from it as much as everyone else - much as 3Com benefited from creating Ethernet and bootstrapping that industry alongside DEC, Intel, Xerox and others. But it would be UTTERLY STUPID for Element to privilege itself over other contributors to Matrix, and would effectively self-sabotage the open ecosystem, thus killing the main thing which could make Element successful in the long term. Hence the huge amount of effort and $ that we put into setting up The Matrix.org Foundation (https://matrix.org/foundation https://matrix.org/foundation) as a neutral independent guardian of the protocol, and protect it from anyone commercially building on Matrix... including Element/New Vector. Sorry you think we suck though :)
- COGlory 6y ago
  Sorry that you have to deal with this on literally every HN thread. I used Matrix every day, along with a number of friends and family. It has been excellent and added a lot of value to our ability to communicate. I also use the same identity to participate in the openSUSE room, PinePhone room, and others. I have some grievances with Matrix. I've aired them in other threads, and on the Matrix.org server. Some have been addressed, others have not. I really hope I was constructive in my criticism. Frankly, some members of the community here act like absolute children every time Matrix is discussed, and hold it to impossibly high standards (like complaining it isn't as usable or polished as privately owned VC competitors, or older standards that have been around for 5x as long and can't do half of what Matrix can). It's disappointing and frustrating because I'm afraid that you will be less willing to engage here, and upset that it detracts from actual critical conversation of the protocol. Anyways, thank you for what you do and continuing to engage here.
- majso 6y ago
  Thanks for your hard work!
- oever 6y ago
  Can you point to a high-level description of Matrix which makes clear where the higher server requirements come from? Obviously a lot of it is not yet optimal code, but even so what are the expensive parts in scaling up?
  - Arathorn 6y ago
    Synapse is relatively RAM heavy because it leans a lot on in-memory caches to speed up DB access, as the DB schema isn't particularly intelligent and the fastest way to get it up and running was to slap caches everywhere. In the past, it could also spike RAM (which python then tends not to release back to the OS) when doing complicated merge operations of complex rooms, but this has largely been solved in the last few releases. The next generation of servers (Dendrite, Conduit, Construct) use way less RAM, having built without the time constraints (and now maintenance constraints) that impact Synapse. Separately, it's true that Matrix intrinsically uses more resources than XMPP, because Matrix's architecture is one of replicating and persisting conversation history and state between servers, rather than passing messages. And this simply involves much more data and processing than just relaying messages between sets of users.
dang 6y ago
Please don't post flamebait to HN or cross into personal attack. You may not feel you owe the other person better—but you owe the community better if you're participating here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

Arathorn 6y ago

sigh The reason I skipped over this was because the author seemed to be pushing an agenda elsewhere in the thread ("Matrix is like XMPP but with a heavier server": https://news.ycombinator.com/item?id=26279816 https://news.ycombinator.com/item?id=26279816, and "Matrix has one reference client/server with really good features but hardware requirements that place it out of ordinary folks' reach.": https://news.ycombinator.com/item?id=26279780 https://news.ycombinator.com/item?id=26279780) which made me wonder whether answering all their posts was going to be a good use of time. In practice, the thread hasn't exploded into a XMPP v. Matrix holy war, so it looks like my concerns were ill-founded. To actually answer the question from the post in question: > Is there a working group talking about vocabulary or interop with other federated networks (ActivityPub/XMPP)? No, there isn't a formal WG - instead there are a set of informal conversations between the 3 projects. I lurk on xsf@muc.xmpp.org and standards@xmpp.org in case interop discussions come up; I also hang around various cross-project rooms where Gargron and Christopher Allen Lemmer and others are active on the AP side. As a result, cross-project initiatives sometimes pop up - for instance Matrix supported Mastodon in adding E2EE at https://github.com/tootsuite/mastodon/pull/13820 https://github.com/tootsuite/mastodon/pull/13820, and we've also tried to work with XMPP on using Olm for E2EE as an alternative to libsignalprotocol in OMEMO. In terms of content vocabulary specifically: we researched JSON-LD a bunch when working on MSC1767 (back when it was MSC1225) - the notes predate us using GFM for MSCs and are archived at https://docs.google.com/document/d/1m4VTRqclB3JEMZBjbr4t5cvIMQUNSGxy6rYN4YtopIk/edit#heading=h.uqsgmp37gqgn https://docs.google.com/document/d/1m4VTRqclB3JEMZBjbr4t5cvI... In practice, I'm not convinced there's a huge amount of value of standardising on the same vocabulary though - whether that's MIME, JSON-LD, RDF-flavoured JSON-LD, IPLD, rich XMPP stanzas, etc - as long as there's a way to map between them. Given you need to map the protocol layer in general when bridging protocols, why not map the content format too? That said, if folks have solved the problem elsewhere satisfactorily, then we'd have no problem with piggybacking on it on Matrix (just as others have piggybacked on Matrix's E2EE). In terms of the parent's comment: > I'll hazard a guess: New Vector LLC sees it is in their financial interest that they own the only functional implementations of the Matrix protocol and what it can interact with, so there's no interest in federation with the wider Fediverse and other protocols. No, my priority is trying to make the wider Matrix ecosystem as diverse and healthy as possible, including bridging nicely other protocols into Matrix. The more different projects building on Matrix (e.g. hummingbard) the more potential in the network and protocol for everyone involved. A rising tide lifts all ships, as it were. My dream for Matrix is to become the realtime communication layer of the open Web, and for there to be as many different parties building on it as do the open Web today. If we pull that off, then Element (formerly known as New Vector) would hopefully benefit from it as much as everyone else - much as 3Com benefited from creating Ethernet and bootstrapping that industry alongside DEC, Intel, Xerox and others. But it would be UTTERLY STUPID for Element to privilege itself over other contributors to Matrix, and would effectively self-sabotage the open ecosystem, thus killing the main thing which could make Element successful in the long term. Hence the huge amount of effort and $ that we put into setting up The Matrix.org Foundation (https://matrix.org/foundation https://matrix.org/foundation) as a neutral independent guardian of the protocol, and protect it from anyone commercially building on Matrix... including Element/New Vector. Sorry you think we suck though :)

COGlory 6y ago
Sorry that you have to deal with this on literally every HN thread. I used Matrix every day, along with a number of friends and family. It has been excellent and added a lot of value to our ability to communicate. I also use the same identity to participate in the openSUSE room, PinePhone room, and others. I have some grievances with Matrix. I've aired them in other threads, and on the Matrix.org server. Some have been addressed, others have not. I really hope I was constructive in my criticism. Frankly, some members of the community here act like absolute children every time Matrix is discussed, and hold it to impossibly high standards (like complaining it isn't as usable or polished as privately owned VC competitors, or older standards that have been around for 5x as long and can't do half of what Matrix can). It's disappointing and frustrating because I'm afraid that you will be less willing to engage here, and upset that it detracts from actual critical conversation of the protocol. Anyways, thank you for what you do and continuing to engage here.
majso 6y ago
Thanks for your hard work!
oever 6y ago
Can you point to a high-level description of Matrix which makes clear where the higher server requirements come from? Obviously a lot of it is not yet optimal code, but even so what are the expensive parts in scaling up?
- Arathorn 6y ago
  Synapse is relatively RAM heavy because it leans a lot on in-memory caches to speed up DB access, as the DB schema isn't particularly intelligent and the fastest way to get it up and running was to slap caches everywhere. In the past, it could also spike RAM (which python then tends not to release back to the OS) when doing complicated merge operations of complex rooms, but this has largely been solved in the last few releases. The next generation of servers (Dendrite, Conduit, Construct) use way less RAM, having built without the time constraints (and now maintenance constraints) that impact Synapse. Separately, it's true that Matrix intrinsically uses more resources than XMPP, because Matrix's architecture is one of replicating and persisting conversation history and state between servers, rather than passing messages. And this simply involves much more data and processing than just relaying messages between sets of users.

dang 6y ago

Please don't post flamebait to HN or cross into personal attack. You may not feel you owe the other person better—but you owe the community better if you're participating here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html