5 ms·

Yup, Matrix as a network has no central authority at all, although it's a common untruth which gets passed around, rather depressingly. Things which could be a

Yup, Matrix as a network has no central authority at all, although it's a common untruth which gets passed around, rather depressingly. Things which could be at the root of this misunderstanding include: * The fact that we operate a *strictly optional* logically centralised directory service to look up matrix IDs by email address or phone number. I really wish we'd never added this to Matrix; it causes way more grief than it solves - we'd have been better off waiting off for someone to build a decentralised keybase service and use that instead. * Element runs a centralised integration manager (aka app store) to make it easier to add bots/bridges/widgets to rooms. Again, this is completely optional - although back in 2019 we had a stupid bug in Element Web which meant it checked the integration manager even if you weren't using it. This was fixed in June 2019 (https://github.com/vector-im/element-web/issues/6802 https://github.com/vector-im/element-web/issues/6802). * Element has to query your server to find out what authentication mechanisms are available before you log in. Therefore if its config points to the default matrix.org homeserver, you end up pinging that every time you log in, even if your homeserver is elsewhere. The solution for now is to change your default homeserver in the config to avoid this. * Servers need to be able to validate events from servers even if those servers are offline. To do this you need to grab the public key for the offline server from somewhere; by default Synapse uses matrix.org as a fallback, but loudly nags the admin to pick a better default when you install the server. None of these actually allow a "central authority to have influence on another server" though; I've just listed the only places I can think of where a typical self-hosted Matrix implementation might interact with centralised services run by Matrix.org or Element. Unfortunately I think the real root of the misinformation is that it's just a bit too tempting and easy to throw around FUD. Some day we may get to the point where alternative FOSS projects will realise that the real enemy are centralised proprietary user-exploiting services rather than one another...

zaik 5y ago
Matrix is a lot better than every proprietary service, but it's poor XMPP compliance currently makes it impossible to have essential features like encrypted communications or adding Matrix users to multi-user chats.
- Arathorn 5y ago
  Unfortunately XMPP<->Matrix end-to-end encryption is a contradiction in terms. You have to speak the same protocol from end-to-end, otherwise you'd have to break the end-to-end encryption to translate from one protocol to the other. The closest you could get would be to run the Matrix<->XMPP bridge clientside alongside your Matrix client... but at that point you might almost as well be running a multihead messenger that speaks both Matrix & XMPP. In terms of adding Matrix to MUCs; i believe we're working on it in Bifrost, which is in active albeit slightly sporadic dev: https://github.com/matrix-org/matrix-bifrost https://github.com/matrix-org/matrix-bifrost.
  - ZoomZoomZoom 5y ago
    > but at that point you might almost as well be running a multihead messenger that speaks both Matrix & XMPP. Yes please! There's sooo much work people put in the clients, and there are multitude of them for each network. I understand they are different and a specialized client may be better suited for unique protocol features, but I also think there should be more general clients with protocol plugins, as it was with Pidgin, Miranda and such.
  - zaik 5y ago
    > Unfortunately XMPP<->Matrix end-to-end encryption is a contradiction in terms. You have to speak the same protocol from end-to-end, otherwise you'd have to break the end-to-end encryption to translate from one protocol to the other. You'd only have to speak the same encryption protocol. There already exists standards like the Signal protocol (on which OMEMO builds) or OpenPGP (which many XMPP clients support besides OMEMO). If messages can cross the border, so can public keys and encrypted messages.
    - Arathorn 5y ago
      Unfortunately this isn't true. Olm (Matrix's encryption protocol) is the same as OMEMO, which in turn is the same as libaxolotl. In fact, the OMEMO XEP even recommended Olm as an implementation for OMEMO at one point due to licensing concerns with libaxolotl/libsignalprotocol! But the fact that both protocols use the same E2EE encryption protocol doesn't help you with interoperability - given that you have to break that E2EE in order to convert between Matrix & XMPP; either on the server (ugh, defeats the whole point of E2EE) or on the client (ugh; you might as well just have a multiheaded messenger).
klabb3 5y ago
> The fact that we operate a strictly optional logically centralised directory service to look up matrix IDs by email address or phone number. I really wish we'd never added this to Matrix; it causes way more grief than it solves (Not a Matrix user so maybe help me understand here.) I think the reason why I have about 30 contacts (instead of like 3) on Signal is due to phone number discovery. It's not decentralized, which is of course subject to criticism, but exchanging & verifying public keys between O(N^2) people hasn't "taken off" in the X decades since it was invented. > we'd have been better off waiting off for someone to build a decentralised keybase service and use that instead. Do you have an example of any mainstream product that has ever achieved that, without either being partly centralized? Decentralized identity is really, really, really hard, and mostly the layer-8 implementation (discovery, name-squatting, impersonation, abuse, spam, key-management and exchange, etc).
- tjoff 5y ago
  And it is also the reason why I'd never considered signal for anything.
  - ikt 5y ago
    because it makes it easy for you to find contacts?
    - johnisgood 5y ago
      I do not want people to know I use X when they have me as a contact. I also do not want my account to be tied to my phone number at all, because my phone number is tied to a contract.
- Arathorn 5y ago
  well, keybase got quite far - it could have been decentralised but they chose to centralise it for commercial reasons. but yes, decentralised identity is hard, hence us cutting that corner (and then getting abuse about being too centralised…)
h0p3 5y ago
Amazing work. Bravo. I'm very much hoping to see P2P Matrix arise. That is no small task, I realize (and I'm grateful to those who have contributed to this). What are you thoughts on that aspect of the ecosystem?
- Arathorn 5y ago
  P2P Matrix is a moonshot, but it's coming together really well. The only bits left at this point are: * Making Pinecone resilient to attacks. It's working well for clear weather scenarios, but we need to properly try to break it and get it audited. Need to test it at scale too. * Improving Dendrite so that it's more stable and efficient clientside * Metadata-protecting store-and-forward, so that clients can pass messages to each other without needing to be online at the same time. You can cheat and run a serverside P2P node to relay, but then it stacks up metadata - whereas half the point of P2P is reduce Matrix's metadata footprint. * Account portability. Currently each node is a separate account; need to support the same logical account being replicated over multiple nodes. * Better than full-mesh routing. This should be pretty easy as we can just follow Pinecone's topology to route traffic between nodes rather than having each node try to simultaneously post to every other node in a given room. You can get testflight & APK builds from #p2p:matrix.org if you want to experiment.
  - mathnmusic 5y ago
    Arathorn, thanks for responding to comments here. Public discourse comes in all flavors and I appreciate you taking time to respond to the queries and comments. :) AFAIK, there are 6 server implementations of Matrix protocol: Synapse (Python), Dendrite (Go), Conduit (Rust), Construct/Charybdis (C++), Plasma (Elixir), Jeon (Java). Are there any more that I have missed? Which ones other than Synapse are most mature? An unbiased comparison post (that remains updated) would be really useful. A second question is: are the APIs between a homeserver and the frontend defined and standardized? I've noticed this in the ActivityPub ecosystem where it is usually very easy to use one frontend with another backend. Are matrix servers also written in the same way?
    - 5y ago
sneak 5y ago
Every Matrix protocol client I've ever tried, including yours, seems to connect to your "strictly optional" centralized directory service. I've never once wanted to use it, and, despite your claims of it being optional, 100% of the times I have logged in to a Matrix instance, via the web or via an app, my computer has talked to your computer. Something's broken here.
- Arathorn 5y ago
  See https://news.ycombinator.com/item?id=29263797 https://news.ycombinator.com/item?id=29263797.
jeroenhd 5y ago
I just tried logging in to Element.io with an account on my own homeserver and I saw several requests to vector.im before the privacy policy popup, including what seems to be some kind of token exchange. I don't think the intent is malicious, but the system is definitely leaky. My homeserver config doesn't mention vector.im (perhaps it's a hidden default somewhere?), I checked. From a protocol standpoint, no identity server is necessary. From a practical endpoint, all feature-rich clients (the ones I know support e2ee) seem to send requests to vector.im. Matrix is decentralized, but a lot of data still seems to get leaked to a centralized place by accident from my short experiments. It's not a great scenario, even if I'm fine with a few requests to the 3PID server every now and then. Client issues are ecosystem issues if all usable clients suffer from them.
- 3np 5y ago
  > I don't think the intent is malicious, but the system is definitely leaky. My homeserver config doesn't mention vector.im (perhaps it's a hidden default somewhere?), I checked. IIRC, identity server is something primarily set on the client (element.io here) rather than the homeserver. It really needn’t and shouldn’t ping an identity server as part of the sign-in process (esp before/without privacy policy agreement) FWIW it seems the issue is that there’s a fallback process for identity server; if neither your account nor your homeserver has one set, it defaults to the client default. if you host your own Element-web instance you can change this. https://github.com/vector-im/element-web/blob/develop/docs/config.md https://github.com/vector-im/element-web/blob/develop/docs/c... Or check out ma1sd. You don’t need to enable all the bells and whistles if you don’t want to.
- Arathorn 5y ago
  So I've just double-checked this by opening https://app.element.io https://app.element.io in an incognito tab and looking at the network inspector. What I see is what I was referring to in the grandparent as: * Element has to query your server to find out what authentication mechanisms are available before you log in. Therefore if its config points to the default matrix.org homeserver, you end up pinging that every time you log in, even if your homeserver is elsewhere. The solution for now is to change your default homeserver in the config to avoid this. The same applies to the identity lookup server as well as the homeserver: the app checks the one in your config to ensure that it exists before the user tries to log in on it. It does this with a single request (a GET to https://vector.im/_matrix/identity/api/v1 https://vector.im/_matrix/identity/api/v1). There is no token exchange there. The workaround is to configure the app with the correct default homeserver & identity server (if any) for your deployment. The underlying bug (i.e. don't validate the config until the user tries to log in) is https://github.com/vector-im/element-web/issues/11655 https://github.com/vector-im/element-web/issues/11655, and it simply hasn't got to the top of the list yet; we've been focusing on narrowing the UX gap between Element & Discord/Slack/Teams rather than reimplementing the login flows. As you can see on the bug, it's been picked up off the backlog 9 days ago and is currently being worked on. Separately, I'm seeing a related bug that Element Web's guest mode is incorrectly kicking in on the login page. Guest mode exists so you can deeplink straight to something like https://app.element.io/#/room/#matrix-dev:matrix.org https://app.element.io/#/room/#matrix-dev:matrix.org and read scrollback without needing to register an account - but it looks like we're spinning up guests on the default homeserver needlessly on the login page. This is https://github.com/vector-im/element-web/issues/11533 https://github.com/vector-im/element-web/issues/11533, and we've put it on the radar too. TL;DR: this is slightly leaky, but is it really any different to Chrome's default config loading up Google as a default homepage? Or Firefox loading up Mozilla.org? Or Thunderbird loading up Thunderbird.org? Eitherway, we'll fix the behaviour, but PLEASE understand that this is a million miles away from the bogus "central authority can have influence on another server" smear from the original post.

zaik 5y ago

Matrix is a lot better than every proprietary service, but it's poor XMPP compliance currently makes it impossible to have essential features like encrypted communications or adding Matrix users to multi-user chats.

Arathorn 5y ago
Unfortunately XMPP<->Matrix end-to-end encryption is a contradiction in terms. You have to speak the same protocol from end-to-end, otherwise you'd have to break the end-to-end encryption to translate from one protocol to the other. The closest you could get would be to run the Matrix<->XMPP bridge clientside alongside your Matrix client... but at that point you might almost as well be running a multihead messenger that speaks both Matrix & XMPP. In terms of adding Matrix to MUCs; i believe we're working on it in Bifrost, which is in active albeit slightly sporadic dev: https://github.com/matrix-org/matrix-bifrost https://github.com/matrix-org/matrix-bifrost.
- ZoomZoomZoom 5y ago
  > but at that point you might almost as well be running a multihead messenger that speaks both Matrix & XMPP. Yes please! There's sooo much work people put in the clients, and there are multitude of them for each network. I understand they are different and a specialized client may be better suited for unique protocol features, but I also think there should be more general clients with protocol plugins, as it was with Pidgin, Miranda and such.
- zaik 5y ago
  > Unfortunately XMPP<->Matrix end-to-end encryption is a contradiction in terms. You have to speak the same protocol from end-to-end, otherwise you'd have to break the end-to-end encryption to translate from one protocol to the other. You'd only have to speak the same encryption protocol. There already exists standards like the Signal protocol (on which OMEMO builds) or OpenPGP (which many XMPP clients support besides OMEMO). If messages can cross the border, so can public keys and encrypted messages.
  - Arathorn 5y ago
    Unfortunately this isn't true. Olm (Matrix's encryption protocol) is the same as OMEMO, which in turn is the same as libaxolotl. In fact, the OMEMO XEP even recommended Olm as an implementation for OMEMO at one point due to licensing concerns with libaxolotl/libsignalprotocol! But the fact that both protocols use the same E2EE encryption protocol doesn't help you with interoperability - given that you have to break that E2EE in order to convert between Matrix & XMPP; either on the server (ugh, defeats the whole point of E2EE) or on the client (ugh; you might as well just have a multiheaded messenger).

klabb3 5y ago

> The fact that we operate a strictly optional logically centralised directory service to look up matrix IDs by email address or phone number. I really wish we'd never added this to Matrix; it causes way more grief than it solves (Not a Matrix user so maybe help me understand here.) I think the reason why I have about 30 contacts (instead of like 3) on Signal is due to phone number discovery. It's not decentralized, which is of course subject to criticism, but exchanging & verifying public keys between O(N^2) people hasn't "taken off" in the X decades since it was invented. > we'd have been better off waiting off for someone to build a decentralised keybase service and use that instead. Do you have an example of any mainstream product that has ever achieved that, without either being partly centralized? Decentralized identity is really, really, really hard, and mostly the layer-8 implementation (discovery, name-squatting, impersonation, abuse, spam, key-management and exchange, etc).

tjoff 5y ago
And it is also the reason why I'd never considered signal for anything.
- ikt 5y ago
  because it makes it easy for you to find contacts?
  - johnisgood 5y ago
    I do not want people to know I use X when they have me as a contact. I also do not want my account to be tied to my phone number at all, because my phone number is tied to a contract.
Arathorn 5y ago
well, keybase got quite far - it could have been decentralised but they chose to centralise it for commercial reasons. but yes, decentralised identity is hard, hence us cutting that corner (and then getting abuse about being too centralised…)

h0p3 5y ago

Amazing work. Bravo. I'm very much hoping to see P2P Matrix arise. That is no small task, I realize (and I'm grateful to those who have contributed to this). What are you thoughts on that aspect of the ecosystem?

Arathorn 5y ago
P2P Matrix is a moonshot, but it's coming together really well. The only bits left at this point are: * Making Pinecone resilient to attacks. It's working well for clear weather scenarios, but we need to properly try to break it and get it audited. Need to test it at scale too. * Improving Dendrite so that it's more stable and efficient clientside * Metadata-protecting store-and-forward, so that clients can pass messages to each other without needing to be online at the same time. You can cheat and run a serverside P2P node to relay, but then it stacks up metadata - whereas half the point of P2P is reduce Matrix's metadata footprint. * Account portability. Currently each node is a separate account; need to support the same logical account being replicated over multiple nodes. * Better than full-mesh routing. This should be pretty easy as we can just follow Pinecone's topology to route traffic between nodes rather than having each node try to simultaneously post to every other node in a given room. You can get testflight & APK builds from #p2p:matrix.org if you want to experiment.
- mathnmusic 5y ago
  Arathorn, thanks for responding to comments here. Public discourse comes in all flavors and I appreciate you taking time to respond to the queries and comments. :) AFAIK, there are 6 server implementations of Matrix protocol: Synapse (Python), Dendrite (Go), Conduit (Rust), Construct/Charybdis (C++), Plasma (Elixir), Jeon (Java). Are there any more that I have missed? Which ones other than Synapse are most mature? An unbiased comparison post (that remains updated) would be really useful. A second question is: are the APIs between a homeserver and the frontend defined and standardized? I've noticed this in the ActivityPub ecosystem where it is usually very easy to use one frontend with another backend. Are matrix servers also written in the same way?
  - 5y ago

sneak 5y ago

Every Matrix protocol client I've ever tried, including yours, seems to connect to your "strictly optional" centralized directory service. I've never once wanted to use it, and, despite your claims of it being optional, 100% of the times I have logged in to a Matrix instance, via the web or via an app, my computer has talked to your computer. Something's broken here.

Arathorn 5y ago
See https://news.ycombinator.com/item?id=29263797 https://news.ycombinator.com/item?id=29263797.

jeroenhd 5y ago

I just tried logging in to Element.io with an account on my own homeserver and I saw several requests to vector.im before the privacy policy popup, including what seems to be some kind of token exchange. I don't think the intent is malicious, but the system is definitely leaky. My homeserver config doesn't mention vector.im (perhaps it's a hidden default somewhere?), I checked. From a protocol standpoint, no identity server is necessary. From a practical endpoint, all feature-rich clients (the ones I know support e2ee) seem to send requests to vector.im. Matrix is decentralized, but a lot of data still seems to get leaked to a centralized place by accident from my short experiments. It's not a great scenario, even if I'm fine with a few requests to the 3PID server every now and then. Client issues are ecosystem issues if all usable clients suffer from them.

3np 5y ago
> I don't think the intent is malicious, but the system is definitely leaky. My homeserver config doesn't mention vector.im (perhaps it's a hidden default somewhere?), I checked. IIRC, identity server is something primarily set on the client (element.io here) rather than the homeserver. It really needn’t and shouldn’t ping an identity server as part of the sign-in process (esp before/without privacy policy agreement) FWIW it seems the issue is that there’s a fallback process for identity server; if neither your account nor your homeserver has one set, it defaults to the client default. if you host your own Element-web instance you can change this. https://github.com/vector-im/element-web/blob/develop/docs/config.md https://github.com/vector-im/element-web/blob/develop/docs/c... Or check out ma1sd. You don’t need to enable all the bells and whistles if you don’t want to.
Arathorn 5y ago
So I've just double-checked this by opening https://app.element.io https://app.element.io in an incognito tab and looking at the network inspector. What I see is what I was referring to in the grandparent as: * Element has to query your server to find out what authentication mechanisms are available before you log in. Therefore if its config points to the default matrix.org homeserver, you end up pinging that every time you log in, even if your homeserver is elsewhere. The solution for now is to change your default homeserver in the config to avoid this. The same applies to the identity lookup server as well as the homeserver: the app checks the one in your config to ensure that it exists before the user tries to log in on it. It does this with a single request (a GET to https://vector.im/_matrix/identity/api/v1 https://vector.im/_matrix/identity/api/v1). There is no token exchange there. The workaround is to configure the app with the correct default homeserver & identity server (if any) for your deployment. The underlying bug (i.e. don't validate the config until the user tries to log in) is https://github.com/vector-im/element-web/issues/11655 https://github.com/vector-im/element-web/issues/11655, and it simply hasn't got to the top of the list yet; we've been focusing on narrowing the UX gap between Element & Discord/Slack/Teams rather than reimplementing the login flows. As you can see on the bug, it's been picked up off the backlog 9 days ago and is currently being worked on. Separately, I'm seeing a related bug that Element Web's guest mode is incorrectly kicking in on the login page. Guest mode exists so you can deeplink straight to something like https://app.element.io/#/room/#matrix-dev:matrix.org https://app.element.io/#/room/#matrix-dev:matrix.org and read scrollback without needing to register an account - but it looks like we're spinning up guests on the default homeserver needlessly on the login page. This is https://github.com/vector-im/element-web/issues/11533 https://github.com/vector-im/element-web/issues/11533, and we've put it on the radar too. TL;DR: this is slightly leaky, but is it really any different to Chrome's default config loading up Google as a default homepage? Or Firefox loading up Mozilla.org? Or Thunderbird loading up Thunderbird.org? Eitherway, we'll fix the behaviour, but PLEASE understand that this is a million miles away from the bogus "central authority can have influence on another server" smear from the original post.