4 ms·

Matrix Client Tutorial

141 points by whereistimbo 2y ago

alwayslikethis 2y ago
The lack of good clients is really holding Matrix back. Element is rather bloated, and most of the other clients are missing significant amounts of features.
- ericjmorey 2y ago
  I'd say that the protocol itself is what's doing Matrix in.
  - alwayslikethis 2y ago
    In order to support bridges to so many different proprietary platforms Matrix needs to have a superset of their features, so the feature creep is probably intentional. It does make it harder for clients to keep up though.
    - cmeacham98 2y ago
      Matrix is the opposite of feature creep - there are several features entirely missing from major clients that users of modern chat apps expect to have. (Examples include: custom emoji, functional search of encrypted chats, pinned messages or some sort of MOTD, forwarding messages, etc)
      - ptman 2y ago
        The matrix developers are experienced IM developers. They've built systems on XMPP and custom protocols before. Their estimate was that because of matrix being federated, every feature was 5-10x harder to implement than in centralized systems.
      - Arathorn 2y ago
        As a protocol, Matrix has all of these features (custom emoji is a proposal, but many clients implement it). As a client, Element has all of these (encrypted search only on Desktop for now) other than custom emoji.
  - isaacaggrey 2y ago
    Could you elaborate?
    - ranger_danger 2y ago
      https://0x0.st/H8h-.png https://0x0.st/H8h-.png
      - Groxx 2y ago
        All that really shows is "the project has an active community". Have you seen how many XMPP has? Or Python[1]? This is just normal open development metadata. [1]: https://peps.python.org/pep-0000/# https://peps.python.org/pep-0000/#
        jeroenhd 2y ago
        > Have you seen how many XMPP has? Or Python[1]? Python has had 664 PEPs in 23 years (29 per year). XMPP has received 495 XEPs in 23 years (22½ per year). Matrix has received about 650 in 8 years (>81 per year). Four times the change rate is quite annoying when writing code against the spec. Plus, most XMPP clients only support a fraction of the full spec, so by that comparison the impact of the rate of change is even worse. Furthermore, most XEPs and PEPs are mere (optional) extensions, whereas a lot of MSCs are alterations of existing APIs. Any JSON parser used for Matrix needs to anticipate fields changing or being added all over the place because you never know when random fields show up all over the input data because of a spec change. The way the Matrix spec is developed feels a lot more like a proprietary company spec that happens to be published on Github than the IETF/XMPP/Python spec process. The rate of change is high and almost all changes are done to serve new features for the two or three major players that bought into the Matrix ecosystem. One recent change that comes to mind is the move from secret, public URLs for media, to authenticated URLs. The setting to force that changeover won't apply everywhere for a while, but it'll completely break every media-supporting client written before the spec change. Nothing wrong with extending the spec to improve the product, but with how fast the protocol is growing, I wouldn't want to be tasked with maintaining a Matrix client and I don't have much faith in the forward compatibility of the few Matrix bots I've written either.
  - Arathorn 2y ago
    What are you thinking of in particular? Fwiw, as project lead, i would be first to admit that there were slow bits - eg sync v2, which is why we built sliding sync… and crypto was easy to get wrong, hence shipping a highly polished implementation in matrix-rust-sdk. But what do you have in mind? Otherwise this doesn’t sound very wellfounded…
    - lifty 2y ago
      It's been a bumpy road but I think now Matrix has gotten to a good place. What's still missing is for the ecosystem to catch up a bit, both on the home server side and on the client side, and implement some of the new usability improvements brought by sliding sync and the rust sdk. Looking forward to see it happen!
- heroprotagonist 2y ago
  I _really_ wish that if Element has an update prompt for me almost every single time I open it, that it would just update during the startup sequence on its own. Or at least have an option for it. Not a 'Oh you loaded, now click this button to load again please' prompt whenever I launch it.
- paulcarroty 2y ago
  cinny/fluffychat/iamb are the greatest.
- itsthejb 2y ago
  Very very true. I self host synapse, and generally speaking the ability to bridge my ~3 most used messengers into one app (ElementX) adds value. However, the lacking features and bizarre feature disjoints between Element (supposedly EOL) and ElementX (suppose next gen) are jarring
- Arathorn 2y ago
  Element X (https://element.io/blog/deep-dive-into-element-x https://element.io/blog/deep-dive-into-element-x) is intended to spell out how lightweight and unbloated an Element-style client can be. The only main missing features are threads and spaces which should land in the coming months.
ranger_danger 2y ago
Finally something besides the matrix foundation's "lol the auto-generated API docs are enough" nonsense. This really helps a whole lot better.
amstan 2y ago
I tried to do something similar. It's infuriating how the client must be stateful and have local storage, for both the access_token and even last message recieved. That's right you must remember as the client where the last events [1] you've seen (even if you already told the server to mark it as read) was or else the server will happily send you the same messages over and over again. I kind of miss making IRC bots where things were much simpler and ... quicker honestly (latency wise). [1] https://uhoreg.gitlab.io/matrix-tutorial/sync.html#:~:text=we%20retrieve%20the-,next_batch,-property%20to%20use https://uhoreg.gitlab.io/matrix-tutorial/sync.html#:~:text=w...
- dzaima 2y ago
  At least that means that you will never have messages get dropped due to network issues, or even restarts if you persist the token across such. (the API will jump forwards if there are too many unsynced messages, but at least it should then be providing a "limited: true" so that one can paginate in the omitted events if so desired (or not if undesired))
deleted 2y ago
[deleted]
Arathorn 2y ago
There was a comment here complaining that Matrix is a failure as an open protocol because encryption in Matrix is too complex and hard and PFS is overrated and "why can't we have a simple protocol for chat like Wireguard is for VPNs"... but it got deleted while i as writing my reply. I'll post the reply anyway: Matrix without encryption is as simple as it gets - e.g. here was a younger, happier me writing a working client in 8 lines of bash: https://news.ycombinator.com/item?id=20948530 https://news.ycombinator.com/item?id=20948530 With encryption, inevitably things get way more complicated - especially in a decentralised network which needs to be byzantine fault tolerant. As you say, we've successfully simplified this by providing best-in-class implementations like matrix-rust-sdk-crypto - which i'd argue is the equivalent to Wireguard (which under the hood is a bunch of gnarly crypto, even if the API it exposes it simple). In the end, encrypting messaging is just way harder than a VPN. The encryption hooks need to know the membership of the room (as users), the membership of the room (as devices), verify identities of all devices and their users to prevent MITM, verify that only the right devices can be added to the room, handle accessing history for new logins and new joiners, handle backing up history if you log out of all devices, handle receiving msgs if you log out of all devices, handle encrypted push notifs and allow multiple processes (push, share extension, etc) to share the same crypto state, scale to thousands of devices, etc etc. Meanwhile if you simplify that by removing PFS - sure, some of it gets better ("the room history gets encrypted by a static password!") but then breaching that secret from any client at any point trivially leaks the whole history of the room. In terms of "Matrix as an open protocol isn't very successful", i suggest taking a look at https://2024.matrix.org/watch/ https://2024.matrix.org/watch/ for the zeitgeist from a few weeks ago. It's working for some folks at least.
- INTPenis 2y ago
  The thing is, while we were all waiting for Matrix clients to handle Encryption several other encrypted messengers came out and seem to handle it just fine. For example I've been using SimpleX in a group chat for a year now and had no issues at all.
  - Arathorn 2y ago
    I'd argue that SimpleX has probably learned from the path Matrix has been on... and so has Matrix. So you end up with multiple viable options to pick from, and to drive each other forwards; sounds like a good outcome to me.
huxflux 2y ago
I've been setting up servers for everything ranging from IRC to Teamspeak, Exchange, and you name it. But how come setting up a Matrix server is still a science in itself? Yes, I'm aware of the Ansible script, but should that really be the only somewhat reasonable option? When Matrix wrote " Matrix 2.0 a new chance if you gave up on us in the past". I listened, but without a somewhat reasonable setup process for the server and the ongoing confusion in terms of clients, it hasn't gotten any easier to onboard to Matrix, unfortunately.
- Arathorn 2y ago
  i think you're asking for https://element.io/blog/experimenting-with-matrix-2-0-using-element-docker-demo/ https://element.io/blog/experimenting-with-matrix-2-0-using-...
- xinayder 2y ago
  I have to agree with you. Not having a proper, understandable documentation is part of what drives people away from Matrix, IMO. And shoving the ansible script into people's faces is not a solution. I never used Ansible, and I won't start using it just because it's "easier" to setup Matrix with it, instead of setting it up by myself. All services I self-host are quite easy to setup, Matrix so far has been a bit more difficult, but not that hard.
  - Arathorn 2y ago
    this is literally why i wrote https://github.com/element-hq/element-docker-demo https://github.com/element-hq/element-docker-demo
- NicolaiS 2y ago
  Self-hosting Conduit as my homeserver using SQLite as by database and Caddy as reverse proxy. That is: 2 binaries (conduit + caddy) + 5 lines of toml config for Conduit and 2 line of config for Caddy + 1 database file. That's all you need for a basic homeserver.
  - 2y ago
solarkraft 2y ago
Oh my, this looks nice and comprehensive! I’ve been wanting to write a Matrix client for a while now, but found myself too caught up in getting even an existing library to work: The Rust SDK, the most maintained and batteries-included SDK, doesn’t have JS bindings and the JS SDK is insufficiently documented and generally seems a bit crusty. In general I find there to be a bit of a lack of guides on implementing Matrix, which is probability a factor in there not being that many implementations.
- factormeta 2y ago
  Would really really be nice to have the -user management -> registering -> device management A hello world example for v3 that will be great. To understand how to actually register a new account on the matrix server with Registration Token enabled server.
deleted 2y ago
[deleted]

141 points by whereistimbo 2y ago

The lack of good clients is really holding Matrix back. Element is rather bloated, and most of the other clients are missing significant amounts of features.

ericjmorey 2y ago
I'd say that the protocol itself is what's doing Matrix in.
- alwayslikethis 2y ago
  In order to support bridges to so many different proprietary platforms Matrix needs to have a superset of their features, so the feature creep is probably intentional. It does make it harder for clients to keep up though.
  - cmeacham98 2y ago
    Matrix is the opposite of feature creep - there are several features entirely missing from major clients that users of modern chat apps expect to have. (Examples include: custom emoji, functional search of encrypted chats, pinned messages or some sort of MOTD, forwarding messages, etc)
    - ptman 2y ago
      The matrix developers are experienced IM developers. They've built systems on XMPP and custom protocols before. Their estimate was that because of matrix being federated, every feature was 5-10x harder to implement than in centralized systems.
    - Arathorn 2y ago
      As a protocol, Matrix has all of these features (custom emoji is a proposal, but many clients implement it). As a client, Element has all of these (encrypted search only on Desktop for now) other than custom emoji.
- isaacaggrey 2y ago
  Could you elaborate?
  - ranger_danger 2y ago
    https://0x0.st/H8h-.png https://0x0.st/H8h-.png
    - Groxx 2y ago
      All that really shows is "the project has an active community". Have you seen how many XMPP has? Or Python[1]? This is just normal open development metadata. [1]: https://peps.python.org/pep-0000/# https://peps.python.org/pep-0000/#
      - jeroenhd 2y ago
        > Have you seen how many XMPP has? Or Python[1]? Python has had 664 PEPs in 23 years (29 per year). XMPP has received 495 XEPs in 23 years (22½ per year). Matrix has received about 650 in 8 years (>81 per year). Four times the change rate is quite annoying when writing code against the spec. Plus, most XMPP clients only support a fraction of the full spec, so by that comparison the impact of the rate of change is even worse. Furthermore, most XEPs and PEPs are mere (optional) extensions, whereas a lot of MSCs are alterations of existing APIs. Any JSON parser used for Matrix needs to anticipate fields changing or being added all over the place because you never know when random fields show up all over the input data because of a spec change. The way the Matrix spec is developed feels a lot more like a proprietary company spec that happens to be published on Github than the IETF/XMPP/Python spec process. The rate of change is high and almost all changes are done to serve new features for the two or three major players that bought into the Matrix ecosystem. One recent change that comes to mind is the move from secret, public URLs for media, to authenticated URLs. The setting to force that changeover won't apply everywhere for a while, but it'll completely break every media-supporting client written before the spec change. Nothing wrong with extending the spec to improve the product, but with how fast the protocol is growing, I wouldn't want to be tasked with maintaining a Matrix client and I don't have much faith in the forward compatibility of the few Matrix bots I've written either.
- Arathorn 2y ago
  What are you thinking of in particular? Fwiw, as project lead, i would be first to admit that there were slow bits - eg sync v2, which is why we built sliding sync… and crypto was easy to get wrong, hence shipping a highly polished implementation in matrix-rust-sdk. But what do you have in mind? Otherwise this doesn’t sound very wellfounded…
  - lifty 2y ago
    It's been a bumpy road but I think now Matrix has gotten to a good place. What's still missing is for the ecosystem to catch up a bit, both on the home server side and on the client side, and implement some of the new usability improvements brought by sliding sync and the rust sdk. Looking forward to see it happen!
heroprotagonist 2y ago
I _really_ wish that if Element has an update prompt for me almost every single time I open it, that it would just update during the startup sequence on its own. Or at least have an option for it. Not a 'Oh you loaded, now click this button to load again please' prompt whenever I launch it.
paulcarroty 2y ago
cinny/fluffychat/iamb are the greatest.
itsthejb 2y ago
Very very true. I self host synapse, and generally speaking the ability to bridge my ~3 most used messengers into one app (ElementX) adds value. However, the lacking features and bizarre feature disjoints between Element (supposedly EOL) and ElementX (suppose next gen) are jarring
Arathorn 2y ago
Element X (https://element.io/blog/deep-dive-into-element-x https://element.io/blog/deep-dive-into-element-x) is intended to spell out how lightweight and unbloated an Element-style client can be. The only main missing features are threads and spaces which should land in the coming months.

ranger_danger 2y ago

Finally something besides the matrix foundation's "lol the auto-generated API docs are enough" nonsense. This really helps a whole lot better.

amstan 2y ago

I tried to do something similar. It's infuriating how the client must be stateful and have local storage, for both the access_token and even last message recieved. That's right you must remember as the client where the last events [1] you've seen (even if you already told the server to mark it as read) was or else the server will happily send you the same messages over and over again. I kind of miss making IRC bots where things were much simpler and ... quicker honestly (latency wise). [1] https://uhoreg.gitlab.io/matrix-tutorial/sync.html#:~:text=we%20retrieve%20the-,next_batch,-property%20to%20use https://uhoreg.gitlab.io/matrix-tutorial/sync.html#:~:text=w...

dzaima 2y ago
At least that means that you will never have messages get dropped due to network issues, or even restarts if you persist the token across such. (the API will jump forwards if there are too many unsynced messages, but at least it should then be providing a "limited: true" so that one can paginate in the omitted events if so desired (or not if undesired))

deleted 2y ago

[deleted]

Arathorn 2y ago

There was a comment here complaining that Matrix is a failure as an open protocol because encryption in Matrix is too complex and hard and PFS is overrated and "why can't we have a simple protocol for chat like Wireguard is for VPNs"... but it got deleted while i as writing my reply. I'll post the reply anyway: Matrix without encryption is as simple as it gets - e.g. here was a younger, happier me writing a working client in 8 lines of bash: https://news.ycombinator.com/item?id=20948530 https://news.ycombinator.com/item?id=20948530 With encryption, inevitably things get way more complicated - especially in a decentralised network which needs to be byzantine fault tolerant. As you say, we've successfully simplified this by providing best-in-class implementations like matrix-rust-sdk-crypto - which i'd argue is the equivalent to Wireguard (which under the hood is a bunch of gnarly crypto, even if the API it exposes it simple). In the end, encrypting messaging is just way harder than a VPN. The encryption hooks need to know the membership of the room (as users), the membership of the room (as devices), verify identities of all devices and their users to prevent MITM, verify that only the right devices can be added to the room, handle accessing history for new logins and new joiners, handle backing up history if you log out of all devices, handle receiving msgs if you log out of all devices, handle encrypted push notifs and allow multiple processes (push, share extension, etc) to share the same crypto state, scale to thousands of devices, etc etc. Meanwhile if you simplify that by removing PFS - sure, some of it gets better ("the room history gets encrypted by a static password!") but then breaching that secret from any client at any point trivially leaks the whole history of the room. In terms of "Matrix as an open protocol isn't very successful", i suggest taking a look at https://2024.matrix.org/watch/ https://2024.matrix.org/watch/ for the zeitgeist from a few weeks ago. It's working for some folks at least.

INTPenis 2y ago
The thing is, while we were all waiting for Matrix clients to handle Encryption several other encrypted messengers came out and seem to handle it just fine. For example I've been using SimpleX in a group chat for a year now and had no issues at all.
- Arathorn 2y ago
  I'd argue that SimpleX has probably learned from the path Matrix has been on... and so has Matrix. So you end up with multiple viable options to pick from, and to drive each other forwards; sounds like a good outcome to me.

huxflux 2y ago

I've been setting up servers for everything ranging from IRC to Teamspeak, Exchange, and you name it. But how come setting up a Matrix server is still a science in itself? Yes, I'm aware of the Ansible script, but should that really be the only somewhat reasonable option? When Matrix wrote " Matrix 2.0 a new chance if you gave up on us in the past". I listened, but without a somewhat reasonable setup process for the server and the ongoing confusion in terms of clients, it hasn't gotten any easier to onboard to Matrix, unfortunately.

Arathorn 2y ago
i think you're asking for https://element.io/blog/experimenting-with-matrix-2-0-using-element-docker-demo/ https://element.io/blog/experimenting-with-matrix-2-0-using-...
xinayder 2y ago
I have to agree with you. Not having a proper, understandable documentation is part of what drives people away from Matrix, IMO. And shoving the ansible script into people's faces is not a solution. I never used Ansible, and I won't start using it just because it's "easier" to setup Matrix with it, instead of setting it up by myself. All services I self-host are quite easy to setup, Matrix so far has been a bit more difficult, but not that hard.
- Arathorn 2y ago
  this is literally why i wrote https://github.com/element-hq/element-docker-demo https://github.com/element-hq/element-docker-demo
NicolaiS 2y ago
Self-hosting Conduit as my homeserver using SQLite as by database and Caddy as reverse proxy. That is: 2 binaries (conduit + caddy) + 5 lines of toml config for Conduit and 2 line of config for Caddy + 1 database file. That's all you need for a basic homeserver.
- 2y ago

solarkraft 2y ago

Oh my, this looks nice and comprehensive! I’ve been wanting to write a Matrix client for a while now, but found myself too caught up in getting even an existing library to work: The Rust SDK, the most maintained and batteries-included SDK, doesn’t have JS bindings and the JS SDK is insufficiently documented and generally seems a bit crusty. In general I find there to be a bit of a lack of guides on implementing Matrix, which is probability a factor in there not being that many implementations.

factormeta 2y ago
Would really really be nice to have the -user management -> registering -> device management A hello world example for v3 that will be great. To understand how to actually register a new account on the matrix server with Registration Token enabled server.

deleted 2y ago

[deleted]