6 ms·

How Prosody developers spent 2020

70 points by upofadown 6y ago

phoe-krk 6y ago
> Prosody does not “phone home” in any way, which means we do not have a lot of insight into how many people are using Prosody. But there are some indicators that we can use to see at least the growth of the project. Great kudos for the authors for writing their software this way; this is analytics done right.
half-kh-hacker 6y ago
I think XMPP is a really good contender in the non-centralized personal-and-public messaging space. It doesn't look like an insurmountable effort from one person to support the XEPs required to create a competent chat application, unlike my impression of Matrix. I think a nice, lean-but-glossy desktop client (we already have Conversations on mobile) would be really good for the platform. It's a lot easier to fight network effects if the end-user-experience is tangibly better (and you can do a LOT better than the current incumbents that are Slack and Discord.)
- feanaro 6y ago
  There are several Matrix projects that are quite full-featured and were built by a ~single person. Leveraging an appropriate SDK makes it feasible.
  - pmlnr 6y ago
    Most of them in alpha state, with some functionality - read not all XEP implemented ;)
    - feanaro 6y ago
      Sure, but that is not unlike the XMPP/XEP situation. That's pretty much unavoidable and comes down to the responsibility of the client. It's natural that there will only be a few full-featured clients since those require a lot of effort with either Matrix or XMPP.
      - pmlnr 6y ago
        That's exactly what I'm saying. XMPP got attacked a lot because of how XEP support was lacking in certain clients, but was implemented in others, yet nearly every federated system out there has the exact same, inevitable problem - email, Matrix, ActivityPub are no exceptions.
        feanaro 6y ago
        I think XMPP got attacked not because of the fragmented ecosystem of clients, but because the standard itself (in the sense of the complete set of XEPs in existence) got fragmented. As far as I know, multiple competing/overlapping XEPs were not an uncommon situation. Compared to this, the Matrix standard(s) have thus far been able to maintain a certain kind of coherence (IMO), allowing for at least a vision of what a complete implementation of a server or client should look like. The standard is versioned so that you can aim for support of a particular version instead of a pick-and-choose subset of functionality. Also, part of the Matrix's coherence story probably lies in the fact that there is an "official" set of clients in the form of the Element clients. Ultimately, you cannot prevent others from writing incomplete servers or clients. But you can show what the goal looks like and have other implementations approach it asymptotically.
        ge0rg 6y ago
        There have been significant efforts to reduce this kind of fragmentation and to outline which XEPs a client or server developer should focus on, given a certain target application. We are reviewing the most important specifications on a yearly base and provide recommendations in the form of Compliance Suites with different profiles like Instant Messaging, Mobile etc.: https://xmpp.org/extensions/xep-0443.html https://xmpp.org/extensions/xep-0443.html
        feanaro 6y ago
        That's great info, I wasn't aware of this. Thanks!
  - remram 6y ago
    If you use an SDK, you're not implementing the protocol or any XEP, and therefore it's not a testament to how easy that is to do.
    - feanaro 6y ago
      That's true, but then again, if everyone uses a small set of SDKs that are robust and known to be compliant, isn't that a strictly better situation? Why is it important to reimplement the standard itself from scratch?
      - remram 6y ago
        Better situation yes, better protocol no.
        feanaro 6y ago
        True, though it doesn't imply it's a worse protocol either.
- kitkat_new 6y ago
  > It doesn't look like an insurmountable effort from one person to support the XEPs required to create a competent chat application, unlike my impression of Matrix. Could you explain how you got this particular impression?
- jiofih 6y ago
  How come there hasn’t been a single really good implementation of it (besides defunct google chat), with presence, synchronized read receipts etc in the past decade?
  - pmlnr 6y ago
    Yes, there has: https://conversations.im/ https://conversations.im/ EDIT: more to read: https://blog.wirelessmoves.com/2020/05/xmpp-voice-and-video-calls-with-conversations-a-dream-come-true.html https://blog.wirelessmoves.com/2020/05/xmpp-voice-and-video-... https://homebrewserver.club/have-you-considered-the-alternative.html https://homebrewserver.club/have-you-considered-the-alternat...
- sweden 6y ago
  That statement doesn't reflect the sad state of XMPP clients available nowadays. I used to run my own XMPP server for years before moving on to Matrix and the reality was that the only good client was Conversations for Android. There wasn't a single one for Linux desktop that felt modern and up to speed with the latest XEPs. And let's not even mention trying to get multiple clients supporting the same history with encryption enabled, that was super chaotic. I'm really glad I abandoned my XMPP server in favor of Matrix. Matrix is modern and it just works and setting up a Synpase server is way easier than setting up a Prosody server.
  - MattJ100 6y ago
    When was "nowadays"? Gajim and Dino are definitely as decent as any native apps available for Matrix. Sorry you feel that way about Prosody. If you have specific feedback we're always looking to improve the setup experience. Snikket (briefly mentioned in the post) is an attempt to bundle everything you need for a modern communication server into a single package (e.g. automatic certs and stuff that you need for audio /video) out of the box. Maybe it sounds like it would suit you more.
    - sweden 6y ago
      I don't feel negative towards Prosody, Prosody is an awesome project. My criticism is about XMPP in general and its ecosystem. I stopped my XMPP server a few years ago, Dino was still in its infancy. It's good to see that the client is reaching some maturity but then, looking at the supported XEPs I can see that MAM is not supported for group chats: - https://github.com/dino/dino/wiki/Supported-XEPs https://github.com/dino/dino/wiki/Supported-XEPs Fortunately Matrix and the respective clients have been evolving quite well, so I don't see the need of going back to XMPP.
      - sam_lowry_ 6y ago
        I stopped my XMPP server some 7..9 years ago after seeing loads of spam in MUCs.
    - aidenn0 6y ago
pmlnr 6y ago
I'm one of them, and I love it. Desktop clients are lagging in support compared to Conversations[^1] on Android, but even Pidgin can be flogged into an acceptable state[^2] - no video or audio though, I couldn't get that working, unlike with Prosody and Conversations[^3]. The thing with XMPP vs Matrix is weight: and XMPP server is featherweight compared to Matrix due to their different nature, so I honestly believe that XMPP still has a future. (And maybe one day, Pidgin 3 will see the light[^4]) [^1]: https://conversations.im/ https://conversations.im/ [^2]: https://github.com/petermolnar/awesome-pidgin-plugins https://github.com/petermolnar/awesome-pidgin-plugins [^3]: https://gist.github.com/petermolnar/10d815eae0b7cbda2b1e5948db157d6a https://gist.github.com/petermolnar/10d815eae0b7cbda2b1e5948... [^4]: https://pidgin.im/development/building/3.0.0/ https://pidgin.im/development/building/3.0.0/
- MattJ100 6y ago
  Yeah, for desktop I'd generally recommend Gajim (recent versions) or Dino (younger project, but progressing well). I personally use a console client (poezio). XMPP has evolved a lot over recent years, but many folk still judge it by the state of Pidgin (where development is focusing on their big 3.0 rather than catching up with modern messaging features) or with the features XMPP had in 2006.
- kitkat_new 6y ago
  > and XMPP server is featherweight compared to Matrix due to their different nature I agree that Synapse, the only currently available Matrix server is resource heavy, but I would not extrapolate this to the future of Matrix servers. There is still quite some optimization possible.
  - swsieber 6y ago
    Dendrite is the next big thing right? Written in Rust? I'd be curious to see how the performance is there with it being in Rust and a rewrite/new implementation that can learn from experience.
    - remram 6y ago
YarickR2 6y ago
Absolutely love what you're doing, and I may come to you later to discuss possible commercial support and collaboration. Right now I'w working on a Prometheus to XMPP gateway with advanced filtering capabilities ( my idea is monitoring and alerting loop has to be tight , so you need means to deliver alerts using only things under your control), and XMPP is the right choice for this task, and I’m using Prosody as an alert delivery server component, with Conversation as a mobile client, and Dino on desktop.
- MattJ100 6y ago
  Sounds interesting :) Long before Prometheus was a thing I was working on a project to tunnel metrics in realtime over XMPP. I used it for some personal monitoring but never got around to polishing and publishing it. These days it totally makes sense to bridge with Prometheus or equivalents.
profsnuggles 6y ago
I'm running ejabberd now, it's been fine for me. Years ago I found it much easier to get all the XEPs conversations wanted working with it. Everything I needed was built into ejabbered and not all the prosody modules I needed were packaged for debian. What is the landscape like now? I want to be convinced to switch, what does prosody do better?
- MattJ100 6y ago
  I'm a Prosody developer. If you're happy with ejabberd, that's fine by me! It used to be a real pain to set up and maintain (one reason we started Prosody), but both ejabberd and Erlang have made progress in that area recently. Prosody has a strong community and one of our priorities is keeping stable, lightweight, and easy to extend and experiment with. We are aware that some useful/popular modules are still in the community repository. We polish and merge new ones with every major release. As detailed in the blog post we are adding a simple way to fetch and install community modules to a Prosody installation. Contrary to popular belief, XMPP is always evolving and adding new features, so this is our way of ensuring you can always keep up with new stuff even if it's not in a release yet. We have a chat, feel free to hang out and ask questions: https://prosody.im/discuss https://prosody.im/discuss
  - kevans91 6y ago
    Huh, cool! Are all Prosody modules pure-lua, or can/do some have a C/binary component to them? I note that the API documentation describes only lua bits, at least.
    - MattJ100 6y ago
      They are generally pure Lua, but they can load binary libraries. Some existing such libraries can be found in most distro repositories or on luarocks.org. Lua also has a well designed API in case you need to write your own bindings to something. If you have questions about how to do something specific we're very approachable and happy to share our knowledge :)
ofrzeta 6y ago
Prosody is also used by the Jitsi videoconferencing solution.

70 points by upofadown 6y ago

> Prosody does not “phone home” in any way, which means we do not have a lot of insight into how many people are using Prosody. But there are some indicators that we can use to see at least the growth of the project. Great kudos for the authors for writing their software this way; this is analytics done right.

half-kh-hacker 6y ago

I think XMPP is a really good contender in the non-centralized personal-and-public messaging space. It doesn't look like an insurmountable effort from one person to support the XEPs required to create a competent chat application, unlike my impression of Matrix. I think a nice, lean-but-glossy desktop client (we already have Conversations on mobile) would be really good for the platform. It's a lot easier to fight network effects if the end-user-experience is tangibly better (and you can do a LOT better than the current incumbents that are Slack and Discord.)

feanaro 6y ago
There are several Matrix projects that are quite full-featured and were built by a ~single person. Leveraging an appropriate SDK makes it feasible.
- pmlnr 6y ago
  Most of them in alpha state, with some functionality - read not all XEP implemented ;)
  - feanaro 6y ago
    Sure, but that is not unlike the XMPP/XEP situation. That's pretty much unavoidable and comes down to the responsibility of the client. It's natural that there will only be a few full-featured clients since those require a lot of effort with either Matrix or XMPP.
    - pmlnr 6y ago
      That's exactly what I'm saying. XMPP got attacked a lot because of how XEP support was lacking in certain clients, but was implemented in others, yet nearly every federated system out there has the exact same, inevitable problem - email, Matrix, ActivityPub are no exceptions.
      - feanaro 6y ago
        I think XMPP got attacked not because of the fragmented ecosystem of clients, but because the standard itself (in the sense of the complete set of XEPs in existence) got fragmented. As far as I know, multiple competing/overlapping XEPs were not an uncommon situation. Compared to this, the Matrix standard(s) have thus far been able to maintain a certain kind of coherence (IMO), allowing for at least a vision of what a complete implementation of a server or client should look like. The standard is versioned so that you can aim for support of a particular version instead of a pick-and-choose subset of functionality. Also, part of the Matrix's coherence story probably lies in the fact that there is an "official" set of clients in the form of the Element clients. Ultimately, you cannot prevent others from writing incomplete servers or clients. But you can show what the goal looks like and have other implementations approach it asymptotically.
        ge0rg 6y ago
        There have been significant efforts to reduce this kind of fragmentation and to outline which XEPs a client or server developer should focus on, given a certain target application. We are reviewing the most important specifications on a yearly base and provide recommendations in the form of Compliance Suites with different profiles like Instant Messaging, Mobile etc.: https://xmpp.org/extensions/xep-0443.html https://xmpp.org/extensions/xep-0443.html
        feanaro 6y ago
        That's great info, I wasn't aware of this. Thanks!
- remram 6y ago
  If you use an SDK, you're not implementing the protocol or any XEP, and therefore it's not a testament to how easy that is to do.
  - feanaro 6y ago
    That's true, but then again, if everyone uses a small set of SDKs that are robust and known to be compliant, isn't that a strictly better situation? Why is it important to reimplement the standard itself from scratch?
    - remram 6y ago
      Better situation yes, better protocol no.
      - feanaro 6y ago
        True, though it doesn't imply it's a worse protocol either.
kitkat_new 6y ago
> It doesn't look like an insurmountable effort from one person to support the XEPs required to create a competent chat application, unlike my impression of Matrix. Could you explain how you got this particular impression?
jiofih 6y ago
How come there hasn’t been a single really good implementation of it (besides defunct google chat), with presence, synchronized read receipts etc in the past decade?
- pmlnr 6y ago
  Yes, there has: https://conversations.im/ https://conversations.im/ EDIT: more to read: https://blog.wirelessmoves.com/2020/05/xmpp-voice-and-video-calls-with-conversations-a-dream-come-true.html https://blog.wirelessmoves.com/2020/05/xmpp-voice-and-video-... https://homebrewserver.club/have-you-considered-the-alternative.html https://homebrewserver.club/have-you-considered-the-alternat...
sweden 6y ago
That statement doesn't reflect the sad state of XMPP clients available nowadays. I used to run my own XMPP server for years before moving on to Matrix and the reality was that the only good client was Conversations for Android. There wasn't a single one for Linux desktop that felt modern and up to speed with the latest XEPs. And let's not even mention trying to get multiple clients supporting the same history with encryption enabled, that was super chaotic. I'm really glad I abandoned my XMPP server in favor of Matrix. Matrix is modern and it just works and setting up a Synpase server is way easier than setting up a Prosody server.
- MattJ100 6y ago
  When was "nowadays"? Gajim and Dino are definitely as decent as any native apps available for Matrix. Sorry you feel that way about Prosody. If you have specific feedback we're always looking to improve the setup experience. Snikket (briefly mentioned in the post) is an attempt to bundle everything you need for a modern communication server into a single package (e.g. automatic certs and stuff that you need for audio /video) out of the box. Maybe it sounds like it would suit you more.
  - sweden 6y ago
    I don't feel negative towards Prosody, Prosody is an awesome project. My criticism is about XMPP in general and its ecosystem. I stopped my XMPP server a few years ago, Dino was still in its infancy. It's good to see that the client is reaching some maturity but then, looking at the supported XEPs I can see that MAM is not supported for group chats: - https://github.com/dino/dino/wiki/Supported-XEPs https://github.com/dino/dino/wiki/Supported-XEPs Fortunately Matrix and the respective clients have been evolving quite well, so I don't see the need of going back to XMPP.
    - sam_lowry_ 6y ago
      I stopped my XMPP server some 7..9 years ago after seeing loads of spam in MUCs.
  - aidenn0 6y ago

pmlnr 6y ago

I'm one of them, and I love it. Desktop clients are lagging in support compared to Conversations[^1] on Android, but even Pidgin can be flogged into an acceptable state[^2] - no video or audio though, I couldn't get that working, unlike with Prosody and Conversations[^3]. The thing with XMPP vs Matrix is weight: and XMPP server is featherweight compared to Matrix due to their different nature, so I honestly believe that XMPP still has a future. (And maybe one day, Pidgin 3 will see the light[^4]) [^1]: https://conversations.im/ https://conversations.im/ [^2]: https://github.com/petermolnar/awesome-pidgin-plugins https://github.com/petermolnar/awesome-pidgin-plugins [^3]: https://gist.github.com/petermolnar/10d815eae0b7cbda2b1e5948db157d6a https://gist.github.com/petermolnar/10d815eae0b7cbda2b1e5948... [^4]: https://pidgin.im/development/building/3.0.0/ https://pidgin.im/development/building/3.0.0/

MattJ100 6y ago
Yeah, for desktop I'd generally recommend Gajim (recent versions) or Dino (younger project, but progressing well). I personally use a console client (poezio). XMPP has evolved a lot over recent years, but many folk still judge it by the state of Pidgin (where development is focusing on their big 3.0 rather than catching up with modern messaging features) or with the features XMPP had in 2006.
kitkat_new 6y ago
> and XMPP server is featherweight compared to Matrix due to their different nature I agree that Synapse, the only currently available Matrix server is resource heavy, but I would not extrapolate this to the future of Matrix servers. There is still quite some optimization possible.
- swsieber 6y ago
  Dendrite is the next big thing right? Written in Rust? I'd be curious to see how the performance is there with it being in Rust and a rewrite/new implementation that can learn from experience.
  - remram 6y ago

YarickR2 6y ago

Absolutely love what you're doing, and I may come to you later to discuss possible commercial support and collaboration. Right now I'w working on a Prometheus to XMPP gateway with advanced filtering capabilities ( my idea is monitoring and alerting loop has to be tight , so you need means to deliver alerts using only things under your control), and XMPP is the right choice for this task, and I’m using Prosody as an alert delivery server component, with Conversation as a mobile client, and Dino on desktop.

MattJ100 6y ago
Sounds interesting :) Long before Prometheus was a thing I was working on a project to tunnel metrics in realtime over XMPP. I used it for some personal monitoring but never got around to polishing and publishing it. These days it totally makes sense to bridge with Prometheus or equivalents.

profsnuggles 6y ago

I'm running ejabberd now, it's been fine for me. Years ago I found it much easier to get all the XEPs conversations wanted working with it. Everything I needed was built into ejabbered and not all the prosody modules I needed were packaged for debian. What is the landscape like now? I want to be convinced to switch, what does prosody do better?

MattJ100 6y ago
I'm a Prosody developer. If you're happy with ejabberd, that's fine by me! It used to be a real pain to set up and maintain (one reason we started Prosody), but both ejabberd and Erlang have made progress in that area recently. Prosody has a strong community and one of our priorities is keeping stable, lightweight, and easy to extend and experiment with. We are aware that some useful/popular modules are still in the community repository. We polish and merge new ones with every major release. As detailed in the blog post we are adding a simple way to fetch and install community modules to a Prosody installation. Contrary to popular belief, XMPP is always evolving and adding new features, so this is our way of ensuring you can always keep up with new stuff even if it's not in a release yet. We have a chat, feel free to hang out and ask questions: https://prosody.im/discuss https://prosody.im/discuss
- kevans91 6y ago
  Huh, cool! Are all Prosody modules pure-lua, or can/do some have a C/binary component to them? I note that the API documentation describes only lua bits, at least.
  - MattJ100 6y ago
    They are generally pure Lua, but they can load binary libraries. Some existing such libraries can be found in most distro repositories or on luarocks.org. Lua also has a well designed API in case you need to write your own bindings to something. If you have questions about how to do something specific we're very approachable and happy to share our knowledge :)

ofrzeta 6y ago

Prosody is also used by the Jitsi videoconferencing solution.