3 ms·

The way I’m thinking of it is that Synapse (not to mention Dendrite) has been badly starved of development over the last few years due to the freerider problem

The way I’m thinking of it is that Synapse (not to mention Dendrite) has been badly starved of development over the last few years due to the freerider problem nuking the $ available to support our dev as the upstream. By making only optimisations that primarily support massive servers proprietary (for now), the $ from Synapse Pro can hopefully then directly fund the features and maintenance of core FOSS Synapse, including core optimisation work which benefits everyone. Concrete examples include: * State Res v3, which hopefully should have algorithmic improvements to perf * State storage, addressing the long standing diskspace consumption problem for state snapshots * Finishing faster room joins so they are actually fast * Shared retry hints on federation, to stop wasting time trying to talk to dead servers * Better than full mesh federation, so small servers can participate in big rooms without melting. EDIT: to spell it out, as it’s been misunderstood: Fixing these issues will likely happen directly in FOSS Synapse (assuming there’s any $ to work on them at all). Meanwhile, FOSS Synapse already has Rust as Python codepaths and has done for years, and that will only increase over time. Only stuff for making workers scale to handle hundreds of thousand concurrent users will remain proprietary for now. Dendrite was meant to be the next big thing, but we (obviously) didn’t have enough $ to work on both it and Synapse. So we focused exclusively on Synapse as of end of Dec 2023, and Dendrite is currently progressing on a best-effort (ie not funded work) basis. > It feels like element is often jumping on new things like big rewrites such as element X instead of improving the existing products. Well, Dendrite was a big rewrite like that. Which is why we’ve ended up improving Synapse instead (both FOSS and the new rust workers).

wkat4242 2y ago
Many of the issues you mention though are real dealbreakers for those trying to use this as a private user. It's really a big adoption barrier not being able to join big rooms without the server 'melting'. Those rooms are big because they have a lot to offer. Those of us promoting matrix as a next gen truly free and federated platform have always expected these to be fixed. It's just not really fit for purpose without those fixes. It doesn't inspire confidence that we have to wait for this to 'trickle down' to the free version. And ok yeah I think it's a good idea to kill concurrent product development that does the same thing. As long as it makes the remaining product better. So there's also a Rust version of FOSS Synapse? I wasn't aware of that. Even the official blog post speaks only of python for the community version. > By making optimisations that primarily support massive servers proprietary (for now), the $ can hopefully then fund features and maintenance of core FOSS Synapse Your own blog post about the pro version (and the one about forking synapse) basically said that you wouldn't be working on the community version at all anymore? Unless I misunderstood. But my takeaway from this is 'it's a dead duck, it's time to pay up'. Which is a bit worrying as this is the kind of price only viable for large corporations.
- Arathorn 2y ago
  You’ve totally misunderstood me - probably my fault; sorry. There is no trickle-down happening here: *these issues will be fixed directly in FOSS synapse, which is the core project*. They are nothing to do with rust worker implementations in Synapse Pro, and FOSS will get them first. The only reason they haven’t been fixed before is due to the freerider nightmare meaning there hasn’t been $ to fund the work. FOSS Synapse has had Rust in it for years now: https://github.com/element-hq/synapse/tree/develop/rust/src https://github.com/element-hq/synapse/tree/develop/rust/src > Your own blog post about the pro version (and the one about forking synapse) basically said that you wouldn't be working on the community version at all anymore? WHAT?! It’s the opposite! We’re trying to fund the FOSS dev!! Where did it say that?!
  - wkat4242 2y ago
    Ok sorry for misunderstanding you. I'm glad those will be fixed and in the FOSS implementation <3 I thought this might happen only if things work out with the pro version first. > WHAT?! It’s the opposite! We’re trying to fund the FOSS dev!! Where did it say that?! Ah sorry it looks like I also misunderstood this blog post from 2023: https://matrix.org/blog/2023/11/06/future-of-synapse-dendrite/ https://matrix.org/blog/2023/11/06/future-of-synapse-dendrit... I thought the community version was left for the community to maintain when it was forked. Especially because the foundation said they wouldn't be able to fund further development. But I see now that it just meant that element would become the official host for synapse. With the other blog post (the recent one about synapse pro) it just goes into so much detail about the shortcomings of the community version that it kinda implies they won't be fixed. After all, if that were the case one could just wait and the problem would solve itself. But I apologise for drawing the wrong conclusions and arguing for them here. I do really understand you're in a difficult position. And I don't have a solution either.
    - Arathorn 2y ago
      np, thanks for listening to my explanation and for the apology - sorry on my side for yelling. i’m terrified now that the various blog posts haven’t been clear enough about the plan though - seems like we should have made it much clearer that FOSS synapse is still the primary project, and everything will happen there first (other than optimisations to support massive scalability)
deleted 2y ago
[deleted]
throwaway48476 2y ago
Maintaining multiple versions of software that have the same functionality is enormously stupid and wasteful. Eventually the excess versions will be abandoned such as dendrite. I foreseeable that the python implementation will be renamed synapse community and paid development on it will cease.

wkat4242 2y ago

Many of the issues you mention though are real dealbreakers for those trying to use this as a private user. It's really a big adoption barrier not being able to join big rooms without the server 'melting'. Those rooms are big because they have a lot to offer. Those of us promoting matrix as a next gen truly free and federated platform have always expected these to be fixed. It's just not really fit for purpose without those fixes. It doesn't inspire confidence that we have to wait for this to 'trickle down' to the free version. And ok yeah I think it's a good idea to kill concurrent product development that does the same thing. As long as it makes the remaining product better. So there's also a Rust version of FOSS Synapse? I wasn't aware of that. Even the official blog post speaks only of python for the community version. > By making optimisations that primarily support massive servers proprietary (for now), the $ can hopefully then fund features and maintenance of core FOSS Synapse Your own blog post about the pro version (and the one about forking synapse) basically said that you wouldn't be working on the community version at all anymore? Unless I misunderstood. But my takeaway from this is 'it's a dead duck, it's time to pay up'. Which is a bit worrying as this is the kind of price only viable for large corporations.

Arathorn 2y ago
You’ve totally misunderstood me - probably my fault; sorry. There is no trickle-down happening here: *these issues will be fixed directly in FOSS synapse, which is the core project*. They are nothing to do with rust worker implementations in Synapse Pro, and FOSS will get them first. The only reason they haven’t been fixed before is due to the freerider nightmare meaning there hasn’t been $ to fund the work. FOSS Synapse has had Rust in it for years now: https://github.com/element-hq/synapse/tree/develop/rust/src https://github.com/element-hq/synapse/tree/develop/rust/src > Your own blog post about the pro version (and the one about forking synapse) basically said that you wouldn't be working on the community version at all anymore? WHAT?! It’s the opposite! We’re trying to fund the FOSS dev!! Where did it say that?!
- wkat4242 2y ago
  Ok sorry for misunderstanding you. I'm glad those will be fixed and in the FOSS implementation <3 I thought this might happen only if things work out with the pro version first. > WHAT?! It’s the opposite! We’re trying to fund the FOSS dev!! Where did it say that?! Ah sorry it looks like I also misunderstood this blog post from 2023: https://matrix.org/blog/2023/11/06/future-of-synapse-dendrite/ https://matrix.org/blog/2023/11/06/future-of-synapse-dendrit... I thought the community version was left for the community to maintain when it was forked. Especially because the foundation said they wouldn't be able to fund further development. But I see now that it just meant that element would become the official host for synapse. With the other blog post (the recent one about synapse pro) it just goes into so much detail about the shortcomings of the community version that it kinda implies they won't be fixed. After all, if that were the case one could just wait and the problem would solve itself. But I apologise for drawing the wrong conclusions and arguing for them here. I do really understand you're in a difficult position. And I don't have a solution either.
  - Arathorn 2y ago
    np, thanks for listening to my explanation and for the apology - sorry on my side for yelling. i’m terrified now that the various blog posts haven’t been clear enough about the plan though - seems like we should have made it much clearer that FOSS synapse is still the primary project, and everything will happen there first (other than optimisations to support massive scalability)

deleted 2y ago

[deleted]

throwaway48476 2y ago

Maintaining multiple versions of software that have the same functionality is enormously stupid and wasteful. Eventually the excess versions will be abandoned such as dendrite. I foreseeable that the python implementation will be renamed synapse community and paid development on it will cease.