6 ms·

A bit annoying how servers meant to be run by people themselves are written in awfully slow languages. Python for matrix server is a terrible choice. Even worse

by rapsey 6y ago

A bit annoying how servers meant to be run by people themselves are written in awfully slow languages. Python for matrix server is a terrible choice. Even worse for home assistant. If it can't fit on raspberry pi it's useless.

ReactiveJelly 6y ago
They're working on a Go server impl, "Dendrite" which is supposed to be far faster and use less RAM. I was hoping for Rust, and I think there is a 3rd-party Rust server, but anything's better than Python. Node.js would probably be better than Python.
- runiq 6y ago
  The 3rd party Rust server is Conduit: https://conduit.rs https://conduit.rs
  - Macha 6y ago
    > Note: This project is work-in-progress and is still missing some features like joining private rooms over federation I'm hopeful it will get there somewhere, but I hestitate to recommend it while "easy to get working without issues" is a bar that it doesn't yet reach.
- eternalban 6y ago
  https://github.com/matrix-org/dendrite https://github.com/matrix-org/dendrite /aside: this is a curious comment in their readme: "A PostgreSQL database engine, which will perform better than SQLite with many users and/or larger rooms"
neolog 6y ago
I can't see how it would matter for home assistant. It's not a CPU-bound application.
- rapsey 6y ago
  Ram usage also important.
bschwindHN 6y ago
I can't take any of these server Python projects seriously. It's like mocking up a car in styrofoam and then just...releasing that mockup as the actual product.
- franga2000 6y ago
  Ah yes, all those styrofoam cars such as Instagram, Reddit, Bitbucket, Eventbrite.....
  - bschwindHN 6y ago
    Notice how the load balancers, databases, and caching layers at those places are _not_ written in Python. They then have to run a wasteful amount of instances to keep their Python services afloat. Modern computers can do an amazing amount of work in a given time slice, but it's a shame when so much of that is thrown away because someone decided to implement it in Python.
    - franga2000 6y ago
      Yes, because those layers aren't theirs. They are products of other companies that specialise in high-performance and stable software. They can afford to use annoying languages because performance is one of the main things they compete on. But for something like a social network, performance isn't something they care about. They care about being able to develop and roll out new features as fast as possible. The overhead of working in a language makes things hard and/or has poor tooling is far greater than the increased server cost of working in something like Python (or, as much as I hate to admit it, Node). (note, that I'm not talking about the client side here. developers should serve their users and a slow fronted is bad for users. a slow backend, however, is not something users care about) And since we're specifically talking about an open source project here, look at the difference in community contributions between any two equally popular projects where one is written in something ridiculous like C++ and the other in JavaScript. Even Go and Rust, which are as close to modern as it gets in the world of compiled languages, are still decades behind Java, Python and Node in features, tooling and support. So until those catch up, anything that doesn't need performance above all else is probably going to stick to more friendly languages and not lose sleep about wasted CPU cycles - after all, we built computers specifically to simplify our work and that's exactly what these languages are doing.
Teever 6y ago
I see this bike-shedding comment about the language choice for the reference Matrix server implementation in every single post about Matrix on Hackernews. Every. Single One. Millions of people use Matrix. It has been adopted by the French civil service, several German states, and the German armed forces. In a previous post I saw someone saying that American Airlines was looking at adopting Matrix for their employees. I assure you that whether or not the reference Matrix server runs well on a raspberry pi did not factor into the decision making process that these organizations used when they picked Matrix.
- canofbars 6y ago
  Its a valid comment. I tried to run synapse on a reasonable VM and it was unusably slow. Essentially you either use the main server or you are a large org that can pay for a powerful machine to run it. The average person can not run a matrix server.
  - Teever 6y ago
    This has not been my experience. I run three servers in separate VMs on a Lenovo T420 laptop. These laptops run for about $200USD on Ebay though you'll need to upgrade the ram and the HDD to an SSD, so the cost is about $300USD.
    - omnimus 6y ago
      Do you federate or these are silos for few users?
      - Teever 6y ago
        The three servers are federated with each other and the one of them is federated to the broader matrix.org network. I've found that loading the larger rooms from the matrix.org network can cause a bit of lag for the few dozen or so users on my servers but once the rooms have synced up that issue goes away. I don't administer the servers personally so I don't know exactly what the issue is but I have a feeling it's due to the limited ram that is provisioned to the VMs running the instances of synapse as there are many other VMs running on that same laptop. If the instances of Synapse were the only things running on the laptop and they had 12gb of ram each (1/3 the total 32gb limit of the laptop) I would imagine that this issue would go away.
franga2000 6y ago
Because nobody except for HN commenters cares what lamguage it's written in. It's about as easy to use and flexible as it gets so development is fast and people can contribute easily and it's entirely fast enough if you run it on reasonable hardware (for proof see other comments around here). The fact that it doesn't run on your toaster is probably a feature, not a bug. To get it running on a Pi, even if written in a compiled language, things would have to be stripped down, making the overall software worse - a for what reason? Running a Matrix server on a Pi in your closet is about the worst thing you can do. It's slow, unstable, has zero redundancy, needs constant maintenance, is physically in your home, on a residential connection.... If the point were for users to run this themselves, something like a local NAS (Synology has 1-click apps, I believe) would be the lowest-end reasonable option, but the whole idea of running your own Matrix server at home is insane! At the very least, it should have basic redundancy for uplink, data and power, which are already near-impossible to achieve at home. And why should servers hosted by individuals? That would be horribly wasteful and inefficient, as well as raising the bar for users. I'll guarantee you that anyone who has enough skill and experience running and maintaining servers will have more than just a Pi available to them and anyone who doesn't have that experience should really not be running such an important service themselves.
kiwidrew 6y ago
Python shouldn't be blamed here, because the real reason that the Matrix homeserver is so slow is due to fundamental protocol design issues. It doesn't matter what language you write it in, it's going to be a resource hog by design.
- rapsey 6y ago
  Python is slow as dirt. If the protocol is heavy, all the more reason a more efficient language should be used.
lokedhs 6y ago
You may want to look at Dendrite which is an alternative implementation that is being developed. It's implemented in Go, and is supposedly a lot lighter on system resources. https://matrix.org/docs/projects/server/dendrite https://matrix.org/docs/projects/server/dendrite

by rapsey 6y ago

ReactiveJelly 6y ago

They're working on a Go server impl, "Dendrite" which is supposed to be far faster and use less RAM. I was hoping for Rust, and I think there is a 3rd-party Rust server, but anything's better than Python. Node.js would probably be better than Python.

runiq 6y ago
The 3rd party Rust server is Conduit: https://conduit.rs https://conduit.rs
- Macha 6y ago
  > Note: This project is work-in-progress and is still missing some features like joining private rooms over federation I'm hopeful it will get there somewhere, but I hestitate to recommend it while "easy to get working without issues" is a bar that it doesn't yet reach.
eternalban 6y ago
https://github.com/matrix-org/dendrite https://github.com/matrix-org/dendrite /aside: this is a curious comment in their readme: "A PostgreSQL database engine, which will perform better than SQLite with many users and/or larger rooms"

neolog 6y ago

I can't see how it would matter for home assistant. It's not a CPU-bound application.

rapsey 6y ago
Ram usage also important.

bschwindHN 6y ago

I can't take any of these server Python projects seriously. It's like mocking up a car in styrofoam and then just...releasing that mockup as the actual product.

franga2000 6y ago
Ah yes, all those styrofoam cars such as Instagram, Reddit, Bitbucket, Eventbrite.....
- bschwindHN 6y ago
  Notice how the load balancers, databases, and caching layers at those places are _not_ written in Python. They then have to run a wasteful amount of instances to keep their Python services afloat. Modern computers can do an amazing amount of work in a given time slice, but it's a shame when so much of that is thrown away because someone decided to implement it in Python.
  - franga2000 6y ago
    Yes, because those layers aren't theirs. They are products of other companies that specialise in high-performance and stable software. They can afford to use annoying languages because performance is one of the main things they compete on. But for something like a social network, performance isn't something they care about. They care about being able to develop and roll out new features as fast as possible. The overhead of working in a language makes things hard and/or has poor tooling is far greater than the increased server cost of working in something like Python (or, as much as I hate to admit it, Node). (note, that I'm not talking about the client side here. developers should serve their users and a slow fronted is bad for users. a slow backend, however, is not something users care about) And since we're specifically talking about an open source project here, look at the difference in community contributions between any two equally popular projects where one is written in something ridiculous like C++ and the other in JavaScript. Even Go and Rust, which are as close to modern as it gets in the world of compiled languages, are still decades behind Java, Python and Node in features, tooling and support. So until those catch up, anything that doesn't need performance above all else is probably going to stick to more friendly languages and not lose sleep about wasted CPU cycles - after all, we built computers specifically to simplify our work and that's exactly what these languages are doing.

Teever 6y ago

I see this bike-shedding comment about the language choice for the reference Matrix server implementation in every single post about Matrix on Hackernews. Every. Single One. Millions of people use Matrix. It has been adopted by the French civil service, several German states, and the German armed forces. In a previous post I saw someone saying that American Airlines was looking at adopting Matrix for their employees. I assure you that whether or not the reference Matrix server runs well on a raspberry pi did not factor into the decision making process that these organizations used when they picked Matrix.

canofbars 6y ago
Its a valid comment. I tried to run synapse on a reasonable VM and it was unusably slow. Essentially you either use the main server or you are a large org that can pay for a powerful machine to run it. The average person can not run a matrix server.
- Teever 6y ago
  This has not been my experience. I run three servers in separate VMs on a Lenovo T420 laptop. These laptops run for about $200USD on Ebay though you'll need to upgrade the ram and the HDD to an SSD, so the cost is about $300USD.
  - omnimus 6y ago
    Do you federate or these are silos for few users?
    - Teever 6y ago
      The three servers are federated with each other and the one of them is federated to the broader matrix.org network. I've found that loading the larger rooms from the matrix.org network can cause a bit of lag for the few dozen or so users on my servers but once the rooms have synced up that issue goes away. I don't administer the servers personally so I don't know exactly what the issue is but I have a feeling it's due to the limited ram that is provisioned to the VMs running the instances of synapse as there are many other VMs running on that same laptop. If the instances of Synapse were the only things running on the laptop and they had 12gb of ram each (1/3 the total 32gb limit of the laptop) I would imagine that this issue would go away.

franga2000 6y ago

Because nobody except for HN commenters cares what lamguage it's written in. It's about as easy to use and flexible as it gets so development is fast and people can contribute easily and it's entirely fast enough if you run it on reasonable hardware (for proof see other comments around here). The fact that it doesn't run on your toaster is probably a feature, not a bug. To get it running on a Pi, even if written in a compiled language, things would have to be stripped down, making the overall software worse - a for what reason? Running a Matrix server on a Pi in your closet is about the worst thing you can do. It's slow, unstable, has zero redundancy, needs constant maintenance, is physically in your home, on a residential connection.... If the point were for users to run this themselves, something like a local NAS (Synology has 1-click apps, I believe) would be the lowest-end reasonable option, but the whole idea of running your own Matrix server at home is insane! At the very least, it should have basic redundancy for uplink, data and power, which are already near-impossible to achieve at home. And why should servers hosted by individuals? That would be horribly wasteful and inefficient, as well as raising the bar for users. I'll guarantee you that anyone who has enough skill and experience running and maintaining servers will have more than just a Pi available to them and anyone who doesn't have that experience should really not be running such an important service themselves.

kiwidrew 6y ago

Python shouldn't be blamed here, because the real reason that the Matrix homeserver is so slow is due to fundamental protocol design issues. It doesn't matter what language you write it in, it's going to be a resource hog by design.

rapsey 6y ago
Python is slow as dirt. If the protocol is heavy, all the more reason a more efficient language should be used.

lokedhs 6y ago

You may want to look at Dendrite which is an alternative implementation that is being developed. It's implemented in Go, and is supposedly a lot lighter on system resources. https://matrix.org/docs/projects/server/dendrite https://matrix.org/docs/projects/server/dendrite