3 ms·

I kinda felt similarly until I found one of my existing web services wasn't quite fast enough (currently serving around 1000req/s at peak). I could either go fo

• vmchale 8y ago
  What are you serving? Some of the Rust frameworks for backend should be able to do more than 1000req/s, but it depends what you're doing.
  • tech2 8y ago
    The system's peak _usage_ is 60kreq/min (avg 1k/s), the Python code it's written in can handle maybe 850/s per instance at 100% CPU. Rust is fine, rust is managing almost 2k/s on the same hardware. And this is when I know practically nothing about Rust, I expect I'm likely able to do better given time. It's acting as a simple filter and cache over JSON data on Postgres.
    • emptysea 8y ago
      Any reason you didn't scale up horizontally with more python instances?
      • steveklabnik 8y ago
        We have seen production users choose to make this port to save on server costs. You can always scale horizontally, but it comes with a price tag.
        • richardwhiuk 8y ago
          Also complexity - managing an order of magnitude more instances is more hassle.
          • cies 8y ago
            But "more hassle" by an order of magnitude that gradualy reduces when growing the number of instances.
            • iopq 8y ago
              But the cost keeps growing, especially when you need to go from 50 servers to 100
          • weberc2 8y ago
            The OP only doubled their performance by rewriting in Rust though...
            • beatgammit 8y ago
              And depending on the amount of code, that's an awesome trade-off if OP can just have one instance instead of managing a cluster. From what OP has said, there's probably a good change they can get more performance with some optimization.
              • weberc2 8y ago
                You still need redundancy, so you’re operating multiple instances either way (or you’re playing fast and loose).
      • tech2 8y ago
        I have multiple instances to keep the latency down overall (and for security should we lose an availability zone or whatever else). But, as others have mentioned, instances cost, I'd like to be able to reduce that to the minimum if possible.
    • weberc2 8y ago
      I’m shocked that Rust only gave you a 2X improvement over Python (I’ve rewritten a handful of Python services to Go and typically see 100-1000X improvement). What is the bottleneck? Was scaling horizontally an option?
      • tech2 8y ago
        I suspect the reason it's not faster is one of the following: 1. backend DB at its query limit 2. this is basically my first rust program and I've done something stupid 3. ab/wrk aren't scaling properly either (I know, not likely, but worth considering) when I'm testing it 4. bandwidth/IO limits on the AWS instance types I'm running this on There are other possibilities too I guess. I was mostly doing this as a means to learn Rust though, I'd only go live with it once I had decent tests and everything else.
        • the_duke 8y ago
          What web framework are you using? Are you using a database? If so, which one? Are you using a cache like Redis? I agree that a 2x speedup is a terribly low for a Python to Rust rewrite and shows that Python is unlikely to be the relevant bottleneck here.
          • tech2 8y ago
            actix/r2d2 on the Rust side (I haven't worked out bb8 yet), aiohttp/uvloop/asyncpg/gunicorn on the Python side (Postgres for the DB). This kinda _is_ the cache at present, it's a distillation of a lot of other data into a given form. Rust's async story re: databases still looks a little up-in-the-air right now from an outside perspective though, so I guess I'll wait and see how it pans out. It's definitely a work in progress and I didn't expect this much interest in my offhand "I tried Rust even though I couldn't see myself using it for real, but..." comment.
          • weberc2 8y ago
            > What web framework are you using? I can't imagine the web framework is the bottleneck...