4 ms·

> If we moved from Python to a language that is an order of magnitude faster, we can both improve how responsive our site is and decrease our server costs drama

• alfalfasprout 7y ago
  Well Brad, that's very dependent on what exactly the service is. Moreover oftentimes low CPU utilization is actually a limitation of the implementation on a slow language (eg; Python technically does have async webservers but adds a lot of idle overhead). Indeed, there are many benefits these more performant languages have over Python aside from raw single-core performance. For starters, more efficient concurrency and parallelism can help reduce average latency when combined with a quality async webserver. Then there's gains due to shared memory across threads. So in many cases-- absolutely, you can only need 1 server vs. 10 when you migrate. It's thus not fair to say that these gains are "very rare".
• pm90 7y ago
  * It’s not just that Go is faster to run but also faster to iterate on. If python can be neither, its offering little benefit. * they moved from a monolith to a microservices architecture; concern that any of the services in the request path could add latency just because of the overall runtime speed is slow is a legitimate one. * their primary deployment method is Google App Engine where you are billed by CPU used. Any change that consumed less CPUs has a tangible effect on their costs
  • andy_ppp 7y ago
    Go is faster to iterate on is absolutely false. Where do you get this idea from?
    • pm90 7y ago
      I can easily ask the vice versa of such a bland question. Answer that first if you want a real answer instead of trying to flame.
      • andy_ppp 7y ago
        It’s not bland it’s direct. What you’re doing is conflating personal preference with actual language features. Most developers who aren’t us would say python, being a higher level, dynamic scripting language rather than Golang which is lower level and extremely explicit about the data your program is using. Iteration in go is simply harder as you have to be more explicit rather than sketching something out. Changing that more explicit stuff is harder if you got it wrong while iterating. Why do you think Golang is better at iteration? Add in extremely poor error messages, lack of generics, having to generate loads of code for various things, the ability to crash whole services if your program does something incorrect, excruciating error handling will all slow you down.
        • danenania 7y ago
          Dynamic languages generally enable faster iteration in the early stages of a project, but once you have a large, mature codebase, static types allow you to work faster and produce fewer bugs. A more performant language will also run your test suite faster which can have a big impact as a project gets very large. Also, while I agree Go’s error handling isn’t very elegant, it does force you to explicitly consider every potential error, which in my experience makes uncaught errors far less likely than a language with bubbling exceptions.
        • krenel 7y ago
          IMHO the affirmation of > Go is faster to iterate holds true considering the total lifespan of the project. Golang is more explicit thus requiring more time to define every type but I've never refactored so fast and safe a codebase. In Python the fact that is dynamic makes it more difficult to safely iterate over it (is more statically-typed Vs dynamically typed). About error handling, it's not perfect, but the code is readable, easy to follow and easy to reason about. > Iteration in go is simply harder as you have to be more explicit rather than sketching something out. I will completely agree that prototyping in Python is way faster. Python is my preferred language for throwaway/prototype code. > Where do you get this idea from? It's solely based in my experience. I hope it contributes to the conversation.
• speedplane 7y ago
  > while other languages are 10x or more faster than python in some benchmarks it's very rare that computation time dominates server latency Most applications spend most of their time waiting for the database or network. I suspect the fastest programming languages are those that have the lowest thread/process overhead. If most apps spend their time waiting, then a language with 10X lower process overhead can handle 10X more processes.
  • golergka 7y ago
    You can also avoid using separate process for each client (NodeJS).
    • speedplane 7y ago
      > You can also avoid using separate process for each client (NodeJS). Yes, NodeJS solves one problem by having low process overhead, but it also fails to take advantage of parallelism in modern processors. Ideally, I'd like to see a system with both.
      • nablaoperator 7y ago
        You can always use the 'cluster' module or worker threads.
      • jacques_chester 7y ago
        > Ideally, I'd like to see a system with both. Java, or Kotlin, using one of the reactive frameworks.
• tgv 7y ago
  I inherited a few python/django servers. One of these has workers that grow to about 1Gb over time, even though they retain absolutely no data in core (or shouldn't). The same server is used to collect data, convert things and analyze the data. Especially the latter can take a bit of time, which means that there is a problem when more than two people try it at the same time, since it severely hinders the other tasks. I'm now converting one server to Go (although not the heavy one), and it really runs fast and uses much, much less memory. It also starts in in less than 1s, whereas the django application takes 5 minutes, because of some stupid problem in static file collection. Python is fine as a teaching tool, to prototype in, and to use in notebooks as a wrapper around numpy, scipy, etc., but not to run in production.
• dangoor 7y ago
  Sorry if this wasn't clear: Go is 10x faster than Python, yes, but we know that we're not going to reduce our server count by 90%. 50% is quite possible, though, given Go's superior threading and its good resource use. Moving away from the monolith should also give us new optimization possibilities.