3 ms·

> I have a couple of services that I consider trivial, but if I were writing them in Python, would be non-trivial. I've worked on two large Python projects whi

> I have a couple of services that I consider trivial, but if I were writing them in Python, would be non-trivial. I've worked on two large Python projects which would have been trivial in Go. In Python, we had to scale them out to multiple machines and ultimately an Apache Spark cluster (after trying all of the multiprocessing / numpy / Cython / etc gimmicks). The naive Go implementation was a dozen times faster and dramatically more maintainable. Unfortunately our principal engineer couldn't be reasoned with--he understood that in general switching languages isn't a panacea, but he didn't understand the specific problems we were encountering due to Python nor how dramatically Go could improve on them. (He was an inexperienced frontend engineer who was rapidly promoted because he would make shiny demos and hand them off to other teams to fix / extend / operate, and predictably that startup failed).

zozbot234 5y ago
> Unfortunately our principal engineer couldn't be reasoned with--he understood that in general switching languages isn't a panacea, but he didn't understand the specific problems we were encountering due to Python nor how dramatically Go could improve on them. It's so interesting that people usually have this complaint about Rust, but here it was a rewrite to Go that was seen as problematic for some reason. Sometimes you just can't win.
- pdimitar 5y ago
  Rewrites are usually a hard sell. Their benefits are most of the time clear only to those who initiate them. The from/to languages hardly matter to bystanders, sadly.
  - throwaway894345 5y ago
    And in general, rewrites often are a bad idea. But Python is hundreds of times slower than languages like Go and it really doesn't have good levers for improving performance--the "rewrite the slow parts in C / multiprocessing / numpy" promises rarely pan out due to the overhead of pickling / marshaling to efficient C representations. Unfortunately, our principle engineer lacked any mechanical sympathy--he could write JS and Python but had no idea how V8/CPython worked under the hood or how other languages might compare. Ultimately, the more interesting story is how this person was able to "fail upward" so effectively. Basically, nontechnical management don't understand backend software, so the person who makes it "real" for them via user interface demos is the person they associate with manifesting the working feature, even if user interface demos are superficial. Moreover, management only interacted with features by way of his demos, so they didn't understand how much work was required to productionize his demos. The features he debuted would be handed to another team to productionize and support. That team would appear to move slowly as they mired through his tech debt while this principle engineer was moving onto yet another demo, giving the appearance that he was our prolific 10x engineer.
- throwaway894345 5y ago
  Yeah, this guy only had a superficial understanding of JS and Python--he had no idea how they worked under the hood, had no low level experience, no CS background, etc. We already sunk so much effort into the Python implementation that he couldn't get behind a rewrite in another language, and moreover I suspect he was intimidated by a "lower level" language (relative to Python) and figured it would take weeks or months to be productive. In his defense, if you work in Python and JavaScript a lot, you get pretty jaded on tools and techniques that promise to be silver bullets and the idea that you could have a language that is orders of magnitude faster than Python, easy to learn, better package management and other tooling, and at least as maintainable just wasn't believable. Frankly, I think it would have been an easier sell if I lied and said that Go was only slightly better than Python.
  - zozbot234 5y ago
    I guess this is the perfect use case for Nim. Same surface syntax as Python, so your senior dev would've had no issue surveying the code at least, even though contributing to it would've been more of a challenge (Nim compiles straight to C; it's not nearly as high level as either Go or Rust).
    - throwaway894345 5y ago
      Probably, although I suspect Nim wouldn't have been a good choice because we were deploying on Lambda and we were using a lot of libraries for which there probably aren't good Nim alternatives (e.g., various cloud databases).

zozbot234 5y ago

> Unfortunately our principal engineer couldn't be reasoned with--he understood that in general switching languages isn't a panacea, but he didn't understand the specific problems we were encountering due to Python nor how dramatically Go could improve on them. It's so interesting that people usually have this complaint about Rust, but here it was a rewrite to Go that was seen as problematic for some reason. Sometimes you just can't win.

pdimitar 5y ago
Rewrites are usually a hard sell. Their benefits are most of the time clear only to those who initiate them. The from/to languages hardly matter to bystanders, sadly.
- throwaway894345 5y ago
  And in general, rewrites often are a bad idea. But Python is hundreds of times slower than languages like Go and it really doesn't have good levers for improving performance--the "rewrite the slow parts in C / multiprocessing / numpy" promises rarely pan out due to the overhead of pickling / marshaling to efficient C representations. Unfortunately, our principle engineer lacked any mechanical sympathy--he could write JS and Python but had no idea how V8/CPython worked under the hood or how other languages might compare. Ultimately, the more interesting story is how this person was able to "fail upward" so effectively. Basically, nontechnical management don't understand backend software, so the person who makes it "real" for them via user interface demos is the person they associate with manifesting the working feature, even if user interface demos are superficial. Moreover, management only interacted with features by way of his demos, so they didn't understand how much work was required to productionize his demos. The features he debuted would be handed to another team to productionize and support. That team would appear to move slowly as they mired through his tech debt while this principle engineer was moving onto yet another demo, giving the appearance that he was our prolific 10x engineer.
throwaway894345 5y ago
Yeah, this guy only had a superficial understanding of JS and Python--he had no idea how they worked under the hood, had no low level experience, no CS background, etc. We already sunk so much effort into the Python implementation that he couldn't get behind a rewrite in another language, and moreover I suspect he was intimidated by a "lower level" language (relative to Python) and figured it would take weeks or months to be productive. In his defense, if you work in Python and JavaScript a lot, you get pretty jaded on tools and techniques that promise to be silver bullets and the idea that you could have a language that is orders of magnitude faster than Python, easy to learn, better package management and other tooling, and at least as maintainable just wasn't believable. Frankly, I think it would have been an easier sell if I lied and said that Go was only slightly better than Python.
- zozbot234 5y ago
  I guess this is the perfect use case for Nim. Same surface syntax as Python, so your senior dev would've had no issue surveying the code at least, even though contributing to it would've been more of a challenge (Nim compiles straight to C; it's not nearly as high level as either Go or Rust).
  - throwaway894345 5y ago
    Probably, although I suspect Nim wouldn't have been a good choice because we were deploying on Lambda and we were using a lot of libraries for which there probably aren't good Nim alternatives (e.g., various cloud databases).