5 ms·

There is a lot of value in leveraging existing programming languages than rewriting in something else. My team rewrote large chunks of code from python -> Go an

There is a lot of value in leveraging existing programming languages than rewriting in something else. My team rewrote large chunks of code from python -> Go and it wasn't a pleasant experience. We were able to justify it, since it _inadvertently_ reduced our infra cost. But, if we could make python a lot faster at compiler level, it doesn't break the dev experience and lengthen dev on-boarding time. This helps the team productivity as a whole. I hope the CPython community is able to leverage few things from Cinder. This would benefit the entire python community.

adsharma 5y ago
Did you rewrite it by hand or use a transpiler?
- ram_rar 5y ago
  Mostly from scratch. I am yet to see a Python -> Go transpiler which generates human readable/maintainable code.
  - adsharma 5y ago
    Try http://github.com/adsharma/py2many http://github.com/adsharma/py2many tests/expected/*.go has generally human readable code.
pm90 5y ago
If you don’t mind sharing, what was it about the experience that made it unpleasant?
antpls 5y ago
Why didn't you profile the Python code and only rewrited the hot loops?
- masklinn 5y ago
  Not op but two reasons would be: 1. The computations are distributed enough that there is no one hot loop 2. Go is shit at integrating with other software without IPC, ipc could be infeasible or have too much overhead, it also complicates building / bundling / shipping a lot
  - antpls 5y ago
    Thanks : 1. In distributed computing, there are usually minimal communication because nodes communication is costly, the bottleneck can still be the hot loop on each nodes. 2. When optimizing the hot loop only, it can be quickly written in Cython or C, since the scope is usually a lot smaller, so no need to use Go.
    - masklinn 5y ago
      > In distributed computing I didn't mean distributed computing, I meant distributed through the program. Not all programs actually have a hot loop. > When optimizing the hot loop only, it can be quickly written in Cython or C, since the scope is usually a lot smaller, so no need to use Go. That assumes proficiency in C, and it still complicates building / bundling / shipping a lot, the first bit of native code into a Python codebase is a huge investment and a significant step back in convenience in many ways.

adsharma 5y ago

Did you rewrite it by hand or use a transpiler?

ram_rar 5y ago
Mostly from scratch. I am yet to see a Python -> Go transpiler which generates human readable/maintainable code.
- adsharma 5y ago
  Try http://github.com/adsharma/py2many http://github.com/adsharma/py2many tests/expected/*.go has generally human readable code.

pm90 5y ago

If you don’t mind sharing, what was it about the experience that made it unpleasant?

antpls 5y ago

Why didn't you profile the Python code and only rewrited the hot loops?

masklinn 5y ago
Not op but two reasons would be: 1. The computations are distributed enough that there is no one hot loop 2. Go is shit at integrating with other software without IPC, ipc could be infeasible or have too much overhead, it also complicates building / bundling / shipping a lot
- antpls 5y ago
  Thanks : 1. In distributed computing, there are usually minimal communication because nodes communication is costly, the bottleneck can still be the hot loop on each nodes. 2. When optimizing the hot loop only, it can be quickly written in Cython or C, since the scope is usually a lot smaller, so no need to use Go.
  - masklinn 5y ago
    > In distributed computing I didn't mean distributed computing, I meant distributed through the program. Not all programs actually have a hot loop. > When optimizing the hot loop only, it can be quickly written in Cython or C, since the scope is usually a lot smaller, so no need to use Go. That assumes proficiency in C, and it still complicates building / bundling / shipping a lot, the first bit of native code into a Python codebase is a huge investment and a significant step back in convenience in many ways.