3 ms·

Why didn't you profile the Python code and only rewrited the hot loops?

• masklinn 5y ago
  Not op but two reasons would be: 1. The computations are distributed enough that there is no one hot loop 2. Go is shit at integrating with other software without IPC, ipc could be infeasible or have too much overhead, it also complicates building / bundling / shipping a lot
  • antpls 5y ago
    Thanks : 1. In distributed computing, there are usually minimal communication because nodes communication is costly, the bottleneck can still be the hot loop on each nodes. 2. When optimizing the hot loop only, it can be quickly written in Cython or C, since the scope is usually a lot smaller, so no need to use Go.
    • masklinn 5y ago
      > In distributed computing I didn't mean distributed computing, I meant distributed through the program. Not all programs actually have a hot loop. > When optimizing the hot loop only, it can be quickly written in Cython or C, since the scope is usually a lot smaller, so no need to use Go. That assumes proficiency in C, and it still complicates building / bundling / shipping a lot, the first bit of native code into a Python codebase is a huge investment and a significant step back in convenience in many ways.