3 ms·

If it was too slow in Python and now moving to Go. Could there a time when there is a need to move to Rust/C/C++ for even faster performance? Go seems an odd ch

If it was too slow in Python and now moving to Go. Could there a time when there is a need to move to Rust/C/C++ for even faster performance? Go seems an odd choice based on performance consideration alone.

tejinderss 7y ago
Especially with node.js/TypeScript one can reach similar performance to Go with arguably much nicer programming language to work with.
- FAKEDETECTOR 7y ago
  Evidence, please.
gwd 7y ago
As he said, given the size and complexity of the transition, one of his primary goals was to do as much of a "literal" translation first as possible. IOW, this wasn't so much of a "rewrite from scratch" (which is almost always a bad idea), as "do a translation first; then do refactoring to be more idiomatic / performant later". Obviously some parts will need to be rewritten, but the lower you can keep the rewrite:translation ratio, the better. I'm not an expert in Rust, but from what I know it seems like moving to Rust would require much more rewriting than Go. The other thing, as he says in the post, is that you get diminishing returns: The Go code was 20x faster than the Python code, but the whole 7-hour operation was only 10x times faster, because at some point the external SVM calls start to dominate. So even if a rewrite into Rust could gain him another dozen percentage points in speed, it's unlikely that a rewrite into Rust would have much of an impact on his end-to-end performance; and it would almost certainly make the code less accessible.
setr 7y ago
Someone once told me a tale of a student who had every variable defined as a single element array. When queried, she responded: "What if I need two of them?" If you consider the could, there's always a way to make things arbitrarily more complex. Reality is that your target is simply "sufficient". Also another provided reason was language consolidation, so ease of developer-project-flexibility was probably not the lowest of concerns

tejinderss 7y ago

Especially with node.js/TypeScript one can reach similar performance to Go with arguably much nicer programming language to work with.

FAKEDETECTOR 7y ago
Evidence, please.

gwd 7y ago

As he said, given the size and complexity of the transition, one of his primary goals was to do as much of a "literal" translation first as possible. IOW, this wasn't so much of a "rewrite from scratch" (which is almost always a bad idea), as "do a translation first; then do refactoring to be more idiomatic / performant later". Obviously some parts will need to be rewritten, but the lower you can keep the rewrite:translation ratio, the better. I'm not an expert in Rust, but from what I know it seems like moving to Rust would require much more rewriting than Go. The other thing, as he says in the post, is that you get diminishing returns: The Go code was 20x faster than the Python code, but the whole 7-hour operation was only 10x times faster, because at some point the external SVM calls start to dominate. So even if a rewrite into Rust could gain him another dozen percentage points in speed, it's unlikely that a rewrite into Rust would have much of an impact on his end-to-end performance; and it would almost certainly make the code less accessible.

setr 7y ago

Someone once told me a tale of a student who had every variable defined as a single element array. When queried, she responded: "What if I need two of them?" If you consider the could, there's always a way to make things arbitrarily more complex. Reality is that your target is simply "sufficient". Also another provided reason was language consolidation, so ease of developer-project-flexibility was probably not the lowest of concerns