3 ms·
I think this is missing the point of the article. Yes, you can implement a faster Python version, but notice also: * This faster version is reading all the fi
by epidemian 5y ago
I think this is missing the point of the article.
Yes, you can implement a faster Python version, but notice also:
* This faster version is reading all the file into memory (except comment lines). The article mentions the data being 150MB, which should fit in memory, but for larger datasets, this approach would be unfeasible
* The faster version is actually delegating a lot of work to Python's C internals by using text.count('G'). All the internal looping and comparisons is done in C, while on the original version, goes through Python
So yes, you can definitely write faster Python by delegating most of the work to C.
The point of the article is not about how to optimize Python, but about how given almost identical implementations in Python and Nim, Nim can outperform Python by 1 or 2 orders of magnitude without resorting to use C internals for basic things like looping or comparing characters.
- soundmasterj 5y agoI didn't try to write optimized code, but idiomatic Python. Which also happens to be 10x faster. To make it streaming, take the second version and remove the readlines (directly iterate over f). Delegating work to Python's C internals is fine IMO because "batteries included" is a key feature of Python. "Nim outperforms unidiomatic Python that deliberately ignores key language features" is perhaps true, but less flashy of a headline. And to be honest, I mainly wrote this because the other top level Python implementations for this one were terrible at the time of the post.