6 ms·

My intention is not to be snarky, but people post all the time about discovering the itertools or collections library. I notice it's a common gap in newer Pytho

by wting 13y ago

My intention is not to be snarky, but people post all the time about discovering the itertools or collections library. I notice it's a common gap in newer Python programmers. Save yourself time and effort down the road and read through both libraries' documentation, they're well worth the effort: http://docs.python.org/3.3/library/itertools.html http://docs.python.org/3.3/library/itertools.html http://docs.python.org/3.3/library/collections.html http://docs.python.org/3.3/library/collections.html I tend to use defaultdict, deque (thread safe), namedtuple, imap, izip, drop/takewhile. In Python 3, map and zip have been replaced with their itertools equivalents. I blame Haskell for all the lazy evaluation influence. :P

naiquevin 13y ago
Author here. I completely agree with you. Could have known all this before had I just read the itertools docs beyond permutations and combinations. It was only after using Scala's groupby while solving an assignment problem that I thought about finding out an equivalent in python. And thanks to having worked with the Stream class from Scala collections, for the first time the rest of the itertools also made sense. Better late than never I guess :-)
- Aqueous 13y ago
  Is there any reason that you don't switch over to Scala for your main programming language? I'm in the process of converting a middle-sized PHP codebase completely over to Scala. It is not uncommon for me to see functions that run between 15 and 30 lines shrink to 3 or 4 line functions thanks to the Collections API. On top of that its performance is something you'll never see in PHP or Python. I love Python for its readability and wonderful syntax, but Scala is starting to have an even greater pull on me for its built in concision and the ability to use the entire Java ecosystem without submitting to java's imperative style.
  - naiquevin 13y ago
    It's very tempting. Frankly, I had enrolled for the course mainly for functional programming and not Scala but towards the end I started liking it. But, at the moment my main languages at work are Python, Erlang and Javascript and I still have a long way to go with these so planning to stick to them for a while.
pjmlp 13y ago
>My intention is not to be snarky, but people post all the time about discovering the itertools or collections library. I notice it's a common gap in newer Python programmers. Not only in Python, but programming languages in general. I still find people writing Java or .NET code that aren't aware of all nice classes that are part of the runtime and end up creating their half baked solutions for their problems. Nowadays developers seem to code without reading.
- Silhouette 13y ago
  Nowadays developers seem to code without reading. When your standard library documentation is so vast that it would take weeks to read and understand it all, and you'd never remember most of it anyway without context and experience using it, I don't think "coding without reading" is really a fair complaint. We as an industry need to get better at documentation, and in particular about separating tutorial/overview documentation that presents a map and summary of what's available from reference documentation, or we're going to keep reinventing wheels like this. Python is a particularly unfortunate example, because while its documentation is vast, it has very little tutorial/overview material beyond the very basics. For example, given that a substantial proportion of Python's standard library actually doesn't work very well in practice, it would be helpful to have a deeper tutorial/map document somewhere that introduced the various areas of the standard library and that also promoted the good ones and suggested popular alternatives for the not so good ones where they exist.
  - Too 13y ago
    Documentation discoverability is one problem. Willingness to learn and trust is another one. People simply want to use things that they themselves have proven to work before. As an example an old colleague wanted to dump some data from python to a csv-file and did this by for-looping through each row and each item and concatenating each cell and a semicolon to a string. Even after pointing out to him that python already has a built in csv writer, that handles all issues of escaping etc, he didn't want to use it because he didn't know what it did and didnt want to learn anything new. His version didn't even do escaping inside the for-loop and he didn't see the issue of not doing it. To him the for-loop gave exactly the same result and didn't require any learning and was thus better, and why change something that works... My last suggestion was to at least use ";".join(...) but it was also a bit too magic so he stuck to his well known for-loop. Usually standard libraries are quite reliable but in some cases, and especially if adding third party libraries, bugs and performance issues inside the library can really give you hell. If the library is supposed to just perform a simple task maybe you would rather implement it yourself as you then also have influence to fix those issues yourself later. Experiences like this can scare you away from even the most reliable libraries in the future.
shavenwarthog2 13y ago
Seconded. I use itertools constantly for data-wrangling on network server and web applications. Specifically: - processing files with imap and ifilter to rapidly grab data, find a subset of it, then process it with a function - defaultdict(list) is incredibly useful for collecting data, arranging it by a certain key (like date or object id), then collecting into a list - namedtuple is occasionally useful for efficiently stuffing data into an object with a few named attributes.

by wting 13y ago

naiquevin 13y ago

Author here. I completely agree with you. Could have known all this before had I just read the itertools docs beyond permutations and combinations. It was only after using Scala's groupby while solving an assignment problem that I thought about finding out an equivalent in python. And thanks to having worked with the Stream class from Scala collections, for the first time the rest of the itertools also made sense. Better late than never I guess :-)

Aqueous 13y ago
Is there any reason that you don't switch over to Scala for your main programming language? I'm in the process of converting a middle-sized PHP codebase completely over to Scala. It is not uncommon for me to see functions that run between 15 and 30 lines shrink to 3 or 4 line functions thanks to the Collections API. On top of that its performance is something you'll never see in PHP or Python. I love Python for its readability and wonderful syntax, but Scala is starting to have an even greater pull on me for its built in concision and the ability to use the entire Java ecosystem without submitting to java's imperative style.
- naiquevin 13y ago
  It's very tempting. Frankly, I had enrolled for the course mainly for functional programming and not Scala but towards the end I started liking it. But, at the moment my main languages at work are Python, Erlang and Javascript and I still have a long way to go with these so planning to stick to them for a while.

pjmlp 13y ago

>My intention is not to be snarky, but people post all the time about discovering the itertools or collections library. I notice it's a common gap in newer Python programmers. Not only in Python, but programming languages in general. I still find people writing Java or .NET code that aren't aware of all nice classes that are part of the runtime and end up creating their half baked solutions for their problems. Nowadays developers seem to code without reading.

Silhouette 13y ago
Nowadays developers seem to code without reading. When your standard library documentation is so vast that it would take weeks to read and understand it all, and you'd never remember most of it anyway without context and experience using it, I don't think "coding without reading" is really a fair complaint. We as an industry need to get better at documentation, and in particular about separating tutorial/overview documentation that presents a map and summary of what's available from reference documentation, or we're going to keep reinventing wheels like this. Python is a particularly unfortunate example, because while its documentation is vast, it has very little tutorial/overview material beyond the very basics. For example, given that a substantial proportion of Python's standard library actually doesn't work very well in practice, it would be helpful to have a deeper tutorial/map document somewhere that introduced the various areas of the standard library and that also promoted the good ones and suggested popular alternatives for the not so good ones where they exist.
- Too 13y ago
  Documentation discoverability is one problem. Willingness to learn and trust is another one. People simply want to use things that they themselves have proven to work before. As an example an old colleague wanted to dump some data from python to a csv-file and did this by for-looping through each row and each item and concatenating each cell and a semicolon to a string. Even after pointing out to him that python already has a built in csv writer, that handles all issues of escaping etc, he didn't want to use it because he didn't know what it did and didnt want to learn anything new. His version didn't even do escaping inside the for-loop and he didn't see the issue of not doing it. To him the for-loop gave exactly the same result and didn't require any learning and was thus better, and why change something that works... My last suggestion was to at least use ";".join(...) but it was also a bit too magic so he stuck to his well known for-loop. Usually standard libraries are quite reliable but in some cases, and especially if adding third party libraries, bugs and performance issues inside the library can really give you hell. If the library is supposed to just perform a simple task maybe you would rather implement it yourself as you then also have influence to fix those issues yourself later. Experiences like this can scare you away from even the most reliable libraries in the future.

shavenwarthog2 13y ago

Seconded. I use itertools constantly for data-wrangling on network server and web applications. Specifically: - processing files with imap and ifilter to rapidly grab data, find a subset of it, then process it with a function - defaultdict(list) is incredibly useful for collecting data, arranging it by a certain key (like date or object id), then collecting into a list - namedtuple is occasionally useful for efficiently stuffing data into an object with a few named attributes.