6 ms·

Sometimes I wish that python strings weren't directly iterable... this article sums it up better than I ever could https://www.xanthir.com/b4wJ1 https://www.xa

by tln 6y ago

Sometimes I wish that python strings weren't directly iterable... this article sums it up better than I ever could https://www.xanthir.com/b4wJ1 https://www.xanthir.com/b4wJ1 ...then str.strip and variants could be cleanly and logically extended to allow this functionality, because passing a string and a sequence of strings would be distinguishable. Alas, clean and logical function design can be hard to do late in a languages life. PEP 593 and PEP 585 are clean and logical... glad to see that :)

joshuamorton 6y ago
On non-iterable strings: The recursive type problem can be solved, with something like what is proposed in [0]. (I have an implementation of a fix on github linked from that thread, there's edge case fixes and PEP's scare me, but technically it's feasible). [0]: https://mail.python.org/archives/list/typing-sig@python.org/thread/OLCQHSNCLM2TOHRQWB3WPSI66AA5A3G7/ https://mail.python.org/archives/list/typing-sig@python.org/...
- spott 6y ago
  While that may solve the recursive type problem, it doesn't really solve the "iterating over strings is rarely what you actually want to do" problem.
BiteCode_dev 6y ago
You can easily distinguish them: if isinstance(msg, str) So I don't think that's a good argument for not accepting iterables of strings in str methods. Things like replace() would benefit a lot and it's not that hard to do, you can even accept regexes optionally: https://wonderful-wrappers.readthedocs.io/en/latest/string_wrapper.html https://wonderful-wrappers.readthedocs.io/en/latest/string_w... I agree that iterating on string is not proper design however. It's not very useful in practice, and the O(1) access has other performance consequences for more important things. Swift did it right IMO, but it's a much younger language. I also wish we stole the file api concepts from swift, and that open() would return a file like object that always gives you bytes. No "b" mode. If you want text, you to open().as_text(), and get a decoding wrapper. The idea that there are text files and binary files has been toxic for a whole generation of coders.
- Skunkleton 6y ago
  > The idea that there are text files and binary files has been toxic for a whole generation of coders. It really is nonsense isn't it? Its like asking a low level api for opening files as a .doc, or as a pdf. Why would that be part of the file io layer?
  - aidos 6y ago
    Well, I guess it’s easy to argue that it’s so common that beginners would expect to open files as text. You can see how it would evolve that way. Now I’m more familiar with it I’m careful to be explicit with the decoding when using text to make it super obvious what’s going on.
    - heavenlyblue 6y ago
      My first mistake I made as a beginner was dumping a bunch of binary data as text. Something would happen in the way and not the whole data would be written because I was writing it in text mode. It just never appeared to me that the default mode of writing the file would _not_ write the array I was passing it. It’s much more important for beginners to be able to learn clear recipes rather than having double standards with a bunch of edge cases.
      - aidos 6y ago
        I’ve done worse. Using MySQL from php and not having the encoding right somewhere along the way so all my content was being mojibaked on the way in and un-mojibaked on the way out so I didn’t notice it until deep into a project when I needed to extract it to another system. EDIT thanks, I knew that didn't look quite right. "Mojibaked" - such a great term.
        mark-r 6y ago
        The term is actually "Mojibake", not "emoji baked". https://en.wikipedia.org/wiki/Mojibake#Etymology https://en.wikipedia.org/wiki/Mojibake#Etymology
danpalmer 6y ago
Strings being it stable can cause problems, and another commenter has pointed out that Swift handles it well. However I think strings being iterable is one of the core ergonomics in the language and basic types of Python that make it so nice for many applications. Scripting, scraping, data cleanup, data science, even basic web development, all benefit hugely from little features like this. Without this sort of thing Python would be a different language with different uses. While I normally like safety and types, I’m personally happy with things like this because it fits with Pythons strengths.
- cauthon 6y ago
  I disagree, I don’t think there’s any meaningful benefit. For example, lets say we iterated over strings as follows. for char in my_str.chars(): foo() That wouldn’t sacrifice any ergonomics, being consistent with how we already iterate over dictionary contents with d.items(), and it’d address all the concerns in the parent comment link
jorams 6y ago
I agree with most of the article you link, but there's one thing I don't understand: The article quickly dismisses the obvious fix for recursive iterability, to make strings be composed of "characters": > And an obvious "fix" for this is worse than the original problem: Common Lisp says that strings are composed of characters, a totally different type, which doesn't implement the same methods and has to be handled specially. It's really annoying. It seems to me this contradicts most of what the article says. Sure, strings are rarely collections, so they should not be iterable by default. But the final solution offered admits that sometimes they are, and then you want to be able to iterate over something. For most instances of something, It does not make sense for the individual "elements" to be strings. Bytes are clearly not strings, code points are clearly not strings, grapheme clusters are clearly not strings. Each of those will provide very different methods, because they are very different things. Only after that point (words, sentences, etc.) does the idea of the element being the same type start making sense again. Clearly the concept of a "character" is too ambiguous, and there is no clear "default" for what it should mean, but the idea of a string consisting of some kind of element that is not string appears obviously correct.
- a1369209993 6y ago
  > for the individual "elements" to be strings. The basic idea is to have sum(string.foobars()) == string. Bytes and characters ('grapheme clusters') are then a specific subset of strings, that can therefore support additional operations like byte.ord(), the same way eg positive numbers support num.sqrt().
emmelaich 6y ago
Didn't Haskell make this move? But I can't find it now so might be misremembering. In Haskell, String is an alias for [Char]. Maybe it's not changing String rather, encouraging other types which don't have this difficulty.
jnxx 6y ago
why not: assert(not isinstance(x, str))

by tln 6y ago

joshuamorton 6y ago

On non-iterable strings: The recursive type problem can be solved, with something like what is proposed in [0]. (I have an implementation of a fix on github linked from that thread, there's edge case fixes and PEP's scare me, but technically it's feasible). [0]: https://mail.python.org/archives/list/typing-sig@python.org/thread/OLCQHSNCLM2TOHRQWB3WPSI66AA5A3G7/ https://mail.python.org/archives/list/typing-sig@python.org/...

spott 6y ago
While that may solve the recursive type problem, it doesn't really solve the "iterating over strings is rarely what you actually want to do" problem.

BiteCode_dev 6y ago

You can easily distinguish them: if isinstance(msg, str) So I don't think that's a good argument for not accepting iterables of strings in str methods. Things like replace() would benefit a lot and it's not that hard to do, you can even accept regexes optionally: https://wonderful-wrappers.readthedocs.io/en/latest/string_wrapper.html https://wonderful-wrappers.readthedocs.io/en/latest/string_w... I agree that iterating on string is not proper design however. It's not very useful in practice, and the O(1) access has other performance consequences for more important things. Swift did it right IMO, but it's a much younger language. I also wish we stole the file api concepts from swift, and that open() would return a file like object that always gives you bytes. No "b" mode. If you want text, you to open().as_text(), and get a decoding wrapper. The idea that there are text files and binary files has been toxic for a whole generation of coders.

Skunkleton 6y ago
> The idea that there are text files and binary files has been toxic for a whole generation of coders. It really is nonsense isn't it? Its like asking a low level api for opening files as a .doc, or as a pdf. Why would that be part of the file io layer?
- aidos 6y ago
  Well, I guess it’s easy to argue that it’s so common that beginners would expect to open files as text. You can see how it would evolve that way. Now I’m more familiar with it I’m careful to be explicit with the decoding when using text to make it super obvious what’s going on.
  - heavenlyblue 6y ago
    My first mistake I made as a beginner was dumping a bunch of binary data as text. Something would happen in the way and not the whole data would be written because I was writing it in text mode. It just never appeared to me that the default mode of writing the file would _not_ write the array I was passing it. It’s much more important for beginners to be able to learn clear recipes rather than having double standards with a bunch of edge cases.
    - aidos 6y ago
      I’ve done worse. Using MySQL from php and not having the encoding right somewhere along the way so all my content was being mojibaked on the way in and un-mojibaked on the way out so I didn’t notice it until deep into a project when I needed to extract it to another system. EDIT thanks, I knew that didn't look quite right. "Mojibaked" - such a great term.
      - mark-r 6y ago
        The term is actually "Mojibake", not "emoji baked". https://en.wikipedia.org/wiki/Mojibake#Etymology https://en.wikipedia.org/wiki/Mojibake#Etymology

danpalmer 6y ago

Strings being it stable can cause problems, and another commenter has pointed out that Swift handles it well. However I think strings being iterable is one of the core ergonomics in the language and basic types of Python that make it so nice for many applications. Scripting, scraping, data cleanup, data science, even basic web development, all benefit hugely from little features like this. Without this sort of thing Python would be a different language with different uses. While I normally like safety and types, I’m personally happy with things like this because it fits with Pythons strengths.

cauthon 6y ago
I disagree, I don’t think there’s any meaningful benefit. For example, lets say we iterated over strings as follows. for char in my_str.chars(): foo() That wouldn’t sacrifice any ergonomics, being consistent with how we already iterate over dictionary contents with d.items(), and it’d address all the concerns in the parent comment link

jorams 6y ago

I agree with most of the article you link, but there's one thing I don't understand: The article quickly dismisses the obvious fix for recursive iterability, to make strings be composed of "characters": > And an obvious "fix" for this is worse than the original problem: Common Lisp says that strings are composed of characters, a totally different type, which doesn't implement the same methods and has to be handled specially. It's really annoying. It seems to me this contradicts most of what the article says. Sure, strings are rarely collections, so they should not be iterable by default. But the final solution offered admits that sometimes they are, and then you want to be able to iterate over something. For most instances of something, It does not make sense for the individual "elements" to be strings. Bytes are clearly not strings, code points are clearly not strings, grapheme clusters are clearly not strings. Each of those will provide very different methods, because they are very different things. Only after that point (words, sentences, etc.) does the idea of the element being the same type start making sense again. Clearly the concept of a "character" is too ambiguous, and there is no clear "default" for what it should mean, but the idea of a string consisting of some kind of element that is not string appears obviously correct.

a1369209993 6y ago
> for the individual "elements" to be strings. The basic idea is to have sum(string.foobars()) == string. Bytes and characters ('grapheme clusters') are then a specific subset of strings, that can therefore support additional operations like byte.ord(), the same way eg positive numbers support num.sqrt().

emmelaich 6y ago

Didn't Haskell make this move? But I can't find it now so might be misremembering. In Haskell, String is an alias for [Char]. Maybe it's not changing String rather, encouraging other types which don't have this difficulty.

jnxx 6y ago

why not: assert(not isinstance(x, str))