3 ms·

Well, yours does iterate over the dictionary twice and generates a temporary list in the meantime. If you have a huge dictionary, this would explode. The equiva

Well, yours does iterate over the dictionary twice and generates a temporary list in the meantime. If you have a huge dictionary, this would explode. The equivalent Java code wouldn't do that (due to lack of list comprehensions).

sur 15y ago
He is using parentheses instead of square brackets, so the code is actually using a generator expression which will produce each value lazily.
- eru 15y ago
  Is mydict.items() also lazy?
  - kragen 15y ago
    In Python 3 yes, in Python 2 no. In Python 2 you'd say mydict.iteritems() if you were concerned about it.
kragen 15y ago
At first glance, your statement is incorrect: there is no Python code to create a list here, and no list comprehensions. On further examination, though, string_join in stringobject.c invokes PySequence_Fast to convert its iterable argument into a sequence --- a temporary list, in this case --- so your statement that this code generates a temporary list is correct, although I suspect that this is more by accident than because you have a deep knowledge of the implementation of the Python standard library. The temporary list of temporary strings probably will use less memory than the original dict did, though, so it's only a very mild sort of "explosion". It will, however, be several times bigger than the final output string, which also has to be huge if the dictionary is huge. Python doesn't have anything like a StringBuffer. If it did, it would be reasonable for string_join to use it instead of generating a temporary list. The Python code above would look the same. But hey, if you just want to print the values, you could say sys.stdout.printlines("%s: %s\n" % (k, v) for k, v in mydict.items()) but frankly I think I would write instead for k, v in mydict.items(): print "%r: %r" % (k, v)
- nyellin 15y ago
  Why doesn't StringIO count as a string buffer?
  - kragen 15y ago
    Sir or madam, you may have an excellent point, there. But I haven't looked at how cStringIO is implemented: does it construct a list of strings and then join them when they are requested, thus keeping alive all the string objects passed to it? Or does it concatenate the bytes into a buffer? I just realized that the Python 3 memoryview object also provides the wherewithal to construct a string buffer, even in pure Python.
StavrosK 15y ago
Aside from the fact that I'm using a generator, where does it iterate twice? I don't see it.

sur 15y ago

He is using parentheses instead of square brackets, so the code is actually using a generator expression which will produce each value lazily.

eru 15y ago
Is mydict.items() also lazy?
- kragen 15y ago
  In Python 3 yes, in Python 2 no. In Python 2 you'd say mydict.iteritems() if you were concerned about it.

kragen 15y ago

At first glance, your statement is incorrect: there is no Python code to create a list here, and no list comprehensions. On further examination, though, string_join in stringobject.c invokes PySequence_Fast to convert its iterable argument into a sequence --- a temporary list, in this case --- so your statement that this code generates a temporary list is correct, although I suspect that this is more by accident than because you have a deep knowledge of the implementation of the Python standard library. The temporary list of temporary strings probably will use less memory than the original dict did, though, so it's only a very mild sort of "explosion". It will, however, be several times bigger than the final output string, which also has to be huge if the dictionary is huge. Python doesn't have anything like a StringBuffer. If it did, it would be reasonable for string_join to use it instead of generating a temporary list. The Python code above would look the same. But hey, if you just want to print the values, you could say sys.stdout.printlines("%s: %s\n" % (k, v) for k, v in mydict.items()) but frankly I think I would write instead for k, v in mydict.items(): print "%r: %r" % (k, v)

nyellin 15y ago
Why doesn't StringIO count as a string buffer?
- kragen 15y ago
  Sir or madam, you may have an excellent point, there. But I haven't looked at how cStringIO is implemented: does it construct a list of strings and then join them when they are requested, thus keeping alive all the string objects passed to it? Or does it concatenate the bytes into a buffer? I just realized that the Python 3 memoryview object also provides the wherewithal to construct a string buffer, even in pure Python.

StavrosK 15y ago

Aside from the fact that I'm using a generator, where does it iterate twice? I don't see it.