I'm not sure about the internals of set(f), but the logically equivalent s = set() for x in f: s.add(x) Does not read everything into memory - by usin

I'm not sure about the internals of set(f), but the logically equivalent s = set() for x in f: s.add(x) Does not read everything into memory - by using iterators, it only reads one line at a time.