3 ms·
better to pickle a frozenset and use that. import pickle # initialize: wordset = frozenset(line.lower().rstrip() for line in open('/usr/share/dict/words
by nerfhammer 15y ago
better to pickle a frozenset and use that.
import pickle
# initialize:
wordset = frozenset(line.lower().rstrip() for line in open('/usr/share/dict/words'))
pickle.dump(wordset, open('/tmp/wordset.pkl', 'wb', -1))
# when you want to use it:
wordset = pickle.load(open('/tmp/wordset.pkl', 'rb'))
'bear' in wordset # == True
- iki23 15y agoThanks for the frozenset. Yet pickle makes the file 2 times bigger and 20 times slower to parse on my machine: >python wordlist.py -w wordlist -p wordlist.pkl >python wordlist.py -T wordlist wordlist.pkl 10 3 2.6.6 (r266:84297, Aug 24 2010, 18:46:32) [MSC v.1500 32 bit (Intel)] words(wordfile): 142.50 best msec/loop words_slower(wordfile): 161.00 best msec/loop words_normalized(wordfile): 295.84 best msec/loop words_pickled(pickled): 2943.66 best msec/loop The winner is: with open(wordfile) as wordlist: return frozenset(wordlist.read().split('\n')) Source: https://gist.github.com/1059725 https://gist.github.com/1059725 Wordlist: http://www.freebsd.org/cgi/cvsweb.cgi/src/share/dict/web2?rev=1.14;content-type=text/plain http://www.freebsd.org/cgi/cvsweb.cgi/src/share/dict/web2?re... Do you have different results?