3 ms·
> Even regular expressions worked. Did you try doing regular expressions for non-ASCII characters? For example, given a file "latin1file" containing the string
by nemetroid 7y ago
> Even regular expressions worked.
Did you try doing regular expressions for non-ASCII characters? For example, given a file "latin1file" containing the string "L£", encoded in Latin-1, Python 2 will fail silently:
$ python2 -c 'import re; s = open("latin1file").read(); print(re.search("L", s));'
<_sre.SRE_Match object at 0x7fb992a44bf8>
$ python2 -c 'import re; s = open("latin1file").read(); print(re.search("£", s));'
None
Python3 forces you to deal with the problem to handle the first case, but if you do, the second case will work as well:
$ python3 -c 'import re; s = open("latin1file").read(); print(re.search("L", s));'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3.7/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa3 in position 1: invalid start byte
$ python3 -c 'import re; s = open("latin1file", encoding="latin1").read(); print(re.search("L", s));'
<re.Match object; span=(0, 1), match='L'>
$ python3 -c 'import re; s = open("latin1file", encoding="latin1").read(); print(re.search("£", s));'
<re.Match object; span=(1, 2), match='£'>
- metalliqaz 7y agoI've never had to do that. Maybe because I live in the U.S.