3 ms·
Mark Pilgrim reversed (or ripped out, can't remember) the encoding detection that Firefox uses. It has done a fairly good job for my web crawling: http://pypi.
by ninjin 15y ago
Mark Pilgrim reversed (or ripped out, can't remember) the encoding detection that Firefox uses. It has done a fairly good job for my web crawling:
http://pypi.python.org/pypi/chardet http://pypi.python.org/pypi/chardet
- e98cuenc 15y agoIn my experience chardet misclassifies very often iso-8859-1 as iso-8859-2. I saw the misclassification even in small spanish pages, which were using only the typical spanish characters.