2 ms·
Has anyone got any more up to date figures than these http://www.w3.org/QA/2008/05/utf8-web-growth.html http://www.w3.org/QA/2008/05/utf8-web-growth.html from 1
by justincormack 15y ago
Has anyone got any more up to date figures than these http://www.w3.org/QA/2008/05/utf8-web-growth.html http://www.w3.org/QA/2008/05/utf8-web-growth.html from 1998, when UTF-8 + ASCII was already 50% of the web? I think there is a lot less ISO 8859-1 than there was back then.
I also think that for Javascript files, almost all will be ASCII, with some UTF-8, and little else. JSON is of course only allowed to be Unicode, and I suspect that almost all of it is UTF8, although that will not generally hit the Javascript parser I presume (although maybe Gecko also normalises it?).
Locating non ASCII chars obviously is an issue, but no worse than the line feed issue mentioned in the article. Doubling the size of the input is generally a big performance hit that will be much more significant.
- bzbarsky 15y ago> Doubling the size of the input is generally a big > performance hit that will be much more significant. Apart from cache locality issues, it's not, really. And for a linear scan like this prefetching does OK. And yes, Gecko normalizes JSON input to the JS engine. I don't have numbers, sorry. But yes, UTF-8 would be the other thing to try normalizing to; it has its own benefits and drawbacks when the system is considered as a whole.