4 ms·
Streaming UTF-8 (with node.js)
- axod 16y agoslightly related - quick way to get the number of bytes in a string in javascript when the string is encoded in utf8: var utf8_length = encodeURIComponent(data).replace(/%../g, 'x').length;
- felixge 16y agoDon't use this, it's broken. Any % sign in your text will screw things. 3-4 byte characters are not handled by it at all.
- axod 16y agoCare to elaborate? % signs are handled fine - they get encoded into %25 which then gets shortened to a single 'x' character - thus counted as a single byte. Similarly, for example € gets encoded to "%E2%82%AC" which then gets correctly counted as 3 bytes. I'm not quite sure what value of 'broken' you're thinking of? Did you try this and find it didn't work in some particular circumstance?
- felixge 16y agoI didn't know % gets encoded. So you're right, your code does work : )
- axod 16y agoFWIW, Checking code or reading API docs is pretty useful :P firebug works pretty well for testing out code snippets. The internet is full of enough incorrect information regarding javascript as it is, and if you think about it, % has to be encoded, otherwise anything containing a % would be decoded wrongly ;)
- BrianHammond 16y agoFor those using Node.js, the same is var len = Buffer.byteLength(a_string, 'utf8')
- deleted 16y ago[deleted]
- Sephr 16y agoThis script is unnecessary, as JavaScript already has functions which can facillitate encoding and decoding of UTF-8. Encoding UTF-8 to a bytestring is unescape(encodeURIComponent(string)) and decoding UTF-8 is decodeURIComponent(escape(string)).
- div 16y agoBut this script handles streaming utf-8 encoded strings correctly.
- codehero 16y agocode does not work, does not check for invalid UTF-8 characters
- felixge 16y agoThat is done in Buffer.toString(), this stream filter just makes sure that no string conversion is attempted on a multibyte character that is still incomplete (valid or not).
- oscarduignan 16y agoWhat an awesome RSS feed icon they have, if you haven't already, try hovering over it a few times (and then a few more.)
- riobard 16y agoWhat's the YouTube video about? It says the video is not available.
- oscarduignan 16y agoI didn't stick around after reading the title when the youtube page loaded, however I'm pretty sure it was Rick Astley singing Never Gonna Give You Up.