3 ms·
Thought: what if Unicode decoding was "lossless" in the face of errors, such that the replacement characters represented the bitstring of non-decodable bytes? (
by derefr 9y ago
Thought: what if Unicode decoding was "lossless" in the face of errors, such that the replacement characters represented the bitstring of non-decodable bytes? (E.g. 256 reserved-codepoints for each possible octet value, that render as e.g. "[FF]" in a box; and then another 255 for the set of 7-bit, 6-bit, 5-bit, etc. overhangs.)
- kazinator 9y agobeen there, done that; it is very useful. 1> [(file-get-string "/bin/ls") 0..15] "\x7F;ELF\x02\x01\x01\xDC00\xDC00\xDC00\xDC00\xDC00\xDC00\xDC00\xDC00" 2> (file-put-string "foo" (file-get-string "/bin/ls")) t 3> (sh "cmp /bin/ls foo") 0 4> (sh "sha256sum /bin/ls foo") a90ba058c747458330ba26b5e2a744f4fc57f92f9d0c9112b1cb2f76c66c4ba0 /bin/ls a90ba058c747458330ba26b5e2a744f4fc57f92f9d0c9112b1cb2f76c66c4ba0 foo 0 There is no need to handle any fractional bytes if the original input is a sequence of bytes; so many whole bytes have to be recovered, not so many whole bytes plus three bits or whatever.
- i336_ 9y agoI was extremely curious what REPL this is, and it appears to be TXR: http://www.nongnu.org/txr/ http://www.nongnu.org/txr/ (My conclusion was based on finding file-get-string and file-put-string in https://fossies.org/linux/misc/txr-176.tar.gz/txr-176/share/txr/stdlib/getput.tl https://fossies.org/linux/misc/txr-176.tar.gz/txr-176/share/...)
- 1_player 9y agoI might be sleep deprived... but what exactly is this script doing? You're making a copy of /bin/ls into foo, and sha256sum the copy and the original. $ head -c 15 /bin/ls $ cat > foo < /bin/ls $ cmp /bin/ls foo $ sha256sum /bin/ls foo I don't get it.
- kazinator 9y ago> You're making a copy of /bin/ls into foo By getting its contents as a character string formed by passing the binary through a UTF-8 decoder, and writing out that string via the UTF-8 encoder.