4 ms·

The arguments in the text: - Valid Unicode strings may contain null-bytes characters. - valid json files may carry null-bytes as part of important and meaning

The arguments in the text: - Valid Unicode strings may contain null-bytes characters. - valid json files may carry null-bytes as part of important and meaningful data

yunruse 6y ago
As weird and inefficient as it may seem at first glance, JSON in a database can have a variety of uses, so I certainly see the benefit. But… is there any use for a null byte explicitly in a JSON string? The de facto standard (easy to use and instantly recognisable) for blobs seems to be base64. I can’t think of any meaningful data benefits other than data efficiency (which is easily mitigated by storing the blob directly in the database).
- corty 6y ago
  JSON strings must conform to UTF8. Storing plain non-base64 blobs in them is abuse of broken JSON parsers.
zaphar 6y ago
That second one is probably the more important one. A null byte in plain text is probably an error. A null byte in a json string may be intentional if perhaps questionable. JSON strings are frequently abused for various transport formats.
- eptcyka 6y ago
  If I was responsible for ingesting and returning JSON strings from a server, my attidue towards people who require nulls in them would be rather juvenile and brash - "lmao fuck em".
lolc 6y ago
Another reason mentioned is consistency with other RDBMS.
corty 6y ago
json carrying important and meaningful data in strings (i.e. blobs in strings) is usually just an exploit on the non-conformance of parsers. JSON strings are defined by ECMA-404 to be UTF8 codepoints. Arbitrary binary data isn't a sequence of UTF8 codepoints. However, that's what it usually is used for, incorrectly. If you use JSON correctly, a JSON string is really just an UTF8 string. Leaving out the null bytes there would be annoying, yes, but usually doesn't hurt the use as a string...
- jfk13 6y ago
  A UTF-8 string can perfectly well contain the character U+0000, which will be encoded as a null byte 0x00. So just "leaving out the null bytes" amounts to changing the string to a different string. Sometimes that may be what you want, but it doesn't sound like a good idea in general.
  - tremon 6y ago
    That was a bit of a rabbit hole. As it turns out, neither the ASCII nul character nor its Unicode equivalent have the semantic meaning "end of stream". ASCII does have control characters for "end of stream", in fact multiple: 0x03 End Of Text, 0x04 End Of Transmission, and 0x17 End Transmission Block. But 0x00 was never intended to be an end marker of any kind. It's really only the C standard that (ab)uses NUL as an end-of-string marker. So yes, after consideration, I'm starting to agree with you. U+0000 is a valid character and should be allowed in text fields.

The arguments in the text: - Valid Unicode strings may contain null-bytes characters. - valid json files may carry null-bytes as part of important and meaningful data

yunruse 6y ago

As weird and inefficient as it may seem at first glance, JSON in a database can have a variety of uses, so I certainly see the benefit. But… is there any use for a null byte explicitly in a JSON string? The de facto standard (easy to use and instantly recognisable) for blobs seems to be base64. I can’t think of any meaningful data benefits other than data efficiency (which is easily mitigated by storing the blob directly in the database).

corty 6y ago
JSON strings must conform to UTF8. Storing plain non-base64 blobs in them is abuse of broken JSON parsers.

zaphar 6y ago

That second one is probably the more important one. A null byte in plain text is probably an error. A null byte in a json string may be intentional if perhaps questionable. JSON strings are frequently abused for various transport formats.

eptcyka 6y ago
If I was responsible for ingesting and returning JSON strings from a server, my attidue towards people who require nulls in them would be rather juvenile and brash - "lmao fuck em".

lolc 6y ago

Another reason mentioned is consistency with other RDBMS.

corty 6y ago

json carrying important and meaningful data in strings (i.e. blobs in strings) is usually just an exploit on the non-conformance of parsers. JSON strings are defined by ECMA-404 to be UTF8 codepoints. Arbitrary binary data isn't a sequence of UTF8 codepoints. However, that's what it usually is used for, incorrectly. If you use JSON correctly, a JSON string is really just an UTF8 string. Leaving out the null bytes there would be annoying, yes, but usually doesn't hurt the use as a string...

jfk13 6y ago
A UTF-8 string can perfectly well contain the character U+0000, which will be encoded as a null byte 0x00. So just "leaving out the null bytes" amounts to changing the string to a different string. Sometimes that may be what you want, but it doesn't sound like a good idea in general.
- tremon 6y ago
  That was a bit of a rabbit hole. As it turns out, neither the ASCII nul character nor its Unicode equivalent have the semantic meaning "end of stream". ASCII does have control characters for "end of stream", in fact multiple: 0x03 End Of Text, 0x04 End Of Transmission, and 0x17 End Transmission Block. But 0x00 was never intended to be an end marker of any kind. It's really only the C standard that (ab)uses NUL as an end-of-string marker. So yes, after consideration, I'm starting to agree with you. U+0000 is a valid character and should be allowed in text fields.