3 ms·

It might be too early for my brain, but splitting on "\n" won't work if any of your json objects have newlines in the strings. Does this regex work? As I see

It might be too early for my brain, but splitting on "\n" won't work if any of your json objects have newlines in the strings. Does this regex work? As I see it, the tricky one is matching a single string json object. /(\]|\})\n|(.")\n/ https://www.debuggex.com/r/LaqAP5wmPFuRLNBO https://www.debuggex.com/r/LaqAP5wmPFuRLNBO

simonw 3y ago
New line characters aren't allowed in JSON strings - JSON uses \n for those.
- jgalt212 3y ago
  What's the difference between a newline character in a string and \n from a regex perspective?
  - yogorenapan 3y ago
    It would be \\n
  - dwaite 3y ago
    JSON doesn't have native regex support, so you would be representing a regex as a string. /\n/ in regex would need the backslash escaped, so "/\\n/"
  - yesco 3y ago
    "\n" is really just two arbitrary characters right? An actual "newline" is just a single invisible character, but we represent it as "\n" for convenience. So in json you are simply putting in the magic character sequence: "\n" in place of the real newline character. Not unlike how many compilers will auto translate "\n" sequences you put in strings into the actual newline character. I believe most json encoders will automatically convert newline characters into the two character sequence, so when you put it all together it goes: character sequence "\n" typed into source file -> compiler converts newline character into actual an actual newline character -> json encoder converts newline character back into "\n" sequence. With all this in mind, since regex is basically doing the same thing as a compiler, treating "\n" sequences as a search for a newline character, you would need to escape the backslash so it is interpreted literally, so "\\n". This gets even weirder depending on the tool/language you are doing this in though, like if you are writing out a regex within a string, the regex engine might not need to have any special logic for "\n" since the compiler is already handling it! In which case: "\\n" is an escape for the compiler not the regex engine. TLDR: You just need to escape it: "\\n"
- jgalt212 3y ago
  yes, you are correct. s = "foo\nbar\nbaz" # 'foo\nbar\nbaz' len(s) # 11 j = json.dumps(s) # '"foo\\nbar\\nbaz"' # \n is escaped as \\n len(j) # 15
throwanem 3y ago
The JSON spec [1] explicitly forbids multiline strings. (Granted JSONL is already outside the spec, but the idea there being each single line contains valid JSON, encountering a multiline string in JSONL would if anything violate more expectations.) [1] https://ecma-international.org/wp-content/uploads/ECMA-404_2nd_edition_december_2017.pdf https://ecma-international.org/wp-content/uploads/ECMA-404_2...

simonw 3y ago

New line characters aren't allowed in JSON strings - JSON uses \n for those.

jgalt212 3y ago
What's the difference between a newline character in a string and \n from a regex perspective?
- yogorenapan 3y ago
  It would be \\n
- dwaite 3y ago
  JSON doesn't have native regex support, so you would be representing a regex as a string. /\n/ in regex would need the backslash escaped, so "/\\n/"
- yesco 3y ago
  "\n" is really just two arbitrary characters right? An actual "newline" is just a single invisible character, but we represent it as "\n" for convenience. So in json you are simply putting in the magic character sequence: "\n" in place of the real newline character. Not unlike how many compilers will auto translate "\n" sequences you put in strings into the actual newline character. I believe most json encoders will automatically convert newline characters into the two character sequence, so when you put it all together it goes: character sequence "\n" typed into source file -> compiler converts newline character into actual an actual newline character -> json encoder converts newline character back into "\n" sequence. With all this in mind, since regex is basically doing the same thing as a compiler, treating "\n" sequences as a search for a newline character, you would need to escape the backslash so it is interpreted literally, so "\\n". This gets even weirder depending on the tool/language you are doing this in though, like if you are writing out a regex within a string, the regex engine might not need to have any special logic for "\n" since the compiler is already handling it! In which case: "\\n" is an escape for the compiler not the regex engine. TLDR: You just need to escape it: "\\n"
jgalt212 3y ago
yes, you are correct. s = "foo\nbar\nbaz" # 'foo\nbar\nbaz' len(s) # 11 j = json.dumps(s) # '"foo\\nbar\\nbaz"' # \n is escaped as \\n len(j) # 15

throwanem 3y ago

The JSON spec [1] explicitly forbids multiline strings. (Granted JSONL is already outside the spec, but the idea there being each single line contains valid JSON, encountering a multiline string in JSONL would if anything violate more expectations.) [1] https://ecma-international.org/wp-content/uploads/ECMA-404_2nd_edition_december_2017.pdf https://ecma-international.org/wp-content/uploads/ECMA-404_2...