3 ms·
In principle, there's no way to fix the interleaving issue. But maybe you can minimise it by reading both from a single process, without monkey patching libc.
by jakobegger 8y ago
In principle, there's no way to fix the interleaving issue. But maybe you can minimise it by reading both from a single process, without monkey patching libc.
The first solution (with the JSON adaptor) is the best solution, the only problem is the implementation (because it doesn't make sure that everything that goes to stdout/stderr goes through the JSON adaptors)
You could fix this problem by using a little C program to do the JSON wrapping.
- The C program would start by setting up two pipes, let's call them pipe_stdout and pipe_stderr.
- Then the C program forks
- The child replaces file descriptors 1 & 2 with the write end of the pipes and closes the read ends.
- The parent process closes file descriptor 0, closes the write end of the pipes, and calls select() on the read end of the pipes
- The child process calls execve to start the interpreter. All output to stdout/stderr now goes into the pipes to the parent process.
- the parent process reads data from the pipes, wraps it in JSON, and sends it to stdout.
Is there a reason why you didn't go this route? This way you should have minimal interleaving, and there's no way the interpreter writes data directly to stdout breaking the JSON stream.
- basicer 8y agoWe tried something like this. Minimal was still confusing a non-trivial percentage of the time. Testing locally it was around 0.9%, but in production under docker where execution is constrained to a single core, it seemed to happen much more often. It is really confusing to the user to have the prompt in the middle of their output.
- jakobegger 8y agoInteresting. I guess the specific issue with the prompt could be fixed by always printing data from the stderr pipe first, but then you probably get incorrect ordering in other cases...