4 ms·

Doesn’t this have an impedance mismatch? Stdin/out are stream based. Websocket is message based. There is no guarantee you can transmit the content of a single

Doesn’t this have an impedance mismatch? Stdin/out are stream based. Websocket is message based. There is no guarantee you can transmit the content of a single WebSocket message inside a single os read or write call. Unless you expect that on both sides messages might be fragmented across multiple calls and callbacks. But I don’t see the docs mentioning that.

xyzzy123 8y ago
Interested to know in what situations you anticipate this would cause problems?
- verroq 8y ago
  In every situation? The API interface doesn’t match. Websocket isn’t a stream, it is framed. Obviously it can be used like a stream but that depends on the client and server. The issue here is that all the client side (browser) APIs expose frames instead of streams. Suppose I send a JSON object I have no idea what framing this socket server will use and if it’ll break it up into multiple pieces and require the client to reassemble it. If I send two JSON objects in a row does it arrive in two websocket frames or could it buffered into one? Secondly the websocket protocol is much more complex than a tcp stream. It has keep alive packets that can be sent, it has close packets that allow a connection to terminate with a given code that can be handled by the client. How are these exposed?
- Matthias247 8y ago
  I estimated that the daemon just tried to send the data from the local application as a websocket messages whenever it received something through the FD. If the application tries to write a 100kB message to stdout, the daemon might pull this out in smaller chunks of it's socket, since the OS buffer might be not as big. If it's 50kB, then the daemon would send 2 50kB messages instead of a single 100kB one. If the JS on the other side would have estimated that everything is inside a single message things would be broken. However it looks like the Daemon might wait for a newline until it forwards everything as a message. Which at least fixes this problem, but might have other side effects.
  - masklinn 8y ago
    > However it looks like the Daemon might wait for a newline until it forwards everything as a message. Which at least fixes this problem, but might have other side effects. That you can't put newlines in messages — whether input or output, you've got to modify the protocol to escape them somehow due to intermediate implementation details.
jampekka 8y ago
It's line based. A newline character separates messages. https://github.com/joewalnes/websocketd/wiki/Ten-minute-tutorial https://github.com/joewalnes/websocketd/wiki/Ten-minute-tuto...
mdavidn 8y ago
Newlines. Each line written to STDOUT is sent as one frame, and each frame received is read from STDIN followed by a synthetic newline. The FAQ explains how to escape multiline messages (or binary data, presumably). https://github.com/joewalnes/websocketd/wiki/FAQ#how-can-i-send-multiple-line-messages https://github.com/joewalnes/websocketd/wiki/FAQ#how-can-i-s...
- Matthias247 8y ago
  Ah thanks that makes sense. I didn't see it when I glimpsed over the site. That should provides reliable framing on both sides. But as shown in the linked it page it also comes with the downside of not being able to send raw websocket messages which contain a new line - so it's not possible to port existing applications from other websocket servers to this one without having to change communication.
  - function_seven 8y ago
    I don't understand why the daemon needs to client to escape newlines in the message. Can't it handle that before feeding the message to STDIN? For example, my browser sends: This is line one... ...and this is line two as a string ("This is line one...\n...and this is line two") and the websocketd receives that, and passes along This is line one...\\n...andthis is line two\n to the STDIN. It knows that the message is one thing, so the newlines aren't the same as the end of message, right? It'd also be nice to use EOF as the indicator to flush STDOUT to the client, so the program can also emit newlines without needing to escape them first. All that being said, this is nifty. I think I can use this for some low-volume ideas I have on an internal app. The newline handling isn't a problem because I'm not replacing anything that already exists.
    - masklinn 8y ago
      > It'd also be nice to use EOF as the indicator to flush STDOUT to the client, so the program can also emit newlines without needing to escape them first. EOF is not an actual control code though (and is implementation dependent). However there are control codes for message framing e.g. ETB, ETX, EOT (often used as terminal EOF), FF (form feed / page break), RS/GS/FS
mpweiher 8y ago
> Stdin/out are stream based. Websocket is message based. WebSocket is implemented on top of TCP/IP, which is stream based.

xyzzy123 8y ago

Interested to know in what situations you anticipate this would cause problems?

verroq 8y ago
In every situation? The API interface doesn’t match. Websocket isn’t a stream, it is framed. Obviously it can be used like a stream but that depends on the client and server. The issue here is that all the client side (browser) APIs expose frames instead of streams. Suppose I send a JSON object I have no idea what framing this socket server will use and if it’ll break it up into multiple pieces and require the client to reassemble it. If I send two JSON objects in a row does it arrive in two websocket frames or could it buffered into one? Secondly the websocket protocol is much more complex than a tcp stream. It has keep alive packets that can be sent, it has close packets that allow a connection to terminate with a given code that can be handled by the client. How are these exposed?
Matthias247 8y ago
I estimated that the daemon just tried to send the data from the local application as a websocket messages whenever it received something through the FD. If the application tries to write a 100kB message to stdout, the daemon might pull this out in smaller chunks of it's socket, since the OS buffer might be not as big. If it's 50kB, then the daemon would send 2 50kB messages instead of a single 100kB one. If the JS on the other side would have estimated that everything is inside a single message things would be broken. However it looks like the Daemon might wait for a newline until it forwards everything as a message. Which at least fixes this problem, but might have other side effects.
- masklinn 8y ago
  > However it looks like the Daemon might wait for a newline until it forwards everything as a message. Which at least fixes this problem, but might have other side effects. That you can't put newlines in messages — whether input or output, you've got to modify the protocol to escape them somehow due to intermediate implementation details.

jampekka 8y ago

It's line based. A newline character separates messages. https://github.com/joewalnes/websocketd/wiki/Ten-minute-tutorial https://github.com/joewalnes/websocketd/wiki/Ten-minute-tuto...

mdavidn 8y ago

Newlines. Each line written to STDOUT is sent as one frame, and each frame received is read from STDIN followed by a synthetic newline. The FAQ explains how to escape multiline messages (or binary data, presumably). https://github.com/joewalnes/websocketd/wiki/FAQ#how-can-i-send-multiple-line-messages https://github.com/joewalnes/websocketd/wiki/FAQ#how-can-i-s...

Matthias247 8y ago
Ah thanks that makes sense. I didn't see it when I glimpsed over the site. That should provides reliable framing on both sides. But as shown in the linked it page it also comes with the downside of not being able to send raw websocket messages which contain a new line - so it's not possible to port existing applications from other websocket servers to this one without having to change communication.
- function_seven 8y ago
  I don't understand why the daemon needs to client to escape newlines in the message. Can't it handle that before feeding the message to STDIN? For example, my browser sends: This is line one... ...and this is line two as a string ("This is line one...\n...and this is line two") and the websocketd receives that, and passes along This is line one...\\n...andthis is line two\n to the STDIN. It knows that the message is one thing, so the newlines aren't the same as the end of message, right? It'd also be nice to use EOF as the indicator to flush STDOUT to the client, so the program can also emit newlines without needing to escape them first. All that being said, this is nifty. I think I can use this for some low-volume ideas I have on an internal app. The newline handling isn't a problem because I'm not replacing anything that already exists.
  - masklinn 8y ago
    > It'd also be nice to use EOF as the indicator to flush STDOUT to the client, so the program can also emit newlines without needing to escape them first. EOF is not an actual control code though (and is implementation dependent). However there are control codes for message framing e.g. ETB, ETX, EOT (often used as terminal EOF), FF (form feed / page break), RS/GS/FS

mpweiher 8y ago

> Stdin/out are stream based. Websocket is message based. WebSocket is implemented on top of TCP/IP, which is stream based.