3 ms·
Yeah, I’ve always felt that the stream API is a leaky abstraction for providing access to networking. I understand the attraction of making network I/O look li
by derriz 10mo ago
Yeah, I’ve always felt that the stream API is a leaky abstraction for providing access to networking. I understand the attraction of making network I/O look like local file access given the philosophy of UNIX.
The API should have been message oriented from the start. This would avoid having the network stack try to compensate for the behavior of the application layer. Then Nagel’s or something like it would just be a library available for applications that might need it.
The stream API is as annoying on the receiving end especially when wrapping (like TLS) is involved. Basically you have to code your layers as if the underlying network is handing you a byte at a time - and the application has to try to figure out where the message boundaries are - adding a great deal of complexity.
- calineczka 10mo ago> message oriented Very well said. I think there is enormous complexity in many layers because we don't have that building block easily available.
- mort96 10mo agoIt's the main reason why I use websockets for a whole lot of things. I don't wanna build my own message chunking layer on top of TCP every time.
- mgaunard 10mo agoWebSocket is full of web-tech silliness; you'd be better off doing your own framing.
- mort96 10mo agoWell, it also has the advantage of providing pretty decent encryption for free through WSS. But yeah, where that's unnecessary, it's probably just as easy to have a 4-byte length prefix, since TCP handles the checksum and retransmit and everything for you.
- mgaunard 10mo agoIt's just a standard TLS layer, works with any TCP protocol, nothing WebSocket-specific in it. You should ideally design your messages to fit within a single Ethernet packet, so 2 bytes is more than enough for the size. Though I have sadly seen an increasing amount of developers send arbitrarily large network messages and not care about proper design.
- mort96 10mo agoMeh I've worked enough with OpenSSL's API to know that I never ever want to implement SSL over TCP myself. Better let the WebSocket library take care of it.
- mgaunard 10mo agothe whole point of TCP is that it is a stream of bytes, not of messages. The problem is that this is not in practice quite what most applications need, but the Internet evolved towards UDP and TCP only. So you can have message-based if you want, but then you have to do sequencing, gap filling or flow control yourself, or you can have the overkill reliable byte stream with limited control or visibility at the application level.
- derriz 10mo agoFor me, the “whole point” of TCP is to add various delivery guarantees on top of IP. It does not mandate or require a particular API. Of course, you can provide a stream API over TCP which suits many applications but it does not suit all and by forcing this abstraction over TCP you end up making message oriented applications (e.g request /response type protocols) more complex to implement than if you had simply exposed the message oriented reality of TCP via an API.
- mgaunard 10mo agoTCP is not message-oriented. Retransmitted bytes can be at arbitrary offsets and do not need to align with the way the original transmission was fragmented or even an earlier retransmission.
- derriz 10mo agoI don’t understand your point here or maybe our understanding of the admittedly vague term “message oriented” differs. I’m not suggesting exposing retransmission, fragmentation, etc to the API user. The sender provides n bytes of data (a message) to the network stack. The receiver API provides the user with the block of n bytes (the message) as part of an atomic operation. Optionally the sender can be provided with notification when the n-bytes have been delivered to the receiver.
- jasomill 10mo agoIs this a TCP API proposal or a protocol proposal? Because TCP, by design, is a stream-oriented protocol, and the only out-of-band signal I'm aware of that's intended to be exposed to applications is the urgent flag/pointer, but a quick Google search suggests that many firewalls clear these by default, so compatibility would almost certainly be an issue if your API tried to use the urgent pointer as a message separator. I suppose you could implement a sort of "raw TCP" API to allow application control of segment boundaries, and force retransmission to respect them, but this would implicitly expose applications to fragmentation issues that would require additional API complexity to address.