6 ms·
Let's Code a TCP/IP Stack: TCP Retransmission
- bogomipz 9y agoWhat a great series of posts! Thanks for sharing. Minor nitpick - I wish the author would link to the previous installments in the series at the bottom of the page :)
- saminiir 9y agoThanks! I'll add the links.
- eddyg 9y agoFor reference, here are links to the first four parts: http://www.saminiir.com/lets-code-tcp-ip-stack-1-ethernet-arp/ http://www.saminiir.com/lets-code-tcp-ip-stack-1-ethernet-ar... http://www.saminiir.com/lets-code-tcp-ip-stack-2-ipv4-icmpv4/ http://www.saminiir.com/lets-code-tcp-ip-stack-2-ipv4-icmpv4... http://www.saminiir.com/lets-code-tcp-ip-stack-3-tcp-handshake/ http://www.saminiir.com/lets-code-tcp-ip-stack-3-tcp-handsha... http://www.saminiir.com/lets-code-tcp-ip-stack-4-tcp-data-flow-socket-api/ http://www.saminiir.com/lets-code-tcp-ip-stack-4-tcp-data-fl...
- bdamm 9y agoThis is an outstanding way to step through understanding TCP. Thank you Sami!
- profpandit 9y agoYes, and in addition, this is a good way to choose projects. We need to "reinvent the wheel"" a lot more rather than just building on top of frameworks that we have inherited from the past. The more work we do at the lower layers of our stacks, the better the quality of our software will be, in terms of its usefulness. There's so many coders around now that there's not enough work at the top of our stacks anymore
- MichaelBurge 9y agoVery interesting. I suppose you could reimplement the Ethernet part of the layer on an FPGA?
- ambrop7 9y agoI don't really intend to promote this yet but anyway here's my in-progress TCP/IP stack implementation: https://github.com/ambrop72/aprinter/tree/master/aipstack https://github.com/ambrop72/aprinter/tree/master/aipstack It is written in C++14, is platform-independent and works within a single-threaded reactor environment. Microcontrollers are the primary target for example the stack does no dynamic memory allocation at all. I consider the TCP part to be "pretty complete" at this point, it even has PMTUD which lwIP lacks. The TCP code is also quite commented.
- Matthias247 9y agoC++14, single-threaded and no dynamic allocations sound like good key parameters! I think reasonable alternatives to lwip for embedded systems would be a good thing.
- CyberDildonics 9y agoCan something like this be used on a linux PC? Does the input come from reading in a file handle of raw ethernet input like these tutorials?
- unmole 9y agoThe easiest way would be using a tun/tap interface.
- ambrop7 9y agoIt is already possible. The current test setup involves my APrinter software (firmware for 3d printers) ported to run on Linux including its web interface. It does indeed use a TAP interface and yes it reads/writes Ethernet frames (the stack does implement Ethernet/ARP). Linux specific code is here: https://github.com/ambrop72/aprinter/tree/master/aprinter/hal/linux https://github.com/ambrop72/aprinter/tree/master/aprinter/ha... However note that the LinuxTapEthernet is not directly for my TCP/IP stack but there is a driver abstraction in between, which is also the part that configures/instantiates the stack (aprinter/net/IpStackNetwork.h). It is possible to try it out if you have the Nix package manager (since the configuration/build system is based on that): python -B config_system/generator/generate.py | nix-build - -o ~/aprinter-build ./aprinter-build/aprinter.elf -ttap0 That will result in trying to get an IP address using DHCP (yes DHCP client is included). Static IP can be configured using these commands: M926 INetworkDhcpEnabled V0 M926 INetworkIpAddress V192.168.64.10 M926 INetworkIpNetmask V255.255.255.0 M926 INetworkIpGateway V192.168.64.1 M930 The software will run a command line usable via nc on port 23, and also an HTTP server on port 80. To get the HTTP server to actually serve files, an SD card image with FAT filesystem needs to be created and the web interface files put into it (I can explain if anyone wants to try). What is to be done is a minimal example for integrating the stack and corresponding documentation. For example how to integrate with the reactor, which is abstract from the perspective of the stack but must follow a specific interface. The stack itself only uses timers since other event sources would be used for I/O by e.g. interface drivers. Currently there are two event loops provided, one for Linux based on epoll (aprinter/system/LinuxEventLoop.h) and one that uses busy looping meant for microcontrollers (aprinter/system/BusyEventLoop.h).
- Dales4 9y agogreat share
- jokoon 9y agoI wonder, is TCP generally integrated in ethernet chips, or is it mainly software ?
- peterwwillis 9y agoAn ethernet chip deals with ethernet, at the link layer. TCP is higher up in the stack and not necessary to process ethernet. A few unique network interfaces are designed to process some aspects of IP, and sometimes TCP, to support specific performance demands. TCP will generally be implemented in software/firmware because it's complex and connection-oriented.
- gricardo99 9y agoThere have been NICs that integrate TCP/IP offload features[1]. I'm not sure how widely used they are anymore. It seems when CPU clock frequencies leveled off, and core count started to increase (~10 years ago), there was less rationale to use dedicated hardware versus consuming more available CPU/core resources to deal with the network protocol processing in software. 1 - https://en.wikipedia.org/wiki/TCP_offload_engine https://en.wikipedia.org/wiki/TCP_offload_engine
- rjsw 9y agoAll Intel NICs do TCP offload, they are quite widely used.
- CyberDildonics 9y agoDo you have a source for more information? I know Intel NICs are mostly regarded pretty highly.
- aarongolliver 9y agoI don't know much about it, but it's briefly described here: https://www.intel.com/content/www/us/en/support/network-and-i-o/ethernet-products/000005593.html https://www.intel.com/content/www/us/en/support/network-and-... (ctrl-f offload) https://wiki.linuxfoundation.org/networking/toe https://wiki.linuxfoundation.org/networking/toe suggests it's done mostly in closed-source firmware.
- partycoder 9y agoProblem with rolling your own TCP stack is all the RFCs that compose what we collectively know as TCP. Congestion control and flow control for instance, there are many RFCs that define them.
- corysama 9y agoI like the explanation I've heard that rolling your own TCP stack is a very educational and worthwhile experience. But, what makes TCP so difficult in practice is that there are so many home-rolled stacks out there in the wild that all have different quirks and errors. So yes, please roll your own. But no, please do not add to the wild zoo of errors by deploying it.
- zik 9y agoThat's literally how the original TCP/IP implementations were tested: by running them against each other in "bake-offs": https://tools.ietf.org/html/rfc1025 https://tools.ietf.org/html/rfc1025 I think there's a lot to be said for testing network code against the myriad of real world implementations out there.
- saminiir 9y agoVery interesting and yes, interoperability is hard. https://www.snellman.net/blog/archive/2014-11-11-tcp-is-harder-than-it-looks.html https://www.snellman.net/blog/archive/2014-11-11-tcp-is-hard...
- hobos_delight 9y agoI love this series - it is what inspired me to have a crack at it in rust (https://github.com/rthomas/rust_net https://github.com/rthomas/rust_net) - however I am only at the stage of responding to arping, and other shiny things have come along to distract me.
- Animats 9y agoIf you're going to write a TCP stack to see how it works, it's useful to log all packets that don't advance the connection. This includes all rejected packets, plus any duplicates or ACK-only packets that don't advance the connection sequence. Examining the contents of the bit bucket is useful. I did that in the early days of TCP, mostly to debug other implementations that had interoperability trouble.
- tr4nc3 9y agoVery informative indeed. Minor nitpick- one should avoid the use of strcpy() - something that (believe it or not) Microsoft outlawed within any code that they wrote which reduced the number of buffer overflows drastically.