13 ms·
Smoltcp: A small TCP/IP stack in Rust
- OskarS 7y agoNaming your product after dumb internet memes is a great way to make me not want to use them. Don’t get me wrong, from first look it seems like a perfectly fine product, but the name is atrocious.
- hajhatten 7y agoComments like these make me not want to read them. Don’t get me wrong, from first look it seems like a perfectly fine comment, but the attitude is atrocious.
- nindalf 7y agoEvaluating others projects with Gordon Ramsay’s attitude is common on HN. These folks must assume that they’re “telling it like it is”.
- noselasd 7y agoFor the uninitiated, which meme is this ?
- Sekhmet 7y agohttps://m.imgur.com/gallery/HPTS0YK https://m.imgur.com/gallery/HPTS0YK
- toby- 7y agoIt's part of DoggoLingo[0], a meme-filled Internet lexicon (or... something). It's all in good fun, and I don't understand why anybody could be upset by it. Sure, he could've called this "SmallTCP" and been bland, but decided to inject some (silly) humour. Plenty of other projects are similarly named.[1] It reminds me of the words "cromulent"[2] and "embiggen"[3]; coined by The Simpsons but used in everything from BBC articles to scientific research. [0]: https://en.wikipedia.org/wiki/DoggoLingo https://en.wikipedia.org/wiki/DoggoLingo [1]: https://github.com/aras-p/smol-v https://github.com/aras-p/smol-v [2]: https://en.wiktionary.org/wiki/cromulent https://en.wiktionary.org/wiki/cromulent [3]: https://en.wiktionary.org/wiki/embiggen https://en.wiktionary.org/wiki/embiggen
- NikkiA 7y ago> It reminds me of the words "cromulent"[2] and "embiggen"[3]; coined by The Simpsons but used in everything from BBC articles to scientific research. popularized by the simpsons, at least cromulent was a valid english word before the episode, I don't think embiggen was.
- toby- 7y agoI don't think that's true. I can find no reference to "cromulent" existing pre-Simpsons, and every source explicitly says it was coined by David Cohen for an episode. From Wiktionary: "A humorous, intentionally morphologically opaque neologism coined by American television writer David X. Cohen for 'Lisa the Iconoclast', a 1996 episode of the animated sitcom The Simpsons." From Merriam-Webster: "It is safe to say that The Simpsons has contributed a great deal to the English language. One famous example is cromulent, which was coined specifically for the 1996 episode 'Lisa the Iconoclast'."
- taneq 7y agoWhile we're at it, the arrangement of spikes on a Stegosaurus' tail is called a thagomizer, after a Far Side cartoon. And it's actually referred to as such by real live paleontologists. https://en.wikipedia.org/wiki/Thagomizer#Etymology https://en.wikipedia.org/wiki/Thagomizer#Etymology
- ignaloidas 7y agoThis is dumb. Many products have dumb names, we just got accustomed to them, i.e. git. If you are hating only on this specific meme - I present you SmolV - the leading(and only?) SpirV compressor. If you aren't going to use good things because of naming, then you deserve to only use bad things.
- zaptheimpaler 7y agoIt's not a product, and if you don't want to use it - don't :)
- pencillr 7y agoAn educational question: If this is a tcp/ip implementation where is it run? On a router? Or on a network interface controller? Both?
- abhishekjha 7y agoI think it would be run on a network interface. Isn't this or an equivalent implementation that comes packaged with every OS so that you can connect to a network? I may be wrong here and others are more than welcome to correct me. EDIT: Added "or an equivalent"
- roblabla 7y agoThis is definitely not the implementation that comes packaged with every OS. Every OS has its own TCP/IP implementation that usually lives in the kernel - though most are derived from BSD's TCP/IP stack. SmolTCP could (in theory) replace the implementation packaged with an OS, or even be used completely from userspace by taking over the raw network interface.
- abhishekjha 7y agoApologies. I didn't mean that this exact implementation is shipped with every OS. I meant an equivalent of it being shipped with OSes.
- chaosite 7y ago"smoltcp is a standalone, event-driven TCP/IP stack that is designed for bare-metal, real-time systems." My understanding based on that description is that it is meant for applications that run directly on the hardware, without an OS in the middle. I'm thinking embedded applications. So I'm thinking that this is meant for IoT-style appliances and the like. Maybe I'm wrong :)
- pencillr 7y agoAh so, it can be something like a "network platform" on my (let's say) smart home thermostat? :D
- haberman 7y agoI have a distant memory of Alan Cox saying something back in the day along the lines of "there are so many things left undocumented in the TCP/IP RFCs that if you just implement it straight from the spec your stack won't work at all." But I can't find any reference to this now. Does anyone else remember this? Is this true? Can you interoperate with the Internet by just implementing the spec?
- nabla9 7y agoTCP/IP is relatively stable ecology where stacks in routers, applications etc. have evolved to survive with each other and intentionally malicious actors by developing coping mechanisms. Introducing a clean implementation of TCP/IP stack is like introducing a new species. It has no immune system against others and others don't know its 'signature behavior' either. You can probably get interoperability at least part of the time. You may have deteriorated throughput and more broken connections with some stack and not with others. But you have introduced a new species. If your brand new stack has a tiny bug or new kind of misconfiguration and you start spreading it fast, the hell can break loose and you may ruin the day of many people before they find you and cut you off.
- maltalex 7y ago> Is this true? Can you interoperate with the Internet by just implementing the spec? I've never implemented the entire stack. Few people have. But from the protocols I've implemented, that's only a slight exaggeration. As soon as a popular implementation has a bug or decides not to behave according to spec, everyone has to adapt. You can typically use the spec to get 95% of the way to full interoperability.
- andy_ppp 7y agoAnd presumably 95% of the work is in the final 5%?
- jsnell 7y agoNo, it's not true. You can implement something that can load a webpage from 99.9% of the world's web servers from a couple of RFCs in a day. But this ease of implementation means that for basically any issue where the specs give any degree of freedom, somebody will have deviated from the norm. (I've implemented TCP from scratch three times, one of those implementations basically ran the traffic of entire countries.) The real problem is that every step toward 100% gets progressively harder to find and debug. See e.g. [0] for a middle box that would mangle connections if it received two identical SYN packets, or [1] for a way in which almost all servers anyone currently runs are accidentally resilient to a certain kind of connection-killing packet corruption, but S3 isn't. [0] https://www.snellman.net/blog/archive/2014-11-11-tcp-is-harder-than-it-looks.html https://www.snellman.net/blog/archive/2014-11-11-tcp-is-hard... [1] https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/ https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/
- gazarullz 7y agoCan anyone point out any resources to acomplish this in the JVM space?
- deleted 7y ago[deleted]
- gazarullz 7y agoWhoever downvoted my question can explain why? or it was just for fun?
- deleted 7y ago[deleted]
- smolder 7y agoI didn’t downvote you but running a TCP/IP stack inside the JVM doesn’t seem terribly useful. Doing user space TCP/IP can be useful for performance reasons but running on top of JVM is likely to wipe out that advantage.
- IAmLiterallyAB 7y agoA zero heap allocation _anything_ in the JVM? Probably impossible
- teddyh 7y agoSomewhat relatedly, there exists a Python-like language for when you only have a few kb of memory, and can’t even run MicroPython. It’s called “Snek”: https://keithp.com/snek/ https://keithp.com/snek/
- littlestymaar 7y agoThis is a really good example of the kind of safety benefit you get when using Rust: this is a fairly low-level piece of software (not a kernel, but I'd say it's lower-level than at least 80% of what's usually written) and yet, only 76 lines of codes are located inside unsafe blocks for the whole code-base. To be safe from memory vulnerabilities, all you need is audit theses 76 lines. It can still be a tough job, and bugs may slip past the review, but it's a huge improvement over having to audit thousands of lines.
- nsajko 7y ago> safety benefit Has "safety" become a buzzword? What we really strive for is correctness. Hypothetically, Rust could still be detrimental to software correctness compared to a more usual language, if some of its traits as a language (eg., complexity) encourage bugs. Sometimes it seems like Rust afficionados think that buffer overflows etc. are the main kind of bugs. And if you need absolute correctness (when human life depends on the software) you would probably use Ada and Spark or something else that enables you to actually prove correctness.
- littlestymaar 7y ago(Memory) safety and correctness are orthogonal issues. If I'm building a missile's aiming system, I absolutely want correctness. But if I run a web service on the cloud I care more about an attacker taking control of my machines than about another kind of bugs. Same if I deploy a fleet of connected devices to thousands of consumers' houses. Computers are all around us, and they are a security nightmare, less because of their lack of correctness than because of their lack of safety. That's why Rust is important. Formal proofs and tools like Ada are important too in their own domain, and Rust isn't competing with them (yet at least, I know there are some people would like to develope something akin to Spark for Rust adding best in class correctness tools to Rust's toolkit).
- ordu 7y ago> Has "safety" become a buzzword? What we really strive for is correctness. Correctness is about program making what I meant as a programmer. Safety is a different thing. I see it as insulator on wires. It helps me to avoid shocks while working with electricity. Safety guarantees of Rust is like that, I have a pointer, I could use that pointer without fear of segmentation fault or data race. It feels like safety. It named "safety". So I believe it is a good coherent name. > if you need absolute correctness (when human life depends on the software) you would probably use Ada and Spark or something else that enables you to actually prove correctness. Maybe. But between absolute buggy code and absolute correct code there is a spectrum of not so buggy and not so correct code. We need a compromise enabling us to create a reasonable reliable software with a reasonable effort.
- adwn 7y ago> Its design anti-goals include complicated compile-time computations, such as macro or type tricks, even at cost of performance degradation. Why? That sounds more like an ideological decision than a pragmatic, engineering-driven one. Especially for a TCP/IP stack, where performance is typically a major concern, be it in a desktop, server, or embedded environment.
- q3k 7y agoIt can be an engineering decision: write readable, easy to understand and reason about code.
- toast0 7y agoMost of the performance of TCP stacks has more to do with the order that comparisons are made for incoming packets (sometimes called fast path -- check for and handle normal packets first), locking of data structures, and congestion strategies (including retransmit behavior, SACK, etc). Macros or typing is less likely to make that faster. For a new stack, though, correctness and readability are more important than performance.
- whitequark_ 7y agoAt the time when I started working on smoltcp, there were a few Rust libraries for working with TCP/IP on the wire level, and they heavily used metaprogramming. Unfortunately, the task of implementing a generic binary protocol serializer/deserializer that can handle TCP/IP is not small, and as far as I could tell, it overtook implementing anything beyond that. So I made the decision to do the simplest possible thing: write the packet serializers/deserializers entirely by hand. It took very little time and adding any new features was easy and predictable. I believe it was the right decision as it allowed me to focus on the hard parts of a TCP/IP stack.
- adwn 7y agoThank you for answering. That sounds sensible, I guess.
- eeZah7Ux 7y agoWhat a poor choice for naming. Please don't use stupid internet memes for naming, it makes all of us look silly in the eyes of non-techies.
- WAHa_06x36 7y agoAre you expecting non-techies to be paying attention to an embedded TCP/IP stack, and also worrying that they think you are having too much fun?
- bananocurrency 7y agoget over it https://z.cash/technology/jubjub https://z.cash/technology/jubjub
- nn3 7y ago>Congestion control is not implemented. Wit that, don't bother with it so far. That's a toy. It would be dangerous on any real network. If you doubt it please read up on "congestion collapse"
- garmaine 7y agoNot sure why you are being down-voted. Congestion control is pretty central to how TCP/IP works. Deploying without it will basically DoS your network.
- blinkingled 7y agoYou might not care for low level TCP/IP details but this project will make for a great learning experience for anyone wanting to dig deeper into Rust and network programming. It works with tun/tap interfaces and there's a tcpdump clone in the example using raw sockets that works on any regular Linux network interfaces. Pretty cool!
- Klasiaster 7y agoHere are memory-safe network services on Linux with smoltcp and an optional switch for running multiple userspace network stacks: https://github.com/ANLAB-KAIST/usnet_sockets https://github.com/ANLAB-KAIST/usnet_sockets → Socket library for Rust using smoltcp as userspace network stack (provides types compatible with the standard library, tokio is unpublished WIP) https://github.com/ANLAB-KAIST/usnetd https://github.com/ANLAB-KAIST/usnetd → Memory-safe L4 Switch for Userspace Network Stacks (firewalls the kernel network stack, see Ideas section for alternatives, e.g., transparently piping the kernel network packets through smoltcp)
- fluffything 7y agoLots of warnings on `cargo build`, undefined behavior on a couple of places (e.g. crating &mut uninitialized), raw calls to libc as opposed to using a safe wrapper over it like nix, ...
- kahlonel 7y agoAny idea what would it take to write this without using _any_ C library function?
- gpm 7y agoI haven't read the code, but probably very little, since libc has rust implementations that you could plug in in it's place.
- whitequark_ 7y agoIt doesn't use any C library functions by itself. (In fact in all of its applications at M-Labs, there is no C code running on the same chip.) The only uses of libc are in the TUN/TAP driver, which is necessary if you want to bind it to a virtual OS interface.