6 ms·
Ubus (OpenWrt micro bus architecture)
- fulafel 3y agoDid they consider using a memory-safe language for this?
- deleted 3y ago[deleted]
- Roark66 3y agoI have no idea if they did. C is a "default choice" in embedded systems. I don't know if we now have a better alternative that is similar in speed, ram use and binary size. If we do I'd would read a discussion on which one to use with interest.
- btgeekboy 3y agoConsidering the repo dates back to 2010 (predating Rust by 4+ years), there probably wasn’t a better option that could run on the embedded devices that OpenWRT targets.
- fulafel 3y agoThe project age is good background info. However Rust didn't invent memory-safety, there were reasonable options for system programming at that time too (eg Ocaml, Go, Ada, etc). edit: Also D in safe mode like mentioend above - not sure if it was around in 2010?
- dikei 3y agoI think any language with a large runtime would be out of the question, because Openwrt has to fit into system with very limited storage.
- josteink 3y agoTo put it in context: On some devices you have 32MB flash storage in total. 32 Megabytes, not gigabytes. This is not a typo. This needs to fit the bootloader, the Linux kernel, the initrd, the rootfs containing all the user-space tools making OpenWRT an actual usable OS and whatever daemons you need to implement your particular network needs. Oh and you probably want the WebUI too. In 32MBs. There are newspapers online which loads more than that just to show the front page! There’s no room for 2MB HelloWorld type languages in this space.
- fulafel 3y agoThis is a good point. But Ocaml and Ada are not in the 2MB helloworld club (eg Ocaml is more like 200k), don't know about Go. Also accounting that to a single binary is not the right perspective I think - adopting a safe language in OpenWRT should be arranged so that all the programs using that language can share the same copy of the runtime.
- phoronixrly 3y ago200k is very large in the OpenWrt world
- fulafel 3y agoAda then? Eg on https://ada.godbolt.org/ https://ada.godbolt.org/ you can see the default program compiled to ~25 lines of assembler. edit: actually it contains a call to a bounds check function that presumably is in the runtime library, i don't know how big that is. A major point in this is that C also has a fairly big runtime library that should be factored into the comparison!
- zokier 3y ago> A major point in this is that C also has a fairly big runtime library that should be factored into the comparison! But libc will be there anyways, unless you are rewriting absolutely all of userland.
- mananaysiempre 3y agoAnother consideration (apart from the 64MB ROM / 8MB RAM configuration OpenWRT targets, up from 32MB / 4MB in 2010) is that the original—and still very popular—platform for OpenWRT is Linux on big-endian MIPS, which does not exactly have ubiquitous compiler support outside of the C world. (Rust treats it as Tier 2, on par with Windows on ARM and Solaris on x86-64, which isn’t bad as these things go.) So—Go is about as realistic as Java, OCaml I don’t think runs on MIPS, neither does MLton, and IIRC the GNU Ada implementation was in a much worse state then. There’s also the part where the devs are by necessity C hackers, what with sorting out the manufacturers’ kernel patches and all.
- raimue 3y agoBack then, your suggestions were niche languages (and some still are) and are still not popular for embedded systems or network equipment. Large runtimes or huge static binaries are not suitable due to the memory and storage constraints. You have to consider the surrounding ecosystem. Those interested in such languages are not necessarily those interested in contributing solutions to the problem space. Any project attempting to use such a language in OpenWrt would very likely not have survived until today.
- lionkor 3y agoThere is no real case to be made against the idea of memory safety, but there is a case to be made for C. You can write C code such that its unit tested, fuzzed, statically analyzed and reviewed, and often embedded code has one or two specific jobs. I would have loved to see Zig here, as it makes all of the above (testing, analysis, code clarity for review) easier, but its also not memory safe necessarily. You could write it in D with @safe (which is SafeD or whatever), which is memory safe, but thats not a very popular language. You could use Ada, but again, thats not as easy to find devs for as C is. You could use Go or Ocaml, though go isn't really memory safe and you can easily cause data races in goroutines, and ocaml has a runtime afaik so thats out of the question for low memory devices. You could use Rust, assuming it had existed back when they started it, and you would get memory safety, but also an entire kitchen sink of useless garbage (like C++). Youd also have to shell out to unsafe{} in a lot of places, unless you use a crate for it, which will then do unsafe{} for you. So chances are, whatever you do, C is a pretty sane choice, or maybe C++ if you want RAII to at least make your resource management and lifetimes easy to handle. I dont think you deserve to be downvoted, since this is an interesting discussion to have. However, I think it would have been helpful for the discussion if you had outlined which language youd suggest and why. To Steelman your argument, I would say you think that C is unsafe to such a degree that even a for loop is UB most of the time (integer overflow as UB), no real way to check array bounds, no real way to catch off-by-one and similarly stupid simple errors, use after free, egc. and that the entire ecosystem relies on raw pointers and macros, and its a shitshow. I think your point would have been to suggest Rust, as it fixes all these issues, while bringing along a stronger type system and a better toolchain.
- jeroenhd 3y ago> You could use Rust, assuming it had existed back when they started it, and you would get memory safety, but also an entire kitchen sink of useless garbage (like C++). Youd also have to shell out to unsafe{} in a lot of places, unless you use a crate for it, which will then do unsafe{} for you. I'm not saying C is a wrong choice here, but this argument doesn't make much sense to me. All the other languages also come with "useless garbage" (some even with a collector for them). I don't see why you'd need that many unsafe blocks either. I feel like Rust, as a language, is much better equipped for writing a daemon like this. I'll even go as far as to say that modern C++ (C++2x) is a better choice than C for this stuff with its superior memory and resource management tools. Of course most of the OpenWRT development is done in C, so doing it in another language requires a very good reason. I wouldn't look at the concept of "a networked daemon exchanging messages between arbitrary services" and think C of all languages is a good fit unless every other developer on the team can only write C. Zig would've been a nice way to meet halfway, but the language isn't finished yet. Of course the point is moot because this particular project existed years before any of the modern alternatives or many of the C++ improvements were even available. Still, if they'd start a project like this again, I'd hope they'd pick a better language for it.
- mkj 3y agoA decent question, I've wondered about Rust programs on openwrt but at the moment they're just too big! Linking the stdlib into every binary doesn't help, and the code is generally larger than the C equivalent. It doesn't seem unsolvable though, I'm hopeful. no_std rust binaries can be near competitive with C. Binary size is sometimes a big deal. Two decades ago I investigated using C++ for an "embedded" linux ssh server, but decided the 30kB overhead was too large (target was a 4MB laptop). The server ended up being used in OpenWRT and other places, I'm curious if it would have happened if I'd gone with C++ instead.
- no_time 3y ago>I'm hopeful. no_std rust binaries can be near competitive with C. I feel like Rust without std might as well be a different language. Almost all of the available mindshare, docs, libraries are dependant on it. Sure, you can use it. But to me it just looks like technological poverty. Even compared to the C ecosystem.
- mkj 3y agoI'd disagree. Using something like Embassy is pretty pleasant for embedded development, less worries than C. The only issue is maturity. There were sufficient crates to write a ssh server for a rp2040 (mostly out of curiosity) using what's available in no_std.
- teunispeters 3y agoFrom working with OpenWRT for years on embedded systems where sometimes 128 bytes made a difference: this is not a useful question. In the embedded space, you use small effective languages like assembly and C. You either design for explicit memory use entirely, or you systematically test for memory usage and waste - especially if you're delivering industrial applications. The so-called "memory safe" systems tend to be very expensive for memory usage and space, and not worth the investment. Usually, anyway. This may change in the future but it's a bad change if it comes with additional power usage requirements and the environment also requires minimal power use.
- xgbi 3y agoStory time.. I coded a similar thing when I started working after graduation for a DTV software company; we needed to have an IPC on Linux and I whipped up some very crude equivalent of protobuf (which I didn’t know about), based on an RLE lib i stumbled upon and without any form of discovery. It was circa 2009, I had only been exposed to plain text protocols and didn’t know about json. In hindsight, we might’ve been better using standard dbus, of protobuf, but I was a rookie and it provided the performance we needed (for DTV metadata). I’m happy to see that these can still thrive, and I just recently figured out that discovery is a net multiplier in these projects; doing this really proved me that any problem is solvable if you can have some time to think about it and can prototype. I long for these moments now, I feel like nearly all computing issues have been solved and we are now just plumbers, connecting libraries and software modules through config files instead of building things.
- IshKebab 3y agoOnly if you choose to be a plumber. You can equally write new things if you want. Also I think "any problem is solvable" needs some qualification - there are a ton of problems that have yet to be solved or are super complicated. I still haven't figured out how unbounded model checking works for example.
- attah_ 3y agoIn case you want to react to stuff connecting to your router via ubus: iw event | awk '/new station/ {print $4}' | xargs -n 1 sh -c 'ubus send new_station {\"mac\":\"$1\"}' _ (Yes, this contains silly hacks to work around busybox limitations)
- phoronixrly 3y agoI'm pretty sure that there already is an event emmitted upon station connection
- attah_ 3y agoCertainly not by default anyway. That would have been great though. Double-checked with ubus listen in case it had changed over the last couple of years.
- peter_d_sherman 3y agoSource code: https://git.openwrt.org/?p=project/ubus.git;a=tree;h=refs/heads/master;hb=refs/heads/master https://git.openwrt.org/?p=project/ubus.git;a=tree;h=refs/he...
- mananaysiempre 3y agoNot to be confused with the (dead) ubus project[1] by the suckless folks. (Mostly I just want to write down the link to that one, because I keep losing it.) [1] https://web.archive.org/web/20131209010702/http://unixbus.org/ubus/ https://web.archive.org/web/20131209010702/http://unixbus.or...
- fatfingerd 3y agoIt seems to have some similarities to RouterOS' IPC and of course they have a similar role and environment. I'm curious if anyone who has looked into both of them in detail has any thoughts? https://news.ycombinator.com/item?id=33904105 https://news.ycombinator.com/item?id=33904105
- PhilipRoman 3y agoAs someone having to debug issues on OpenWRT derivatives, I wish I had a time machine to tell the inventors of Unix to never add other forms of IPC than pipes. It's all a big pile of stateful daemons notifying other daemons with a billion race conditions and zero debugging capabilities, like a parody of how not to create reliable systems. When you have shell scripts parsing JSON messages, you know it's over.
- westurner 3y agoBut then how to add datatypes - schema - to pipes? How do we add a column to the messages on the pipe? Newline-delimited != SOTA https://en.wikipedia.org/wiki/Apache_Arrow https://en.wikipedia.org/wiki/Apache_Arrow : > Arrow allows for zero-copy reads and fast data access and interchange without serialization overhead between these languages and systems JSON lines formatted messages can be UTF-8 JSON-LD: https://jsonlines.org/ https://jsonlines.org/ Linux networking is now all built on eBPF.
- packetlost 3y agoThe same way you add it to any other SOCK_STREAM? How do we add types and schemas to TCP connections?
- westurner 3y agoBut then when I need to scale to more than one node with multiple cores, do pipes scale?
- packetlost 3y agoYes, with some work: https://mazzo.li/posts/fast-pipes.html https://mazzo.li/posts/fast-pipes.html
- westurner 3y agohttps://mazzo.li/posts/fast-pipes.html#splicing https://mazzo.li/posts/fast-pipes.html#splicing : > In general when we write to a socket, or a file, or in our case a pipe, we’re first writing to a buffer somewhere in the kernel, and then let the kernel do its work. In the case of pipes, the pipe is a series of buffers in the kernel. All this copying is undesirable if we’re in the business of performance. > Luckily, Linux includes system calls to speed things up when we want to move data to and from pipes, without copying. Specifically: > - splice moves data from a pipe to a file descriptor, and vice-versa. > - vmsplice moves data from user memory into a pipe. > Crucially, both operations work without copying anything. But then to scale to more than one node, everything has to be copied from the pipe buffer to the network socket buffer; unless splice()'ing from a pipe to a socket is Zero-copy like sendfile() (which doesn't work with pipes IIRC). "SOCKMAP - TCP splicing of the future" (2019) https://blog.cloudflare.com/sockmap-tcp-splicing-of-the-future/ https://blog.cloudflare.com/sockmap-tcp-splicing-of-the-futu...