7 ms·
Safe and Secure Drivers in High-Level Languages [video]
- saagarjha 8y agoPersonally, I'd call this writing "Safer and More Secure Drivers in High-Level Languages", because there are still unsafe operations going on (for DMA, etc.): https://github.com/ixy-languages/ixy.swift/search?q=Unsafe https://github.com/ixy-languages/ixy.swift/search?q=Unsafe While it's great that you get improved safety (and often nicer and easier to reason about code) by using something other than C, you can still have memory safety issues from cavalier or incorrect usage of unsafe APIs, since they undermine the guarantees the language provides with regards to correctness. Also, unrelated: does anyone actually have the slides (you know, the presentation file with text in it, rather than the mp4 that CCC is offering me) for this presentation? It's really annoying to scrub through a video to find stuff on a slow internet connection :(
- qznc 8y agoUsually they would be linked on this page: https://fahrplan.events.ccc.de/congress/2018/Fahrplan/events/9670.html https://fahrplan.events.ccc.de/congress/2018/Fahrplan/events...
- voltagex_ 8y agoThe slides may not be up yet, but you could demux just the video stream out of https://mirrors.dotsrc.org/cdn.media.ccc.de//congress/2018/slides-h264-hd/35c3-9670-eng-deu-Safe_and_Secure_Drivers_in_High-Level_Languages_hd-slides.mp4 https://mirrors.dotsrc.org/cdn.media.ccc.de//congress/2018/s..., or give me a yell in the next 30 minutes or so and I can do it. Don't have a nice script to de-dupe images though.
- emmericp 8y agoI've added them to the git repo: https://github.com/ixy-languages/ixy-languages/blob/master/slides/35C3.pdf https://github.com/ixy-languages/ixy-languages/blob/master/s...
- saagarjha 8y agoThanks!
- edwintorok 8y agoDirect link for download (GitHub won't render the entire PDF by default): https://github.com/ixy-languages/ixy-languages/raw/master/slides/35C3.pdf https://github.com/ixy-languages/ixy-languages/raw/master/sl...
- pjmlp 8y agoQuite true, however the unsafety code coverage gets reduced quite significantly. On the last Linux Kernel Summit, according to Google, 68% of Linux kernel security exploits are caused by C memory corruption issues due to lack of bounds checking.
- zozbot123 8y agoThe talk mentions making DMA operations safe using IOMMU. (Unfortunately, IOMMU is a hardware feature and not always supported.)
- Santosh83 8y ago> Our drivers in Rust, C#, go, and Swift are completely finished, tuned for performance, evaluated, and benchmarked. And all of them except for Swift are about 80-90% as fast as our user space C driver and 6-10 times faster than the kernel C driver. Interesting. What is the reason for higher performance of user space C driver (and the other user space drivers for that matter) when compared to the kernel C driver? Will this hold for all driver types or is this a rather uncommon property of this particular kind of driver?
- usaphp 8y ago> A main driver of performance for network drivers is sending/receiving packets in batches from/to the NIC. Ixy can already achieve a high performance with relatively low batch sizes of 32-64 because it is a full user space driver. Other user space packet processing frameworks like netmap that rely on a kernel driver need larger batch sizes of 512 and above to amortize the larger overhead of communicating with the driver in the kernel.
- mpweiher 8y agoInteresting! As our devices have gotten faster, the user-space/kernel boundary is becoming more and more of an issue. I was shocked when my supposedly super-fast MacBook Pro SSD (2+GB/s) was only giving me around 250MB/s. Turned out mkfile(8), which I was using without thinking much about it, is only using 512 byte buffers... https://blog.metaobject.com/2017/02/mkfile8-is-severely-syscall-limited-on.html https://blog.metaobject.com/2017/02/mkfile8-is-severely-sysc...
- zozbot123 8y agoThe main thing affecting performance at the userspace/kernel boundary these days is Spectre and Meltdown mitigations, FWIW...
- mpweiher 8y agoYeah, when I ran the same tests a little later and with the Spectre/Meltdown patches (+APFS, which also didn't help), the mkfile I/O rate had a further precipitous drop to ~80MB/s.
- pjmlp 8y agoAs addition to the talk, Android Things userspace drivers are done in Java, and since Treble, it is also possible to write Android drivers in Java. MicroEJ and Windows IoT Core also allow for such capabilities. Loved the talk.
- emmericp 8y agoCode on GitHub: https://github.com/ixy-languages/ixy-languages/ https://github.com/ixy-languages/ixy-languages/ Discussion about the C version on GitHub in 2017: https://news.ycombinator.com/item?id=16014307 https://news.ycombinator.com/item?id=16014307
- guerby 8y agoGreat work! Do you have by any chance the C driver latency measurements? It would be nice to have them on the same graph as those for Rust and other languages.
- emmericp 8y agoWould look the same as rust; I haven't run the full measurement with all the packets. But sampling 1k packets/second yields the same result for C and Rust. You can't really get a faster speed than Rust here; the only minor thing that could be improved is the worst-case latency by isolating the core that the driver is running on (isolcpus kernel option) to prevent other random interrupts or scheduling weirdness. But that optimization is the same for all languages and should get rid of the (tiny) long tail.
- guerby 8y agoThanks!
- mpweiher 8y agoGreat direction! One of the lesser known but really nifty bits of NeXTStep was DriverKit: http://www.nextcomputers.org/NeXTfiles/Software/OPENSTEP/Developer/DriverKit/DriverKit.pdf http://www.nextcomputers.org/NeXTfiles/Software/OPENSTEP/Dev... Objective-C was a great match for driver development, devices tended to have a very natural OO flavour to them and naturally sorted into classes. Putting things in user-space seemed a natural extension, that sadly didn't happen at the time even though Mach had the hooks for it (we never got user-level pagers either, which would have rocked together with garbage collected languages). There certainly didn't seem to be a good reason why I had to reboot the machine and wait for fsck when there was a minor bug in the little driver I was writing to talk to an EISA printer controller that had nothing to do with the basic functioning of the system... (Why would a printer controller be on an EISA controller, you ask? It directly drove a Canon Laser Copier, so, yes!) Oh, and not surprised by the abysmal Swift performance. Alas, Apple's marketing of Swift as a "high-performance" language has been very successful despite all the crushing evidence to the contrary.
- saagarjha 8y ago> Objective-C was a great match for driver development, devices tended to have a very natural OO flavour to them and naturally sorted into classes. How do you feel about the current state of driver development on macOS, with Objective-C basically being replaced with Embedded C++ with partial reflection?
- mpweiher 8y agoIt was part appeasement of the "never Objective-C" crowd (hello CoreFoundation, hello CocoaJava, hello Swift) and part the exact sentiment discussed here, that you cannot possibly do kernel development in a higher level language. What I heard (quite some time ago) is that this move is now seen as a mistake.
- mrpippy 8y agoObjC was definitely seen as a dead-end at the time: it would either be replaced with Java, or maybe Mac developers would just stick with Carbon and C/C++. Either way, all driver development (on classic Mac, Windows, Unix) was in C, and C++ would be much more familiar than the “weird obsolete square-brackets NeXT language” I made a post a few years ago discussing the issue: https://news.ycombinator.com/item?id=10006411 https://news.ycombinator.com/item?id=10006411
- drasah 8y agoYou have to visit Maktabtk site to get all about theoretical framework in research plan you need to use in scientific research https://www.maktabtk.com/ https://www.maktabtk.com/ https://www.maktabtk.com/service/7/%D8%A5%D8%B9%D8%AF%D8%A7%D8%AF-%D8%A7%D9%84%D8%A5%D8%B7%D8%A7%D8%B1-%D8%A7%D9%84%D9%86%D8%B8%D8%B1%D9%8A.html https://www.maktabtk.com/service/7/%D8%A5%D8%B9%D8%AF%D8%A7%...
- fulafel 8y agoIs there a plan for letting applications using the normal networking APIs see this, or is it currently just for "process raw ethernet packets in userspace" kind of apps? (The latter is a great thing to have built, of course, just thinking aloud about how this might replace existing drivers)
- emmericp 8y agoYeah, I'd love to port a Go TCP stack like https://github.com/google/netstack https://github.com/google/netstack to use our driver to build a microkernel-style service offering network connectivity. Running taps (https://datatracker.ietf.org/wg/taps/about/ https://datatracker.ietf.org/wg/taps/about/) on top of that would be ideal for a modern setup. But that's a lot of work...
- duneroadrunner 8y agoAnother option is to use a memory safe subset of C++ [1]. It should be less work to migrate existing C drivers as (reasonable) C code maps directly into the safe C++ subset. And the migration can be done incrementally with corresponding incremental safety benefits. [1] shameless plug: https://github.com/duneroadrunner/SaferCPlusPlus https://github.com/duneroadrunner/SaferCPlusPlus
- pjmlp 8y agoOnly if the team plays along regarding static analysers and compiler warnings as errors.
- newnewpdro 8y agoWouldn't you simply enforce this with automation if you were making a serious effort? It's already quite common for github PR's to require myriad CI tests to pass before anything can be merged... those can incorporate static analysis and warnings as errors.
- pjmlp 8y agoStill needs to have the buy-in from the team. Try to be that clever guy putting such gates into place without having the team be on the same wave length. Github is a bubble, there are tons of software projects out there, using a myriad of build infrastructures, or even just doing plain old IDE builds (yes I know, but it is as it is).
- newnewpdro 8y agoConvincing your team to switch languages is infinitely more difficult than adding infrastructure to enforce good hygiene. So I don't really see what your point is, it's moot.
- pjmlp 8y agoThis whole thread started about enforcing behaviors that are largely ignored by enterprise developers outside HN bubble. In no point there was a mention of switching languages.
- pjmlp 8y agoSwift loosing heavily due to reference counting GC instead of a tracing GC. Now I have a talk to point out to those that always trump that referencing counting GC are so much better than tracing ones.
- zozbot123 8y agoRust also uses refcounting, not tracing GC. The problem with Swift is its use of obligate GC for things that don't need it in other languages (C/C++, Rust).
- pjmlp 8y agoRust doesn't do refcounting, it makes use of an affine type system, which is something completly different. If you make use of the refcounting library types in Rust, from the std::rc and std::sync crates, the performace impact would be quite similar.
- zozbot123 8y agoSure, but you don't need refcounting for cases that are covered by Rust's affine types, even in C/C++. You can use the patterns described in the C++ Core Guidelines, and end up with something quite rusty, only without fully-automated checking. (Besides, I think std::rc has better performance than the refcounts found in Swift and C++, because it's used in cases that don't need atomic update, and yes this is statically checked too.)
- pjmlp 8y ago> You can use the patterns described in the C++ Core Guidelines, and end up with something quite rusty, only without fully-automated checking. Also known as the mythical "only other programmers commit errors, I am always correct".
- asdkhadsj 8y agoYea, I'm not sure what point was being made there. Rust isn't about being faster than C, it's about not letting you make mistakes you can make in C/etc. If you could somehow write perfect code in a timely manner, you'd have no need for Rust. You'd likely also be a unicorn.
- emmericp 8y agoPrevious discussions about our Rust and Go drivers: https://news.ycombinator.com/item?id=18405515 https://news.ycombinator.com/item?id=18405515 https://news.ycombinator.com/item?id=18399389 https://news.ycombinator.com/item?id=18399389
- henrikeh 8y agoIt is disappointing that Ada seems to be completely shunned from all discussions of safe high-level languages despite having both a track record (several successful safety critical systems) and a unique feature set (multiple compilers, provably correct subset, ranged subtypes). Is it purely a matter of what is well-branded?
- agumonkey 8y agoAda also suffered from bad roots and bad timing. We're off a decade of simple dynamic languages. Ada seems like an old and immense ruin. Maybe there will be a julia/rust equivalent for Ada.
- henrikeh 8y agoAnd it is very true -- Ada never truly gained any foothold in anything but situations where it truly delivered upon a requirement. If anything, I can take solace in the fact that it is very much alive despite the exaggerated rumors of its death. I'd encourage everyone to try it out and steal all the ideas.
- zozbot123 8y ago"Multiple compilers" will do little good if the compilers aren't generally available. The Ada problem is all about availability of good implementations. (And no, GCC doesn't cut it. Not in 2018 at least.)
- henrikeh 8y agoPaying for a compiler isn't really as crazy as it sounds. Companies pay for CI, static analyzers etc. etc. For non-commercial applications it is obviously different.* Yet, I'm genuinely curious, why isn't GCC up to snuff? * Edit: Which of course limits general adoption.
- pjmlp 8y agoSomehow it feels strange how many devs expect to be paid for their work, while blantly refusing to pay for the work of others.
- volkisch 8y agoFinally a talk that isn't women in tech or tranny agenda.
- heyjudy 8y agoWhat would make device drivers safer: - Microkernel OSes - one driver failing is okay, because it runs in mostly in user-space. The big gotcha in microkernel design is transactions that touch multiple components. Some sort of across-driver transaction API is needed (start, commit, rollback) in order to undo changes across several userspace subsystems. - A standard language (like the talk suggests) and shipped as portable bytecode to run on a VM or compile to native, so that drivers are portable and runnable without knowing the architecture. - Devices themselves containing OS-signed drivers rather than each OS having a kitchen-sink installation of all drivers. Each bus would have an interrogation call to fetch the driver.