7 ms·
Linux 3.13
- bjackman 13y agoGreat summaries, thanks! I love when kernel news is made as accessible as this (thanks also to LWN). Looks like a pretty major release. I only wish I had a rackfull of cutting-edge SSDs to try out the new block layer on!
- dded 13y agoOne significant user-visible feature of 3.13 is nftables: https://lwn.net/Articles/564095/ https://lwn.net/Articles/564095/
- sp332 13y agoNote libnftables was just renamed to libnftnl http://www.marshut.com/imyzmp/libnftables-renamed-to-libnftnl.html http://www.marshut.com/imyzmp/libnftables-renamed-to-libnftn... A later, higher-level library will probably get the name libnftables.
- esbranson 13y agoI've been wanting to implement a dynamic ARP filter (DHCP snooping + ARP filtering) for ages now, but arptables/ebtables just didn't cut it. Hopefully this will be easier/viable now. Because I still don't think ArpON sniffs DHCP leases for its mapping (it intercepts and replays ARP requests or something) and it doesn't filter rogue DHCP servers. I'm just amazed ARP spoofing/ARP cache poisoning is still a viable attack vector on home networks in 2014.
- adekok 13y agoPlease see the "master" branch of FreeRADIUS. https://github.com/FreeRADIUS/freeradius-server/ https://github.com/FreeRADIUS/freeradius-server/ It can accept both DHCP and ARP protocols, and will decode them into attribute-value pairs. Those can then be referenced in a policy language, and stored to / read from a database. I'm the author. :) It's no longer just a RADIUS server. I've been looking for a DHCP / ARP checker for a while, and couldn't find anything useful. Rather than writing something from scratch, I decided it was easier to just add ~2K LoC to FreeRADIUS. I could then leverage the policy language and database integration, so I didn't have to re-write all of that, either.
- esbranson 13y agoExcellent. :) I will look into this. I am targeting embedded devices on OpenWRT, which means it needs to be as simple and small as possible, so I hope the code is tight. But on the other hand, I would prefer to not reinvent the wheel.
- adekok 13y agoThe code is tiny. It already runs on OpenWRT, so there's no issue there.
- smutticus 13y agoYou need to implement DHCP Snooping at Layer 2 for it to really work, and these days I think all major switch vendors support it. Unless you're building a Linux L2 switch I don't see why you would want to implement DHCP Snooping.
- esbranson 13y agoYou need DHCP snooping for proper ARP reply filtering. It is the cleanest way of determining MAC address<->IP address mappings.
- makmanalp 13y ago> It adds a simple virtual machine to the kernel that is able to execute bytecode to inspect a network packet and make decisions on how that packet should be handled. I wonder how long it's going to take until someone figures out a way to craft a specific sequence of packets that remotely do something nasty at the kernel level :P
- diegocg 13y agoIt's not different than the rest of the network stack, most of it consists in "inspecting packets" in one way or another, and security bugs can be (and some times are) introduced. But nftables is actually a big win from a security perspective, because it simplifies the current code (lots of duplicated code goes away) and moves other parts to userspace. Old netfilter system: 70.000 LoC in kernel + 50.000 in userspace nftables: 7.000 LoC in kernel + 50.000 in userspace (source: http://www.slideshare.net/ennael/2013-kernel-recipesnftables http://www.slideshare.net/ennael/2013-kernel-recipesnftables) Also note that it's not really a "virtual machine" comparable with java, this is how the developers actually describe it In a nutshell, nftables provides a pseudo-state machine with 4 general purpose registers of 128 bits and 1 specific purpose register to store verdicts. This pseudo-machine comes with an extensible instruction set, a.k.a. "expressions" in the nftables jargon. The expressions included in this patch provide the basic functionality, they are: * bitwise: to perform bitwise operations. * byteorder: to change from host/network endianess. * cmp: to compare data with the content of the registers. * counter: to enable counters on rules. * ct: to store conntrack keys into register. * exthdr: to match IPv6 extension headers. * immediate: to load data into registers. * limit: to limit matching based on packet rate. * log: to log packets. * meta: to match metainformation that usually comes with the skbuff. * nat: to perform Network Address Translation. * payload: to fetch data from the packet payload and store it into registers. * reject (IPv4 only): to explicitly close connection, eg. TCP RST. Using this instruction-set, the userspace utility 'nft' can transform the rules expressed in human-readable text representation (using a new syntax, inspired by tcpdump) to nftables bytecode.
- krakensden 13y agoI don't even think this is the first vm in the networking part of the kernel.
- netcraft 13y agoDo we know what the next LTS kernel release will be and when it is expected?
- sp332 13y agoGreg Kroah-Hartman hasn't mentioned it on his blog. He's doing one per year, but he didn't announce last years' until it had been out for a month already. http://www.kroah.com/log/blog/2013/08/04/longterm-kernel-3-dot-10/ http://www.kroah.com/log/blog/2013/08/04/longterm-kernel-3-d... Edit: more official link https://www.kernel.org/category/releases.html https://www.kernel.org/category/releases.html
- higherpurpose 13y agoThat will probably be 3.15.
- dimadima 13y agoThis is so nice!
- shimon_e 13y agoGreat finally some drivers I've wanted have been merged. http://kernelnewbies.org/Linux_3.13-DriversArch http://kernelnewbies.org/Linux_3.13-DriversArch
- contingencies 13y agoFor the block layer update, before anyone gets excited like I did, the paper actually suggests it's not useful to most people at all with current era hardware. In this paper, we have established that the current design of the Linux block layer does not scale beyond one million IOPS per device. This is sucient for today's SSD, but not for tomorrow's. We proposed a new design for the Linux block layer. This design is based on two levels of queues in order to reduce contention and promote thread locality. Our experiments have shown the superiority of our design and its scalability on multi-socket systems. Our multiqueue design leverages the new capabilities of NVM-Express or high-end PCI-E devices, while still providing the common interface and convenience features of the block layer.
- nly 13y agoIndeed. There were even some benchmarks on Phoronix a while ago that hinted that these block layer changes had resulted in some performance regressions. Anyone know if these regressions were tracked down and fixed?
- mbjorling 13y agoThis statement should be seen as there's no way to scale the old block layer to new devices. To current SSDs, its already useful, in that it decreases latency and CPU usage for current generation of drives. It's currently only enabled using the virtioblk driver. But there's work underway to make the scsi layer and all the others drivers use it (already patches out for the mtip and nvme driver).
- contingencies 13y agoThanks for the clarification. IIRC you are a co-author of the paper, so perhaps you can answer a follow-up question. What kind of latency or CPU usage change should a typical modern SSD on an amd64 class multicore processor observe when using the new block layer? Also correct me if I'm wrong but since Linux aggressively caches already and SSDs are already way faster than older drives for normal (ie. ~random access) loads, plus RAM is cheap and plentiful these days, I am guessing that very few applications will honestly be IO-bound enough to see that benefit.
- arielweisberg 13y agoGreat to see the NUMA balancing in. My question has always been what workloads require NUMA balancing in the first place? If I present the kernel with the same number of threads as cores and keep all data local to a thread would the existing approach of allocating on the NUMA node the thread is running on have been enough?
- fsaintjacques 13y agoWhen your process use more than half of the memory, e.g. DBs.
- arielweisberg 13y agoThe kernel will balance loaded threads across cores so with a policy of allocating off the local NUMA node you actually end up with balanced allocations in practice if you run shared nothing thread per core. In memory databases are my day job so I am pretty interested in cases where things go south because memory isn't balanced. To date it appears like no special actions were necessary, stuff just ends up balanced across nodes. That's why it would be great of someone could characterize when balancing is necessary outside of obvious cases like allocating an entire buffer pool from one thread.
- pmenon 13y agoYou should take a look at "OLTP on Hardware Islands" in VLDB 2012: http://vldb.org/pvldb/vol5/p1447_danicaporobic_vldb2012.pdf http://vldb.org/pvldb/vol5/p1447_danicaporobic_vldb2012.pdf
- MichaelGG 13y agoIt applies to more than DBs. Running VoIP software, we found that just by setting CPU affinity, we got a major increase in performance. The software in question, FreeSWITCH, is inanely-threaded in a misguided believe that "more threads=more performance" (well that, and it's also just easier to program). When there's thousands of threads going, keeping them and their data local to one NUMA node or less really makes a huge difference.
- saosebastiao 13y agoI noticed a couple of commits regarding btrfs. Can anybody summarize them to someone that doesn't know anything about kernels and very little about file systems?
- dsr_ 13y agoThe big improvements to btrfs came in 3.12. The 3.13 changes are rather minor: a mount option to specify the maximum delay before committing writes to storage (default 30 seconds, no maximum, warning for 300+ seconds -- make sure you have battery coverage for whatever this is set to plus a few seconds, and try not to crash...) a mount option for emergency use that will force the rebuild of the UUID tree userspace tools that read FIEMAP_EXTENT_SHARED can now use that on btrfs; no functionality change, really, just making the info available in the same way that ocfs2 does it.
- wtbob 13y ago> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=6e9fa2c8a630e6d0882828012431038abce285b9 http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.g... I really, really, really wish that the Linux CSPRNG would quit having its flaws papered over. A fellow submitted a patch to implement the Fortuna CSPRNG years ago, and it wasn't accepted because of a misguided belied in entropy estimation. I'm not saying that Fortuna is the One True CSPRNG—it's not—but any clean design would be preferable to the current Rube Goldberg mechanism. I'm pretty sure that /dev/random as it currently stands is secure enough, but 'pretty sure' isn't very reassuring.
- forgottenpass 13y agoHave you brought any of this up on lkml?
- wtbob 13y agoJean-Luc Cooke tried to submit a patch back in September 2004 (yes, a decaded ago) and was shot down. I don't know if anyone has tried since.
- zokier 13y agoThe current "Rude-Goldberg mechanism" has one significant advantage over anything that could replace it: it has been reviewed by experts.
- stouset 13y agoThe implementation may have. But the cryptography certainly hasn't (otherwise, it wouldn't be changing so often).
- sandGorgon 13y ago3.13 is the first release with full opensource Intel Broadwell drivers - switched off by default though. So by the time Broadwell actually lands in Q3-Q4, the kernel should have stabilized nicely. Perfect for a cheap Steambox.
- higherpurpose 13y agoI doubt Broadwell will have any performance improvements over Haswell at the same price points. You're probably better off buying a cheaper Haswell then if you want a cheap Steambox. Intel has pretty much given up on improving overall performance of its chips. IVB was only 10 percent improvement over SNB, and Haswell only 5 percent over IVB, and the difference in price between new-gen and last-gen is probably more like 30 percent.
- sandGorgon 13y agoIntel’s Ben Widawsky, who works on Intel’s Linux graphics driver efforts, says that “Broadwell graphics bring some of the biggest changes we’ve seen on the execution and memory management side of the GPU… [the changes] dwarf any other silicon iteration during my tenure, and certainly can compete with the likes of the gen3->gen4 changes.” This combined with the fully opensource Linux driver for Broadwell would mean that there is a very high chance it would perform significantly better.
- wmf 13y agoBroadwell will perform better, but Intel may decide to charge more for it. That's what they did with Ivy Bridge and Haswell.
- delroth 13y agoSingle core performance in CPU intensive applications really improved between Nehalem/SNB/IVB/Haswell actually. [1] is a benchmark of the Dolphin Emulator (mostly single threaded performance) on a variety of recent CPUs that shows this. [1] https://docs.google.com/spreadsheet/ccc?key=0AunYlOAfGABxdFQ0UzJyTFAxbzZhYWtGcGwySlRFa1E#gid=0 https://docs.google.com/spreadsheet/ccc?key=0AunYlOAfGABxdFQ...
- pshc 13y agoAnyone heard any news on Google's user mode thread[1] kernel syscalls? I was really excited for that when it was announced but haven't heard a peep about it since. [1]http://youtube.com/watch?v=KXuZi9aeGTw http://youtube.com/watch?v=KXuZi9aeGTw
- nopaste7 13y agoThey never showed up on LKML.
- gcr 13y agoIs this similar to the old "Scheduler Activations" idea? http://homes.cs.washington.edu/~tom/pubs/sched_act.pdf http://homes.cs.washington.edu/~tom/pubs/sched_act.pdf I'd love to see that idea wind up in a mainstream kernel.
- twic 13y agoIt was in FreeBSD! http://www.freebsd.cz/kse/index.html http://www.freebsd.cz/kse/index.html For some reason, it didn't work out, and FreeBSD switched back to conventional threads, around 7.0, i think.