7 ms·

Hist Triggers in Linux 4.7

98 points by helper 10y ago

cyphar 10y ago
Glad to see that GNU/Linux is getting a DTrace-equivalent. Sucks that it's all a bunch of string and staples rather than a single ioctl interface like DTrace -- but hey, that's the Linux way.
- wkz 10y ago
  Yeah the UX is user hostile to say the least. Shameless plug: I've started working on a more coherent tool. Inspired by awk and DTrace, but it is still in its infancy. https://wkz.github.io/ply/ https://wkz.github.io/ply/
- netneutralish 10y ago
  Reminds me of developing countries that leapfrog industrialized G8 countries in certain technologies, like skipping landline deployments and going straight for wireless networks.
  - qb45 10y ago
    OTOH, industrialized countries leapfrog them in certain other technologies, like wired networks. I'm not sure if wireless would scale to the traffic capacity of our developed and industrialized telephone and TV cabling, especially in dense cities.
  - venomsnake 10y ago
    That is not leapfrogging. To have decent internet access that can power economy - you need some form of wire to the home. S/FTP,coaxial, twisted pair or fiber.
    - mikecb 10y ago
      At the moment.
      - venomsnake 10y ago
        Laws of physics tend to change infrequently. Sharing spectrum means low speed for all.
        lisivka 10y ago
        So beams must be directional, to avoid sharing of spectrum with all, but only with some.
        cyphar 10y ago
        The problem is that broadcasts are exactly that: broadcasting. They transmit in all directions. While you have bandwidth constraints in wires, they're all localised to the wire. Even parabolic dishes (which make getting a signal a huge PITA) still aren't as localised as cables. Wired connections get a bad rap, but they're actually a much more preferable (and scalable) medium for transmission than radio waves. Not to mention that "bidirectional beams" is another way of saying "noise".
        dmm 10y ago
        A fiber of glass is the ultimate directional antenna.
        mikecb 10y ago
        I'm not suggesting that, but that economics changes things much more rapidly, c.f. 5G, which may roll out multi-gigabit speeds to consumers far before landline.
        venomsnake 10y ago
        And how you will provide multi gigabit speeds to a 100 people in close proximity simultaneously? There is the question of throughput of the whole network.
    - anonbanker 10y ago
      why can't TCP/BATMAN be defined as "decent internet access"?
  - cm3 10y ago
    It's a shared medium and can never reach the qualities of wired networking, though it might get better enough that we don't notice it as much as we do now. This reminds me of efforts like xDSL G.fast. People only talk about approaching the speeds of common fiber configurations, but miss the point of very low latency in FTTH and easy upgrade to, say, 10 or 40 Gbs, whereas that would require a technological enhancement in DSL or CABLE. With fibre it would only require switching out the "modem" and network gear in the local uplink station (hut).
    - deleted 10y ago
      [deleted]
    - Tiksi 10y ago
      Eh, you're not gonna get 10 or 40Gbps over most FTTH installs. Most run GPON with beam splitters which maxes out at 2.5Gbps/line. To get 10/40/100 you need OS1/2 singlemode or OM3/4 multimode fiber which I've never seen used in residential/small business FTTP setups.
      - cm3 10y ago
        GPON isn't used everywhere, but it's used a lot, yes. Still, getting 2.5Gbps, exclusively, with low latency isn't possible with the other connections (yet?).
        Tiksi 10y ago
        No doubt fiber is definitely the way to go if it's an option. Other than fiber, the closest you'll get is DOCSIS 3 theoretically runs at ~1gig downstream. Anecdotally, I've had Fios for a couple years and get faster than advertised speeds consistently and never had an outage, so I'm all about fiber. I think it's due time that having an actual fiber drop becomes as common place as having a phone line or cable line. With how much people use the internet these days, it makes sense to have it's own connection instead of piggybacking on whatever physical connection people already have.
cm3 10y ago
Sorry, this is going to be rather long for HN, but it's been bugging me for a while now, and this is a good opportunity to talk about it. Can't enhanced BPF do this too? Yes, if you program it to. I feel like we've been waiting for advanced tracing in Linux for years, and now two busses have arrived at the same time. This overlap concern was raised and discussed on lkml, and it was eventually deemed that hist triggers was different enough to be included. This is what's wrong with Linux kernel development. We have pluggable subsystems like I/O scheduling or CPU scheduling where BFQ or BFS are not mainlined and have to live as external patchsets. Then we have more than a few competing tracing/probing frameworks all somehow magically in torvalds/linux.git despite their overlap and unfinished (partly experimental) state. I used to think that Linux kernel development is about who comes up with good, working code, but more and more I'm starting to get the impression it's that you need to have the right connections for a competing implementation to land. I won't argue that linux needs DTrace or a similarly coherent solution, although I absolutely believe that. However, seeing the impossible to explain difference in what experimental, half-implemented code gets mainlined and which widely used patches have to live outside the kernel, I cannot find another explanation than requiring the right street cred and connections for competing implementations to land. This is especially inexplicable for BFQ because it's a in an already pluggable system. To be clear, I'm not referring to projects like GRsec whose maintainers are unwilling to split it into smaller chunks, and I'm primarily talking about simple things like BFQ, which has solved my global lockups during large (long) USB mass storage flushing. With BFQ, it's at most the application that called fsync() and the rest of the applications are still responsive. With all that being said, the evolutionary style of kernel subsystem development doesn't work for everything, especially not for a DTrace replacement, which would do well by having a single well thought out design and broad availability of probes. While I can see that maybe people are afraid of Oracle patents, DTrace isn't complicated enough that a similar effort couldn't be implemented while keeping the same benefits. A simple transpiler for DTrace scripts can be more than sufficient if the syntax is also preferred to be different. In summary, I don't understand how we have so many tracing system alternatives, experimental and unfinished as well, while a pluggable subsystem has only plugins personally accepted by that subsystem's maintainer.
- gajjanag 10y ago

98 points by helper 10y ago

Glad to see that GNU/Linux is getting a DTrace-equivalent. Sucks that it's all a bunch of string and staples rather than a single ioctl interface like DTrace -- but hey, that's the Linux way.

wkz 10y ago
Yeah the UX is user hostile to say the least. Shameless plug: I've started working on a more coherent tool. Inspired by awk and DTrace, but it is still in its infancy. https://wkz.github.io/ply/ https://wkz.github.io/ply/
netneutralish 10y ago
Reminds me of developing countries that leapfrog industrialized G8 countries in certain technologies, like skipping landline deployments and going straight for wireless networks.
- qb45 10y ago
  OTOH, industrialized countries leapfrog them in certain other technologies, like wired networks. I'm not sure if wireless would scale to the traffic capacity of our developed and industrialized telephone and TV cabling, especially in dense cities.
- venomsnake 10y ago
  That is not leapfrogging. To have decent internet access that can power economy - you need some form of wire to the home. S/FTP,coaxial, twisted pair or fiber.
  - mikecb 10y ago
    At the moment.
    - venomsnake 10y ago
      Laws of physics tend to change infrequently. Sharing spectrum means low speed for all.
      - lisivka 10y ago
        So beams must be directional, to avoid sharing of spectrum with all, but only with some.
        cyphar 10y ago
        The problem is that broadcasts are exactly that: broadcasting. They transmit in all directions. While you have bandwidth constraints in wires, they're all localised to the wire. Even parabolic dishes (which make getting a signal a huge PITA) still aren't as localised as cables. Wired connections get a bad rap, but they're actually a much more preferable (and scalable) medium for transmission than radio waves. Not to mention that "bidirectional beams" is another way of saying "noise".
        dmm 10y ago
        A fiber of glass is the ultimate directional antenna.
      - mikecb 10y ago
        I'm not suggesting that, but that economics changes things much more rapidly, c.f. 5G, which may roll out multi-gigabit speeds to consumers far before landline.
        venomsnake 10y ago
        And how you will provide multi gigabit speeds to a 100 people in close proximity simultaneously? There is the question of throughput of the whole network.
  - anonbanker 10y ago
    why can't TCP/BATMAN be defined as "decent internet access"?
- cm3 10y ago
  It's a shared medium and can never reach the qualities of wired networking, though it might get better enough that we don't notice it as much as we do now. This reminds me of efforts like xDSL G.fast. People only talk about approaching the speeds of common fiber configurations, but miss the point of very low latency in FTTH and easy upgrade to, say, 10 or 40 Gbs, whereas that would require a technological enhancement in DSL or CABLE. With fibre it would only require switching out the "modem" and network gear in the local uplink station (hut).
  - deleted 10y ago
    [deleted]
  - Tiksi 10y ago
    Eh, you're not gonna get 10 or 40Gbps over most FTTH installs. Most run GPON with beam splitters which maxes out at 2.5Gbps/line. To get 10/40/100 you need OS1/2 singlemode or OM3/4 multimode fiber which I've never seen used in residential/small business FTTP setups.
    - cm3 10y ago
      GPON isn't used everywhere, but it's used a lot, yes. Still, getting 2.5Gbps, exclusively, with low latency isn't possible with the other connections (yet?).
      - Tiksi 10y ago
        No doubt fiber is definitely the way to go if it's an option. Other than fiber, the closest you'll get is DOCSIS 3 theoretically runs at ~1gig downstream. Anecdotally, I've had Fios for a couple years and get faster than advertised speeds consistently and never had an outage, so I'm all about fiber. I think it's due time that having an actual fiber drop becomes as common place as having a phone line or cable line. With how much people use the internet these days, it makes sense to have it's own connection instead of piggybacking on whatever physical connection people already have.

cm3 10y ago

Sorry, this is going to be rather long for HN, but it's been bugging me for a while now, and this is a good opportunity to talk about it. Can't enhanced BPF do this too? Yes, if you program it to. I feel like we've been waiting for advanced tracing in Linux for years, and now two busses have arrived at the same time. This overlap concern was raised and discussed on lkml, and it was eventually deemed that hist triggers was different enough to be included. This is what's wrong with Linux kernel development. We have pluggable subsystems like I/O scheduling or CPU scheduling where BFQ or BFS are not mainlined and have to live as external patchsets. Then we have more than a few competing tracing/probing frameworks all somehow magically in torvalds/linux.git despite their overlap and unfinished (partly experimental) state. I used to think that Linux kernel development is about who comes up with good, working code, but more and more I'm starting to get the impression it's that you need to have the right connections for a competing implementation to land. I won't argue that linux needs DTrace or a similarly coherent solution, although I absolutely believe that. However, seeing the impossible to explain difference in what experimental, half-implemented code gets mainlined and which widely used patches have to live outside the kernel, I cannot find another explanation than requiring the right street cred and connections for competing implementations to land. This is especially inexplicable for BFQ because it's a in an already pluggable system. To be clear, I'm not referring to projects like GRsec whose maintainers are unwilling to split it into smaller chunks, and I'm primarily talking about simple things like BFQ, which has solved my global lockups during large (long) USB mass storage flushing. With BFQ, it's at most the application that called fsync() and the rest of the applications are still responsive. With all that being said, the evolutionary style of kernel subsystem development doesn't work for everything, especially not for a DTrace replacement, which would do well by having a single well thought out design and broad availability of probes. While I can see that maybe people are afraid of Oracle patents, DTrace isn't complicated enough that a similar effort couldn't be implemented while keeping the same benefits. A simple transpiler for DTrace scripts can be more than sufficient if the syntax is also preferred to be different. In summary, I don't understand how we have so many tracing system alternatives, experimental and unfinished as well, while a pluggable subsystem has only plugins personally accepted by that subsystem's maintainer.

gajjanag 10y ago