3 ms·

"...you have naive low-level "mechanism" models making the mundane non-edge-case decisions, with no awareness of the edge-cases; and then you have smarter case-

"...you have naive low-level "mechanism" models making the mundane non-edge-case decisions, with no awareness of the edge-cases; and then you have smarter case-specific "policy" models, trained to recognize when the conditions apply for some particular policy to be enforced, whose job is to change/override the decision the low-level agent makes, by MITMing ("subsuming") the lower-level model." It seems like handling the edge cases at "the top" has the inherent problem that in a complex system, the edge-cases will grow faster than the normal cases. Moreover, the edge-cases are more or less the sum of what the normal approximators can't handle, "the top" has to be aware of every lower part and compensate for it, meaning "the top" will grow in an unmanageable way. Also, a higher function telling a lower function how to handle errors isn't quite following the handle errors at the top approach, since this implies lower level systems do have error handlers.

derefr 7y ago
> this implies lower level systems do have error handlers I'd picture it like the lower-level system having a try{} block (with no corresponding catch clauses), the postdominant block of which would be where you'd resume from if there'd an error within that block that gets caught and resolved. The mechanism knows that the policy needs to have a clean "cut point" (in AOP terms) to subsume it by; but the mechanism doesn't need to know what the policy is going to do at that cut point. > Moreover, the edge-cases are more or less the sum of what the normal approximators can't handle, "the top" has to be aware of every lower part and compensate for it The normal case isn't having A, B, C, and D all overriding E's behavior at different scopes. It's possible and useful to have that capability, but it'd be uncommon to use it. More common with subsumption would be that {B, C, D}—all siblings—override A's behavior in complementary ways for different edge-cases; and then {E, F, G}—all siblings—might override B's behavior, including overriding the inputs it uses to compute its override values for A, but not including the logic it uses to compute that override. Or, in short: you subsume your "direct reports" by changing their goals and modifying their work products; but you don't micromanage down the org-chart. (Except when micro-managing gets you something.) Also keep in mind, we're not talking about a tree with one central top-level agent. Robotics systems don't work like that, for several good reasons. It's better viewed upside-down: each one low-level agent is the root of a tree of patches to its behavior, where 1 gets overridden by 2 and 3, 2 gets overridden by {4, 5, 6}, etc. Notice that a node like 5, that determines how to bias the input into 2, isn't modelling the behavior of 1; it's just trying to directly influence 2 on 2's own terms, with everything below 2 being a black box. The full system is, then, a forest of agents, where all agents are sourcing shared inputs and are capable of observing any other tree-node's outputs; but there are no agents shared between different trees in the forest, only distinct per-tree copies with similar functions, trying to do the same job while embedded in different local environments. A good example of subsumption—without any Machine Learning to get in the way—is how a subway system works: 1. Each wheel has a low-level agent in its motor firmware that "wants" to calibrate power output so that the train will continue along the track at a constant speed. It is built as a simple control system: it can see the train's speed, and it will provide more or less power if the train is going faster or slower than its reference. (As well, there's another low-level control system attached to the brakes, that also observe the train's current speed and set-point speed, and will engage the brakes as long as the current speed is much higher than the reference speed.) 2. There's a higher-level agent, still one per motor, responsible for not colliding with things on the track (e.g. other trains.) It can observe when it enters contested sections of track (and, more recently, can read off a front-facing depth-sensor); and will bias the reference-point values for the lower-level wheel-motor and brake systems, so that they "seek to stop" more fervently, the closer the train gets to any obstacle. You can sort of see this like an agent that stands between a gas pedal in a car and the actual signal to the gearbox; but rather than influencing the signal's output for everything, it's an individualized influence for each wheel. (This is important in robotic systems, because hardware is built to varying tolerances—different wheels on the same train can require different amounts of power to generate the same torque!) 3. There's a higher-level agent still—again, still one per motor!—responsible for stopping at stations, which does this by biasing the collision agent to temporarily think there's another train just far-enough away that it must stop where the station is in order to avoid it. Note how the stopping-at-stations agent doesn't know how the collision-avoidance agent actually does collision avoidance. It doesn't know it's overriding a wheel-power agent further down. It just "delegates" the requirement, by changing the requirements for its subsumed subordinate agent. And, yes, the stopping-at-stations agent wouldn't get a very good output without some careful fine-tuning of where to produce the illusionary train in the collision-detection agent's input model. That's why ML is used with these systems: to tune inputs like that, in accordance with what parameter the upper-level agent can emit that will best make the lower-level agent do what it wants to accomplish its own goal, without actually needing to understand how the lower-level agent is accomplishing that goal—only the ability to observe how closely the train manages to line up with the station. If you're wondering why it's all duplicated so much: consider how this system reacts if there's a bug in one motor's sensors that makes it observe the wrong input values. It'll start misbehaving as a result. Certainly, with some kind of online-learning ML in play, the agent above it might realize it needs to apply some sort of gamma curve to ramp the input/output from the agent to get the results it expects. But even without ML, as long as the other wheels are all functioning, they'll fight the misbehaving wheel, and they'll likely win. I.e., even if one wheel-motor is going full tilt because it thinks the train is stopped, the other ones will notice that the train is actually going too fast, and apply their brakes. It is theorized that this is, in some ways, a model for how synapses work, especially for connections projecting between different parts of your brain (e.g. your neocortex vs. paleocortex.)
- im3w1l 7y ago
  > 3. There's a higher-level agent still—again, still one per motor!—responsible for stopping at stations, which does this by biasing the collision agent to temporarily think there's another train just far-enough away that it must stop where the station is in order to avoid it. This sounds clever in the worst sense of the word. Introducing dependencies and complexities for no reason. Want to increase safety margins from trains? Suddenly it doesn't stop correctly at stations any more.
  - derefr 7y ago
    You don't change internal variables; you change top-level goal conditions (in imperative programming terms, the "test suite"), and retrain the system, and all the internal variables change together to find a new equilibrium. That's kind of... how ML works. It's how brains seem to work, too. (Think about the order in which you learn to do something like typing. Think about what "levels" of that learning that you need to practice over again, if you switch to a different keyboard layout.) Also, to be clear, this is already what subway systems do. And roombas. And a bunch of other types of robots. It turns out to work better, in practice, than the alternative (= networks of leaf-node controllers with individual top-down-issued config parameters, like computer networks are.) Although, they don't do it because it's an easy system to develop in. Subsumption is operationally robust—it continues to do the right thing in the face of failure of components, or even "subverted" components producing malicious input. This is a little important when you're a Mars rover whose firmware chips could be hit by gamma rays; but it's even more important when you're an organic lifeform with parasites constantly attempting to hijack your nervous system to their own ends (e.g. the Toxoplasma bacterium and the Cordyceps fungi, two cases where the host isn't strong enough to resist the hijacking. In the 99.999% of other cases of parasitic onslaught that you don't hear about, the host does still manage to "do the right thing" despite the parasite.)
    - im3w1l 7y ago
      I didn't mean it to be a critique of the concept of subsumption itself, only your specific example. For instance having the train avoidance system provide a reference speed to the wheel controllers seems solid. Having the station stopper create fake trains (or having the train avoider create fake stations) does not seem robust. Would be better to have the station stopper set reference speed, and give the train avoider the authority to override it.

derefr 7y ago

> this implies lower level systems do have error handlers I'd picture it like the lower-level system having a try{} block (with no corresponding catch clauses), the postdominant block of which would be where you'd resume from if there'd an error within that block that gets caught and resolved. The mechanism knows that the policy needs to have a clean "cut point" (in AOP terms) to subsume it by; but the mechanism doesn't need to know what the policy is going to do at that cut point. > Moreover, the edge-cases are more or less the sum of what the normal approximators can't handle, "the top" has to be aware of every lower part and compensate for it The normal case isn't having A, B, C, and D all overriding E's behavior at different scopes. It's possible and useful to have that capability, but it'd be uncommon to use it. More common with subsumption would be that {B, C, D}—all siblings—override A's behavior in complementary ways for different edge-cases; and then {E, F, G}—all siblings—might override B's behavior, including overriding the inputs it uses to compute its override values for A, but not including the logic it uses to compute that override. Or, in short: you subsume your "direct reports" by changing their goals and modifying their work products; but you don't micromanage down the org-chart. (Except when micro-managing gets you something.) Also keep in mind, we're not talking about a tree with one central top-level agent. Robotics systems don't work like that, for several good reasons. It's better viewed upside-down: each one low-level agent is the root of a tree of patches to its behavior, where 1 gets overridden by 2 and 3, 2 gets overridden by {4, 5, 6}, etc. Notice that a node like 5, that determines how to bias the input into 2, isn't modelling the behavior of 1; it's just trying to directly influence 2 on 2's own terms, with everything below 2 being a black box. The full system is, then, a forest of agents, where all agents are sourcing shared inputs and are capable of observing any other tree-node's outputs; but there are no agents shared between different trees in the forest, only distinct per-tree copies with similar functions, trying to do the same job while embedded in different local environments. A good example of subsumption—without any Machine Learning to get in the way—is how a subway system works: 1. Each wheel has a low-level agent in its motor firmware that "wants" to calibrate power output so that the train will continue along the track at a constant speed. It is built as a simple control system: it can see the train's speed, and it will provide more or less power if the train is going faster or slower than its reference. (As well, there's another low-level control system attached to the brakes, that also observe the train's current speed and set-point speed, and will engage the brakes as long as the current speed is much higher than the reference speed.) 2. There's a higher-level agent, still one per motor, responsible for not colliding with things on the track (e.g. other trains.) It can observe when it enters contested sections of track (and, more recently, can read off a front-facing depth-sensor); and will bias the reference-point values for the lower-level wheel-motor and brake systems, so that they "seek to stop" more fervently, the closer the train gets to any obstacle. You can sort of see this like an agent that stands between a gas pedal in a car and the actual signal to the gearbox; but rather than influencing the signal's output for everything, it's an individualized influence for each wheel. (This is important in robotic systems, because hardware is built to varying tolerances—different wheels on the same train can require different amounts of power to generate the same torque!) 3. There's a higher-level agent still—again, still one per motor!—responsible for stopping at stations, which does this by biasing the collision agent to temporarily think there's another train just far-enough away that it must stop where the station is in order to avoid it. Note how the stopping-at-stations agent doesn't know how the collision-avoidance agent actually does collision avoidance. It doesn't know it's overriding a wheel-power agent further down. It just "delegates" the requirement, by changing the requirements for its subsumed subordinate agent. And, yes, the stopping-at-stations agent wouldn't get a very good output without some careful fine-tuning of where to produce the illusionary train in the collision-detection agent's input model. That's why ML is used with these systems: to tune inputs like that, in accordance with what parameter the upper-level agent can emit that will best make the lower-level agent do what it wants to accomplish its own goal, without actually needing to understand how the lower-level agent is accomplishing that goal—only the ability to observe how closely the train manages to line up with the station. If you're wondering why it's all duplicated so much: consider how this system reacts if there's a bug in one motor's sensors that makes it observe the wrong input values. It'll start misbehaving as a result. Certainly, with some kind of online-learning ML in play, the agent above it might realize it needs to apply some sort of gamma curve to ramp the input/output from the agent to get the results it expects. But even without ML, as long as the other wheels are all functioning, they'll fight the misbehaving wheel, and they'll likely win. I.e., even if one wheel-motor is going full tilt because it thinks the train is stopped, the other ones will notice that the train is actually going too fast, and apply their brakes. It is theorized that this is, in some ways, a model for how synapses work, especially for connections projecting between different parts of your brain (e.g. your neocortex vs. paleocortex.)

im3w1l 7y ago
> 3. There's a higher-level agent still—again, still one per motor!—responsible for stopping at stations, which does this by biasing the collision agent to temporarily think there's another train just far-enough away that it must stop where the station is in order to avoid it. This sounds clever in the worst sense of the word. Introducing dependencies and complexities for no reason. Want to increase safety margins from trains? Suddenly it doesn't stop correctly at stations any more.
- derefr 7y ago
  You don't change internal variables; you change top-level goal conditions (in imperative programming terms, the "test suite"), and retrain the system, and all the internal variables change together to find a new equilibrium. That's kind of... how ML works. It's how brains seem to work, too. (Think about the order in which you learn to do something like typing. Think about what "levels" of that learning that you need to practice over again, if you switch to a different keyboard layout.) Also, to be clear, this is already what subway systems do. And roombas. And a bunch of other types of robots. It turns out to work better, in practice, than the alternative (= networks of leaf-node controllers with individual top-down-issued config parameters, like computer networks are.) Although, they don't do it because it's an easy system to develop in. Subsumption is operationally robust—it continues to do the right thing in the face of failure of components, or even "subverted" components producing malicious input. This is a little important when you're a Mars rover whose firmware chips could be hit by gamma rays; but it's even more important when you're an organic lifeform with parasites constantly attempting to hijack your nervous system to their own ends (e.g. the Toxoplasma bacterium and the Cordyceps fungi, two cases where the host isn't strong enough to resist the hijacking. In the 99.999% of other cases of parasitic onslaught that you don't hear about, the host does still manage to "do the right thing" despite the parasite.)
  - im3w1l 7y ago
    I didn't mean it to be a critique of the concept of subsumption itself, only your specific example. For instance having the train avoidance system provide a reference speed to the wheel controllers seems solid. Having the station stopper create fake trains (or having the train avoider create fake stations) does not seem robust. Would be better to have the station stopper set reference speed, and give the train avoider the authority to override it.