4 ms·

Reminds me after the GFC how some of the rules that were brought in were around banning automated real time trading systems. This was in some similar ways recog

by zmmmmm 6y ago

Reminds me after the GFC how some of the rules that were brought in were around banning automated real time trading systems. This was in some similar ways recognizing that automated algorithmic treatment can have extremely harmful side effects - even when it successfully executes the goals of its owner (for the stock market - once a certain threshold is breached, get me out of the market as quickly as possible - as an individual, its exactly what I want, for the overall market it is a disaster if everyone does it suddenly).

ocdtrekkie 6y ago
Indeed. Generally along with blaming the algorithm, as we start using AI, the problem is the algorithm is no longer even understandable in many cases. Tech companies should be able to explain and demonstrate the logic their systems use. These algorithms should probably be public. And any system which cannot be transparently explained should be shut down.
- dane-pgp 6y ago
  > These algorithms should probably be public. And any system which cannot be transparently explained should be shut down. But what would even count as an explanation, and what does a "public" algorithm reveal? There is no line of code that says "if (video.content == extreme) { show_to(EVERYONE) }". A lot of the dangers are possible from a simple algorithm which merely performs A/B testing on whether a certain (randomly chosen, at first) video increases the amount of time a user spends on a site. You could pass a law against A/B testing, or require companies to provide deliberately bad suggestions to make their users frustrated, but I'm not sure if that is a proportionate legislative response.
  - zmmmmm 6y ago
    > But what would even count as an explanation That sort of gets to the heart of it - saying "it's impossible to explain my algorithm because it is so complicated" no longer cuts it. It's the same as if I said to the FDA "This drug works in our testing for the indicated purpose but we have no idea if there are side effects" they aren't go to say "well, OK then, you can sell it". They'll say "come back when you have tested for all possible side effects and you can precisly define its behavior" Currently Facebook, Google et al, will measure a narrow set of metrics to define the success of their algorithms (primarily relating to $$$) and then declare victory based on empirical optimisation of those metrics. They have to show that not only does the algorithm work for its stated purpose, it doesn't have undesirable side effects. They could do massive testing to demonstrate a huge range of side effects are not present. That would be similar to and probably as expensive as drug development. But an easier way to do that is to transparently explain what the algorithms do so that side effects can be predicted. If that is actually impossible for a given algorithm - well, maybe that algorithm shouldn't be used on the public at all.
  - ocdtrekkie 6y ago
    If a company can't explain why a given user was shown a given video, at a given time, that system is faulty and should be prohibited. Which is to say, there's nothing particularly wrong with complicated rules and scoring systems making recommendations, but things like black box neural networks should not be able to be used for this stuff. Let me give you an example of a good system: Spam filters not operated by Google. Algorithms decide whether or not mail gets through my corporate mail filter. It determines it based on a score, which is tallied from a set of rules. And you can drill down and see the score a spam email received, and then you can see the rules and factors that created that score. As a user, you can even generally see this, because the results are included in the message's headers in your inbox. And if you're the admin, you can then adjust those rules to fix errant behavior. That's how a recommendation system should work. A system which can't be analyzed in that matter should not exist, and the rules and scoring applied to such systems should be disclosed in some sort of header.
    - cercatrova 6y ago
      In many cases making a neural network explainable is literally impossible. Logic based AI failed in the 90s and only though neural nets have we gotten to human and superhuman level advances. If you want explainability it's going to require giving up a lot of innovation that could help people. Especially when we ourselves can't explain our own behavior psychologically.
      - ocdtrekkie 6y ago
        > In many cases making a neural network explainable is literally impossible. Then it should be illegal to use on consumer websites. It's that simple. > If you want explainability it's going to require giving up a lot of innovation that could help people. Four people died in the US Capitol because of this "innovation". It's time to stop pretending technology doesn't cause more harm than good in these cases.
        cercatrova 6y ago
        I don't like when people try to stop or slow down technological innovation. Technologies are just tools, a nuclear bomb could just as easily have been made into a nuclear power plant. But to fully restrict them shows a lack of foresight. Yes, recommender systems can be used for bad, as yesterday, but they can also be good.

by zmmmmm 6y ago

ocdtrekkie 6y ago

Indeed. Generally along with blaming the algorithm, as we start using AI, the problem is the algorithm is no longer even understandable in many cases. Tech companies should be able to explain and demonstrate the logic their systems use. These algorithms should probably be public. And any system which cannot be transparently explained should be shut down.

dane-pgp 6y ago
> These algorithms should probably be public. And any system which cannot be transparently explained should be shut down. But what would even count as an explanation, and what does a "public" algorithm reveal? There is no line of code that says "if (video.content == extreme) { show_to(EVERYONE) }". A lot of the dangers are possible from a simple algorithm which merely performs A/B testing on whether a certain (randomly chosen, at first) video increases the amount of time a user spends on a site. You could pass a law against A/B testing, or require companies to provide deliberately bad suggestions to make their users frustrated, but I'm not sure if that is a proportionate legislative response.
- zmmmmm 6y ago
  > But what would even count as an explanation That sort of gets to the heart of it - saying "it's impossible to explain my algorithm because it is so complicated" no longer cuts it. It's the same as if I said to the FDA "This drug works in our testing for the indicated purpose but we have no idea if there are side effects" they aren't go to say "well, OK then, you can sell it". They'll say "come back when you have tested for all possible side effects and you can precisly define its behavior" Currently Facebook, Google et al, will measure a narrow set of metrics to define the success of their algorithms (primarily relating to $$$) and then declare victory based on empirical optimisation of those metrics. They have to show that not only does the algorithm work for its stated purpose, it doesn't have undesirable side effects. They could do massive testing to demonstrate a huge range of side effects are not present. That would be similar to and probably as expensive as drug development. But an easier way to do that is to transparently explain what the algorithms do so that side effects can be predicted. If that is actually impossible for a given algorithm - well, maybe that algorithm shouldn't be used on the public at all.
- ocdtrekkie 6y ago
  If a company can't explain why a given user was shown a given video, at a given time, that system is faulty and should be prohibited. Which is to say, there's nothing particularly wrong with complicated rules and scoring systems making recommendations, but things like black box neural networks should not be able to be used for this stuff. Let me give you an example of a good system: Spam filters not operated by Google. Algorithms decide whether or not mail gets through my corporate mail filter. It determines it based on a score, which is tallied from a set of rules. And you can drill down and see the score a spam email received, and then you can see the rules and factors that created that score. As a user, you can even generally see this, because the results are included in the message's headers in your inbox. And if you're the admin, you can then adjust those rules to fix errant behavior. That's how a recommendation system should work. A system which can't be analyzed in that matter should not exist, and the rules and scoring applied to such systems should be disclosed in some sort of header.
  - cercatrova 6y ago
    In many cases making a neural network explainable is literally impossible. Logic based AI failed in the 90s and only though neural nets have we gotten to human and superhuman level advances. If you want explainability it's going to require giving up a lot of innovation that could help people. Especially when we ourselves can't explain our own behavior psychologically.
    - ocdtrekkie 6y ago
      > In many cases making a neural network explainable is literally impossible. Then it should be illegal to use on consumer websites. It's that simple. > If you want explainability it's going to require giving up a lot of innovation that could help people. Four people died in the US Capitol because of this "innovation". It's time to stop pretending technology doesn't cause more harm than good in these cases.
      - cercatrova 6y ago
        I don't like when people try to stop or slow down technological innovation. Technologies are just tools, a nuclear bomb could just as easily have been made into a nuclear power plant. But to fully restrict them shows a lack of foresight. Yes, recommender systems can be used for bad, as yesterday, but they can also be good.