6 ms·

I think transparency matters more. I liked Andrew Yang’s suggestion to require the recommendation algorithms of the largest social networks to be open sourced g

I think transparency matters more. I liked Andrew Yang’s suggestion to require the recommendation algorithms of the largest social networks to be open sourced given how they can shape public discourse and advertising in all mass media is regulated to prevent outright lies from being spread by major institutions (although an individual certainly may do so).

anigbrowl 6y ago
Not the recommendation engines. The graph. All the social media companies (and indeed Google and others) profit by putting up a wall and then allowing people to look at individual leaves of a tree behind the wall, 50% of which is grown with the help of people's own requests. You go to the window, submit your query, and receive a small number of leaves. These companies do provide some value by building the infrastructure and so on. But the graph itself is kept proprietary, most likely because it is not copyrightable.
- annadane 6y ago
  Yeah, pretty much. It's easy for Facebook to claim that it's popular and the best thing going when you specifically need a FB account for contacting people
- devonkim 6y ago
  The graph in itself is pretty close to privacy issues that border closely as well. Even if FB et al were government funded that wouldn’t make it good either. And said data could be considered competitive advantages but perhaps not. If everyone got a copy of various social networks’ friends lists, the number of viable alternatives would skyrocket quickly because the lock-in effect would be gone. Perhaps this needs to be theorized more along modernized anti-trust laws (which don’t work in a tech given anti-trust laws were based around trying to lower consumer prices).
SkyBelow 6y ago
Does that actually work? If they create some complex AI and then show us the trained model, it doesn't really give much insight into the AI doing the recommendation. You could potentially test certain articles to see if it is recommended, but reverse engineering how the AI recommends it would be far more time consuming than updating the AI. As such Facebook would just need to regularly update the AI faster than researchers can determine how it works to hide how their code works. Older versions of the AI would eventually be cracked open (as much as a large matrix of numbers representing a neural network could be), but between it being a trained model with a bunch of numbers and Facebook having a never version I think they'll be able to hide behind "oops there was a problem, but don't worry our training has made the model much better now".
- devonkim 6y ago
  It would make it clear or not whether the site tries at all to restrict certain recommendations like harmful content at least and the model would be different and less subject to top-down rules / policies like recommending government propaganda sites over independent sources. It could be used in later, better worded and targeted subpoenas for how said filtering and censoring works. It would also show if there exists a special promotion system for a company’s own products and so forth. In many respects, it acts like an org chart and to determine _what_ to scrutinize with more concrete actions as regulators and the public. It provides a map and that’s better than a black box or Skinner Box where we are the subjects.
s1t5 6y ago
Open sourcing the algorithms (however we define it) does absolutely nothing. What use is a neural network architecture? Or a trained NN with some weights? Or an explanation that says - we measure similar posts by this metric and after you click on something we start serving you similar posts? None of those things are secret. More transparency wouldn't change anything because even if completely different algorithms were used, the fundamental problems with the platform would be exactly the same.
- banads 6y ago
  It's silly to so confidently assert that opening up a closed source algorithm to 3rd party analysis will "do absolutely nothing". How could you possibly know there is nothing unusual in the code without having audited it yourself? Seeing how the sausage gets made certainly can make lots of people lose their taste for it.
  - ChrisLomont 6y ago
    A lot of how big systems work is embodied in large neural networks, and the detailed structure of how they make decisions is an open research problem. So it’s not silly for OP to state that at all; it’s empirical fact. It’s also not possible to audit the code for anything unusual without taking it all the way back through all tools source, all hardware, down through the chips, through the doping in the chips, and even lower. This stack is such a hard problem that DARPA has run programs for a long time to address this. Start by reading Thompson’s ACM article titled something like “Reflections on Trustimg Trust” where he shows code audits don’t catch program behavior, then follow the past few decades where these holes have been pushed through the entire computing stack.
    - banads 6y ago
      >It’s also not possible to audit the code for anything unusual without taking it all the way back through all tools source, all hardware, down through the chips, through the doping in the chips, and even lower. Are you implying that since we can't audit every single thing, auditing anything is useless?
closeparen 6y ago
>advertising in all mass media is regulated to prevent outright lies from being spread Advertising in mass media is regulated. You are very much allowed to publish claims that the government would characterize as outright lies, you just can't do it to sell a product.
root_axis 6y ago
Setting aside the concerns about the efficacy of the idea, it also seems like an arbitrary encroachment on business prerogatives. I think everyone agrees that social media companies need more regulation, but mandating technical business process directives based on active user totals isn't workable, not the least of which because the definition of "active user" is highly subjective (especially if there is an incentive to get creative about the numbers), but also because something like "open source the recommendation algorithm" isn't a simple request that can be made on demand, especially with the inevitable enfilade of corporate lawyering to establish battle lines around the bounds of intellectual property that companies would still be allowed to control vs that which they would be forced to abdicate to the public domain.

anigbrowl 6y ago

Not the recommendation engines. The graph. All the social media companies (and indeed Google and others) profit by putting up a wall and then allowing people to look at individual leaves of a tree behind the wall, 50% of which is grown with the help of people's own requests. You go to the window, submit your query, and receive a small number of leaves. These companies do provide some value by building the infrastructure and so on. But the graph itself is kept proprietary, most likely because it is not copyrightable.

annadane 6y ago
Yeah, pretty much. It's easy for Facebook to claim that it's popular and the best thing going when you specifically need a FB account for contacting people
devonkim 6y ago
The graph in itself is pretty close to privacy issues that border closely as well. Even if FB et al were government funded that wouldn’t make it good either. And said data could be considered competitive advantages but perhaps not. If everyone got a copy of various social networks’ friends lists, the number of viable alternatives would skyrocket quickly because the lock-in effect would be gone. Perhaps this needs to be theorized more along modernized anti-trust laws (which don’t work in a tech given anti-trust laws were based around trying to lower consumer prices).

SkyBelow 6y ago

Does that actually work? If they create some complex AI and then show us the trained model, it doesn't really give much insight into the AI doing the recommendation. You could potentially test certain articles to see if it is recommended, but reverse engineering how the AI recommends it would be far more time consuming than updating the AI. As such Facebook would just need to regularly update the AI faster than researchers can determine how it works to hide how their code works. Older versions of the AI would eventually be cracked open (as much as a large matrix of numbers representing a neural network could be), but between it being a trained model with a bunch of numbers and Facebook having a never version I think they'll be able to hide behind "oops there was a problem, but don't worry our training has made the model much better now".

devonkim 6y ago
It would make it clear or not whether the site tries at all to restrict certain recommendations like harmful content at least and the model would be different and less subject to top-down rules / policies like recommending government propaganda sites over independent sources. It could be used in later, better worded and targeted subpoenas for how said filtering and censoring works. It would also show if there exists a special promotion system for a company’s own products and so forth. In many respects, it acts like an org chart and to determine _what_ to scrutinize with more concrete actions as regulators and the public. It provides a map and that’s better than a black box or Skinner Box where we are the subjects.

s1t5 6y ago

Open sourcing the algorithms (however we define it) does absolutely nothing. What use is a neural network architecture? Or a trained NN with some weights? Or an explanation that says - we measure similar posts by this metric and after you click on something we start serving you similar posts? None of those things are secret. More transparency wouldn't change anything because even if completely different algorithms were used, the fundamental problems with the platform would be exactly the same.

banads 6y ago
It's silly to so confidently assert that opening up a closed source algorithm to 3rd party analysis will "do absolutely nothing". How could you possibly know there is nothing unusual in the code without having audited it yourself? Seeing how the sausage gets made certainly can make lots of people lose their taste for it.
- ChrisLomont 6y ago
  A lot of how big systems work is embodied in large neural networks, and the detailed structure of how they make decisions is an open research problem. So it’s not silly for OP to state that at all; it’s empirical fact. It’s also not possible to audit the code for anything unusual without taking it all the way back through all tools source, all hardware, down through the chips, through the doping in the chips, and even lower. This stack is such a hard problem that DARPA has run programs for a long time to address this. Start by reading Thompson’s ACM article titled something like “Reflections on Trustimg Trust” where he shows code audits don’t catch program behavior, then follow the past few decades where these holes have been pushed through the entire computing stack.
  - banads 6y ago
    >It’s also not possible to audit the code for anything unusual without taking it all the way back through all tools source, all hardware, down through the chips, through the doping in the chips, and even lower. Are you implying that since we can't audit every single thing, auditing anything is useless?

closeparen 6y ago

>advertising in all mass media is regulated to prevent outright lies from being spread Advertising in mass media is regulated. You are very much allowed to publish claims that the government would characterize as outright lies, you just can't do it to sell a product.

root_axis 6y ago

Setting aside the concerns about the efficacy of the idea, it also seems like an arbitrary encroachment on business prerogatives. I think everyone agrees that social media companies need more regulation, but mandating technical business process directives based on active user totals isn't workable, not the least of which because the definition of "active user" is highly subjective (especially if there is an incentive to get creative about the numbers), but also because something like "open source the recommendation algorithm" isn't a simple request that can be made on demand, especially with the inevitable enfilade of corporate lawyering to establish battle lines around the bounds of intellectual property that companies would still be allowed to control vs that which they would be forced to abdicate to the public domain.