4 ms·

The project is very interesting, but I'm not sure why they are doing it. How is that protecting user rights? This doesn't measure AI/ML progress that's availabl

by comboy 9y ago

The project is very interesting, but I'm not sure why they are doing it. How is that protecting user rights? This doesn't measure AI/ML progress that's available to state actors.

mlinksva 9y ago
> EFF’s interest in AI progress is primarily from a policy perspective. We want to know what types of AI we need to start engaging with on legal, political, and technical safety fronts. Beyond that, we’re also just excited to see how many things computers are learning to do over time. > Given that machine learning tools and AI techniques are increasingly part of our everyday lives, it is critical that journalists, policy makers, and technology users understand the state of the field. When improperly designed or deployed, machine learning methods can violate privacy, threaten safety, and perpetuate inequality and injustice. Stakeholders must be able to anticipate such risks and policy questions before they arise, rather than playing catch-up with the technology. To this end, it’s part of the responsibility of researchers, engineers, and developers in the field to help make information about their life-changing research widely available and understandable. We hope you’ll join us.
- comboy 9y ago
  Yes, it's not that I didn't read the page. But even in case of corporations, it's not really possible to gauge what t's available to them, because they don't share everything. Well on the other hand whatever is available publicly, they have at least that. So maybe the whole research does make some sense in this context.
  - mlinksva 9y ago
    Perhaps forcing governments and corporations to share more will be explored if it seems that this publicly documented baseline is far from the state of the art. Impossible to gauge, perhaps, but even realizing that highlights the risk of public ignorance.
    - ethbro 9y ago
      A question mark is more valuable than simply reporting nothing. And I think as the Samsung / Intel / semiconductor fab article (about birth defects and cancer caused by chemical exposure) the other day noted, traditional lawmaking is ill-equiped to deal with fast-moving technical fields. The first step is public awareness.
PeterisP 9y ago
Looking from within the AI/ML field, I don't think that state actors (or the major tech megacorps) are somehow special in terms of progress in a manner that makes them unmeasurable. Yes, they have certain datasets that are somewhat larger/better than those available to the research community and somewhat easier access to larger clusters of hardware. However, that's generally a minor and (more importantly) predictable advantage - I may not be able to train as good a model of X as Google does because they have more data and hardware, but I can estimate that their model is likely to have e.g. 20% lower error rate; which gives a meaningful advantage in product quality but doesn't really enable anything conceptually different. Yes, there are certain technologies that are developed by them and not yet published, but looking at all the competition, what is seen in the products they build, and and what is seen in what state actors are funding/attempting to buy, they're not significantly beyond what is known to the public, at there seems to be no more than 6-12 months lag between when a capability becomes available and when it's seen by everyone else. It may take more than that for others to replicate, but within 6-12 months, people would have a general idea on what it'd take to replicate and what are the limits of such capabilities. Yes, the bigger actors may have invested lots of effort to polish things that are important to them - so if you're seeing a researcher's proof of concept held together by duct tape then you should keep in mind that somebody can have a reasonably working prototype of that hidden in their labs; but if you're not seeing anything close to a proof of concept in the research community (or at least claims "we know how to make a proof of concept quickly if there was funding for that"), then the capability doesn't exit yet; somebody may have unpublished results but that's included in the "6-12 months advantage" I mentioned above. Regarding user rights, IMHO you don't need to worry about still unknown capabilities of ML/AI but instead about how (and how widely) they'll apply and deploy ML/AI capabilities that are already well known. E.g. there's no question that we can run face recognition to everyone seen by a sufficiently high quality camera and try to match those places to social media profiles; all serious state actors definitely have such a capability iff they are willing (and have the funding/manpower) to build and run the required infrastructure. Whether they will do so is a question of economics and politics, not about AI/ML tech. In order to massively (ab)use something, it needs to have a certain technological readiness level; and there's a long gap between invention and sufficient maturity for that.
Iv 9y ago
Algorithmic sorting is a way to circumvent privacy protections by laws enforcement. For instance: "I only capture the emails that contain the word Al-Quaeda inside". For this an algorithm is fed all the emails of a given ISP but only selects a few. Aren't you interested in knowing the capabilities and bias of the algorithms used to filter out these data? Also, there is another important question in AI: what is the status of the neural networks generated through feeding some other data inside? Is it a derivative work? If you feed an AI some GPL work does it make the resulting NN GPL too?

by comboy 9y ago

The project is very interesting, but I'm not sure why they are doing it. How is that protecting user rights? This doesn't measure AI/ML progress that's available to state actors.

mlinksva 9y ago

> EFF’s interest in AI progress is primarily from a policy perspective. We want to know what types of AI we need to start engaging with on legal, political, and technical safety fronts. Beyond that, we’re also just excited to see how many things computers are learning to do over time. > Given that machine learning tools and AI techniques are increasingly part of our everyday lives, it is critical that journalists, policy makers, and technology users understand the state of the field. When improperly designed or deployed, machine learning methods can violate privacy, threaten safety, and perpetuate inequality and injustice. Stakeholders must be able to anticipate such risks and policy questions before they arise, rather than playing catch-up with the technology. To this end, it’s part of the responsibility of researchers, engineers, and developers in the field to help make information about their life-changing research widely available and understandable. We hope you’ll join us.

comboy 9y ago
Yes, it's not that I didn't read the page. But even in case of corporations, it's not really possible to gauge what t's available to them, because they don't share everything. Well on the other hand whatever is available publicly, they have at least that. So maybe the whole research does make some sense in this context.
- mlinksva 9y ago
  Perhaps forcing governments and corporations to share more will be explored if it seems that this publicly documented baseline is far from the state of the art. Impossible to gauge, perhaps, but even realizing that highlights the risk of public ignorance.
  - ethbro 9y ago
    A question mark is more valuable than simply reporting nothing. And I think as the Samsung / Intel / semiconductor fab article (about birth defects and cancer caused by chemical exposure) the other day noted, traditional lawmaking is ill-equiped to deal with fast-moving technical fields. The first step is public awareness.

PeterisP 9y ago

Looking from within the AI/ML field, I don't think that state actors (or the major tech megacorps) are somehow special in terms of progress in a manner that makes them unmeasurable. Yes, they have certain datasets that are somewhat larger/better than those available to the research community and somewhat easier access to larger clusters of hardware. However, that's generally a minor and (more importantly) predictable advantage - I may not be able to train as good a model of X as Google does because they have more data and hardware, but I can estimate that their model is likely to have e.g. 20% lower error rate; which gives a meaningful advantage in product quality but doesn't really enable anything conceptually different. Yes, there are certain technologies that are developed by them and not yet published, but looking at all the competition, what is seen in the products they build, and and what is seen in what state actors are funding/attempting to buy, they're not significantly beyond what is known to the public, at there seems to be no more than 6-12 months lag between when a capability becomes available and when it's seen by everyone else. It may take more than that for others to replicate, but within 6-12 months, people would have a general idea on what it'd take to replicate and what are the limits of such capabilities. Yes, the bigger actors may have invested lots of effort to polish things that are important to them - so if you're seeing a researcher's proof of concept held together by duct tape then you should keep in mind that somebody can have a reasonably working prototype of that hidden in their labs; but if you're not seeing anything close to a proof of concept in the research community (or at least claims "we know how to make a proof of concept quickly if there was funding for that"), then the capability doesn't exit yet; somebody may have unpublished results but that's included in the "6-12 months advantage" I mentioned above. Regarding user rights, IMHO you don't need to worry about still unknown capabilities of ML/AI but instead about how (and how widely) they'll apply and deploy ML/AI capabilities that are already well known. E.g. there's no question that we can run face recognition to everyone seen by a sufficiently high quality camera and try to match those places to social media profiles; all serious state actors definitely have such a capability iff they are willing (and have the funding/manpower) to build and run the required infrastructure. Whether they will do so is a question of economics and politics, not about AI/ML tech. In order to massively (ab)use something, it needs to have a certain technological readiness level; and there's a long gap between invention and sufficient maturity for that.

Iv 9y ago

Algorithmic sorting is a way to circumvent privacy protections by laws enforcement. For instance: "I only capture the emails that contain the word Al-Quaeda inside". For this an algorithm is fed all the emails of a given ISP but only selects a few. Aren't you interested in knowing the capabilities and bias of the algorithms used to filter out these data? Also, there is another important question in AI: what is the status of the neural networks generated through feeding some other data inside? Is it a derivative work? If you feed an AI some GPL work does it make the resulting NN GPL too?