5 ms·

I can't understand why people keep saying "AI has been democratized", when all the big research is being done by highly credentialed Phd scientists working for

I can't understand why people keep saying "AI has been democratized", when all the big research is being done by highly credentialed Phd scientists working for rich American tech companies.

deleted 10y ago
[deleted]
chronolitus 10y ago
Today, I downloaded the model weights for a state-of-the-art R-CNN architecture, ran some object recognition on some pictures. Took me 15 minutes to install the dependencies (bunch of one-liners), modify the code a little, and get it running on my data; all the internals are explained in the papers, and can be modified in the scripts. Just saying, could be worse. To be fair, I agree that the big papers mostly come from google & MS, and I wish research was a real democracy.
- taneq 10y ago
  Go and do some cutting edge research, and publish it. Nobody's stopping you.
  - skeletonjelly 10y ago
    I imagine the need to pay bills is what's stopping most people
existencebox 10y ago
Driving a car has largely been democratized, yet most of the car driving research is done by highly credentialed phd scientists working for rich american tech companies. Although it may sound like one, I honestly don't intend this as a strawman, but as a, "I pretty much ALWAYS expect research to occur in that way." I pessimistically think most very deep fields that require significant domain background are largely out of reach of "citizen scientists" nowadays. But even by the standard of doing research, I'm a rather uneducated (by the standards of a PHD AI researcher) programmer who has given multiple talks on AI models I developed, and uses relatively deep functionality on a day to day basis that I couldn't hope to implement the end to end of by hand. Depending on where you draw the line of democratization, I think we're already well into the promised land, and I'm only seeing continued positive improvements in this respect as well.
- argonaut 10y ago
  The analogy is a bit mixed up. Driving a car has been democratized. Actually going in and fixing/modifying a modern car is increasingly less accessible/democratized, though still possible with quite a bit of training. Car internals are getting more complex every generation. Likewise, using Google search / getting things recommended to you is obviously widely accessible. It is unclear how accessible ML research / going into the guts of a model actually is. While I don't doubt you've given talks about applying ML, I do have a little skepticism that the talks went any deeper than just application-related topics.
  - existencebox 10y ago
    "While I don't doubt you've given talks about applying ML, I do have a little skepticism that the talks went any deeper than just application-related topics." Let me correct that misconception; every aspect of the algorithm was custom, from tokenization through classification through smoothing and postprocessing. It was a more simple model and process, but it was not exclusively application related. Far simpler than many of the deeper models I consume as a black box, for sure, but I'd defend my competencies in that I'm not exclusively a plug-and-play data engineer :) Let me also emphasize that I intentionally called out "driving" as opposed to "fixing". In fact I can't think of any car-fixing research going on just about anywhere, although I'm sure there is. My core point was that _using_ the tooling is key, and has been democratized in both driving and AI (the application layer aspect as you put it), the research point was merely an add-on that I've also seen a democratization of the knowledge needed to do even the basic level of novel algorithm development, even if I'm writing deep network papers or something of that ilk; I just find that to be the exception rather than the rule (per my "hardcore research is hard for the non-deeply-initiated once low hanging fruit has been pruned" comment)
    - argonaut 10y ago
      To be clear, many of the things you just mentioned are feature engineering-related and don't have much to do with ML. Car construction/modification research is going on. It's what car companies are doing... Driving is not a valid analogy because by that logic AI was democratized 10 years ago because everyone "uses" Google search.
tomnipotent 10y ago
There's is a vast ecosystem of free and open source projects that offer access to statistical and machine learning techniques now usable by non-specialists. I can now easily build recommendation systems scaling to billions of ratings, do facial recognition with a high degree of accuracy, fraud detection, or identify all the things in a photo (thanks ImageNet!). I can build models to predict customer churn or ad CTR performance, extract summaries from large text documents, use unsupervised clustering to identify potential customer segments, determine if an image contains nudity, or if a 140 character tweet contains positive or negative language. Open access has come a long way in a short period of time. It was only five years ago that Facebook introduced auto-tagging friends in photos (facial recognition), a feature that's now taken for granted but was once the realm of bad CSI TV shows and Las Vegas casinos. Google, Facebook, Amazon, Microsoft, and IBM even started a consortium this September to share research [0]. "AI has been democratized" might be a bit overzealous, but I'm perfectly happy with the offerings on the table right now. [0] https://www.partnershiponai.org/ https://www.partnershiponai.org/
mahyarm 10y ago
There are also a lot of PhD scientists working at rich Chinese tech companies.
h4nkoslo 10y ago
Because those techniques are then available to anyone with a similar problem. Research isn't democratized but the product is.
freyir 10y ago
The tools and results are now freely available to the masses. But talent isn't democratized.
genericpseudo 10y ago
This is impossible to respond to without engaging in political analysis (particularly around equality of opportunity), but the reason that a small number of people do the research is largely that it's both expensive and requires a lot of highly specialized knowledge, which typically implies extensive postgraduate education. Credentials can be abused, sure, and they're a shorthand for a more complex set of concepts, but they are on the whole a very good thing.
taneq 10y ago
Detailed explanations of state of the art techniques. Huge archives of learning data. Massive amounts of cheaply available computing power. All of these are available to anyone on the internet. Anyone can get involved. However, not everyone has equal skills, experience, and insight. And as you'd expect, the people with the best of all three are highly credentialed and are being paid highly for their valuable efforts by companies which an afford to do so. The days of an untrained tinkerer in a back shed contributing to the state of the art are long gone.
nl 10y ago
Because that isn't true? Looking at the first thing (GANs): developed by Ian Goodfellow, who is now at OpenAI. The first linked paper (https://arxiv.org/pdf/1605.05396v2.pdf https://arxiv.org/pdf/1605.05396v2.pdf) is out of University of Michigan and Max Planck Institute. The second (https://phillipi.github.io/pix2pix/ https://phillipi.github.io/pix2pix/) is out of Berkeley, the third (https://arxiv.org/pdf/1609.04802v3.pdf https://arxiv.org/pdf/1609.04802v3.pdf) is out of Twitter (which I guess counts as a rich American tech company). Google Brain/DeepMind/FAIR/MS Research do great work, but there is plenty of great work elsewhere too. For example, just yesterday an implementation[1] of Fast Layer Normalization for TensorFlow was posted on Reddit[2] but someone, who says they don't really do C++ ("I am really new to CUDA and C++"). This can speed up training (sometimes) by 5-10X (!). That's democratization. [1] https://github.com/MycChiu/fast-LayerNorm-TF https://github.com/MycChiu/fast-LayerNorm-TF [2] https://www.reddit.com/r/MachineLearning/comments/5gt0wm/p_fast_and_efficient_layernorm_gpu_kernel_for/ https://www.reddit.com/r/MachineLearning/comments/5gt0wm/p_f...
- argonaut 10y ago
  No? Goodfellow has a PhD from UMontreal (one of the deep learning academic powerhouses), advised by Bengio. He then joined Google Research and then left for OpenAI (which has a billion dollars in funding and is backed by YC/Elon Musk). The first author of that paper has a PhD from MIT and is a postdoc at Berkeley. Berkeley is also an academic ML powerhouse, and receives millions per year in industry funding just for ML. The first two authors are also former Microsoft Research interns.
  - nl 10y ago
    It was mostly in response to the "working for rich American tech companies" part. Yes, well credentialed people continue to do great work, even when more people are working in the field.
    - argonaut 10y ago
      Sure, but my point still stands about all of these people having worked at rich tech companies or being funded by rich tech companies.

I can't understand why people keep saying "AI has been democratized", when all the big research is being done by highly credentialed Phd scientists working for rich American tech companies.

deleted 10y ago

[deleted]

chronolitus 10y ago

Today, I downloaded the model weights for a state-of-the-art R-CNN architecture, ran some object recognition on some pictures. Took me 15 minutes to install the dependencies (bunch of one-liners), modify the code a little, and get it running on my data; all the internals are explained in the papers, and can be modified in the scripts. Just saying, could be worse. To be fair, I agree that the big papers mostly come from google & MS, and I wish research was a real democracy.

taneq 10y ago
Go and do some cutting edge research, and publish it. Nobody's stopping you.
- skeletonjelly 10y ago
  I imagine the need to pay bills is what's stopping most people

existencebox 10y ago

Driving a car has largely been democratized, yet most of the car driving research is done by highly credentialed phd scientists working for rich american tech companies. Although it may sound like one, I honestly don't intend this as a strawman, but as a, "I pretty much ALWAYS expect research to occur in that way." I pessimistically think most very deep fields that require significant domain background are largely out of reach of "citizen scientists" nowadays. But even by the standard of doing research, I'm a rather uneducated (by the standards of a PHD AI researcher) programmer who has given multiple talks on AI models I developed, and uses relatively deep functionality on a day to day basis that I couldn't hope to implement the end to end of by hand. Depending on where you draw the line of democratization, I think we're already well into the promised land, and I'm only seeing continued positive improvements in this respect as well.

argonaut 10y ago
The analogy is a bit mixed up. Driving a car has been democratized. Actually going in and fixing/modifying a modern car is increasingly less accessible/democratized, though still possible with quite a bit of training. Car internals are getting more complex every generation. Likewise, using Google search / getting things recommended to you is obviously widely accessible. It is unclear how accessible ML research / going into the guts of a model actually is. While I don't doubt you've given talks about applying ML, I do have a little skepticism that the talks went any deeper than just application-related topics.
- existencebox 10y ago
  "While I don't doubt you've given talks about applying ML, I do have a little skepticism that the talks went any deeper than just application-related topics." Let me correct that misconception; every aspect of the algorithm was custom, from tokenization through classification through smoothing and postprocessing. It was a more simple model and process, but it was not exclusively application related. Far simpler than many of the deeper models I consume as a black box, for sure, but I'd defend my competencies in that I'm not exclusively a plug-and-play data engineer :) Let me also emphasize that I intentionally called out "driving" as opposed to "fixing". In fact I can't think of any car-fixing research going on just about anywhere, although I'm sure there is. My core point was that _using_ the tooling is key, and has been democratized in both driving and AI (the application layer aspect as you put it), the research point was merely an add-on that I've also seen a democratization of the knowledge needed to do even the basic level of novel algorithm development, even if I'm writing deep network papers or something of that ilk; I just find that to be the exception rather than the rule (per my "hardcore research is hard for the non-deeply-initiated once low hanging fruit has been pruned" comment)
  - argonaut 10y ago
    To be clear, many of the things you just mentioned are feature engineering-related and don't have much to do with ML. Car construction/modification research is going on. It's what car companies are doing... Driving is not a valid analogy because by that logic AI was democratized 10 years ago because everyone "uses" Google search.

tomnipotent 10y ago

There's is a vast ecosystem of free and open source projects that offer access to statistical and machine learning techniques now usable by non-specialists. I can now easily build recommendation systems scaling to billions of ratings, do facial recognition with a high degree of accuracy, fraud detection, or identify all the things in a photo (thanks ImageNet!). I can build models to predict customer churn or ad CTR performance, extract summaries from large text documents, use unsupervised clustering to identify potential customer segments, determine if an image contains nudity, or if a 140 character tweet contains positive or negative language. Open access has come a long way in a short period of time. It was only five years ago that Facebook introduced auto-tagging friends in photos (facial recognition), a feature that's now taken for granted but was once the realm of bad CSI TV shows and Las Vegas casinos. Google, Facebook, Amazon, Microsoft, and IBM even started a consortium this September to share research [0]. "AI has been democratized" might be a bit overzealous, but I'm perfectly happy with the offerings on the table right now. [0] https://www.partnershiponai.org/ https://www.partnershiponai.org/

mahyarm 10y ago

There are also a lot of PhD scientists working at rich Chinese tech companies.

h4nkoslo 10y ago

Because those techniques are then available to anyone with a similar problem. Research isn't democratized but the product is.

freyir 10y ago

The tools and results are now freely available to the masses. But talent isn't democratized.

genericpseudo 10y ago

This is impossible to respond to without engaging in political analysis (particularly around equality of opportunity), but the reason that a small number of people do the research is largely that it's both expensive and requires a lot of highly specialized knowledge, which typically implies extensive postgraduate education. Credentials can be abused, sure, and they're a shorthand for a more complex set of concepts, but they are on the whole a very good thing.

taneq 10y ago

Detailed explanations of state of the art techniques. Huge archives of learning data. Massive amounts of cheaply available computing power. All of these are available to anyone on the internet. Anyone can get involved. However, not everyone has equal skills, experience, and insight. And as you'd expect, the people with the best of all three are highly credentialed and are being paid highly for their valuable efforts by companies which an afford to do so. The days of an untrained tinkerer in a back shed contributing to the state of the art are long gone.

nl 10y ago

Because that isn't true? Looking at the first thing (GANs): developed by Ian Goodfellow, who is now at OpenAI. The first linked paper (https://arxiv.org/pdf/1605.05396v2.pdf https://arxiv.org/pdf/1605.05396v2.pdf) is out of University of Michigan and Max Planck Institute. The second (https://phillipi.github.io/pix2pix/ https://phillipi.github.io/pix2pix/) is out of Berkeley, the third (https://arxiv.org/pdf/1609.04802v3.pdf https://arxiv.org/pdf/1609.04802v3.pdf) is out of Twitter (which I guess counts as a rich American tech company). Google Brain/DeepMind/FAIR/MS Research do great work, but there is plenty of great work elsewhere too. For example, just yesterday an implementation[1] of Fast Layer Normalization for TensorFlow was posted on Reddit[2] but someone, who says they don't really do C++ ("I am really new to CUDA and C++"). This can speed up training (sometimes) by 5-10X (!). That's democratization. [1] https://github.com/MycChiu/fast-LayerNorm-TF https://github.com/MycChiu/fast-LayerNorm-TF [2] https://www.reddit.com/r/MachineLearning/comments/5gt0wm/p_fast_and_efficient_layernorm_gpu_kernel_for/ https://www.reddit.com/r/MachineLearning/comments/5gt0wm/p_f...

argonaut 10y ago
No? Goodfellow has a PhD from UMontreal (one of the deep learning academic powerhouses), advised by Bengio. He then joined Google Research and then left for OpenAI (which has a billion dollars in funding and is backed by YC/Elon Musk). The first author of that paper has a PhD from MIT and is a postdoc at Berkeley. Berkeley is also an academic ML powerhouse, and receives millions per year in industry funding just for ML. The first two authors are also former Microsoft Research interns.
- nl 10y ago
  It was mostly in response to the "working for rich American tech companies" part. Yes, well credentialed people continue to do great work, even when more people are working in the field.
  - argonaut 10y ago
    Sure, but my point still stands about all of these people having worked at rich tech companies or being funded by rich tech companies.