5 ms·

Controversial opinion as a machine learning researcher: Well roundedness (or teaching basic elements to many) will not pay off as much, it would likely yield b

Controversial opinion as a machine learning researcher: Well roundedness (or teaching basic elements to many) will not pay off as much, it would likely yield better results for the country to invest heavily at the top. Observation based on success of machine learning research in highly concentrated organisations (certain universities in the US, DeepMind, etc) versus countries with scattered expertise and resources (e.g. Germany). If you can afford to, do both, but observationally, this does not seem to happen. Not for a lack of resources but rather the mindset and a society's perspective.

deleted 8y ago
[deleted]
PandaRider 8y ago
If the objective is for applied research (research for the benefit of the economy), I disagree. My main argument is that we need domain experts outside of academia. Moreover, theoretical research in AI is very open and countries with limited resources are better off stand on the shoulders of giants than try and be number 1 themselves. Finland might never beat Russia/China/US AI arms race but they just need to defend themselves should any of these 3 countries monopolize or weaponize on AI.
- jbottoms 8y ago
  I have a more informed perspective. First, Finland introduced a browser (Erwise) two years before the W3c browser. Second, the U.S. is stumbling in terms of getting beyond analytic and ANN designs, and AGI is an area where few in the U.S. work. I have worked with Finland on cutting-edge tech in the past, and I wouldn't be surprised if they made some serious breakthroughs. It does not take a lot of researchers to excell in AGI, it only takes one bricoleur. (source, inventor of the Internet browser.)
openIce 8y ago
Well I'm surprised you are a researcher if you can't see the benefits of educating non-IT people with domain knowledge what tools IT people can provide in order for both to work together on solving real world issues with AI.
LeanderK 8y ago
well, i don't know whether this is true. I am quite frustrated with germany, because our scattered expertise and resources are just not that competitive in this area. We were unable to retain our top people (Schmidhuber, Hochreiter, etc.), don't invest enough into our universities and our AI-initiative is also just not enough. A serious comparison to the top countries (United States, more interesting: Canada) does not end well. If the TUM (Technical University of Munich) would have been sucessfull in retaining the top talents they developed, they would count Schmidhuber, Hochreiter, Schaal, etc. to their faculty and be competitive with top institutions. But they didn't and it's a symptom of a larger lack of investment. My university (KIT) also developed some pretty sucessfull researchers in "AI", but they left (for example to Carnegie Mellon, where there's a lot cooperation, especially in machine learning). Prof. Alex Waibel made a pretty important contribution to ConvNets, one could argue they developed out of his time-delay NNs. But, although he's still a Prof at KIT, he's not really that present in karlsruhe. Just compare the total numbers of papers on NeurIPS of germany to other countries. We are getting obliterated. I still think that a "reasonably" distributed approach is best for germany (and probably for finnland), but we have to get our shit together! I don't think this is due to our more distributed approach. It's all pretty frustrating, i think there's a lot of wasted potential here.
pasabagi 8y ago
On the other hand, Finland punches massively above its weight when it comes to tech in general - while the US, being the world's largest economy, punches at its weight. That would indicate the generally egalitarian Finish approach is better than the darwinian US approach. My feeling is that while the US is phenomenally good at delivering an excellent education to a small set of extremely talented people, I get the feeling that these people go on to not have enough free time to do really interesting work, and rather end up devoting their lives to industry, which is predominantly very conservative[1]. [1]: Obviously, I think this is offset by the fact that US industry is less conservative and more interested in abstract research than it is in most other countries, so you do get really cool research being done. But I think generally speaking, this model isn't as effective as just letting most people in society do more of the stuff that interests them, and less of the stuff that doesn't.
- hoaw 8y ago
  The whole idea that a country like Finland could create and maintain talent without a domestic market is completely silly, if not entirely unsupported. Which is also true for much of the US. Arguably even the Soviet Union couldn't. And if you actually had to choose between "top talent" and "mainstream appeal" the latter has orders of magnitude better returns. A larger part of the modern economy is built on that fact, including much of Silicon Valley. If you could just invest in talent many countries would pass the US in no time. But unsurprisingly those investments mostly yield handshakes and cocktail parties.
  - pasabagi 8y ago
    I think Silicon Valley is quite a good refutation of your point, actually -- it was almost entirely founded on DARPA investing in talent. After all, they funded everything from the first ICs to Chomsky. I don't think mainstream appeal is irrelevant. I just think that if you fund scientists and technologists to do the work they find interesting, they'll produce something more fundamentally interesting and generally useful than if you fund them to make great products. Great products, after all, include things like Minecraft, pet rocks, and fidget spinners.
heyjudy 8y ago
It makes sense... almost Power law distribution of talent. The wisest move is to hedge bets accordingly and accurately classify potential talent for said investment. The worst move would be to do what the US is doing: - nationalist xenophobia turning off expats from emigrating - exacerbating massive inequality disparity/impoverishing it with people living on highway on-ramp embankment worse than S. American favelas (i.e., go to the Bay Area, drive around.. 'nuf said) - ban the export of new technology with arms controls, killing industries and making a country radioactive to invest/do business in - make it extraordinarily difficult for moderately wealthy entrepreneurs to work in a country for even a limited time because their passport isn't from one of a dozen or so favored countries - ban everyone entirely, regardless of their background or achievements, if they happen to have been born in another set of countries - double-down on fossil fuels, end clean energy tax credits To me, the best things from Finland are Linus Torvalds, Jarkko Oikarinen and The Hydraulic Press Channel. There's probably others but the wikipedia page for famous Finns was too long.
notahacker 8y ago
Surely this depends on the results you're after? Sure, giving everyone a basic intro to AI isn't going to result in [m]any AI breakthrough discoveries or give you a workforce of AI researchers (though it might engage a few people in the subject that might otherwise have chosen to be quants). But it might help innoculate them against a lot of misconceptions about what the current state of AI can and can't do and encourage them to think critically about data sources and inferences that might be drawn from them.
fermienrico 8y ago
It is 1965. Imagine if computer science was left for the top elites and never propagated to the masses. This kind of thinking is delusional, gate-keeping that only hurts. ML/AI research can continue, but its application and implementation needs to be democratized. Everyone in CS should know ML - it is a tool just like learning algorithms. Why would you condone not learning a tool that an engineer would use? I completely and utterly disagree.
- high_derivative 8y ago
  1. They are not teaching CS people, but a random 1% slice of the population. 2. Where did I not condone? I made an observation what I think leads to better outcomes. I said both is desirable but culturally only one thing seems to happen. Before you angrily accuse me of "Delusional gatekeeping", maybe re-read (or maybe for the first time read) both the article and my comment. Your confrontational tone does not help the discussion and is not in the spirit of hackernews to assume the best intentions.
  - fermienrico 8y ago
    I apologize for my tone but no one is accusing you. I am criticizing your thinking and your argument. Photography is democratized to the point where there is so much noise. Everyone has a camera with them at all times. If you want to be a professional photographer today, it is almost impossible barring journalism or wedding photography. I feel like ML researchers (again, not particularly you so don't take it personally) feel threatened by the mass adoption, marketing buzz and the whole ML/AI enchilada. Everyone is getting on it including the topic of this thread - i.e. the Finnish government. This is honestly what I see when I read your parent comment. If you're offended by it, then may be you can balance your argument so it doesn't appear to be gate-keeping elitism. In fact, you've admitted it is controversial!
    - high_derivative 8y ago
      Thank you. Well here is a potential counterpoint: I think one could also imagine a world where there will be a cannibalization/consolidation on the lower end of the market because of the democratization (which I am not saying is bad or good because I am not sure about the effects ML has on people and democracies, but that is another discussion). Rather, I observe that nobody today would hire someone to (in essence) build numpy functionality. Yet there are a lot of jobs in ML/AI currently which amount to building things that will be commoditized within years (and be used by everyone like numpy today). So in my opinion learning AI basics will not necessarily lead to more AI jobs or startups because these basic features are commoditized via big tech open source. Second, on applications: My research is actually in the applied domain and that's where my argument mostly stems from. Doing novel, valuable applications in AI requires concentrated resources and expertise. For example, if you think about self-driving cars as an extremely high-value application, they require highly specialized domain expertise in various domains and machine learning combined with massive compute resources. Simple applications (e.g. applying basic machine learning tools to common workflows and processes) are prone to undergo commoditization precisely because they are obvious. So I do not think the gatekeeping argument holds because there is no way around obtaining deep and wide expertise to generate valuable application IP. So for a country wanting to focus on applications, I still think there is a good argument that concentrating resources is a more successful approach.
nabla9 8y ago
Education that provides basic outline for larger audience can be beneficial in different ways. If it can make people and decision makers more resistant to AI snake oil, it's a good thing.
3pt14159 8y ago
Eh—disagree. Unlike the majority of our sciences, ML tech disperses extremely quickly. Software being software. Patents aside, there's nothing that machine learning research is going to give as an advantage nationally other than a couple years of warning that something is coming. On the other hand, if more software devs understood the basics of statistics and ML they'd be able to apply the tech into specialized domains. The same thing is happening with ML that happened with web dev. Abstracting over the annoying stuff allows developers to attack the problem at the conceptual level. But you can't think conceptually without understanding thing from first principles. For example, I expect developers to understand the definition of accuracy and why maximizing it usually isn't useful.
currymj 8y ago
Depends on the goal. If the goal is to maximize publications in good venues and have the occasional spectacular result, I think you are probably right. However, if I grant that there is a lot of low-hanging fruit out there in the economy that can be attacked with already-existing approaches, then the Finnish approach makes sense to me. Having a bunch of people who understand statistics, basic machine learning algorithms, and have learned the discipline/soft skills to use them successfully (cross-validate, don't overfit, etc.) might result in a lot of interesting problems being solved.
dqpb 8y ago
I imagine certain people said that about reading and writing back in the day.

deleted 8y ago

[deleted]

PandaRider 8y ago

If the objective is for applied research (research for the benefit of the economy), I disagree. My main argument is that we need domain experts outside of academia. Moreover, theoretical research in AI is very open and countries with limited resources are better off stand on the shoulders of giants than try and be number 1 themselves. Finland might never beat Russia/China/US AI arms race but they just need to defend themselves should any of these 3 countries monopolize or weaponize on AI.

jbottoms 8y ago
I have a more informed perspective. First, Finland introduced a browser (Erwise) two years before the W3c browser. Second, the U.S. is stumbling in terms of getting beyond analytic and ANN designs, and AGI is an area where few in the U.S. work. I have worked with Finland on cutting-edge tech in the past, and I wouldn't be surprised if they made some serious breakthroughs. It does not take a lot of researchers to excell in AGI, it only takes one bricoleur. (source, inventor of the Internet browser.)

openIce 8y ago

Well I'm surprised you are a researcher if you can't see the benefits of educating non-IT people with domain knowledge what tools IT people can provide in order for both to work together on solving real world issues with AI.

LeanderK 8y ago

well, i don't know whether this is true. I am quite frustrated with germany, because our scattered expertise and resources are just not that competitive in this area. We were unable to retain our top people (Schmidhuber, Hochreiter, etc.), don't invest enough into our universities and our AI-initiative is also just not enough. A serious comparison to the top countries (United States, more interesting: Canada) does not end well. If the TUM (Technical University of Munich) would have been sucessfull in retaining the top talents they developed, they would count Schmidhuber, Hochreiter, Schaal, etc. to their faculty and be competitive with top institutions. But they didn't and it's a symptom of a larger lack of investment. My university (KIT) also developed some pretty sucessfull researchers in "AI", but they left (for example to Carnegie Mellon, where there's a lot cooperation, especially in machine learning). Prof. Alex Waibel made a pretty important contribution to ConvNets, one could argue they developed out of his time-delay NNs. But, although he's still a Prof at KIT, he's not really that present in karlsruhe. Just compare the total numbers of papers on NeurIPS of germany to other countries. We are getting obliterated. I still think that a "reasonably" distributed approach is best for germany (and probably for finnland), but we have to get our shit together! I don't think this is due to our more distributed approach. It's all pretty frustrating, i think there's a lot of wasted potential here.

pasabagi 8y ago

On the other hand, Finland punches massively above its weight when it comes to tech in general - while the US, being the world's largest economy, punches at its weight. That would indicate the generally egalitarian Finish approach is better than the darwinian US approach. My feeling is that while the US is phenomenally good at delivering an excellent education to a small set of extremely talented people, I get the feeling that these people go on to not have enough free time to do really interesting work, and rather end up devoting their lives to industry, which is predominantly very conservative[1]. [1]: Obviously, I think this is offset by the fact that US industry is less conservative and more interested in abstract research than it is in most other countries, so you do get really cool research being done. But I think generally speaking, this model isn't as effective as just letting most people in society do more of the stuff that interests them, and less of the stuff that doesn't.

hoaw 8y ago
The whole idea that a country like Finland could create and maintain talent without a domestic market is completely silly, if not entirely unsupported. Which is also true for much of the US. Arguably even the Soviet Union couldn't. And if you actually had to choose between "top talent" and "mainstream appeal" the latter has orders of magnitude better returns. A larger part of the modern economy is built on that fact, including much of Silicon Valley. If you could just invest in talent many countries would pass the US in no time. But unsurprisingly those investments mostly yield handshakes and cocktail parties.
- pasabagi 8y ago
  I think Silicon Valley is quite a good refutation of your point, actually -- it was almost entirely founded on DARPA investing in talent. After all, they funded everything from the first ICs to Chomsky. I don't think mainstream appeal is irrelevant. I just think that if you fund scientists and technologists to do the work they find interesting, they'll produce something more fundamentally interesting and generally useful than if you fund them to make great products. Great products, after all, include things like Minecraft, pet rocks, and fidget spinners.

heyjudy 8y ago

It makes sense... almost Power law distribution of talent. The wisest move is to hedge bets accordingly and accurately classify potential talent for said investment. The worst move would be to do what the US is doing: - nationalist xenophobia turning off expats from emigrating - exacerbating massive inequality disparity/impoverishing it with people living on highway on-ramp embankment worse than S. American favelas (i.e., go to the Bay Area, drive around.. 'nuf said) - ban the export of new technology with arms controls, killing industries and making a country radioactive to invest/do business in - make it extraordinarily difficult for moderately wealthy entrepreneurs to work in a country for even a limited time because their passport isn't from one of a dozen or so favored countries - ban everyone entirely, regardless of their background or achievements, if they happen to have been born in another set of countries - double-down on fossil fuels, end clean energy tax credits To me, the best things from Finland are Linus Torvalds, Jarkko Oikarinen and The Hydraulic Press Channel. There's probably others but the wikipedia page for famous Finns was too long.

notahacker 8y ago

Surely this depends on the results you're after? Sure, giving everyone a basic intro to AI isn't going to result in [m]any AI breakthrough discoveries or give you a workforce of AI researchers (though it might engage a few people in the subject that might otherwise have chosen to be quants). But it might help innoculate them against a lot of misconceptions about what the current state of AI can and can't do and encourage them to think critically about data sources and inferences that might be drawn from them.

fermienrico 8y ago

It is 1965. Imagine if computer science was left for the top elites and never propagated to the masses. This kind of thinking is delusional, gate-keeping that only hurts. ML/AI research can continue, but its application and implementation needs to be democratized. Everyone in CS should know ML - it is a tool just like learning algorithms. Why would you condone not learning a tool that an engineer would use? I completely and utterly disagree.

high_derivative 8y ago
1. They are not teaching CS people, but a random 1% slice of the population. 2. Where did I not condone? I made an observation what I think leads to better outcomes. I said both is desirable but culturally only one thing seems to happen. Before you angrily accuse me of "Delusional gatekeeping", maybe re-read (or maybe for the first time read) both the article and my comment. Your confrontational tone does not help the discussion and is not in the spirit of hackernews to assume the best intentions.
- fermienrico 8y ago
  I apologize for my tone but no one is accusing you. I am criticizing your thinking and your argument. Photography is democratized to the point where there is so much noise. Everyone has a camera with them at all times. If you want to be a professional photographer today, it is almost impossible barring journalism or wedding photography. I feel like ML researchers (again, not particularly you so don't take it personally) feel threatened by the mass adoption, marketing buzz and the whole ML/AI enchilada. Everyone is getting on it including the topic of this thread - i.e. the Finnish government. This is honestly what I see when I read your parent comment. If you're offended by it, then may be you can balance your argument so it doesn't appear to be gate-keeping elitism. In fact, you've admitted it is controversial!
  - high_derivative 8y ago
    Thank you. Well here is a potential counterpoint: I think one could also imagine a world where there will be a cannibalization/consolidation on the lower end of the market because of the democratization (which I am not saying is bad or good because I am not sure about the effects ML has on people and democracies, but that is another discussion). Rather, I observe that nobody today would hire someone to (in essence) build numpy functionality. Yet there are a lot of jobs in ML/AI currently which amount to building things that will be commoditized within years (and be used by everyone like numpy today). So in my opinion learning AI basics will not necessarily lead to more AI jobs or startups because these basic features are commoditized via big tech open source. Second, on applications: My research is actually in the applied domain and that's where my argument mostly stems from. Doing novel, valuable applications in AI requires concentrated resources and expertise. For example, if you think about self-driving cars as an extremely high-value application, they require highly specialized domain expertise in various domains and machine learning combined with massive compute resources. Simple applications (e.g. applying basic machine learning tools to common workflows and processes) are prone to undergo commoditization precisely because they are obvious. So I do not think the gatekeeping argument holds because there is no way around obtaining deep and wide expertise to generate valuable application IP. So for a country wanting to focus on applications, I still think there is a good argument that concentrating resources is a more successful approach.

nabla9 8y ago

Education that provides basic outline for larger audience can be beneficial in different ways. If it can make people and decision makers more resistant to AI snake oil, it's a good thing.

3pt14159 8y ago

Eh—disagree. Unlike the majority of our sciences, ML tech disperses extremely quickly. Software being software. Patents aside, there's nothing that machine learning research is going to give as an advantage nationally other than a couple years of warning that something is coming. On the other hand, if more software devs understood the basics of statistics and ML they'd be able to apply the tech into specialized domains. The same thing is happening with ML that happened with web dev. Abstracting over the annoying stuff allows developers to attack the problem at the conceptual level. But you can't think conceptually without understanding thing from first principles. For example, I expect developers to understand the definition of accuracy and why maximizing it usually isn't useful.

currymj 8y ago

Depends on the goal. If the goal is to maximize publications in good venues and have the occasional spectacular result, I think you are probably right. However, if I grant that there is a lot of low-hanging fruit out there in the economy that can be attacked with already-existing approaches, then the Finnish approach makes sense to me. Having a bunch of people who understand statistics, basic machine learning algorithms, and have learned the discipline/soft skills to use them successfully (cross-validate, don't overfit, etc.) might result in a lot of interesting problems being solved.

dqpb 8y ago

I imagine certain people said that about reading and writing back in the day.