6 ms·

Machine learning isn't Kaggle competitions

81 points by akbarnama 12y ago

happyhammy 12y ago
except building models is the essence of machine learning
splike 12y ago
I'm selecting my final year (undergrad) project in the next couple of weeks. One of my professors posted a project that will simply be entering a competition on Kaggle. I don't know much about machine learning yet and I thought that this would be a great way to pick up something I've always been interested in. What does HN think of doing this?
- QuantumChaos 12y ago
  It sounds like a great idea. Even though the article is entirely correct in that Kaggle competitions are only a small part of machine learning, they are still a good way to learn some aspects. Even the data-cleaning that the author claims is missing from Kaggle competitions, is not really missing, it's just Kaggle have made your data cleaning much simpler. The main advice I would give is to find a problem where you can get an understanding of the data, i.e. not just treat it like a black box.
pedrosorio 12y ago
It sounds like someone justifying their own shortcomings. "Since kaggle proves I am not that good at the 'math' part of machine learning, I will dismiss it by pointing out that it does not include all of the steps necessary to make a machine learning approach work for a business"
gentrificatron 12y ago
This is exactly my experience and that of most people in the field I've worked with. In fact, many people have said the quality/processing of data is much more important than the machine learning model you use. The exception is those fields that have physical data, like computer vision or speech recognition. In those fields, the actual model matters a lot more. I think every job has this tendency, where the public focus on the most exciting and interesting part, and ignore the mundane but also extremely important parts. And on the matter of skill/ability, in spite of not being a Kaggle winner, the author couldn't do their job without a good understanding of machine learning models. To do machine learning in practice, you must know both the software development/systems side and the maths/stats/ML side.
mjawa 12y ago
Machine Learning is just a part of many steps required to solve a practical problem. In my opinion although data cleansing/pre processing is important , but so are other things like feature engineering, model selection, understanding of machine learning models. And kaggle competitions are great way to practically learn these other things.
shazeline 12y ago
I think it is more accurate to say that data science isn't Kaggle. The process of taking a data set and fitting it to the most robust model is certainly machine learning.
agibsonccc 12y ago
It's a shame there is a huge disconnect between what data scientists ACTUALLY do in their day to day vs what's published in the media. Many people see machine learning as synonymous with data science when a data scientist's real job is to leverage data to achieve business objectives. This has a wide range of implications from producing reports, to visualizations for understanding different kinds of trends. Model making is, imo, the fun part of the process, but by far from what actually needs to be done. That being said, kaggle is a great platform to learn on. There is a lot of value in understanding how to train a model, but it's only as valuable as understanding how to take data from disparate sources, cleaning and normalizing it, doing some EDA, and then understanding AFTER all that if you need a model to achieve your goals.
krick 12y ago
"Once I played against an NBA player and I couldn't do anything against him. I felt sad and demoralized. I still don't think I could win against an NBA player, but everyday I do (among other things) play basketball with my friends! And, you know, there're so many problems in basketball aside of dribbling and stuff: you have to find space to play with your friends, you have to convince the guy that doesn't want to play this evening. Once I didn't have time to play that day, and if it ever happened to you: it doesn't matter how good you are at dribbling if you don't have time to play. So I decided to write an article for you guys to know: that stuff they do at NBA has nothing to do with basketball!"
- pfortuny 12y ago
  To me it sounded more like "the problems they do at Mathematical Olympiads are not the real mathematics done at academia/the enterprise," which turns out to be quite true in many ways.
  - chubot 12y ago
    Not just math -- even the problems done at programming contests have fairly little to do with programming! I've worked at Google for nearly 10 years -- not sure I can solve the Google code Jams :) Or at least I haven't been motivated to do so. I know all my basic data structures and algorithms, but the questions don't seem that motivating.
    - radicality 12y ago
      Terminology would probably be important – programming contests have a lot to do with programming, you literally sit and write programs that pass the test cases. I'm sure you meant it has fairly little to do with day-to-day programming for the majority of software engineers, which is correct. Regarding the article, I also disagree with it. I feel like the author is saying "algorithms != programming contests" which is false. Algorithms are the essence of programming contests, just like 'machine learning' is the essence of Kaggle competitions. Open up Bishop's "Pattern Recognition and Machine learning" and you won't find anything about "deploying your model to production", but you will find lots of math on how to build models because that is what ML is. What the author seems to be describing is data-science.
yankoff 12y ago
Hm, looks like this argument about Kaggle pops up all the time. This is true, that getting and cleaning data is an important part of a workflow in data science and it's a necessary skill to have. But I don't believe anyone ever claimed Kaggle is enough to become a good data scientist. That's like studying algorithms is very useful, but who said it's enough to become good at software engineering? The skill consist of multiple parts which can be practiced separately or in combination in a real project where you do everything from scratch.
- gentrificatron 12y ago
  It's not as straightforward as Kaggle representing one stage in a pipeline. Kaggle also fixes this stage in an unnatural way. In real life, there is a feedback loop from modelling, to data processing. In Kaggle all you can do is improve your model. You can't go back and change how data was processed, or collect different sorts of data. Also, some flows simply don't fit into Kaggle's train/test paradigm. E.g. suppose you have an online algorithm that continuously updates parameters. Or situations where the model is used to generate the train/test data. Finally some situations may have stringent computational requirements, e.g. <100ms to classify a single instance.
  - yankoff 12y ago
    Sure, everything "in real life" is more complicated. I think Kaggle to data science is as programming contests to Software Engineering. Will you say that doing programming challenges is "unnatural"? Is it useless or bad? Because you may also argue that you never have to solve problems for speed, you rarely see clearly-defined problems, you rarely get to apply complicated or rare algorithms in real life situation. The thing is Kaggle helps developing certain skills. It doesn't mean it helps you developing all necessary skills for your "real life". Your real life and your job may require unique combination of skills and the only way to fully prepare for it is to actually do real project that are specific to your job.
onislandtime 12y ago
There is too much hype, solving data-related problems requires many different skills and it depends on the problem. In general you need background in quite a few of these areas linear algebra, probability, statistical modeling, stochastic processes, computer science, programming, signal processing, etc. We used to call this electrical/computer engineering :-)
deleted 12y ago
[deleted]
blauwbilgorgel 12y ago
Why Machine Learning is Kaggle competitions. I used an out-of-the-box algorithm, messed around a bit, and definitely did not make the leaderboard. Because that is not Kaggle competitions. Nearly everyone on the leaderboard is proficient in data analysis and machine learning. They all tried that out-of-the-box algorithm for their attempt. But they did not give up so easily. Understand the business problem If you want to predict flight arrival times, what are you really trying to do? This is not different from Kaggle competitions, this is a tip for performing better in Kaggle competitions. See also the GE Flight Quest: https://www.gequest.com/c/flight https://www.gequest.com/c/flight Those winners used industry-standard machine learning, optimization techniques, but also creative insights and hunches, like tweaking the target labels: "A next step is to ask, “What should I actually be predicting?”. This is an important step that is often missed by many – they just throw the raw dependent variable into their favorite algorithm and hope for the best. But sometimes you want to create a derived dependent variable. I’ll use the GE Flight Quest as an example: you don't want to predict the actual time the airplane will land; you want to predict the length of the flight; and maybe the best way to do that is to use the ratio of how long the flight actually was to how long it was originally estimated to be and then multiply that times the original estimate." - Steve Donoho - http://blog.kaggle.com/2014/08/01/learning-from-the-best/ http://blog.kaggle.com/2014/08/01/learning-from-the-best/ Furthermore, it is entirely clear to everyone that data science in a business setting and in a competitive sport setting is different. To say they are equal, would be to say something like: paintball is equal to being in the military. But to say that Kaggle is not machine learning is to say: paintball requires no marksmanship. There are some very messy, unwieldy datasets on Kaggle right now. For example the Seizure Detection challenge has many GBs of raw sensor data, from just a few patients. This would require a competitor to clean, understand problem domain, understand evaluation metrics, measure cross validation and put your model in production on your laptop in the evening hours. The author of that blogpost is invited to team up, with me or others. Let's see if we can use machine learning to improve some pressing issues. I'd also love it if Stripe can host a contest on Kaggle.
- mkrump 12y ago
  Completely agree. Everything that is supposedly not addressed in Kaggle actually is, aside from productionizing and monitoring your model in production. Sure it's a simplified less open ended version of what you'll encounter in the real world, but that doesn't mean that many of the core concepts don't translate. It's kind of like saying that you shouldn't do your calculus practice problems, because math in the real world is never so straightforward. I think the most legitimate knock against Kaggle is that in many business settings there probably isn't much value in improving that extra .00001 (but obviously there are exceptions). Anyway, I think that that even someone very experienced in machine learning would learn something doing a Kaggle competition, especially if the competition is in an area outside their core expertise.
TheAlchemist 12y ago
I'm a quite new Kaggler, but did some similar machine learning competitions on similar plateforms and I have to say that this article is quite bullshit. How can a data scientist actually complain that the low value data munging work is already done for him ? Gosh, it should be the other way around - we should complain about how much of this we have to do at work !
deleted 12y ago
[deleted]
deleted 12y ago
[deleted]

81 points by akbarnama 12y ago

except building models is the essence of machine learning

I'm selecting my final year (undergrad) project in the next couple of weeks. One of my professors posted a project that will simply be entering a competition on Kaggle. I don't know much about machine learning yet and I thought that this would be a great way to pick up something I've always been interested in. What does HN think of doing this?

QuantumChaos 12y ago
It sounds like a great idea. Even though the article is entirely correct in that Kaggle competitions are only a small part of machine learning, they are still a good way to learn some aspects. Even the data-cleaning that the author claims is missing from Kaggle competitions, is not really missing, it's just Kaggle have made your data cleaning much simpler. The main advice I would give is to find a problem where you can get an understanding of the data, i.e. not just treat it like a black box.

pedrosorio 12y ago

It sounds like someone justifying their own shortcomings. "Since kaggle proves I am not that good at the 'math' part of machine learning, I will dismiss it by pointing out that it does not include all of the steps necessary to make a machine learning approach work for a business"

gentrificatron 12y ago

This is exactly my experience and that of most people in the field I've worked with. In fact, many people have said the quality/processing of data is much more important than the machine learning model you use. The exception is those fields that have physical data, like computer vision or speech recognition. In those fields, the actual model matters a lot more. I think every job has this tendency, where the public focus on the most exciting and interesting part, and ignore the mundane but also extremely important parts. And on the matter of skill/ability, in spite of not being a Kaggle winner, the author couldn't do their job without a good understanding of machine learning models. To do machine learning in practice, you must know both the software development/systems side and the maths/stats/ML side.

mjawa 12y ago

Machine Learning is just a part of many steps required to solve a practical problem. In my opinion although data cleansing/pre processing is important , but so are other things like feature engineering, model selection, understanding of machine learning models. And kaggle competitions are great way to practically learn these other things.

shazeline 12y ago

I think it is more accurate to say that data science isn't Kaggle. The process of taking a data set and fitting it to the most robust model is certainly machine learning.

agibsonccc 12y ago

It's a shame there is a huge disconnect between what data scientists ACTUALLY do in their day to day vs what's published in the media. Many people see machine learning as synonymous with data science when a data scientist's real job is to leverage data to achieve business objectives. This has a wide range of implications from producing reports, to visualizations for understanding different kinds of trends. Model making is, imo, the fun part of the process, but by far from what actually needs to be done. That being said, kaggle is a great platform to learn on. There is a lot of value in understanding how to train a model, but it's only as valuable as understanding how to take data from disparate sources, cleaning and normalizing it, doing some EDA, and then understanding AFTER all that if you need a model to achieve your goals.

krick 12y ago

"Once I played against an NBA player and I couldn't do anything against him. I felt sad and demoralized. I still don't think I could win against an NBA player, but everyday I do (among other things) play basketball with my friends! And, you know, there're so many problems in basketball aside of dribbling and stuff: you have to find space to play with your friends, you have to convince the guy that doesn't want to play this evening. Once I didn't have time to play that day, and if it ever happened to you: it doesn't matter how good you are at dribbling if you don't have time to play. So I decided to write an article for you guys to know: that stuff they do at NBA has nothing to do with basketball!"

pfortuny 12y ago
To me it sounded more like "the problems they do at Mathematical Olympiads are not the real mathematics done at academia/the enterprise," which turns out to be quite true in many ways.
- chubot 12y ago
  Not just math -- even the problems done at programming contests have fairly little to do with programming! I've worked at Google for nearly 10 years -- not sure I can solve the Google code Jams :) Or at least I haven't been motivated to do so. I know all my basic data structures and algorithms, but the questions don't seem that motivating.
  - radicality 12y ago
    Terminology would probably be important – programming contests have a lot to do with programming, you literally sit and write programs that pass the test cases. I'm sure you meant it has fairly little to do with day-to-day programming for the majority of software engineers, which is correct. Regarding the article, I also disagree with it. I feel like the author is saying "algorithms != programming contests" which is false. Algorithms are the essence of programming contests, just like 'machine learning' is the essence of Kaggle competitions. Open up Bishop's "Pattern Recognition and Machine learning" and you won't find anything about "deploying your model to production", but you will find lots of math on how to build models because that is what ML is. What the author seems to be describing is data-science.

yankoff 12y ago

Hm, looks like this argument about Kaggle pops up all the time. This is true, that getting and cleaning data is an important part of a workflow in data science and it's a necessary skill to have. But I don't believe anyone ever claimed Kaggle is enough to become a good data scientist. That's like studying algorithms is very useful, but who said it's enough to become good at software engineering? The skill consist of multiple parts which can be practiced separately or in combination in a real project where you do everything from scratch.

gentrificatron 12y ago
It's not as straightforward as Kaggle representing one stage in a pipeline. Kaggle also fixes this stage in an unnatural way. In real life, there is a feedback loop from modelling, to data processing. In Kaggle all you can do is improve your model. You can't go back and change how data was processed, or collect different sorts of data. Also, some flows simply don't fit into Kaggle's train/test paradigm. E.g. suppose you have an online algorithm that continuously updates parameters. Or situations where the model is used to generate the train/test data. Finally some situations may have stringent computational requirements, e.g. <100ms to classify a single instance.
- yankoff 12y ago
  Sure, everything "in real life" is more complicated. I think Kaggle to data science is as programming contests to Software Engineering. Will you say that doing programming challenges is "unnatural"? Is it useless or bad? Because you may also argue that you never have to solve problems for speed, you rarely see clearly-defined problems, you rarely get to apply complicated or rare algorithms in real life situation. The thing is Kaggle helps developing certain skills. It doesn't mean it helps you developing all necessary skills for your "real life". Your real life and your job may require unique combination of skills and the only way to fully prepare for it is to actually do real project that are specific to your job.

onislandtime 12y ago

There is too much hype, solving data-related problems requires many different skills and it depends on the problem. In general you need background in quite a few of these areas linear algebra, probability, statistical modeling, stochastic processes, computer science, programming, signal processing, etc. We used to call this electrical/computer engineering :-)

deleted 12y ago

[deleted]

blauwbilgorgel 12y ago

Why Machine Learning is Kaggle competitions. I used an out-of-the-box algorithm, messed around a bit, and definitely did not make the leaderboard. Because that is not Kaggle competitions. Nearly everyone on the leaderboard is proficient in data analysis and machine learning. They all tried that out-of-the-box algorithm for their attempt. But they did not give up so easily. Understand the business problem If you want to predict flight arrival times, what are you really trying to do? This is not different from Kaggle competitions, this is a tip for performing better in Kaggle competitions. See also the GE Flight Quest: https://www.gequest.com/c/flight https://www.gequest.com/c/flight Those winners used industry-standard machine learning, optimization techniques, but also creative insights and hunches, like tweaking the target labels: "A next step is to ask, “What should I actually be predicting?”. This is an important step that is often missed by many – they just throw the raw dependent variable into their favorite algorithm and hope for the best. But sometimes you want to create a derived dependent variable. I’ll use the GE Flight Quest as an example: you don't want to predict the actual time the airplane will land; you want to predict the length of the flight; and maybe the best way to do that is to use the ratio of how long the flight actually was to how long it was originally estimated to be and then multiply that times the original estimate." - Steve Donoho - http://blog.kaggle.com/2014/08/01/learning-from-the-best/ http://blog.kaggle.com/2014/08/01/learning-from-the-best/ Furthermore, it is entirely clear to everyone that data science in a business setting and in a competitive sport setting is different. To say they are equal, would be to say something like: paintball is equal to being in the military. But to say that Kaggle is not machine learning is to say: paintball requires no marksmanship. There are some very messy, unwieldy datasets on Kaggle right now. For example the Seizure Detection challenge has many GBs of raw sensor data, from just a few patients. This would require a competitor to clean, understand problem domain, understand evaluation metrics, measure cross validation and put your model in production on your laptop in the evening hours. The author of that blogpost is invited to team up, with me or others. Let's see if we can use machine learning to improve some pressing issues. I'd also love it if Stripe can host a contest on Kaggle.

mkrump 12y ago
Completely agree. Everything that is supposedly not addressed in Kaggle actually is, aside from productionizing and monitoring your model in production. Sure it's a simplified less open ended version of what you'll encounter in the real world, but that doesn't mean that many of the core concepts don't translate. It's kind of like saying that you shouldn't do your calculus practice problems, because math in the real world is never so straightforward. I think the most legitimate knock against Kaggle is that in many business settings there probably isn't much value in improving that extra .00001 (but obviously there are exceptions). Anyway, I think that that even someone very experienced in machine learning would learn something doing a Kaggle competition, especially if the competition is in an area outside their core expertise.

TheAlchemist 12y ago

I'm a quite new Kaggler, but did some similar machine learning competitions on similar plateforms and I have to say that this article is quite bullshit. How can a data scientist actually complain that the low value data munging work is already done for him ? Gosh, it should be the other way around - we should complain about how much of this we have to do at work !

deleted 12y ago

[deleted]

deleted 12y ago

[deleted]