4 ms·

I'm an open source supporter, but why do a open version of Kaggle? Is not needed. Stop cloning ideas and claim an open source version, a good product is much m

I'm an open source supporter, but why do a open version of Kaggle? Is not needed. Stop cloning ideas and claim an open source version, a good product is much more than just the source code.

jonbaer 5y ago
They make better points on their readme, https://github.com/Cloud-CV/EvalAI https://github.com/Cloud-CV/EvalAI ... I wonder about the automation of the entire model and putting the winning solution right into production and it can be shared and openly available. Imagine you just describe a problem and let the crowd produce the best and the top solution is always available with zero work.
malux85 5y ago
I agree that a good product is more than just the source code but you shouldn’t discourage cloning ideas, because building more specialised versions, or enhancing features is much easier if the source code is available.
croes 5y ago
Because Kaggle is owned by Google, so it's unpredictable how long it will exist.
- wodenokoto 5y ago
  Since 2017! I had no idea until now. I guess that explains the access to computing power for all the notebooks.
dexter1691 5y ago
I am one of the members managing EvalAI. We started EvalAI in order to meet the demands of our own research lab (https://github.com/batra-mlp-lab https://github.com/batra-mlp-lab). In 2016, 2017, we were running the VQA competition on Codalab and it was not a pleasant experience -- the submissions would get stuck; the evaluation was slow; etc. We had the in-house expertise (two grad students) to build something better. We could manage the entire stack ourselves and easily add custom features (code-upload; custom metrics; private/remote evaluation; human-in-the-loop evaluation). Turns out that a lot of folks in the computer vision community were looking for something similar (130+ challenges; 30+ organizations). As we became more mature, multiple companies (Ebay, IBM, Mapillary) cloned EvalAI to host their own versions. They often contributed back their features and it was overall a net positive to have an open-source version. It also allows for faster iteration, experimentation (human-in-the-loop evaluation, Challenge Entries to Demo). For instance, recently PapersWithCode collaborated with us to deep-link leaderboard results on EvalAI with their own leaderboard tables https://twitter.com/paperswithcode/status/1341085285975789570 https://twitter.com/paperswithcode/status/134108528597578957...). I agree that the comparison to Kaggle is a bit old and we have removed it (https://github.com/Cloud-CV/EvalAI/pull/3502 https://github.com/Cloud-CV/EvalAI/pull/3502). :-)

I'm an open source supporter, but why do a open version of Kaggle? Is not needed. Stop cloning ideas and claim an open source version, a good product is much more than just the source code.

jonbaer 5y ago

They make better points on their readme, https://github.com/Cloud-CV/EvalAI https://github.com/Cloud-CV/EvalAI ... I wonder about the automation of the entire model and putting the winning solution right into production and it can be shared and openly available. Imagine you just describe a problem and let the crowd produce the best and the top solution is always available with zero work.

malux85 5y ago

I agree that a good product is more than just the source code but you shouldn’t discourage cloning ideas, because building more specialised versions, or enhancing features is much easier if the source code is available.

croes 5y ago

Because Kaggle is owned by Google, so it's unpredictable how long it will exist.

wodenokoto 5y ago
Since 2017! I had no idea until now. I guess that explains the access to computing power for all the notebooks.

dexter1691 5y ago

I am one of the members managing EvalAI. We started EvalAI in order to meet the demands of our own research lab (https://github.com/batra-mlp-lab https://github.com/batra-mlp-lab). In 2016, 2017, we were running the VQA competition on Codalab and it was not a pleasant experience -- the submissions would get stuck; the evaluation was slow; etc. We had the in-house expertise (two grad students) to build something better. We could manage the entire stack ourselves and easily add custom features (code-upload; custom metrics; private/remote evaluation; human-in-the-loop evaluation). Turns out that a lot of folks in the computer vision community were looking for something similar (130+ challenges; 30+ organizations). As we became more mature, multiple companies (Ebay, IBM, Mapillary) cloned EvalAI to host their own versions. They often contributed back their features and it was overall a net positive to have an open-source version. It also allows for faster iteration, experimentation (human-in-the-loop evaluation, Challenge Entries to Demo). For instance, recently PapersWithCode collaborated with us to deep-link leaderboard results on EvalAI with their own leaderboard tables https://twitter.com/paperswithcode/status/1341085285975789570 https://twitter.com/paperswithcode/status/134108528597578957...). I agree that the comparison to Kaggle is a bit old and we have removed it (https://github.com/Cloud-CV/EvalAI/pull/3502 https://github.com/Cloud-CV/EvalAI/pull/3502). :-)