7 ms·
Show HN: Datasets.co – Share and discover new machine learning datasets
- tryitnow 11y agoI was skeptical when I first saw this because there seem to be many other efforts to collect datasets. But most of those efforts are pretty poor because they're unnecessarily difficult to understand. I like how the data is clearly described with "Feature/Description/Example", that gives me most of what I want to know at a first glance.
- mrborgen 11y agoThanks! Would you also be interested in having a 'data preview' feature, where you'd see a tiny part of the data in the browser?
- stared 11y agoIt would be nice to have some estimate of the data size (e.g. number of rows).
- mrborgen 11y agoHey, on datasets added from now on, this'll be an option. Check it out in action here (displayed under the image). http://www.datasets.co/dataset/All-UFC-Fights http://www.datasets.co/dataset/All-UFC-Fights Was that what you were thinking?
- stared 11y agoYes, it's what I meant. But in this case there are 19 features (as said) or 10 features (no of columns)?
- danso 11y agoA couple things I would suggest. Besides places like Stanford's SNAP, I would check out Socrata's OpenData portal [0], which is a place where anyone can post datasets, and the Open Data Network portal [1], which searches across all of the Socrata city portals...it's stunning what's out there. The NYPD's release of felony data is nice (if many years late), but it's nothing compared to what cities such as Chicago, Dallas, and Los Angeles have, in terms of quantity of records and detail (and the NYPD's stop-and-frisk data -- which the ACLU forced them to release -- is also much more voluminous). I would consider providing numbers about the datasets in the default view, such as number of observations and variables. That's probably the biggest weakness, IMO, of current data portals (including data.gov)...you have to click through every link to then find out there's not much data in the set. In your situation, this applies to several of the things you've included...that Gun Ownership and Crime Rates set, for example...unless I'm missing something, but that has fewer than 40 observations, and relies on highly questionable numbers from the FBI [2] on a nationwide level...that can't possibly be of any use in a machine learning context, can it? I'm surprised it's even the basis for an academic paper (though kudos to the authors for posting their work). If you still think it's worth keeping that dataset, it'd be nice to know # of observations before having to click through. [0] https://opendata.socrata.com https://opendata.socrata.com [1] http://www.opendatanetwork.com/ http://www.opendatanetwork.com/ [2] http://www.jsonline.com/watchdog/watchdogreports/fbi-crimereporting-audits-are-shallow-infrequent-cg5uvel-166665516.html http://www.jsonline.com/watchdog/watchdogreports/fbi-crimere...
- mrborgen 11y agoThanks for the input! The variables are already described, if it was the features you were thinking about?
- mrborgen 11y agoWas a bit quick comment last time. Yes, having the amount of observations is a very natural feature to add. Shiuld be fairly simple as well, so it'll be added soon!
- mrborgen 11y agoHey all!I built this site because I think it should be easier and more fun to discover new datasets. I'd love hear your thoughts, suggestions and critisism!
- phatbyte 11y agoThis is awesome, as someone who is currently playing with ML this is a great resource for playing with Datasets. Thanks to everyone involved in this project. Keep it up.
- danso 11y agoAwhile ago I put together a list of big (100,000+ rows) and public datasets...not all of them are ideal for machine learning applications but you'll probably find a few worth sharing: http://cjlab.stanford.edu/2015/09/30/lab-launch-and-data-sets/ http://cjlab.stanford.edu/2015/09/30/lab-launch-and-data-set...
- mrborgen 11y agoAwesome list, thanks!
- uberneo 11y agoGood datasets source .. one point - you can register an empty data set . You might want to fix that.. rest all looks great .. good job