4 ms·

Ask HN: What are the funnest data sets you’ve played with?

Trying to battle test our startup product to help users prepare data for data science, looking for some interesting data sets to play with. Ideally something publically downloadable, non controversial and under 1 GB.

gennarro 4y ago
US auto data is a lot of fun. Useful for all sorts of projects. Data: https://www.transportation.gov/developer https://www.transportation.gov/developer Example: https://transportation.report/ https://transportation.report/ Well under 1gb if you focus on models and specs, not safety!
giraffe_lady 4y ago
Destiny 2 has an api that a lot of great third-party organizational tools are built around. But you can also download a sqlite file with all the set game info in it: weapons, abilities, perks, items, upgrades etc. I haven't messed with it in a while but I remember it being pretty small, one or two hundred mb. It's highly normalized, to the point where it goes backwards and starts to actually interfere with human usability. Not sure if that helps your goal or hinders it. I used to use it for teaching both sql and the pros/cons/usefulness of the normal forms.
vital_beach 4y ago
Wahpedia has a pretty unusual Warhammer 40k dataset general overview: https://wahapedia.ru/wh40k9ed/the-rules/data-export/ https://wahapedia.ru/wh40k9ed/the-rules/data-export/ spreadsheet with dataset links: https://wahapedia.ru/wh40k9ed/Export%20Data%20Specs.xlsx https://wahapedia.ru/wh40k9ed/Export%20Data%20Specs.xlsx
- neerd 4y ago
  Damn, I wish I’d known this existed. I ended up building a parser for the raw battle-scribe xml files
  - vital_beach 4y ago
    that's honestly a better route. battlescribe has more data, but the schema is a nightmare and gives me headaches.
mhh__ 4y ago
Not super exciting but tire data# is both hard to model and very expensive. # Data from spinning a tire while crushing it into a "road" at an angle
connordoner 4y ago
I took a list of UK Parliamentary constituencies and took a while to match each with their International Territorial Levels (the post-Brexit version of NUTS). It’s here as a CSV if you’d like it: https://gist.github.com/connordoner/9cda1857b8fff5b8e042013d9ad8c17d https://gist.github.com/connordoner/9cda1857b8fff5b8e042013d.... There’s no license attached so do as you wish.
chunkyks 4y ago
Early in covid, I played with a famous 65-year-venerated dataset from where I work. Found some interesting minutiae and ended up on the front page of the wsj over it. https://github.com/RANDCorporation/milliondigits https://github.com/RANDCorporation/milliondigits It's probably not what you're looking for, but it's my own favorite dataset.
trynewideas 4y ago
I learned GIS from John Mechalas's effort to georeference and create a base world map for the Pathfinder Roleplaying Game: https://dungeonetics.com/golarion-geography/index.html https://dungeonetics.com/golarion-geography/index.html
hnthrowaway1099 4y ago
Kaggle has a fair amount of datasets https://www.kaggle.com/datasets?fileType=csv https://www.kaggle.com/datasets?fileType=csv

gennarro 4y ago

US auto data is a lot of fun. Useful for all sorts of projects. Data: https://www.transportation.gov/developer https://www.transportation.gov/developer Example: https://transportation.report/ https://transportation.report/ Well under 1gb if you focus on models and specs, not safety!

giraffe_lady 4y ago

Destiny 2 has an api that a lot of great third-party organizational tools are built around. But you can also download a sqlite file with all the set game info in it: weapons, abilities, perks, items, upgrades etc. I haven't messed with it in a while but I remember it being pretty small, one or two hundred mb. It's highly normalized, to the point where it goes backwards and starts to actually interfere with human usability. Not sure if that helps your goal or hinders it. I used to use it for teaching both sql and the pros/cons/usefulness of the normal forms.

vital_beach 4y ago

Wahpedia has a pretty unusual Warhammer 40k dataset general overview: https://wahapedia.ru/wh40k9ed/the-rules/data-export/ https://wahapedia.ru/wh40k9ed/the-rules/data-export/ spreadsheet with dataset links: https://wahapedia.ru/wh40k9ed/Export%20Data%20Specs.xlsx https://wahapedia.ru/wh40k9ed/Export%20Data%20Specs.xlsx

neerd 4y ago
Damn, I wish I’d known this existed. I ended up building a parser for the raw battle-scribe xml files
- vital_beach 4y ago
  that's honestly a better route. battlescribe has more data, but the schema is a nightmare and gives me headaches.

mhh__ 4y ago

Not super exciting but tire data# is both hard to model and very expensive. # Data from spinning a tire while crushing it into a "road" at an angle

connordoner 4y ago

I took a list of UK Parliamentary constituencies and took a while to match each with their International Territorial Levels (the post-Brexit version of NUTS). It’s here as a CSV if you’d like it: https://gist.github.com/connordoner/9cda1857b8fff5b8e042013d9ad8c17d https://gist.github.com/connordoner/9cda1857b8fff5b8e042013d.... There’s no license attached so do as you wish.

chunkyks 4y ago

Early in covid, I played with a famous 65-year-venerated dataset from where I work. Found some interesting minutiae and ended up on the front page of the wsj over it. https://github.com/RANDCorporation/milliondigits https://github.com/RANDCorporation/milliondigits It's probably not what you're looking for, but it's my own favorite dataset.

trynewideas 4y ago

I learned GIS from John Mechalas's effort to georeference and create a base world map for the Pathfinder Roleplaying Game: https://dungeonetics.com/golarion-geography/index.html https://dungeonetics.com/golarion-geography/index.html

hnthrowaway1099 4y ago

Kaggle has a fair amount of datasets https://www.kaggle.com/datasets?fileType=csv https://www.kaggle.com/datasets?fileType=csv