5 ms·
Ask HN: List of open/public databases
A year or two ago somebody posted a list of open/free/accessible datasources for hackers to download and play around with. I thought this was a great resource, so I saved it, but heck if I can find it now.
Does anybody have such a list? Things like zip-codes for the US, locations of Starbucks stores, current weather forecasts, list of major newspapers, list of publicly-traded stocks, etc. I know there are tons of open/free databases waiting for us to mashup, just can't seem to find a list of them.
EDIT: The goal is a downloadable chunk of data to mashup, reformat, and use. That means CSV/XML/etc format and a public/anonymous FTP or something.
- robin_reala 16y agoFreebase is a big one: http://www.freebase.com/ http://www.freebase.com/ . Also, if you’re in the UK the government datasets might interest you: http://data.gov.uk/ http://data.gov.uk/
- gspyrou 16y agoYou may try taking a look at Microsoft Codename "Dallas" https://www.sqlazureservices.com/Catalog.aspx https://www.sqlazureservices.com/Catalog.aspx
- randomtask 16y agoYou might find the datasets/opendata sub-reddits interesting. http://www.reddit.com/r/datasets http://www.reddit.com/r/datasets http://www.reddit.com/r/opendata http://www.reddit.com/r/opendata
- sidmitra 16y agoDelicious is a good place for the same. Eg. http://delicious.com/sidmitra/datasets http://delicious.com/sidmitra/datasets
- IgorPartola 16y agohttp://themoviedb.org http://themoviedb.org, http://thetvdb.com/ http://thetvdb.com/ and my own API on top of these: http://igorpartola.com/projects/disciddb http://igorpartola.com/projects/disciddb
- DanielBMarkham 16y agoAs I continue to research this today, I grow more and more amazed at the lack of public data sets. There are what? A million different websites that keep some kind of list of something? And I'm finding only a dozen or two databases publicly accessible? It's crazy. Everybody either wants to charge or actively wants to prevent you from getting their data -- even when the data is common knowledge, was gathered at public expense, etc. Anybody from Google on? There has to be some shortcuts here somewhere.
- tszming 16y agoAmazon Public Data Sets: http://aws.amazon.com/publicdatasets/ http://aws.amazon.com/publicdatasets/ It is fast and easy to use if you are already using EC2.
- brown9-2 16y agohttp://flowingdata.com/2009/10/01/30-resources-to-find-the-data-you-need/ http://flowingdata.com/2009/10/01/30-resources-to-find-the-d... US Govt data on the stimulus: http://www.recovery.gov/FAQ/Pages/DownloadCenter.aspx http://www.recovery.gov/FAQ/Pages/DownloadCenter.aspx
- joubert 16y agoI have links to a few open (gov) data sets at the top of http://elev.at http://elev.at
- vili 16y agoYou can get all of Wikipedia here: http://en.wikipedia.org/wiki/Wikipedia_database http://en.wikipedia.org/wiki/Wikipedia_database
- iamelgringo 16y agohttp://infochimps.org/ http://infochimps.org/ http://www.data.gov/ http://www.data.gov/ http://data.gov.uk/ http://data.gov.uk/
- truiu 16y agoThe openstreetmap project lets you download its whole dataset: http://wiki.openstreetmap.org/wiki/Planet http://wiki.openstreetmap.org/wiki/Planet They also got a API ... see the wiki.
- warp 16y agoMusicBrainz has postgresql dumps available (partly public domain, partly cc-by-nc-sa): http://musicbrainz.org/doc/MusicBrainz_Database http://musicbrainz.org/doc/MusicBrainz_Database
- tszming 16y agoOpen Directory RDF Dump (e.g. Google Directory using it) http://rdf.dmoz.org/ http://rdf.dmoz.org/
- tomhaney 16y agoTry http://www.programmableweb.com/ http://www.programmableweb.com/
- binarymax 16y agohttp://data.worldbank.org/ http://data.worldbank.org/
- zackham 16y agoHow about weather - I have been looking around for an API to access historical weather data with no luck so far. Anyone familiar with the landscape of weather data options?
- joubert 16y agohttp://www.ncdc.noaa.gov/oa/mpp/freedata.html http://www.ncdc.noaa.gov/oa/mpp/freedata.html
- lostbit 16y agoLottery in Brazil (MegaSena) allow download of all results ever: http://www1.caixa.gov.br/loterias/loterias/megasena/download.asp http://www1.caixa.gov.br/loterias/loterias/megasena/download...
- lpolovets 16y agowww.factual.com has a lot of data, much of which is freely downloadable. There's also an API for data access. Examples: zip codes in the US: http://www.factual.com/t/MwwkkU/US_Zip_Codes http://www.factual.com/t/MwwkkU/US_Zip_Codes stock symbols: http://www.factual.com/t/jyqdWC/market_symbols http://www.factual.com/t/jyqdWC/market_symbols I work at Factual, so if you have feature requests/comments/complaints please let me know! I'm leo at factual dot com
- kinlane 16y agoA list of several data sources and tools to work with the data: http://www.kinlane.com/data/ http://www.kinlane.com/data/
- helwr 16y agohttp://www.datawrangling.com/some-datasets-available-on-the-web http://www.datawrangling.com/some-datasets-available-on-the-... http://archive.ics.uci.edu/ml/datasets.html http://archive.ics.uci.edu/ml/datasets.html http://data.worldbank.org/developers http://data.worldbank.org/developers
- cullenking 16y agoWe are running SRTM and NED elevation datasets, and have a web service (sinatra FTW!) exposing the data. I setup a variety of interpolation algorithms, which are fun to play with. If someone needs access to it and has a cool project, we are happy to provide it. Get ahold of me via email for info. The SRTM data is < 20 gigs compressed, and I downloaded it from some source, can't find the link now. It was a SLOWWWWWW download, so took many days. Reminded me of my old 2400 baud modem. the NED dataset is just shy of 500 gigs. We shipped a drive off to the USGS and they populated it and shipped it back. The 2 month wait sucked, but it was the only way to get it without paying a thousand bucks from a private company. I am happy to share either the SRTM or the NED dataset we have, but sharing it is pretty much limited to shipping hard drives around :(
- cjoh 16y agohttp://data.sunlightlabs.com http://data.sunlightlabs.com http://services.sunlightlabs.com http://services.sunlightlabs.com
- Cherian 16y agoStackoverflow(includes sister sites) dump: http://blog.stackoverflow.com/2010/07/creative-commons-data-dump-jul-10/ http://blog.stackoverflow.com/2010/07/creative-commons-data-...
- aslam333_333 16y agoaslam55555@yahoo.com
- aslam55555 16y agoaslam55555@yahoo.com
- aslam55555 16y agoaslam2222_2222@yahoo.com
- aslam55555 16y agoaslam11111_111111@yahoo.com
- aslam55555 16y agoaslam4444_4444@yahoo.com
- aslam55555 16y agokabir.hossen@rocketmail.com
- aslam55555 16y agoaslam666_666@yahoo.com
- aslam55555 16y agotaanu_t@yahoo.com
- aslam55555 16y agofari_ala2002@yahoo.com
- aslam55555 16y agofari_ala2002@yahoo.com
- aslam55555 16y agoa5555555555555559270@yahoo.com
- what 16y agohttp://dataincubator.org/ http://dataincubator.org/ Datasets about some Canadian cities: http://www.toronto.ca/open/datasets/web-map-services/ http://www.toronto.ca/open/datasets/web-map-services/ http://data.edmonton.ca/ http://data.edmonton.ca/ http://data.vancouver.ca/ http://data.vancouver.ca/