10 ms·

Finding the perfect house using open data

202 points by phiggy 12y ago

cjoh 12y ago
The biggest users of the open data movement are, in fact, real estate professionals. It's companies like Trulia and Estately and Zillow and what not that are actually taking open data from government and putting it to good use. It'd be interesting to see some kind of startup come out of this. Though it's fairly hands-on, real estate is still a fairly hands on market, and with real estate lobbying being what it is, it's likely that agents aren't going anywhere anytime soon. Building them tools that allow them to make better recommendations to their clients or to find better buyers might be something really interesting.
- toomuchtodo 12y ago
  Don't forget http://www.redfin.com http://www.redfin.com ; their presentation of properties with the associated public data is beautiful. No association, just a huge fan.
  - atourgates 12y ago
    Redfin really is a great real-estate app. A few years ago, I shopped for a home in an area that Redfin serviced. The data they had and the way the presented it made it easy to find the house we wanted, and figure out what we would be comfortable paying for it. A few weeks ago we closed on a house in a non-Redfin serviced area. The shopping experience was night and day. Here, none of the local MLS databases, or Zillow or Truila, list things like past purchase prices, and even generally hide past listing prices. I'm not sure if that has to do with local regulations here (ID) vs. the last place we tried to buy a house (CA), or if it's something specific to Redfin. Either way, buying a house in a non-Redfin service area involved quite a few extra trips to the local courthouse, and made me really appreciate the data it offers.
    - toomuchtodo 12y ago
      > The shopping experience was night and day. Here, none of the local MLS databases, or Zillow or Truila, list things like past purchase prices, and even generally hide past listing prices. I'm not sure if that has to do with local regulations here (ID) vs. the last place we tried to buy a house (CA), or if it's something specific to Redfin. Yes, this is typically because of realtor and MLS database regulations.
    - mrfusion 12y ago
      Was this comment corporate sponsored?
  - opendais 12y ago
    Ya, I used them to buy a condo and find my realtor. It was a good experience. :D
- Hovertruck 12y ago
  I know it's anecdotal, but most realtors I have met actively dislike Trulia/Zillow/etc
  - smacktoward 12y ago
    That's right, because they generally have their own proprietary source of data (http://en.wikipedia.org/wiki/Multiple_listing_service http://en.wikipedia.org/wiki/Multiple_listing_service), which is a big part of the appeal of working with a realtor -- they learn of new listings faster than you can by relying on public, "open source" data. And realtors aren't stupid; they noticed what happened when other professions that were built on closed data opened their databases up to the public. Travel agents, for instance, used to be the only ones who could search the SABRE database (http://en.wikipedia.org/wiki/Sabre_%28computer_system%29 http://en.wikipedia.org/wiki/Sabre_%28computer_system%29), which made it valuable to work with an agent when making your travel plans. Then in the '80s the company behind SABRE started opening it up to the public, first via services like CompuServe and AOL, and then later via the Web. Suddenly travelers didn't need a travel agent anymore; they could get the same booking data right from their computer. And travel agencies as a business sector promptly collapsed (http://www.cnn.com/2013/10/03/travel/travel-agent-survival/ http://www.cnn.com/2013/10/03/travel/travel-agent-survival/). For this reason, the real estate agents have little desire to see MLS data show up in publicly-accessible services like Zillow. The current state of affairs appears to be that it does eventually flow out to them, but with a significant (1 week+) delay -- which lets the agents preserve their status as the people you have to talk to if you want the freshest data.
    - Domenic_S 12y ago
      > The current state of affairs appears to be that it does eventually flow out to them, but with a significant (1 week+) delay I don't think that's true, at least not for Redfin. According to them: "Homes on the MLS show up on Redfin in 30 minutes or less." - http://forums.redfin.com/t5/Inland-Empire/How-quickly-does-MLS-info-get-posted-on-redfin/td-p/433041 http://forums.redfin.com/t5/Inland-Empire/How-quickly-does-M... "Local brokerage sites like Redfin show homes for sale immediately, as opposed to the seven to nine day delay typical for the national portals." - http://blog.redfin.com/blog/2013/03/you-deserve-to-see-it-all.html#.U5Yj9S-yPjU http://blog.redfin.com/blog/2013/03/you-deserve-to-see-it-al... "Updates every 15-30 minutes" - http://www.redfin.com/about/data-quality-on-redfin http://www.redfin.com/about/data-quality-on-redfin The biggest advantage of a local (real human) broker is they know what's about to go up for sale, have detailed neighborhood info, can help grease the wheels of your deal, etc. If they're good.
- kaylarose 12y ago
  I don't know if Redfin uses the same feeds, but their update times seem vastly superior to Trulia, Zillow & Estately. After spending 1 1/2 years trying to find a house in a turnkey-scarce market, and using RedFin (after it launched in our market a few months ago), Estately, Zillow, and Trulia in parallel - this was the killer feature of Redfin for me. For instance, I knew an offer we had made on a house was going to fall through when I got an alert from Redfin half hour after sending the offer, that the house was pending. Similarly - I found, toured, offered, and signed on a house (and was alerted to the "new listing", "pending", and "sold" status by redfin) before Zillow even had is registered as "For Sale". So I am genuinely curious how their feeds differ from the rest. I originally assumed they all just scraped the MLS, but maybe this isn't the case...?
  - zippergz 12y ago
    Redfin is an actual real estate brokerage, and are therefore real members of the MLS, with access to the data that provides. The others are simply scraping via publicly available data. This is a major difference.
    - gdudeman 12y ago
      Close, except Estately gets access to MLS data too. (Founder & CEO here)
      - mrfusion 12y ago
        Do you guys have an API? I have a couple cool ideas for apps bit don't know where to get data?
        gdudeman 12y ago
        Hi mrfusion, we unfortunately have strict licensing restrictions on our data. Sorry!
    - smackfu 12y ago
      Is it a major difference, or just a matter of timeliness? The actual data that Zillow etc. has seems identical to any other service, and the same as the printouts that realtors give out that they are printing from MLS.
ChuckFrank 12y ago
Can you add a legend to your final map? I'm not sure what the different colors represent.
- Caged 12y ago
  I'll try to get one on there. In the meantime, the colors represent rankings (scored by feature intersections). You can see the ranking selectors here (pink, blue, green, yellow-green): https://cloud.githubusercontent.com/assets/25/3075424/5497c28a-e376-11e3-9d74-3d8430519db2.png https://cloud.githubusercontent.com/assets/25/3075424/5497c2....
  - jaggederest 12y ago
    Thought I'd mention to you that your 'supermarkets' data is pretty out of date or marginal quality. I see at least 3-4 supermarkets opened in the last year not on there.
frik 12y ago
Great, way to go! Reminds me of Bill Gates book "The Road Ahead" (1st ed, 1995, p. 158) - it's about his vision of a next-gen network that would replace the middle-man beside other innovative things: [...] most market places are very inefficient. For instance, if you are trying to find a doctor, lawyer, accountant, or similar professional, or are buying a house, information is incomplete and comparisons are difficult to make. The information highway will extend the electronic marketplace and make it the ultimate go-between, the universal middleman. Often the only human involved in a transaction will be the actual buyer and seller. All the goods for sale in the world will be available for you to examine, compare, and, often, customize. When you want to buy something you'll be able to tell your computer to find it for you at the best price offered by any acceptable source or ask your computer to 'haggle' with the computers of various sellers. Information about venders and their products will be available to any computer connected to the highway. Servers distributed worldwide will accept bids, resolve offers into completed transactions, control authentication and security, and handle all other aspects of the marketplace, including the transfer of funds. This will carry us into a new world of low-friction, low-overhead capitalism, in which market information will be plentiful and transaction costs low. It will be a shopper's heaven. [...]
- gohrt 12y ago
  Amazing that the book was written by a man who failed to communicate that (rather obvious) idea to the employees of the multi-billion dollar company he owned and directly managed.
  - drzaiusapelord 12y ago
    He had the financial incentive to not make that happen, namely to lock people onto IE/Windows and direct them out of AOL and into a 1990's MSN walled garden. If anything, Gates the futurist vs Gates the businessperson shows us that, regardless of what the pro-lassiez faire types say, most often corporations left unfettered don't product the best results, but only results that directly benefit themselves and that often translates into things like vendor lock-in, poor software, poor support, lack of progress. I mean, does Gates even question why the market for the things he complains about are so inefficient? Its because its full of guys like himself.
thrownaway2424 12y ago
Might be worth combining with walkscore data? http://www.walkscore.com/professional/research.php http://www.walkscore.com/professional/research.php Not sure how freely they distribute this.
- oniTony 12y ago
  walkscore can already map out all "within walking distance of grocery + rail" locations in Portland (and elsewhere). E.g. http://www.walkscore.com/apartments/search/Portland-OR?zoom=12&sort=14_low&hood=off&nearby=%7B%7D&places=%5B%5B%22Groceries%22%2C%2210%22%5D%5D&transit=%5B30%2C%2219.81.83-85.99%22%2C45.537169%2C-122.65015900000003%5D&lat=45.535517621410435&lng=-122.64287605285642 http://www.walkscore.com/apartments/search/Portland-OR?zoom=... The actual scores would have to be looked up one at a time, on per-address basis (e.g. http://www.walkscore.com/score/320-pioneer-way-mountain-view-ca-94041 http://www.walkscore.com/score/320-pioneer-way-mountain-view... )
mrfusion 12y ago
Where do you get the house listing data? Do you then geo code the addresses and filter them against your acceptable locations? (Sorry if I missed reading that part)
- Caged 12y ago
  Are you referring to real estate listings? In my case, I use Trulia to search the zones after I narrow down the locations. However, if you're referring to the building footprints I have a Make target that pulls GIS data from public sources: https://github.com/caged/portland-atlas https://github.com/caged/portland-atlas
  - mrfusion 12y ago
    I guess I'm asking how you find only listings in the zones you want? Does trulia have an API?
- vlandham 12y ago
  Started a similar repo for Seattle, WA data here: https://github.com/vlandham/seattle-atlas https://github.com/vlandham/seattle-atlas
schooldistrict 12y ago
http://schooldistrictfinder.com/ http://schooldistrictfinder.com/ Finding the perfect school (one of the many steps to find the perfect house) using open data
- huherto 12y ago
  Great, Just today I was thinking about this. So, how do I know the district area. I can see the pin. Is that the center? Sorry, I am just getting familiar with this subject in the US.
akgerber 12y ago
The "moneyball" move is buying land right before the mass transit extension is completed and the neighborhood develops: http://en.wikipedia.org/wiki/MAX_Orange_Line http://en.wikipedia.org/wiki/MAX_Orange_Line But this is a really fun article on how to use open-source GIS tools to do something practical, which is something I've wanted to delve into for a while— I'll try it later tonight, thanks!
- jweir 12y ago
  Yeah, Governor Christie's brother likes that strategy. "Governor Christie's brother invested in real estate near new PATH station in Harrison" http://www.northjersey.com/news/governor-christie-s-brother-invested-in-real-estate-near-new-path-station-in-harrison-1.667498 http://www.northjersey.com/news/governor-christie-s-brother-...
  - akgerber 12y ago
    It's only problematic when you're doing it before the extension is public. Likewise, property on Second Avenue on the Upper East Side of Manhattan has been cheap lately, because blasting out subway stations is unpleasant. But a new subway coming to an area that was recently relatively far from one means property values are shooting up.
aembleton 12y ago
Wow; this is great to see. I have recently done something similar in the Greater Manchester, UK area. I've been looking for a house and wrote some code to scrape rightmove.co.uk and zoopla.com to get the house data. Checked their locations against data from http://data.police.uk/ http://data.police.uk/. Checked time to walk to the nearest metro or railway station and time to cycle to work. Metro and Railway station locations were scraped from wikipedia and time to travel was taken from Google Maps. From all this, as well as £s per m^2 of floor space I ordered by the least compromise. Currently trying to clean it all up and turn it into a webapp to make it useful for anyone else who wants to find property in an area. I'm about to complete on the purchase of my first house and am really looking forward to it. It's not perfect; but I know for the price, I can't get much better. If I finish the webapp; I might try and include data such as walking distance to the nearest pub from CAMRAs Good Beer Guide and maybe something similar for restaurants.
- freyfogle 12y ago
  You may wish to check out Nestoria's API of property listings and house price trends http://www.nestoria.co.uk/help/api http://www.nestoria.co.uk/help/api
Jemaclus 12y ago
I work in online real estate, and this is exactly the kind of thing we love to do when it comes to utilizing data. How can we take X, Y and Z data points and generate something useful? Taking into account crime reports, graffiti reports, available parking spaces, etc, can all help you decide whether this place is good or not. The requirements here are kinda strange (proximity to a grocery store? not crime rates or anything like that?), but to each his own, I suppose. I'm really impressed with the resourcefulness of the author, though. I think I would probably just go to Zillow or Trulia and see what they say...
- pdx 12y ago
  > (proximity to a grocery store? not crime rates or anything > like that?), but to each his own, Some people are not able to, or do not wish to drive every time they want something to eat. Proximity to a grocery store sounds like a good metric. It's the european model of shopping every day for that days meal, vs the U.S. model of stocking up at Costco.
  - Jemaclus 12y ago
    True. I was more remarking on the fact that the grocery store metric seemed to be the primary one, as opposed to just one of a great many. It's definitely the kind of thing I would consider, but I'd also consider more things like price per square foot, number of bedrooms, neighborhood quality, distance to schools, etc. That's why I said "kinda strange" and not something harsher.
- Caged 12y ago
  > The requirements here are kinda strange (proximity to a grocery store? not crime rates or anything like that?), but to each his own, I suppose. You're right. I've been working with open data in the Portland area for years now [1][2][3][4] and have a lot of built up internal knowledge of the neighborhoods and locations so I chose not to muddy the map with those things. 1: https://github.com/Caged/portlandcrime https://github.com/Caged/portlandcrime 2: https://github.com/caged/portland-atlas#examples https://github.com/caged/portland-atlas#examples 3: https://github.com/Caged/portland-census https://github.com/Caged/portland-census 4: http://labratrevenge.com/pdx http://labratrevenge.com/pdx
qrybam 12y ago
I'm working on something exactly like this at the moment (not US) - it's super exciting to see people make use of the data available to them. Kudos :)
misuba 12y ago
Finding unaffordable neighborhoods, more like.
- Eric_WVGG 12y ago
  ikr. Three problems about this article. First, he better be rich if he’s serious about living in these zones. Second, his data analysis is blowing past a multitude of equally livable, more affordable PDX neighborhoods. Third, within his zones are actual unlivable subzones (nobody wants to live in actual-downtown-portland). Impressive analysis, but he'd be better served by living for a month in a PDX AirBnB and learning his way around the city.
fryguy 12y ago
When I was looking for a place to live, I made my own program to scrape the MLS, and did similar things to this. My rectangles were hand drawn, not scraped from open data though, and were more based on the neighborhoods that had houses I knew I didn't like.
gdudeman 12y ago
This is awesome. I love analysis like this. Funny enough, something very similar to this was the impetus for me starting Estately. Early on, we had a bunch of wild spatial queries that would let you even filter by distance to specific transit line stops: http://blog.estately.com/wp-content/uploads/2008/06/near_transit.gif http://blog.estately.com/wp-content/uploads/2008/06/near_tra... When we ran stats on what searches people were actually doing, we found that very few consumers actually used these kinds of filters. We found we had a small and passionate audience for specific geo features and a much larger audience for "great site to search for and learn about homes." It took me way to long to come to terms with this. We still have walkscore search and distance from a neighborhood, school or address search even though they are very little used. Why wasn't it popular? My guess is it's 80% because the market was small and 20% because we didn't present them right. The thing about real estate - especially in the current market - is there just are few enough homes for sale that meet your price / proximity to work / school / bedrooms criteria that most people want to just glance at all of them and rule them in or out. Additionally, most people start with a bunch of "rules," then they bend one, change their mind on another and finally compromise on the last one. The typical home search starts so clean, but dissolves over time as you tour houses, learn more about what you do and don't like and find something that works.
- Dwolb 12y ago
  Did you ever try to give people the option to filter by school district or school district ranking?
  - gdudeman 12y ago
    We let people search by specific school and we are considering adding school district ranking soon. We've had a bunch of users requesting it.
    - Dwolb 12y ago
      seems like one of the big factors that affects property values. maybe since it's so important some people would rather search by top school within a radius or boundary than know the district ranking associated with the house.
chrisamiller 12y ago
I was worried about commute times from three different points when looking for a house, so I wrote a quick and dirty little ruby script to hit the Google Driving Directions API. It tiles across the area you specify and spits out a kml file that can be loaded into google earth and lets you see which areas fit your parameters (https://github.com/chrisamiller/commute-times https://github.com/chrisamiller/commute-times). There's lots of room for improvement in the code, but it's in the same vein as what this guy did and helped us quite a bit.
joshdotsmith 12y ago
I didn't do anything this intense, but back in 2008 I took some data I gathered from city-data and mapped neighborhood ratings for San Diego. After living here for 6 years they're still pretty accurate. https://www.google.com/maps/@32.7740233,-117.1799609,12z/data=!4m2!6m1!4s215611903328390102803.0004515da0f446355193e https://www.google.com/maps/@32.7740233,-117.1799609,12z/dat... I'd just take an address from Craigslist and paste it in to quickly see whether I wanted to live there. Today I live in one of the "bit dumpy and rough, but OK" neighborhoods.
- petersellers 12y ago
  Thanks, this is really interesting. I love how Kensington turns from "Affluent" to "Gang-influenced" as soon as you cross El Cajon Blvd. That might be a slight exaggeration, but I've definitely noticed a few times walking through there how drastically the feel of the neighborhood changes after just a block or two.
andbberger 12y ago
My startup does exactly this and we just released! https://www.dwellaware.com/ https://www.dwellaware.com/

202 points by phiggy 12y ago

cjoh 12y ago

The biggest users of the open data movement are, in fact, real estate professionals. It's companies like Trulia and Estately and Zillow and what not that are actually taking open data from government and putting it to good use. It'd be interesting to see some kind of startup come out of this. Though it's fairly hands-on, real estate is still a fairly hands on market, and with real estate lobbying being what it is, it's likely that agents aren't going anywhere anytime soon. Building them tools that allow them to make better recommendations to their clients or to find better buyers might be something really interesting.

toomuchtodo 12y ago
Don't forget http://www.redfin.com http://www.redfin.com ; their presentation of properties with the associated public data is beautiful. No association, just a huge fan.
- atourgates 12y ago
  Redfin really is a great real-estate app. A few years ago, I shopped for a home in an area that Redfin serviced. The data they had and the way the presented it made it easy to find the house we wanted, and figure out what we would be comfortable paying for it. A few weeks ago we closed on a house in a non-Redfin serviced area. The shopping experience was night and day. Here, none of the local MLS databases, or Zillow or Truila, list things like past purchase prices, and even generally hide past listing prices. I'm not sure if that has to do with local regulations here (ID) vs. the last place we tried to buy a house (CA), or if it's something specific to Redfin. Either way, buying a house in a non-Redfin service area involved quite a few extra trips to the local courthouse, and made me really appreciate the data it offers.
  - toomuchtodo 12y ago
    > The shopping experience was night and day. Here, none of the local MLS databases, or Zillow or Truila, list things like past purchase prices, and even generally hide past listing prices. I'm not sure if that has to do with local regulations here (ID) vs. the last place we tried to buy a house (CA), or if it's something specific to Redfin. Yes, this is typically because of realtor and MLS database regulations.
  - mrfusion 12y ago
    Was this comment corporate sponsored?
- opendais 12y ago
  Ya, I used them to buy a condo and find my realtor. It was a good experience. :D
Hovertruck 12y ago
I know it's anecdotal, but most realtors I have met actively dislike Trulia/Zillow/etc
- smacktoward 12y ago
  That's right, because they generally have their own proprietary source of data (http://en.wikipedia.org/wiki/Multiple_listing_service http://en.wikipedia.org/wiki/Multiple_listing_service), which is a big part of the appeal of working with a realtor -- they learn of new listings faster than you can by relying on public, "open source" data. And realtors aren't stupid; they noticed what happened when other professions that were built on closed data opened their databases up to the public. Travel agents, for instance, used to be the only ones who could search the SABRE database (http://en.wikipedia.org/wiki/Sabre_%28computer_system%29 http://en.wikipedia.org/wiki/Sabre_%28computer_system%29), which made it valuable to work with an agent when making your travel plans. Then in the '80s the company behind SABRE started opening it up to the public, first via services like CompuServe and AOL, and then later via the Web. Suddenly travelers didn't need a travel agent anymore; they could get the same booking data right from their computer. And travel agencies as a business sector promptly collapsed (http://www.cnn.com/2013/10/03/travel/travel-agent-survival/ http://www.cnn.com/2013/10/03/travel/travel-agent-survival/). For this reason, the real estate agents have little desire to see MLS data show up in publicly-accessible services like Zillow. The current state of affairs appears to be that it does eventually flow out to them, but with a significant (1 week+) delay -- which lets the agents preserve their status as the people you have to talk to if you want the freshest data.
  - Domenic_S 12y ago
    > The current state of affairs appears to be that it does eventually flow out to them, but with a significant (1 week+) delay I don't think that's true, at least not for Redfin. According to them: "Homes on the MLS show up on Redfin in 30 minutes or less." - http://forums.redfin.com/t5/Inland-Empire/How-quickly-does-MLS-info-get-posted-on-redfin/td-p/433041 http://forums.redfin.com/t5/Inland-Empire/How-quickly-does-M... "Local brokerage sites like Redfin show homes for sale immediately, as opposed to the seven to nine day delay typical for the national portals." - http://blog.redfin.com/blog/2013/03/you-deserve-to-see-it-all.html#.U5Yj9S-yPjU http://blog.redfin.com/blog/2013/03/you-deserve-to-see-it-al... "Updates every 15-30 minutes" - http://www.redfin.com/about/data-quality-on-redfin http://www.redfin.com/about/data-quality-on-redfin The biggest advantage of a local (real human) broker is they know what's about to go up for sale, have detailed neighborhood info, can help grease the wheels of your deal, etc. If they're good.
kaylarose 12y ago
I don't know if Redfin uses the same feeds, but their update times seem vastly superior to Trulia, Zillow & Estately. After spending 1 1/2 years trying to find a house in a turnkey-scarce market, and using RedFin (after it launched in our market a few months ago), Estately, Zillow, and Trulia in parallel - this was the killer feature of Redfin for me. For instance, I knew an offer we had made on a house was going to fall through when I got an alert from Redfin half hour after sending the offer, that the house was pending. Similarly - I found, toured, offered, and signed on a house (and was alerted to the "new listing", "pending", and "sold" status by redfin) before Zillow even had is registered as "For Sale". So I am genuinely curious how their feeds differ from the rest. I originally assumed they all just scraped the MLS, but maybe this isn't the case...?
- zippergz 12y ago
  Redfin is an actual real estate brokerage, and are therefore real members of the MLS, with access to the data that provides. The others are simply scraping via publicly available data. This is a major difference.
  - gdudeman 12y ago
    Close, except Estately gets access to MLS data too. (Founder & CEO here)
    - mrfusion 12y ago
      Do you guys have an API? I have a couple cool ideas for apps bit don't know where to get data?
      - gdudeman 12y ago
        Hi mrfusion, we unfortunately have strict licensing restrictions on our data. Sorry!
  - smackfu 12y ago
    Is it a major difference, or just a matter of timeliness? The actual data that Zillow etc. has seems identical to any other service, and the same as the printouts that realtors give out that they are printing from MLS.

ChuckFrank 12y ago

Can you add a legend to your final map? I'm not sure what the different colors represent.

Caged 12y ago
I'll try to get one on there. In the meantime, the colors represent rankings (scored by feature intersections). You can see the ranking selectors here (pink, blue, green, yellow-green): https://cloud.githubusercontent.com/assets/25/3075424/5497c28a-e376-11e3-9d74-3d8430519db2.png https://cloud.githubusercontent.com/assets/25/3075424/5497c2....
- jaggederest 12y ago
  Thought I'd mention to you that your 'supermarkets' data is pretty out of date or marginal quality. I see at least 3-4 supermarkets opened in the last year not on there.

frik 12y ago

Great, way to go! Reminds me of Bill Gates book "The Road Ahead" (1st ed, 1995, p. 158) - it's about his vision of a next-gen network that would replace the middle-man beside other innovative things: [...] most market places are very inefficient. For instance, if you are trying to find a doctor, lawyer, accountant, or similar professional, or are buying a house, information is incomplete and comparisons are difficult to make. The information highway will extend the electronic marketplace and make it the ultimate go-between, the universal middleman. Often the only human involved in a transaction will be the actual buyer and seller. All the goods for sale in the world will be available for you to examine, compare, and, often, customize. When you want to buy something you'll be able to tell your computer to find it for you at the best price offered by any acceptable source or ask your computer to 'haggle' with the computers of various sellers. Information about venders and their products will be available to any computer connected to the highway. Servers distributed worldwide will accept bids, resolve offers into completed transactions, control authentication and security, and handle all other aspects of the marketplace, including the transfer of funds. This will carry us into a new world of low-friction, low-overhead capitalism, in which market information will be plentiful and transaction costs low. It will be a shopper's heaven. [...]

gohrt 12y ago
Amazing that the book was written by a man who failed to communicate that (rather obvious) idea to the employees of the multi-billion dollar company he owned and directly managed.
- drzaiusapelord 12y ago
  He had the financial incentive to not make that happen, namely to lock people onto IE/Windows and direct them out of AOL and into a 1990's MSN walled garden. If anything, Gates the futurist vs Gates the businessperson shows us that, regardless of what the pro-lassiez faire types say, most often corporations left unfettered don't product the best results, but only results that directly benefit themselves and that often translates into things like vendor lock-in, poor software, poor support, lack of progress. I mean, does Gates even question why the market for the things he complains about are so inefficient? Its because its full of guys like himself.

thrownaway2424 12y ago

Might be worth combining with walkscore data? http://www.walkscore.com/professional/research.php http://www.walkscore.com/professional/research.php Not sure how freely they distribute this.

oniTony 12y ago
walkscore can already map out all "within walking distance of grocery + rail" locations in Portland (and elsewhere). E.g. http://www.walkscore.com/apartments/search/Portland-OR?zoom=12&sort=14_low&hood=off&nearby=%7B%7D&places=%5B%5B%22Groceries%22%2C%2210%22%5D%5D&transit=%5B30%2C%2219.81.83-85.99%22%2C45.537169%2C-122.65015900000003%5D&lat=45.535517621410435&lng=-122.64287605285642 http://www.walkscore.com/apartments/search/Portland-OR?zoom=... The actual scores would have to be looked up one at a time, on per-address basis (e.g. http://www.walkscore.com/score/320-pioneer-way-mountain-view-ca-94041 http://www.walkscore.com/score/320-pioneer-way-mountain-view... )

mrfusion 12y ago

Where do you get the house listing data? Do you then geo code the addresses and filter them against your acceptable locations? (Sorry if I missed reading that part)

Caged 12y ago
Are you referring to real estate listings? In my case, I use Trulia to search the zones after I narrow down the locations. However, if you're referring to the building footprints I have a Make target that pulls GIS data from public sources: https://github.com/caged/portland-atlas https://github.com/caged/portland-atlas
- mrfusion 12y ago
  I guess I'm asking how you find only listings in the zones you want? Does trulia have an API?
vlandham 12y ago
Started a similar repo for Seattle, WA data here: https://github.com/vlandham/seattle-atlas https://github.com/vlandham/seattle-atlas

schooldistrict 12y ago

http://schooldistrictfinder.com/ http://schooldistrictfinder.com/ Finding the perfect school (one of the many steps to find the perfect house) using open data

huherto 12y ago
Great, Just today I was thinking about this. So, how do I know the district area. I can see the pin. Is that the center? Sorry, I am just getting familiar with this subject in the US.

akgerber 12y ago

The "moneyball" move is buying land right before the mass transit extension is completed and the neighborhood develops: http://en.wikipedia.org/wiki/MAX_Orange_Line http://en.wikipedia.org/wiki/MAX_Orange_Line But this is a really fun article on how to use open-source GIS tools to do something practical, which is something I've wanted to delve into for a while— I'll try it later tonight, thanks!

jweir 12y ago
Yeah, Governor Christie's brother likes that strategy. "Governor Christie's brother invested in real estate near new PATH station in Harrison" http://www.northjersey.com/news/governor-christie-s-brother-invested-in-real-estate-near-new-path-station-in-harrison-1.667498 http://www.northjersey.com/news/governor-christie-s-brother-...
- akgerber 12y ago
  It's only problematic when you're doing it before the extension is public. Likewise, property on Second Avenue on the Upper East Side of Manhattan has been cheap lately, because blasting out subway stations is unpleasant. But a new subway coming to an area that was recently relatively far from one means property values are shooting up.

aembleton 12y ago

Wow; this is great to see. I have recently done something similar in the Greater Manchester, UK area. I've been looking for a house and wrote some code to scrape rightmove.co.uk and zoopla.com to get the house data. Checked their locations against data from http://data.police.uk/ http://data.police.uk/. Checked time to walk to the nearest metro or railway station and time to cycle to work. Metro and Railway station locations were scraped from wikipedia and time to travel was taken from Google Maps. From all this, as well as £s per m^2 of floor space I ordered by the least compromise. Currently trying to clean it all up and turn it into a webapp to make it useful for anyone else who wants to find property in an area. I'm about to complete on the purchase of my first house and am really looking forward to it. It's not perfect; but I know for the price, I can't get much better. If I finish the webapp; I might try and include data such as walking distance to the nearest pub from CAMRAs Good Beer Guide and maybe something similar for restaurants.

freyfogle 12y ago
You may wish to check out Nestoria's API of property listings and house price trends http://www.nestoria.co.uk/help/api http://www.nestoria.co.uk/help/api

Jemaclus 12y ago

I work in online real estate, and this is exactly the kind of thing we love to do when it comes to utilizing data. How can we take X, Y and Z data points and generate something useful? Taking into account crime reports, graffiti reports, available parking spaces, etc, can all help you decide whether this place is good or not. The requirements here are kinda strange (proximity to a grocery store? not crime rates or anything like that?), but to each his own, I suppose. I'm really impressed with the resourcefulness of the author, though. I think I would probably just go to Zillow or Trulia and see what they say...

pdx 12y ago
> (proximity to a grocery store? not crime rates or anything > like that?), but to each his own, Some people are not able to, or do not wish to drive every time they want something to eat. Proximity to a grocery store sounds like a good metric. It's the european model of shopping every day for that days meal, vs the U.S. model of stocking up at Costco.
- Jemaclus 12y ago
  True. I was more remarking on the fact that the grocery store metric seemed to be the primary one, as opposed to just one of a great many. It's definitely the kind of thing I would consider, but I'd also consider more things like price per square foot, number of bedrooms, neighborhood quality, distance to schools, etc. That's why I said "kinda strange" and not something harsher.
Caged 12y ago
> The requirements here are kinda strange (proximity to a grocery store? not crime rates or anything like that?), but to each his own, I suppose. You're right. I've been working with open data in the Portland area for years now [1][2][3][4] and have a lot of built up internal knowledge of the neighborhoods and locations so I chose not to muddy the map with those things. 1: https://github.com/Caged/portlandcrime https://github.com/Caged/portlandcrime 2: https://github.com/caged/portland-atlas#examples https://github.com/caged/portland-atlas#examples 3: https://github.com/Caged/portland-census https://github.com/Caged/portland-census 4: http://labratrevenge.com/pdx http://labratrevenge.com/pdx

qrybam 12y ago

I'm working on something exactly like this at the moment (not US) - it's super exciting to see people make use of the data available to them. Kudos :)

misuba 12y ago

Finding unaffordable neighborhoods, more like.

Eric_WVGG 12y ago
ikr. Three problems about this article. First, he better be rich if he’s serious about living in these zones. Second, his data analysis is blowing past a multitude of equally livable, more affordable PDX neighborhoods. Third, within his zones are actual unlivable subzones (nobody wants to live in actual-downtown-portland). Impressive analysis, but he'd be better served by living for a month in a PDX AirBnB and learning his way around the city.

fryguy 12y ago

When I was looking for a place to live, I made my own program to scrape the MLS, and did similar things to this. My rectangles were hand drawn, not scraped from open data though, and were more based on the neighborhoods that had houses I knew I didn't like.

gdudeman 12y ago

This is awesome. I love analysis like this. Funny enough, something very similar to this was the impetus for me starting Estately. Early on, we had a bunch of wild spatial queries that would let you even filter by distance to specific transit line stops: http://blog.estately.com/wp-content/uploads/2008/06/near_transit.gif http://blog.estately.com/wp-content/uploads/2008/06/near_tra... When we ran stats on what searches people were actually doing, we found that very few consumers actually used these kinds of filters. We found we had a small and passionate audience for specific geo features and a much larger audience for "great site to search for and learn about homes." It took me way to long to come to terms with this. We still have walkscore search and distance from a neighborhood, school or address search even though they are very little used. Why wasn't it popular? My guess is it's 80% because the market was small and 20% because we didn't present them right. The thing about real estate - especially in the current market - is there just are few enough homes for sale that meet your price / proximity to work / school / bedrooms criteria that most people want to just glance at all of them and rule them in or out. Additionally, most people start with a bunch of "rules," then they bend one, change their mind on another and finally compromise on the last one. The typical home search starts so clean, but dissolves over time as you tour houses, learn more about what you do and don't like and find something that works.

Dwolb 12y ago
Did you ever try to give people the option to filter by school district or school district ranking?
- gdudeman 12y ago
  We let people search by specific school and we are considering adding school district ranking soon. We've had a bunch of users requesting it.
  - Dwolb 12y ago
    seems like one of the big factors that affects property values. maybe since it's so important some people would rather search by top school within a radius or boundary than know the district ranking associated with the house.

chrisamiller 12y ago

I was worried about commute times from three different points when looking for a house, so I wrote a quick and dirty little ruby script to hit the Google Driving Directions API. It tiles across the area you specify and spits out a kml file that can be loaded into google earth and lets you see which areas fit your parameters (https://github.com/chrisamiller/commute-times https://github.com/chrisamiller/commute-times). There's lots of room for improvement in the code, but it's in the same vein as what this guy did and helped us quite a bit.

joshdotsmith 12y ago

I didn't do anything this intense, but back in 2008 I took some data I gathered from city-data and mapped neighborhood ratings for San Diego. After living here for 6 years they're still pretty accurate. https://www.google.com/maps/@32.7740233,-117.1799609,12z/data=!4m2!6m1!4s215611903328390102803.0004515da0f446355193e https://www.google.com/maps/@32.7740233,-117.1799609,12z/dat... I'd just take an address from Craigslist and paste it in to quickly see whether I wanted to live there. Today I live in one of the "bit dumpy and rough, but OK" neighborhoods.

petersellers 12y ago
Thanks, this is really interesting. I love how Kensington turns from "Affluent" to "Gang-influenced" as soon as you cross El Cajon Blvd. That might be a slight exaggeration, but I've definitely noticed a few times walking through there how drastically the feel of the neighborhood changes after just a block or two.

andbberger 12y ago

My startup does exactly this and we just released! https://www.dwellaware.com/ https://www.dwellaware.com/