4 ms·

I have done something similar with with a python script that basically: - parsed locations from an html file and stored them in a list - parsed them into this

I have done something similar with with a python script that basically: - parsed locations from an html file and stored them in a list - parsed them into this url: 'https://maps.google.com/maps?q=%s' https://maps.google.com/maps?q=%s' % (location) -parsed the html for an address -parsed the address into this url: 'https://maps.google.com/maps/api/geocode/json?address=%s&sensor=false' https://maps.google.com/maps/api/geocode/json?address=%s&sen... % (address) -parse the coordinates I used it to get oil refineries in the united states and it worked pretty well… i can put it on github if you would like. It probably can also be done between the front-end and back-end with a little bit of trickery (the calls to google on the front-end, POST'ed html processing on the back from the front-end), so that users ip is using the service and not the servers.

ColinWright 13y ago
That's very similar to what I've been doing so far, but it's broken again, so I'm looking for something less fragile and more "official" so I can stop having to fix it every few months, or every few weeks. I'll probably fix it this time while I'm searching for something more robust. Thanks.
- cinquemb 13y ago
  Good luck with your search, everything seems so fractured when it comes to doing things like this. I'm curious, how does your method break? Is it the query string not finding any results to a location, wrong locations, or something else?
  - ColinWright 13y ago
    I've been sending query to: 'https://maps.google.com/maps?q=%s' % sanitise(location) and then parsing the result to find the lat/lng. However, this isn't an "official" way of doing it, and the format of the html has changed. The result used to match this: 'viewport_center_lat=([-0-9.]*);var viewport_center_lng=([-0-9.]*)' but now it looks like this: 'latlng:{lat:([-0-9.]*),lng:([-0-9.]*)}' Yes, I'm using regexs to parse HTML. However, now that's working again, but I might switch to the explicit two phase process used in https://news.ycombinator.com/item?id=6058751 https://news.ycombinator.com/item?id=6058751 - or I might directly use the second link mentioned there, instead of calling the URL I currently use. Does that all make sense?
    - cinquemb 13y ago
      Yeah it makes sense. If you are getting locations in specific, then it makes more sense to just use the second url as you state. I've been using BeautifulSoup, then searching for the span tags where the "class" attribute is set to "pp-headline-item pp-headline-address", and getting the formatted address of the possible locations that show up and using that in the query to the second url (I'm lazy so I only get the first result if multiple exist). I don't know if that would work better than regexs because google could still change the html every so often (but maybe not as often as javascript).

ColinWright 13y ago

That's very similar to what I've been doing so far, but it's broken again, so I'm looking for something less fragile and more "official" so I can stop having to fix it every few months, or every few weeks. I'll probably fix it this time while I'm searching for something more robust. Thanks.

cinquemb 13y ago
Good luck with your search, everything seems so fractured when it comes to doing things like this. I'm curious, how does your method break? Is it the query string not finding any results to a location, wrong locations, or something else?
- ColinWright 13y ago
  I've been sending query to: 'https://maps.google.com/maps?q=%s' % sanitise(location) and then parsing the result to find the lat/lng. However, this isn't an "official" way of doing it, and the format of the html has changed. The result used to match this: 'viewport_center_lat=([-0-9.]*);var viewport_center_lng=([-0-9.]*)' but now it looks like this: 'latlng:{lat:([-0-9.]*),lng:([-0-9.]*)}' Yes, I'm using regexs to parse HTML. However, now that's working again, but I might switch to the explicit two phase process used in https://news.ycombinator.com/item?id=6058751 https://news.ycombinator.com/item?id=6058751 - or I might directly use the second link mentioned there, instead of calling the URL I currently use. Does that all make sense?
  - cinquemb 13y ago
    Yeah it makes sense. If you are getting locations in specific, then it makes more sense to just use the second url as you state. I've been using BeautifulSoup, then searching for the span tags where the "class" attribute is set to "pp-headline-item pp-headline-address", and getting the formatted address of the possible locations that show up and using that in the query to the second url (I'm lazy so I only get the first result if multiple exist). I don't know if that would work better than regexs because google could still change the html every so often (but maybe not as often as javascript).