7 ms·

Mozilla Location Service: Geolocation lookups from public cell tower, WiFi data

98 points by hackhackhack 13y ago

TheHippo 13y ago
Some one should get all the Ingress players to install the app and contribute data. Larger city would be mapped in no time.
- yeukhon 13y ago
  Huh? How do you map a city? Or what do you mean by mapping a city. We know a city's approximate lat, long and we know lat and long of various locations except we don't know exactly what is at a particular coordinate, or the road condition. edit downvote without a comment is a crime.
  - Arelius 13y ago
    I Think you are getting down-voted by misunderstanding what the parent was saying. The parent is talking about mapping out WiFi and Cell data through a city. Mapping out it's radio spectrum spatially. Not a traditional human usable map such as a street map, road conditions, or whatever else you may be thinking.
    - yeukhon 13y ago
      I asked specifically to address his point.
- cpeterso 13y ago
  The client UI a little geeky now, but we have some gamification ideas akin to Google's Ingress or Nintendo's Treasure World games: https://en.wikipedia.org/wiki/Ingress_%28game%29 https://en.wikipedia.org/wiki/Ingress_%28game%29 https://en.wikipedia.org/wiki/Treasure_World https://en.wikipedia.org/wiki/Treasure_World
  - aaron42net 13y ago
    Turn your API users into data suppliers by adding an optional GPS lat/long and accuracy radius to the API request. You could potentially enhance the position they already have with wifi data (like Google Maps claims to do), and collect the GPS to wifi correlations to help other users. This is potentially a much larger data source than people running dedicated stumblers.
    - hannosch 13y ago
      Yes, we are aware of this approach, thx for reminding us. It wasn't a priority so far, as we don't have anyone using the search API's just yet.
- andrewaylett 13y ago
  Isn't this likely to be at least part of the point of Ingress in the first place?
  - cpeterso 13y ago
    I'm sure it is. Google obscures their involvement with Ingress by funding Niantic Labs, a "Google start-up company" that just happens to have the same address as Google's San Francisco office.
    - magicalist 13y ago
      Both the Ingress website and the play store download page list "NianticLabs@Google" as the creator. Was it ever a secret? The ads were meant to be mysterious in general, but I can't remember any coverage that didn't mention the Google connection pretty prominently. On a different note: I can't believe the game is still invite only. Do people still play it? I tried to get into it, but quickly grew tired of it because everyone around me was much better than me and I couldn't sustain the interest needed to learn the ins and outs of getting good at it.
      - cpeterso 13y ago
        Yeah, you are right. Google's participation is not "secret", but it's unclear (to me) how much access Google gets to Niantic Labs' user data. Also, Ingress is "invitation only" in the sense that they will email you an invitation code if you register on their website. <:)
ChuckMcM 13y ago
Hmm, do you get extra points if you ship back a few captured unencrypted packets? I realize its a bit snarky but the amazing flap over Google Streetview cars collecting WiFi access point data was, to me, a bit over the top.
floatboth 13y ago
Would be cool if you could download the data for your city to your Android phone and use the location service without sending your cell towers and BSSIDs to anyone, not even Mozilla.
- daleharvey 13y ago
  When Andreas Gal talked about this at the recent mozsummit, he specifically mentioned this use case.
  - cpeterso 13y ago
    Yes. Downloading offline data sets also means you can geolocate without network connectivity, so it's faster and saves power too.
    - chimeracoder 13y ago
      You really should emphasize this more prominently. It's a huge selling point, both for practicality and privacy.
hendi_ 13y ago
Sounds similar to http://www.openwlanmap.org http://www.openwlanmap.org. While OWM only collects WiFi APs, Mozilla also collects cell towers. Still, I couldn't find anything about whether Mozilla collaborated with these guys or not.
- cpeterso 13y ago
  We've reached out to some other wifi mapping projects like OpenWLANMap, OpenCellID, and WiGLE. None of them collected all the data we were looking for and their databases are published with different data licenses that may not be compatible.
  - Maxious 13y ago
    What about http://openbmap.org/ http://openbmap.org/ Open Database License (ODbL) and (last time I tried) you can download the raw data XML submitted by each user in the footer of that page
    - cpeterso 13y ago
      We've talked to openBmap and their data is interesting, but we're still figuring out data license compatibility.
  - mlinksva 13y ago
    Interesting, I see these and a few others listed at https://wiki.mozilla.org/Services/Location/Bootstrap https://wiki.mozilla.org/Services/Location/Bootstrap (all one of copyleft, (non)commercial, or unknown). While https://wiki.mozilla.org/Services/Location#Goals https://wiki.mozilla.org/Services/Location#Goals says "Make as much of the data as possible available under permissive open licenses." Assuming permissive means what it usually does, the above clearly aren't compatible. Curious what permissive licenses will be used, when data will be available.
    - cpeterso 13y ago
      One of our original project goals was to publish an open, downloadable database. Some smaller geolocation projects have downloadable databases, but Mozilla's legal and privacy teams were not ready to take that step yet. Geolocation is a very sensitive privacy issue for "stumbler" users, WiFi access point owners, and end users. Once you publish data, it's hard to take it back. <:) For now, we're just hosting a public API. You can read some early ideas for obscuring access point identities (using hashes of one or more MAC addresses) on the mozilla.dev.security newsgroup: https://groups.google.com/forum/#!topic/mozilla.dev.security/0qT0grS_PIY https://groups.google.com/forum/#!topic/mozilla.dev.security... You can read some of the Mozilla discussion about data licenses on Bugzilla: https://bugzil.la/911306 https://bugzil.la/911306
chanux 13y ago
I think the code behind this is ichnaea[1]. I've been reading the code in past few days and learning some stuff. (OK I learn a lot of things reading other's code on github.) Off topic: I found ichnaea while looking for circusd[2] configuration for celery. Mozilla circus is a nice process (& socket) management tool. [1] https://github.com/mozilla/ichnaea https://github.com/mozilla/ichnaea [2] http://circus.readthedocs.org/en/0.9.2/ http://circus.readthedocs.org/en/0.9.2/
cpeterso 13y ago
I work on this Mozilla project. If you have any questions, please ask! Mozilla's network "stumbler" client is installable from GitHub: https://github.com/dougt/MozStumbler/releases https://github.com/dougt/MozStumbler/releases. The client code is at https://github.com/dougt/MozStumbler/ https://github.com/dougt/MozStumbler/ and the server code is at https://github.com/mozilla/ichnaea/ https://github.com/mozilla/ichnaea/.
- bane 13y ago
  I'd like to download the data for offline (non-internet connected) use, will it be made available under an open license or is there some complex contractual stuff preventing it?
- tuxracer 13y ago
  Is the data gathered going to be made public? Or does this data collected solely benefit FirefoxOS users?
- graue 13y ago
  So, this is not exactly related, but I see that you also work on Firefox OS, and I assume part of the reason for this project is to improve location services on FxOS. I got a ZTE Open to play around with, and was disappointed to find that it can't figure out where I am. Of course, I know it's a cheap phone and the OS is only in version 1.0. So if the location were a little inaccurate, or took a long time to come up, I would understand. But it literally never gets my location at all. (I live in NYC.) Why do you think that would happen? And tying it back into this project, how soon do you think FxOS end-users will start seeing improvements from what you're doing?
  - abrowne 13y ago
    I've read 1.1 helps with this, for the Keon at least.
  - cpeterso 13y ago
    Yeah, the geolocation on some early Firefox OS models is a known problem. Unfortunately, Mozilla does not control the final bits that ship on devices. This Mozilla wiki page specifically addresses these GPS problem: https://wiki.mozilla.org/B2G/Geolocation https://wiki.mozilla.org/B2G/Geolocation
hamburglar 13y ago
Anybody gotten this to work at all? I'm trying to locate based on wifi info alone, but all I ever get back is "not_found". A working example would be nice, too, since the example data and the API doc seem to disagree on what goes in the "key" field (example request shows "3680873e9b83738eb72946d19e971e023e51fd01" but the docs say it should be a BSSID and "look similar to 01:23:45:67:89:ab"). edit: I'll save anyone who wants to try a lot of trouble and give you the one piece of information that their API docs and announcements all inexplicably avoid telling you, and which I found by digging through their client code: The API endpoint URL is https://location.services.mozilla.com/v1/ https://location.services.mozilla.com/v1/ :)
- cpeterso 13y ago
  The docs should be fixed now. Thanks for pointing out the problem. <:) https://github.com/mozilla/ichnaea/compare/3a3ed20df078...4e344c7f5764 https://github.com/mozilla/ichnaea/compare/3a3ed20df078...4e... The location API depends on having wifi measurements in our database, which is still tiny compared to big players.
- hannosch 13y ago
  Thx for the feedback, I've fixed the example to have the right format and provided full URL's in the API docs. The docs where written under the assumption of being reusable for anyone installed a ichnaea server and not just the specific instance we currently run. But that apparently isn't all too helpful right now :)
  - hamburglar 13y ago
    Thanks for the fix. It was clear that the ichnaea docs were intended to be generic, but there's also this bit: "More information about the specific instance hosted by Mozilla can be found at https://wiki.mozilla.org/Services/Location" https://wiki.mozilla.org/Services/Location" which leads to a page that also doesn't list the endpoint API.
lorax 13y ago
It is pretty cool that I could see a difference in coverage between when I read the article and when I got home from work. A bunch of people must have installed the app after reading about it here.

98 points by hackhackhack 13y ago

Some one should get all the Ingress players to install the app and contribute data. Larger city would be mapped in no time.

yeukhon 13y ago
Huh? How do you map a city? Or what do you mean by mapping a city. We know a city's approximate lat, long and we know lat and long of various locations except we don't know exactly what is at a particular coordinate, or the road condition. edit downvote without a comment is a crime.
- Arelius 13y ago
  I Think you are getting down-voted by misunderstanding what the parent was saying. The parent is talking about mapping out WiFi and Cell data through a city. Mapping out it's radio spectrum spatially. Not a traditional human usable map such as a street map, road conditions, or whatever else you may be thinking.
  - yeukhon 13y ago
    I asked specifically to address his point.
cpeterso 13y ago
The client UI a little geeky now, but we have some gamification ideas akin to Google's Ingress or Nintendo's Treasure World games: https://en.wikipedia.org/wiki/Ingress_%28game%29 https://en.wikipedia.org/wiki/Ingress_%28game%29 https://en.wikipedia.org/wiki/Treasure_World https://en.wikipedia.org/wiki/Treasure_World
- aaron42net 13y ago
  Turn your API users into data suppliers by adding an optional GPS lat/long and accuracy radius to the API request. You could potentially enhance the position they already have with wifi data (like Google Maps claims to do), and collect the GPS to wifi correlations to help other users. This is potentially a much larger data source than people running dedicated stumblers.
  - hannosch 13y ago
    Yes, we are aware of this approach, thx for reminding us. It wasn't a priority so far, as we don't have anyone using the search API's just yet.
andrewaylett 13y ago
Isn't this likely to be at least part of the point of Ingress in the first place?
- cpeterso 13y ago
  I'm sure it is. Google obscures their involvement with Ingress by funding Niantic Labs, a "Google start-up company" that just happens to have the same address as Google's San Francisco office.
  - magicalist 13y ago
    Both the Ingress website and the play store download page list "NianticLabs@Google" as the creator. Was it ever a secret? The ads were meant to be mysterious in general, but I can't remember any coverage that didn't mention the Google connection pretty prominently. On a different note: I can't believe the game is still invite only. Do people still play it? I tried to get into it, but quickly grew tired of it because everyone around me was much better than me and I couldn't sustain the interest needed to learn the ins and outs of getting good at it.
    - cpeterso 13y ago
      Yeah, you are right. Google's participation is not "secret", but it's unclear (to me) how much access Google gets to Niantic Labs' user data. Also, Ingress is "invitation only" in the sense that they will email you an invitation code if you register on their website. <:)

ChuckMcM 13y ago

Hmm, do you get extra points if you ship back a few captured unencrypted packets? I realize its a bit snarky but the amazing flap over Google Streetview cars collecting WiFi access point data was, to me, a bit over the top.

floatboth 13y ago

Would be cool if you could download the data for your city to your Android phone and use the location service without sending your cell towers and BSSIDs to anyone, not even Mozilla.

daleharvey 13y ago
When Andreas Gal talked about this at the recent mozsummit, he specifically mentioned this use case.
- cpeterso 13y ago
  Yes. Downloading offline data sets also means you can geolocate without network connectivity, so it's faster and saves power too.
  - chimeracoder 13y ago
    You really should emphasize this more prominently. It's a huge selling point, both for practicality and privacy.

hendi_ 13y ago

Sounds similar to http://www.openwlanmap.org http://www.openwlanmap.org. While OWM only collects WiFi APs, Mozilla also collects cell towers. Still, I couldn't find anything about whether Mozilla collaborated with these guys or not.

cpeterso 13y ago
We've reached out to some other wifi mapping projects like OpenWLANMap, OpenCellID, and WiGLE. None of them collected all the data we were looking for and their databases are published with different data licenses that may not be compatible.
- Maxious 13y ago
  What about http://openbmap.org/ http://openbmap.org/ Open Database License (ODbL) and (last time I tried) you can download the raw data XML submitted by each user in the footer of that page
  - cpeterso 13y ago
    We've talked to openBmap and their data is interesting, but we're still figuring out data license compatibility.
- mlinksva 13y ago
  Interesting, I see these and a few others listed at https://wiki.mozilla.org/Services/Location/Bootstrap https://wiki.mozilla.org/Services/Location/Bootstrap (all one of copyleft, (non)commercial, or unknown). While https://wiki.mozilla.org/Services/Location#Goals https://wiki.mozilla.org/Services/Location#Goals says "Make as much of the data as possible available under permissive open licenses." Assuming permissive means what it usually does, the above clearly aren't compatible. Curious what permissive licenses will be used, when data will be available.
  - cpeterso 13y ago
    One of our original project goals was to publish an open, downloadable database. Some smaller geolocation projects have downloadable databases, but Mozilla's legal and privacy teams were not ready to take that step yet. Geolocation is a very sensitive privacy issue for "stumbler" users, WiFi access point owners, and end users. Once you publish data, it's hard to take it back. <:) For now, we're just hosting a public API. You can read some early ideas for obscuring access point identities (using hashes of one or more MAC addresses) on the mozilla.dev.security newsgroup: https://groups.google.com/forum/#!topic/mozilla.dev.security/0qT0grS_PIY https://groups.google.com/forum/#!topic/mozilla.dev.security... You can read some of the Mozilla discussion about data licenses on Bugzilla: https://bugzil.la/911306 https://bugzil.la/911306

chanux 13y ago

I think the code behind this is ichnaea[1]. I've been reading the code in past few days and learning some stuff. (OK I learn a lot of things reading other's code on github.) Off topic: I found ichnaea while looking for circusd[2] configuration for celery. Mozilla circus is a nice process (& socket) management tool. [1] https://github.com/mozilla/ichnaea https://github.com/mozilla/ichnaea [2] http://circus.readthedocs.org/en/0.9.2/ http://circus.readthedocs.org/en/0.9.2/

cpeterso 13y ago

I work on this Mozilla project. If you have any questions, please ask! Mozilla's network "stumbler" client is installable from GitHub: https://github.com/dougt/MozStumbler/releases https://github.com/dougt/MozStumbler/releases. The client code is at https://github.com/dougt/MozStumbler/ https://github.com/dougt/MozStumbler/ and the server code is at https://github.com/mozilla/ichnaea/ https://github.com/mozilla/ichnaea/.

bane 13y ago
I'd like to download the data for offline (non-internet connected) use, will it be made available under an open license or is there some complex contractual stuff preventing it?
tuxracer 13y ago
Is the data gathered going to be made public? Or does this data collected solely benefit FirefoxOS users?
graue 13y ago
So, this is not exactly related, but I see that you also work on Firefox OS, and I assume part of the reason for this project is to improve location services on FxOS. I got a ZTE Open to play around with, and was disappointed to find that it can't figure out where I am. Of course, I know it's a cheap phone and the OS is only in version 1.0. So if the location were a little inaccurate, or took a long time to come up, I would understand. But it literally never gets my location at all. (I live in NYC.) Why do you think that would happen? And tying it back into this project, how soon do you think FxOS end-users will start seeing improvements from what you're doing?
- abrowne 13y ago
  I've read 1.1 helps with this, for the Keon at least.
- cpeterso 13y ago
  Yeah, the geolocation on some early Firefox OS models is a known problem. Unfortunately, Mozilla does not control the final bits that ship on devices. This Mozilla wiki page specifically addresses these GPS problem: https://wiki.mozilla.org/B2G/Geolocation https://wiki.mozilla.org/B2G/Geolocation

hamburglar 13y ago

Anybody gotten this to work at all? I'm trying to locate based on wifi info alone, but all I ever get back is "not_found". A working example would be nice, too, since the example data and the API doc seem to disagree on what goes in the "key" field (example request shows "3680873e9b83738eb72946d19e971e023e51fd01" but the docs say it should be a BSSID and "look similar to 01:23:45:67:89:ab"). edit: I'll save anyone who wants to try a lot of trouble and give you the one piece of information that their API docs and announcements all inexplicably avoid telling you, and which I found by digging through their client code: The API endpoint URL is https://location.services.mozilla.com/v1/ https://location.services.mozilla.com/v1/ :)

cpeterso 13y ago
The docs should be fixed now. Thanks for pointing out the problem. <:) https://github.com/mozilla/ichnaea/compare/3a3ed20df078...4e344c7f5764 https://github.com/mozilla/ichnaea/compare/3a3ed20df078...4e... The location API depends on having wifi measurements in our database, which is still tiny compared to big players.
hannosch 13y ago
Thx for the feedback, I've fixed the example to have the right format and provided full URL's in the API docs. The docs where written under the assumption of being reusable for anyone installed a ichnaea server and not just the specific instance we currently run. But that apparently isn't all too helpful right now :)
- hamburglar 13y ago
  Thanks for the fix. It was clear that the ichnaea docs were intended to be generic, but there's also this bit: "More information about the specific instance hosted by Mozilla can be found at https://wiki.mozilla.org/Services/Location" https://wiki.mozilla.org/Services/Location" which leads to a page that also doesn't list the endpoint API.

lorax 13y ago

It is pretty cool that I could see a difference in coverage between when I read the article and when I got home from work. A bunch of people must have installed the app after reading about it here.