5 ms·

So let me get this straight -- for these "unnofficial" APIs, someone just scraped a bunch of packets from their phone while letting Pokemon Go run on it? Then

So let me get this straight -- for these "unnofficial" APIs, someone just scraped a bunch of packets from their phone while letting Pokemon Go run on it? Then investigated to see what the communication from client to server looks like, then implemented an API that mimicks that communication? If that's all so, could the PoGo devs simply enforce some type of device authentication to 'shut down' these APIs, or otherwise take different steps to make unofficial APIs less compatible/more difficult/effectively impossible?

ry_ry 10y ago
Basically, yes, but as the other guy mentioned it's an ongoing process of fix/break. It seems that the devs are firefighting scalability issues atm, and it would make sense that unsanctioned 3rd party APIs will be targeted Soon™ but probably heuristically. They're also in violation of the ToS I believe, but bans in their previous title came in waves, and I've not heard of any pogo banwaves yet.
- drac 10y ago
  they certainly ban people who do GPS spoofing although I can't speak to how rigorously enforced those checks might be
  - ry_ry 10y ago
    That is auto-triggered by software detection at a device level isn't it?
    - BryantD 10y ago
      No clear answer to this as far as anyone can tell. From the evidence it definitely looks like there's a server component, though.
    - pbhjpbhj 10y ago
      I tried spoofing GPS with Niantic's Ingress game, just about the second time I played, and the spoofing apps relied on the fake location flag in the dev settings - of course Niantic checked to see if that was set and refuse to work if it was [until the system was rebooted, with the flag disabled, AFAICT]. Haven't tried with PoGo as I'm more interested in playing it [with my kids, honest!] than working out how it works.
    - therein 10y ago
      Not necessarily. At least on Android, it is rather simple to just use Xposed Framework to basically "mock syscalls".
- pbhjpbhj 10y ago
  >They're also in violation of the ToS I believe // Only if they signed up to it ;o) If you're reverse engineering an app and someone else is using the device with the app installed there's no need for you to have submitted to a contract of obligations to the app provider. Depending how you orchestrate things the signee might be guilty of giving their credentials away. It would probably come in as unauthorised access of a computer system with respect to USA/England&Wales legislation.
Retr0spectrum 10y ago
How would you differentiate between a "real" device and "fake" device?
- ry_ry 10y ago
  I'd assume by looking at the usage patterns - has the device teleported instantly across huge areas, does it display any GPS drift and organic movement rather than a series of static locations, is it moving 24/7, are the reaction times inhumanly fast etc. All that can be spoofed tho, and whether they have the resources to apply that level of analysis to XXX million users is questionable.
  - sleepychu 10y ago
    Or the motivation, I guess as botted users start claiming gyms it becomes a problem but until they implement trading each user is silo'd by the game mechanics
    - pbhjpbhj 10y ago
      >I guess as botted users start claiming gyms it becomes a problem // It might be a nice problem for Niantic if that means non-hacking users pay to improve their own pokemon to try and beat the botted gym users? Provided the numbers of bots is relatively low it probably won't be a problem they feel needs fixing. I guess if someone is claiming more than a few gyms it's going to get flagged and they can probably easily spot if it's genuine or not?!?
      - jack9 10y ago
        Gyms can't be held for very long, in any scenario other than remote destinations that nobody visits (like a reef 2 miles from the coast or a desert monument). 3 Perfect Pokemon of the highest tier will be burned down by 6 Pokemon of various quality (multiple people using 6 at a time), especially with the rock-paper-scissors counters available.
        10y ago
evook 10y ago
> If that's all so, could the PoGo devs simply enforce some type of device authentication to 'shut down' these APIs, or otherwise take different steps to make unofficial APIs less compatible/more difficult/effectively impossible? That's impossible. If you try this you'll either have a bunch of false positives or even more likely a patched api around 12 hours later. Never underestimate the dedication of botters. There are multiple headless WoW Apis around for 10 years now and Blizzard isn't able to close them out.
- ry_ry 10y ago
  Agreed. Googling 'pokemon go lvl20' shows a bunch of accounts for sale already, if people are buying them the motivation to keep botting will be there.
- deleted 10y ago
  [deleted]
- FungalRaincloud 10y ago
  The best way to handle this is with account-specific API keys. Even that would just mean creating an account, and the only real benefit to that is that you could track the API key, and if it did "bot-like" things, ban it. That's not really a fix, just a barrier for entry, preventing poorly thought out bots from working.
  - danielrmay 10y ago
    I'm pretty sure this already exists. These APIs have been in development for a couple of weeks now and as far as I can tell (from watching the /r/pokemongodev subreddit) there haven't been any bans. Niantic, at present, don't appear to have any automated system attempting to catch bot-like behavior.
    - Fiahil 10y ago
      They can't produce a bug-free version of their game, already. Why would they start implementing automated systems to ban people?
grrowl 10y ago
A bit more complex, it uses Google's protobuf protocol so it likely reimplements the API using reverse engineered data structures. See: https://www.fknsrs.biz/blog/reverse-engineering-pokemon-go.html https://www.fknsrs.biz/blog/reverse-engineering-pokemon-go.h...
markhowe 10y ago
There's one field in the app's request that's still unknown. It's a header of seemingly encrypted data, along with a varying number of encrypted blocks (all the same length). In those blocks could be anything, detailed gps co-ords, device details, there's a fair chance they can ban all these API users at the push of a button based on whatever's in those blocks. Everything else is unencrypted - sent back and forth using the protobuf format, the formatting of the protobuf's were dropped on pastebin a few weeks ago.
- tlrobinson 10y ago
  Even then it's just an arms race. Someone will reverse engineer the app and figure out how to encrypt that block of data.
dom96 10y ago
> If that's all so, could the PoGo devs simply enforce some type of device authentication to 'shut down' these APIs, or otherwise take different steps to make unofficial APIs less compatible/more difficult/effectively impossible? I sincerely hope that they don't do that, or at least not 'shut it down' completely. The reason is that there are some use cases for this API which don't involve cheating, for example the many different Pokemon Go server status pages including one that I built myself [1]. 1 - https://pokemongostatus.org/ https://pokemongostatus.org/
elliottcarlson 10y ago
Majority of the reverse engineering started with packet inspection and reversing the APK's from the Android version. From there a lot of effort was put in to protobuf's and reversing the protobuf messages. Two things learned from the packet inspection was the authentication schemes for Pokemon Trainer Club and Google; and then how to use the JWT's in the API requests. All of the protobuf requests have been figured out, but sometimes it's about implementation details in how things are called (for example S2 cellid's for map object requests, etc).
blackice 10y ago
One of the better ways to find these people using "unofficial" APIs is to look for VPN / hosting IPs. Running all of it on their home connection is very obvious so they'll branch out and let their bots run on servers 24x7. Based on the IP address and their account activity, I think it'll be a lot easier to catch botters or at least catch the big fish.
- corobo 10y ago
  Aren't bots like this often run on botnets? After the classic DDoS of course, I'd have thought this sort of thing would be what botnets are used for these days.

ry_ry 10y ago

Basically, yes, but as the other guy mentioned it's an ongoing process of fix/break. It seems that the devs are firefighting scalability issues atm, and it would make sense that unsanctioned 3rd party APIs will be targeted Soon™ but probably heuristically. They're also in violation of the ToS I believe, but bans in their previous title came in waves, and I've not heard of any pogo banwaves yet.

drac 10y ago
they certainly ban people who do GPS spoofing although I can't speak to how rigorously enforced those checks might be
- ry_ry 10y ago
  That is auto-triggered by software detection at a device level isn't it?
  - BryantD 10y ago
    No clear answer to this as far as anyone can tell. From the evidence it definitely looks like there's a server component, though.
  - pbhjpbhj 10y ago
    I tried spoofing GPS with Niantic's Ingress game, just about the second time I played, and the spoofing apps relied on the fake location flag in the dev settings - of course Niantic checked to see if that was set and refuse to work if it was [until the system was rebooted, with the flag disabled, AFAICT]. Haven't tried with PoGo as I'm more interested in playing it [with my kids, honest!] than working out how it works.
  - therein 10y ago
    Not necessarily. At least on Android, it is rather simple to just use Xposed Framework to basically "mock syscalls".
pbhjpbhj 10y ago
>They're also in violation of the ToS I believe // Only if they signed up to it ;o) If you're reverse engineering an app and someone else is using the device with the app installed there's no need for you to have submitted to a contract of obligations to the app provider. Depending how you orchestrate things the signee might be guilty of giving their credentials away. It would probably come in as unauthorised access of a computer system with respect to USA/England&Wales legislation.

Retr0spectrum 10y ago

How would you differentiate between a "real" device and "fake" device?

ry_ry 10y ago
I'd assume by looking at the usage patterns - has the device teleported instantly across huge areas, does it display any GPS drift and organic movement rather than a series of static locations, is it moving 24/7, are the reaction times inhumanly fast etc. All that can be spoofed tho, and whether they have the resources to apply that level of analysis to XXX million users is questionable.
- sleepychu 10y ago
  Or the motivation, I guess as botted users start claiming gyms it becomes a problem but until they implement trading each user is silo'd by the game mechanics
  - pbhjpbhj 10y ago
    >I guess as botted users start claiming gyms it becomes a problem // It might be a nice problem for Niantic if that means non-hacking users pay to improve their own pokemon to try and beat the botted gym users? Provided the numbers of bots is relatively low it probably won't be a problem they feel needs fixing. I guess if someone is claiming more than a few gyms it's going to get flagged and they can probably easily spot if it's genuine or not?!?
    - jack9 10y ago
      Gyms can't be held for very long, in any scenario other than remote destinations that nobody visits (like a reef 2 miles from the coast or a desert monument). 3 Perfect Pokemon of the highest tier will be burned down by 6 Pokemon of various quality (multiple people using 6 at a time), especially with the rock-paper-scissors counters available.
      - 10y ago

evook 10y ago

> If that's all so, could the PoGo devs simply enforce some type of device authentication to 'shut down' these APIs, or otherwise take different steps to make unofficial APIs less compatible/more difficult/effectively impossible? That's impossible. If you try this you'll either have a bunch of false positives or even more likely a patched api around 12 hours later. Never underestimate the dedication of botters. There are multiple headless WoW Apis around for 10 years now and Blizzard isn't able to close them out.

ry_ry 10y ago
Agreed. Googling 'pokemon go lvl20' shows a bunch of accounts for sale already, if people are buying them the motivation to keep botting will be there.
deleted 10y ago
[deleted]
FungalRaincloud 10y ago
The best way to handle this is with account-specific API keys. Even that would just mean creating an account, and the only real benefit to that is that you could track the API key, and if it did "bot-like" things, ban it. That's not really a fix, just a barrier for entry, preventing poorly thought out bots from working.
- danielrmay 10y ago
  I'm pretty sure this already exists. These APIs have been in development for a couple of weeks now and as far as I can tell (from watching the /r/pokemongodev subreddit) there haven't been any bans. Niantic, at present, don't appear to have any automated system attempting to catch bot-like behavior.
  - Fiahil 10y ago
    They can't produce a bug-free version of their game, already. Why would they start implementing automated systems to ban people?

grrowl 10y ago

A bit more complex, it uses Google's protobuf protocol so it likely reimplements the API using reverse engineered data structures. See: https://www.fknsrs.biz/blog/reverse-engineering-pokemon-go.html https://www.fknsrs.biz/blog/reverse-engineering-pokemon-go.h...

markhowe 10y ago

There's one field in the app's request that's still unknown. It's a header of seemingly encrypted data, along with a varying number of encrypted blocks (all the same length). In those blocks could be anything, detailed gps co-ords, device details, there's a fair chance they can ban all these API users at the push of a button based on whatever's in those blocks. Everything else is unencrypted - sent back and forth using the protobuf format, the formatting of the protobuf's were dropped on pastebin a few weeks ago.

tlrobinson 10y ago
Even then it's just an arms race. Someone will reverse engineer the app and figure out how to encrypt that block of data.

dom96 10y ago

> If that's all so, could the PoGo devs simply enforce some type of device authentication to 'shut down' these APIs, or otherwise take different steps to make unofficial APIs less compatible/more difficult/effectively impossible? I sincerely hope that they don't do that, or at least not 'shut it down' completely. The reason is that there are some use cases for this API which don't involve cheating, for example the many different Pokemon Go server status pages including one that I built myself [1]. 1 - https://pokemongostatus.org/ https://pokemongostatus.org/

elliottcarlson 10y ago

Majority of the reverse engineering started with packet inspection and reversing the APK's from the Android version. From there a lot of effort was put in to protobuf's and reversing the protobuf messages. Two things learned from the packet inspection was the authentication schemes for Pokemon Trainer Club and Google; and then how to use the JWT's in the API requests. All of the protobuf requests have been figured out, but sometimes it's about implementation details in how things are called (for example S2 cellid's for map object requests, etc).

blackice 10y ago

One of the better ways to find these people using "unofficial" APIs is to look for VPN / hosting IPs. Running all of it on their home connection is very obvious so they'll branch out and let their bots run on servers 24x7. Based on the IP address and their account activity, I think it'll be a lot easier to catch botters or at least catch the big fish.

corobo 10y ago
Aren't bots like this often run on botnets? After the classic DDoS of course, I'd have thought this sort of thing would be what botnets are used for these days.