4 ms·

Not anymore. Now it’s mobile IP addresses.

varenc 5y ago
This! Mobile IPs are far more lucrative. Many services will drop captchas and other anti-bot stuff for consumer mobile IPs. I recall Plaid at some point would run their bank scraping through mobile IPs. This sketchy company lets mobile app developers monetize user base by letting other people pay $$ to route requests through random people’s mobile IPs: https://brightdata.com/ https://brightdata.com/
- cute_boi 5y ago
  This is really bad. Imagine if someone plants these proxy inside app how user are even going to know? I think every OS should come with firewall so if app tries to make connection it should prompt with Accept | Accept Forever | Deny | Deny Forever. I think these companies used to go for extension developer now it seems they have found new idea to implant malware on apps which is not easy to detect.
  - TedDoesntTalk 5y ago
    This is EXACTLY how mobile proxy companies like Luminati and OxyLabs acquire their ip address pool. They pay devs to embed a lib inside their app.
    - dzhiurgis 5y ago
      They hack IP cams and routers too…
- dzhiurgis 5y ago
  Brightdata is formerly known as as Luminati who is owned by same company as Hola VPN. Similarly NordVPN owns Oxylabs (who mostly hack routers and cameras and sell those as residential IP’s).
  - varenc 5y ago
    But they have a white paper on how ethical their proxies are! https://oxylabs.io/Oxylabs_Residential_Proxy_Acquisition_Handbook.pdf https://oxylabs.io/Oxylabs_Residential_Proxy_Acquisition_Han...
  - herbst 5y ago
    I never checked their business practice. But I use Luminati rotating residential proxies for a lot of my scraping work. Mostly to avoid hitting 'ddos protections' or other security bullshit that doesn't really make sense on 1 daily request or so.
walrus01 5y ago
One of the reasons for this is that the vast majority of the time, mobile LTE data users are behind cgnat for ipv4. You can't block one ip without possibly blocking hundreds of innocent IPs using the same exit point. As a scraper operator on a mobile data connection all you need is a new useragent and browser fingerprint, there's no easy way for a scraper-blocker-operator to tell that you're not a totally new person.
- cute_boi 5y ago
  This is reason why most of services uses App instead of Browser. When there is App it can use lot of thing like phone fingerprint derived from various sources.
  - dzhiurgis 5y ago
    Apple provides a framework or some sort of unique id
    - varenc 5y ago
      Note that Apple explicitly tries to prevent apps from generating any sort of overall device fingerprint besides the ad tracking identifier one which now requires user consent in iOS 14. You can still generate an app-scoped device ID though. (Not sure if these persist across re-installs of the same app or not)

This! Mobile IPs are far more lucrative. Many services will drop captchas and other anti-bot stuff for consumer mobile IPs. I recall Plaid at some point would run their bank scraping through mobile IPs. This sketchy company lets mobile app developers monetize user base by letting other people pay $$ to route requests through random people’s mobile IPs: https://brightdata.com/ https://brightdata.com/

cute_boi 5y ago
This is really bad. Imagine if someone plants these proxy inside app how user are even going to know? I think every OS should come with firewall so if app tries to make connection it should prompt with Accept | Accept Forever | Deny | Deny Forever. I think these companies used to go for extension developer now it seems they have found new idea to implant malware on apps which is not easy to detect.
- TedDoesntTalk 5y ago
  This is EXACTLY how mobile proxy companies like Luminati and OxyLabs acquire their ip address pool. They pay devs to embed a lib inside their app.
  - dzhiurgis 5y ago
    They hack IP cams and routers too…
dzhiurgis 5y ago
Brightdata is formerly known as as Luminati who is owned by same company as Hola VPN. Similarly NordVPN owns Oxylabs (who mostly hack routers and cameras and sell those as residential IP’s).
- varenc 5y ago
  But they have a white paper on how ethical their proxies are! https://oxylabs.io/Oxylabs_Residential_Proxy_Acquisition_Handbook.pdf https://oxylabs.io/Oxylabs_Residential_Proxy_Acquisition_Han...
- herbst 5y ago
  I never checked their business practice. But I use Luminati rotating residential proxies for a lot of my scraping work. Mostly to avoid hitting 'ddos protections' or other security bullshit that doesn't really make sense on 1 daily request or so.

walrus01 5y ago

One of the reasons for this is that the vast majority of the time, mobile LTE data users are behind cgnat for ipv4. You can't block one ip without possibly blocking hundreds of innocent IPs using the same exit point. As a scraper operator on a mobile data connection all you need is a new useragent and browser fingerprint, there's no easy way for a scraper-blocker-operator to tell that you're not a totally new person.

cute_boi 5y ago
This is reason why most of services uses App instead of Browser. When there is App it can use lot of thing like phone fingerprint derived from various sources.
- dzhiurgis 5y ago
  Apple provides a framework or some sort of unique id
  - varenc 5y ago
    Note that Apple explicitly tries to prevent apps from generating any sort of overall device fingerprint besides the ad tracking identifier one which now requires user consent in iOS 14. You can still generate an app-scoped device ID though. (Not sure if these persist across re-installs of the same app or not)