5 ms·

As the CTO of an API SaaS who sees a lot of promotion fraud on our service (i.e. bots attempting to sign up for thousands of free-tier accounts, because enough

by derefr 4y ago

As the CTO of an API SaaS who sees a lot of promotion fraud on our service (i.e. bots attempting to sign up for thousands of free-tier accounts, because enough free-tier API-keys lashed together ≡ the capabilities of one paid-tier account), I see fraudulent sign-ups coming from IP addresses reallocated by these auction providers all the time. Addresses sold on IPXO (https://www.ipxo.com/ https://www.ipxo.com/) are seemingly especially bad for this. If there was a big list of all reallocatable / not-permanently-allocated IP ranges, I'd willingly — gleefully! — just dump it into Cloudflare as an IP blocklist for our website / registration flow. And there'd be zero fear in my mind of getting any false positives or user complaints by doing so. After all, none of these reallocatable ranges are ever purchased by residential/commercial broadband ISPs; those customers would much rather solve their IPv4 problems permanently, by doing either IPv6 enablement or CGNAT, than solve them for only the next 1024 customers, by buying a piddly /22 — especially at an unpredictable, un-budget-able price! And for registrations, any IP other than ISP IPs doesn't matter. IaaS IP? Bot. VPS IP? Bot. Colo IP? Bot. "Internal-use" IP? Bot. if someone's traffic isn't coming from an IP address owned by an ISP, then for purposes of registration, it's not good traffic, 99.999% of the time†. Those IPs are perfectly fine when it comes to actual requests — obviously, your application backend using our API is a "bot" in some sense — but your application backend shouldn't be going to our website and filling out a sign-up form. :) († The other 0.0001% are people using "workstation in the cloud" services. But people using those are used to being treated as second-class Internet citizens when trying to do web browsing from their cloud workstation; and they already know the solution is always to switch back to their real computer's web browser for anything requiring IP reputation.)

leshenka 4y ago
> solution is always to switch back to their real computer's web browser for anything requiring IP reputation … only to find out that you’re through no fault of your own blocked by region because we suddenly have a splinternet in the great year of 2022
hnov 4y ago
So much for running a personal VPN in the cloud.
- jimmydorry 4y ago
  It gets worse every year. I use a personal VPN, especially when travelling.
5e92cb50239222b 4y ago
> VPS IP? Bot. > The other 0.0001% are people using "workstation in the cloud" services. Yeah, no. I've been using VPNs/SSH tunnels/proxies set up on various VPS hosters/"cloud" vendors for more than a decade to bypass heavy internet censorship. It is much cheaper than using ready-made VPN solutions, and the server can be reused for other purposes. I understand that may be <0.1% of your market, but it's not 0. We're definitely used to being treated as third-class internet citizens not only by our own government, but also by various internet companies, which your comment demonstrates pretty clearly. Here's a nice write-up. It's not mine, and it's not even the same country, but the experience he describes is very relatable. https://shahinsorkh.ir/2019/07/20/how-is-it-like-to-be-a-dev-in-iran https://shahinsorkh.ir/2019/07/20/how-is-it-like-to-be-a-dev...
- taf2 4y ago
  Just curious which cloud based software as a service subscriptions have you purchase and actively maintain? Also approximately how many $ a year to you spend on those services? I’m curious because blocking IPs from bad actors is often the most effective and immediate solution to protect a service from a bad actor… it would be nice to understand the financial impact of this.
- derefr 4y ago
  If there was a list of commercial VPN provider IP ranges, we’d block that too. A key property of people who aren’t willing to relinquish their anonymity for even a moment in registering for your service, is that they’ll never be able to pay money for your service, either, because doing so would reveal their identity through their credit card. (Yes, they can use Bitcoin or whatever, but then you’re basically signing up to be a dark web business, having to deal with being a money-laundering sink for stolen funds.) So, if your goal in getting users to sign up, is to feed them through a funnel that eventually ends up with them paying you money, then you can take “signs up using a VPN” (or an email anonymization service, or…) as an immediate self-disqualification. Of course, most such failed attempts are innocent first-time “I didn’t realize you cared” attempts; so we just throw a clear error message, telling them to try again with their real email+IP+etc. But when a user responds to that error by just trying more and more email proxy domains, or repeatedly rotating their VPN country… well, that’s someone who’ll never be a customer. May as well just block them and be done with it.
1vuio0pswjnm7 4y ago
Whats the purpose of "free-tier" API SaaS. For example, does "free-tier" provide an opportunity for the "API SaaS" operators to collect personal information, or at least email addresses, belonging to users who have no intent of paying the "API SaaS". The notion of "fraudulent sign-ups" for something allegedly offered for "free" sounds funny. Fraud implies that the alleged perpetrator is obtaining something unlawfully or unfairly. However if that thing is "free" then how could this ever be true. Perhaps the operator of the "free" service actually is expecting to receive (or take) something from the user. When the user fails to provide or allow the operator to take this thing from her, then the operator complains of "fraud". The problem with this reasoning is that it fails to account for value of the right to privacy. Consider a telephone analogy, imagine a _toll-free_ number for free information. Imagine the operator referring to callers with unlisted numbers or ones who did not allow caller-ID when calling a toll-free number to obtain free information as comitting "fraud on the service". As a user, I see no point in "free-tier services" that require "sign-up" because they are really not "free". The user is being asked to give up some of her rights in exchange for the "service". Consider a public square analogy, imagine someone standing on a street corner handing out _free pamphlets_ but requiring people to "sign-up" to receive one. Then imagine the pamphlet publisher accusing those who did not provide real names and other details when "signing-up" as comitting "fraud". If the operator ceases the "free-tier" tactics to gather data/information about users and simply sells paid service then this problem "fraud on our service" becomes moot. The question becomes why the operator feels it _must_ provide "free" service. Consider a www analogy, imagine the "CTO" of HN accusing users who read HN but do not create accounts or provide their real names to HN as committing "fraud on our service". The www provides an enormous amount of free information. Beyond purchasing service from an ISP, no other "sign-up" is required. Accusing www users of committing "fraud" for retrieving such information without "signing-up" is silly. The majority of "fraud" occuring via the internet is not being perpetrated by users on "tech" companies. It's being perpetrated by "tech" companies on users.
- derefr 4y ago
  In our specific case, we’re providing users query access to datasets supplied to us by channel partners; where these partners pay us to make this data available through our API, because it makes their own ecosystems more valuable for our service to be offered as a part of them. So, technically, our revenue is entirely independent of customers. Which is why free-tier users still benefit us: the more DAU our platform receives, the more our partners value it, and the more they’re willing to pay us to make their data available through it. And before you ask, no, we don’t give our customer data to our partners; all they get to know is DAU and API call count for their network. > Whats the purpose of "free-tier" API SaaS. Mostly? Giving people a taste without the friction of paying, so that they get hooked, and start paying later. Like how you can have one Heroku VM for free. Or how GCP gives you a free month of usage when you sign up. I can’t think of a direct analogy within the API SaaS vertical specifically; but a hypothetical analogy would be if Twilio gave N free credits on each sign-up. > Fraud implies that the alleged perpetrator is obtaining something unlawfully or unfairly. However if that thing is "free" then how could this ever be true. Our free offer is intended to give people a certain amount of credits to spend to try out the service. By registering multiple accounts, you get multiples of that promotional amount. And, in fact, by registering multiple accounts, you escape the rate-limiting we place on every account, to ensure QoS and prevent Denial-of-Service attacks against our service. So we prevent people from registering multiple accounts. Analogy: someone who sees a free sample display in a grocery store, and stuffs the entire display into their bag. And then camps out the display each day, taking the whole thing again the instant it gets restocked. While also going around to N other stores in the chain, camping them out as well. It may not be illegal, but it’s 1. unethical, 2. against the spirit in which the promotion was intended, and 3. screws all the other customers out of their opportunity to get the free thing, because you’re using up their “share.”
leach 4y ago
I’m one of the people who uses cloud workstations sometimes. Lots of places ban you from doing stuff. Even Minecraft lol
iancarroll 4y ago
Worth noting that most IP blocks are going to be transferrable in some form, and thus most IP blocks can be sold on this site. IPXO is of course different since it's temporary leasing, but you couldn't really build a list of IPs that could be transferred, or you'd block most of the internet.
- derefr 4y ago
  That’s what I figured was at the root of there not already being a DNSRBL-alike list of these IP ranges, yeah. Maybe the closest thing would be an ASN reputation score, where one of the factors is the ASN’s newness, and another is the average recency of their acquisition of the IP blocks they own.
  - duskwuff 4y ago
    I don't think you have a good sense of what actually plays into IP reputation. Some of the sketchiest ASNs I know of have been assigned for many years, and perfectly reputable ISPs have to acquire new IP space occasionally.
    - derefr 4y ago
      > Some of the sketchiest ASNs I know of have been assigned for many years ...which can be addressed perfectly fine with a static, human-audited blacklist, because the old bad actors don't change. The goal of a reputation score would be to complement such a static blacklist, by recognizing the fly-by-night operations that constantly crop up and disappear for what they are; or, conversely, by rewarding the ASNs who manage to stick around long enough without ever doing anything to land them on the static blacklist. In other words, just like how mail domain reputation works. If your domain sticks around long enough without ending up on a DNSRBL, then it builds up a positive reputation, which acts as a counterweight in spam-filtering algorithms. If you sell the domain, though, then its reputation resets. > perfectly reputable ISPs have to acquire new IP space occasionally. Yeah, but the average age "by volume" of all their IPs would still be pretty old, as the new acquisitions would be marginal compared to their existing holdings.
Rasbora 4y ago
I am here to answer your prayers: https://rasbora.dev/blog/detecting-residential-proxy-networks https://rasbora.dev/blog/detecting-residential-proxy-network... While my tool is just a proof of concept, it can easily be scaled up to run checks on a sign up form to prevent bad actors from abusing the system.
- derefr 4y ago
  Interesting. "Anti-scalping" is a novel keyword to me; I've always just seen these kinds of techinques discussed under the "anti promotion-fraud" banner. Time for more research!
vbezhenar 4y ago
I guess similar reasoning caused cilium.io blocking Kazakhstani traffic. Never heard of this country? Bot. Thankfully they didn’t manage to block VPS and I still can use my VPN to circumvent the blocking. As if it isn’t enough for my government to block websites.
kortilla 4y ago
> or CGNAT, than solve them for only the next 1024 customers CGNAT requires buying IPs and a /22 is big enough for 100k customers. Your assumptions are wrong.
iforgotpassword 4y ago
Hmmm, those services selling you residential IP address proxying have become pretty cheap and easy to use. I guess if you'd block all those non-ISP addresses people would just start to use them. Im using one for a personal scraping project and it's very simple to set up, most allow you to pick a country of origin and other nice features.
Nextgrid 4y ago
Back when I had a leased line the outgoing IP was resolving to a colo facility right next to the dedicated servers they also offered. I'm glad I haven't and will never have a need for your company's services.
fomine3 4y ago
It's fortunate if the ISP already has enough IPv4 addresses even for CGNAT. Probably it's not the case for ISP in developing country.
stephen123 4y ago
Hi, At [Upollo](1) we do a lot for detecting dodgy signups and helping users to convert. IP lists are a part of what we use and we are planning on making our IP API's public in the near future. We find IP lists alone dont really cut it though. Its easy for someone to just tether to their phone to signup. If your interested we would love to chat :) [1] https://upollo.ai https://upollo.ai
vincentas 4y ago
Those are entirely wrong assumptions you have stated in your comment. IPXO works with many well-known telcos and infrastructure providers. Many projects use the IPs from the IPXO platform and announce them on AWS, Azure, or Google Cloud (those are actual SaaS companies) and not to mention Cloudflare, where we see IPs brought to their network using CF's BYOIP. All that information is visible and available on the BGP table. Regarding the abuse, bots, etc. Yes, we have abuse cases, and you are partially right about that. However, we take every abuse case very seriously since we are responsible for hundreds of LIRs who brought their IPs to IPXO. If we ignore the abuse, we would not be boarding tens of thousands or hundreds of thousands of new IPv4 every week. - Vincentas, CEO @ IPXO

by derefr 4y ago

leshenka 4y ago

> solution is always to switch back to their real computer's web browser for anything requiring IP reputation … only to find out that you’re through no fault of your own blocked by region because we suddenly have a splinternet in the great year of 2022

hnov 4y ago

So much for running a personal VPN in the cloud.

jimmydorry 4y ago
It gets worse every year. I use a personal VPN, especially when travelling.

5e92cb50239222b 4y ago

> VPS IP? Bot. > The other 0.0001% are people using "workstation in the cloud" services. Yeah, no. I've been using VPNs/SSH tunnels/proxies set up on various VPS hosters/"cloud" vendors for more than a decade to bypass heavy internet censorship. It is much cheaper than using ready-made VPN solutions, and the server can be reused for other purposes. I understand that may be <0.1% of your market, but it's not 0. We're definitely used to being treated as third-class internet citizens not only by our own government, but also by various internet companies, which your comment demonstrates pretty clearly. Here's a nice write-up. It's not mine, and it's not even the same country, but the experience he describes is very relatable. https://shahinsorkh.ir/2019/07/20/how-is-it-like-to-be-a-dev-in-iran https://shahinsorkh.ir/2019/07/20/how-is-it-like-to-be-a-dev...

taf2 4y ago
Just curious which cloud based software as a service subscriptions have you purchase and actively maintain? Also approximately how many $ a year to you spend on those services? I’m curious because blocking IPs from bad actors is often the most effective and immediate solution to protect a service from a bad actor… it would be nice to understand the financial impact of this.
derefr 4y ago
If there was a list of commercial VPN provider IP ranges, we’d block that too. A key property of people who aren’t willing to relinquish their anonymity for even a moment in registering for your service, is that they’ll never be able to pay money for your service, either, because doing so would reveal their identity through their credit card. (Yes, they can use Bitcoin or whatever, but then you’re basically signing up to be a dark web business, having to deal with being a money-laundering sink for stolen funds.) So, if your goal in getting users to sign up, is to feed them through a funnel that eventually ends up with them paying you money, then you can take “signs up using a VPN” (or an email anonymization service, or…) as an immediate self-disqualification. Of course, most such failed attempts are innocent first-time “I didn’t realize you cared” attempts; so we just throw a clear error message, telling them to try again with their real email+IP+etc. But when a user responds to that error by just trying more and more email proxy domains, or repeatedly rotating their VPN country… well, that’s someone who’ll never be a customer. May as well just block them and be done with it.

1vuio0pswjnm7 4y ago

Whats the purpose of "free-tier" API SaaS. For example, does "free-tier" provide an opportunity for the "API SaaS" operators to collect personal information, or at least email addresses, belonging to users who have no intent of paying the "API SaaS". The notion of "fraudulent sign-ups" for something allegedly offered for "free" sounds funny. Fraud implies that the alleged perpetrator is obtaining something unlawfully or unfairly. However if that thing is "free" then how could this ever be true. Perhaps the operator of the "free" service actually is expecting to receive (or take) something from the user. When the user fails to provide or allow the operator to take this thing from her, then the operator complains of "fraud". The problem with this reasoning is that it fails to account for value of the right to privacy. Consider a telephone analogy, imagine a _toll-free_ number for free information. Imagine the operator referring to callers with unlisted numbers or ones who did not allow caller-ID when calling a toll-free number to obtain free information as comitting "fraud on the service". As a user, I see no point in "free-tier services" that require "sign-up" because they are really not "free". The user is being asked to give up some of her rights in exchange for the "service". Consider a public square analogy, imagine someone standing on a street corner handing out _free pamphlets_ but requiring people to "sign-up" to receive one. Then imagine the pamphlet publisher accusing those who did not provide real names and other details when "signing-up" as comitting "fraud". If the operator ceases the "free-tier" tactics to gather data/information about users and simply sells paid service then this problem "fraud on our service" becomes moot. The question becomes why the operator feels it _must_ provide "free" service. Consider a www analogy, imagine the "CTO" of HN accusing users who read HN but do not create accounts or provide their real names to HN as committing "fraud on our service". The www provides an enormous amount of free information. Beyond purchasing service from an ISP, no other "sign-up" is required. Accusing www users of committing "fraud" for retrieving such information without "signing-up" is silly. The majority of "fraud" occuring via the internet is not being perpetrated by users on "tech" companies. It's being perpetrated by "tech" companies on users.

derefr 4y ago
In our specific case, we’re providing users query access to datasets supplied to us by channel partners; where these partners pay us to make this data available through our API, because it makes their own ecosystems more valuable for our service to be offered as a part of them. So, technically, our revenue is entirely independent of customers. Which is why free-tier users still benefit us: the more DAU our platform receives, the more our partners value it, and the more they’re willing to pay us to make their data available through it. And before you ask, no, we don’t give our customer data to our partners; all they get to know is DAU and API call count for their network. > Whats the purpose of "free-tier" API SaaS. Mostly? Giving people a taste without the friction of paying, so that they get hooked, and start paying later. Like how you can have one Heroku VM for free. Or how GCP gives you a free month of usage when you sign up. I can’t think of a direct analogy within the API SaaS vertical specifically; but a hypothetical analogy would be if Twilio gave N free credits on each sign-up. > Fraud implies that the alleged perpetrator is obtaining something unlawfully or unfairly. However if that thing is "free" then how could this ever be true. Our free offer is intended to give people a certain amount of credits to spend to try out the service. By registering multiple accounts, you get multiples of that promotional amount. And, in fact, by registering multiple accounts, you escape the rate-limiting we place on every account, to ensure QoS and prevent Denial-of-Service attacks against our service. So we prevent people from registering multiple accounts. Analogy: someone who sees a free sample display in a grocery store, and stuffs the entire display into their bag. And then camps out the display each day, taking the whole thing again the instant it gets restocked. While also going around to N other stores in the chain, camping them out as well. It may not be illegal, but it’s 1. unethical, 2. against the spirit in which the promotion was intended, and 3. screws all the other customers out of their opportunity to get the free thing, because you’re using up their “share.”

leach 4y ago

I’m one of the people who uses cloud workstations sometimes. Lots of places ban you from doing stuff. Even Minecraft lol

iancarroll 4y ago

Worth noting that most IP blocks are going to be transferrable in some form, and thus most IP blocks can be sold on this site. IPXO is of course different since it's temporary leasing, but you couldn't really build a list of IPs that could be transferred, or you'd block most of the internet.

derefr 4y ago
That’s what I figured was at the root of there not already being a DNSRBL-alike list of these IP ranges, yeah. Maybe the closest thing would be an ASN reputation score, where one of the factors is the ASN’s newness, and another is the average recency of their acquisition of the IP blocks they own.
- duskwuff 4y ago
  I don't think you have a good sense of what actually plays into IP reputation. Some of the sketchiest ASNs I know of have been assigned for many years, and perfectly reputable ISPs have to acquire new IP space occasionally.
  - derefr 4y ago
    > Some of the sketchiest ASNs I know of have been assigned for many years ...which can be addressed perfectly fine with a static, human-audited blacklist, because the old bad actors don't change. The goal of a reputation score would be to complement such a static blacklist, by recognizing the fly-by-night operations that constantly crop up and disappear for what they are; or, conversely, by rewarding the ASNs who manage to stick around long enough without ever doing anything to land them on the static blacklist. In other words, just like how mail domain reputation works. If your domain sticks around long enough without ending up on a DNSRBL, then it builds up a positive reputation, which acts as a counterweight in spam-filtering algorithms. If you sell the domain, though, then its reputation resets. > perfectly reputable ISPs have to acquire new IP space occasionally. Yeah, but the average age "by volume" of all their IPs would still be pretty old, as the new acquisitions would be marginal compared to their existing holdings.

Rasbora 4y ago

I am here to answer your prayers: https://rasbora.dev/blog/detecting-residential-proxy-networks https://rasbora.dev/blog/detecting-residential-proxy-network... While my tool is just a proof of concept, it can easily be scaled up to run checks on a sign up form to prevent bad actors from abusing the system.

derefr 4y ago
Interesting. "Anti-scalping" is a novel keyword to me; I've always just seen these kinds of techinques discussed under the "anti promotion-fraud" banner. Time for more research!

vbezhenar 4y ago

I guess similar reasoning caused cilium.io blocking Kazakhstani traffic. Never heard of this country? Bot. Thankfully they didn’t manage to block VPS and I still can use my VPN to circumvent the blocking. As if it isn’t enough for my government to block websites.

kortilla 4y ago

> or CGNAT, than solve them for only the next 1024 customers CGNAT requires buying IPs and a /22 is big enough for 100k customers. Your assumptions are wrong.

iforgotpassword 4y ago

Hmmm, those services selling you residential IP address proxying have become pretty cheap and easy to use. I guess if you'd block all those non-ISP addresses people would just start to use them. Im using one for a personal scraping project and it's very simple to set up, most allow you to pick a country of origin and other nice features.

Nextgrid 4y ago

Back when I had a leased line the outgoing IP was resolving to a colo facility right next to the dedicated servers they also offered. I'm glad I haven't and will never have a need for your company's services.

fomine3 4y ago

It's fortunate if the ISP already has enough IPv4 addresses even for CGNAT. Probably it's not the case for ISP in developing country.

stephen123 4y ago

Hi, At [Upollo](1) we do a lot for detecting dodgy signups and helping users to convert. IP lists are a part of what we use and we are planning on making our IP API's public in the near future. We find IP lists alone dont really cut it though. Its easy for someone to just tether to their phone to signup. If your interested we would love to chat :) [1] https://upollo.ai https://upollo.ai

vincentas 4y ago

Those are entirely wrong assumptions you have stated in your comment. IPXO works with many well-known telcos and infrastructure providers. Many projects use the IPs from the IPXO platform and announce them on AWS, Azure, or Google Cloud (those are actual SaaS companies) and not to mention Cloudflare, where we see IPs brought to their network using CF's BYOIP. All that information is visible and available on the BGP table. Regarding the abuse, bots, etc. Yes, we have abuse cases, and you are partially right about that. However, we take every abuse case very seriously since we are responsible for hundreds of LIRs who brought their IPs to IPXO. If we ignore the abuse, we would not be boarding tens of thousands or hundreds of thousands of new IPv4 every week. - Vincentas, CEO @ IPXO