4 ms·

Ask HN: How can a website copy mine in real time?

• RecycledEle 2y ago
  My guess is that your web hosting provider sold you out. DirectNIC did it to me. You can test this by adding a file like inxed.htm (note the deliberate misspelling) in your web root folder without any links to it anywhere and seeing if you can hit it in their web server.
• cranberryturkey 2y ago
  its easy. is your site accessible on the internet? it can be copied in realtime.
  • calculito 2y ago
    ok, it's easy for you to understand. For some other people, me for example, not that easy. The question is also, how is is possible. So, I don't expect a detailed explanation or a step-by-step howTo but do you have any hints?
    • cranberryturkey 2y ago
      its called 'web scraping' look it up. You can send a cease and desist or block their ips.
      • jrvieira 2y ago
        Server side code is not publicly accessible, that's not what webscraping means, and nothing is being copied in this case, it's being proxied.
    • ahazred8ta 2y ago
      Other commenters have suggested that the xyz site is not using telepathy to instantly update itself whenever you make a change, but rather they have no local copy of your site at all, and every time you load an xyz page, it fetches the contents directly from your site. Some are assuming that the foo.xyz DNS is pointing at your own server, but you say you've checked for that.
• sim7c00 2y ago
  not sure about nextjs / vercel, but don't you have access logs for your webserver which you can check to see who's doing the cloning? Perhaps you can find some user-agent or certain ip-range or something is responsible, and block that. there are tons of automated systems copying everything everywhere unfortunately. a lot are fairly ok to block. There's also services you can hire or hosting providers who do some filtering for you, but those are generally more expensive than basic ones. (added security tooling around their platform)
• petercooper 2y ago
  Might they just be reverse proxying it? Make a request for an unusual URL and see if it appears in your logs.
  • francescochi 2y ago
    Yes, that's exactly what they are doing. How can I stop this?
    • mlhpdx 2y ago
      Put CORS in place and a strict CSP. Look the Host header in your API backend and block it. It might be forged, but probably not. Identify their host (reverse IP lookup) and block requests from that ASN. If they host in same place as you, complain and get them shut down (or move). The obfuscated JS based domain checking others suggested. None of these is perfect, but they don’t have to be. Just keep adding layers until they go away for good.
    • gary_0 2y ago
      If your site is only meant for human end users, you can get lists of datacenter IPs and use that to block them (assuming they are running in a datacenter that's on the list). That tends to block all sorts of abuse. If you are worried about blocking legit users with this method, you can show a CAPTCHA instead on every request from a suspect IP. At any rate, get their IP from your logs and run a whois on it to get their ISP and location. You could also try emailing the ISP's abuse contact or filing a DMCA complaint with their hosting provider, although of course that might not work.
    • petercooper 2y ago
      One naive approach would be to look at what IP address(es) they're using when proxying and blocking them. (Or detecting them in code and redirecting them to your domain, perhaps.)
    • roosgit 2y ago
      I had a similar thing happen to one of my websites. In Varnish I used something like this: if (req.http.host ~ "^(?i)(example.com|www.example.com)") { #redirect to https } else { return(synth(403, "Not allowed.")); } It basically checks if the host is my domain. I don’t know know what the equivalent of `req.http.host` is on the web server you use. This "solution" might run into issues with Google Translate, but I’m not sure.
• beardyw 2y ago
  Check the domain and if it's not yours redirect it. If they copy that you are done.
• maremmano 2y ago
  Realtime? DNS?
• ActorNightly 2y ago
  Just add obfuscated java script code that checks the host and errors out if its not your domain. Can still be bypassed, just will take some reverse engineering. Ive added JSFuck code for this specifically on a few occasions
• curtisblaine 2y ago
  Insert some heavily obfuscated javascript code that checks the domain at runtime and, if it's the wrong one, redirects to distasteful images. If the site is in China, redirecting to prohibited content is another fun thing to do.