10 ms·
I accidentally made my link shortener into a malware honeypot
- deleted 3y ago[deleted]
- goth60000 3y agoVery interesting, thanks for sharing. Wish you had made it into an actual honeypot though!
- hilux 3y agoSuch an interesting read. I prompts me to wonder whether abuse was one reason that Heroku removed their beloved (among students) free tier.
- y_gy 3y agoMost likely. We see a large amount of abuse coming from the replit free tier. And that's pretty similar to what Heroku used to offer. If you're going to provide people with free compute online, there are just a lot of ways to exploit that.
- dividendpayee 3y agoI was shocked that even with the free tier gone, there's still some level of abuse. Even the paywall doesn't totally cure the problem.
- sodality2 3y agoPassing malicious URL filters is crucial to operations like ransomware, phishing, etc - hiding a bad domain behind a good one is extremely valuable to hackers and relatively cheap. Though I am surprised they'd pay for it due to the payment -> identity link (maybe it's stolen CCs but Stripe is pretty good about blocking that).
- bell-cot 3y ago> Though I am surprised they'd pay for it due to the payment -> identity... Between gift cards, money mules, shell corporations, and "that country doesn't cooperate with investigations"...I'd guess that this is no more than a minor problem for serious criminals.
- arccy 3y agohoneypot indicates some sort of intention to do it, but as the post states, they don't want any of it
- schleck8 3y agothis is the second time I'm seeing someone point out Replit being used for obvious phishing and I'm pretty sure I've even seen it myself before
- VyseofArcadia 3y agoIs there a name for this phenomenon? It's sort of like the dark forest, but not exactly. As soon as a free service becomes discovered, it is immediately swamped by scammers and spammers. Many many years ago I ran a small forum for a small webcomic, and one day it was just full of low effort scams and spam. For an audience of, I dunno, a dozen people? I just shut the whole thing down because it wasn't worth our time to do anything about it. We just can't have nice things, and if you run across something that is actually nice, make sure to thank whoever runs it for all their behind the scenes effort to deal with the scumbags that clog everything, and I mean everything, up with s(p|c)am.
- dhosek 3y agoI had a similar thing happen with a mediawiki site that I run. There was some “shrinkwrap” software behind the abuse, though, and a trivial capcha on account creation was sufficient to turn the abuse from a flood into a manageable trickle (I haven’t had to deal with spam since December, and when I do get spamming, it’s typically happening no more than once a month).
- Vt71fcAqt7 3y agoThis is a great writeup. If you are just looking to deter scammers I bet $1 would have the same affect. I don't think scammers are worried about the price as much as having to give any amount of information to you. I could be wrong though as I am not a scammer!
- nubinetwork 3y agoThe article said that a few scammers tried to pay them to look more legitimate.
- thimkerbell 3y agoPayment or email&phone also gives the site owner sellable info. Maybe not a good idea for the customer.
- Karellen 3y agoPSA: https://en.wikipedia.org/wiki/URL_shortening#Disadvantages https://en.wikipedia.org/wiki/URL_shortening#Disadvantages (Also note the difference between the length of the "Advantages" and "Disadvantages" sections)
- kornhole 3y agoI generally always run any shortened link through a link checker before opening. So they are an inconvenience to me. The time it took you to write all this evidences the problem with hosting the service publicly. Yesterday I ran into problem with sharing a link to a simplex.chat group which was so long my website builder translated it incorrectly. I looked at link shorteners publicly available and now understand from your writeup why they are somewhat limited now. I found it easier to just spin up my own link shortener on my webserver using Shuri. It took less than a minute for me install. I won't publicize its availability now that I have read this.
- doakes 3y agoWhich link checker(s) do you use?
- SushiHippie 3y agoNot OP but I use https://wheregoes.com https://wheregoes.com, as it shows you all redirects that happen.
- kornhole 3y agohttps://f-droid.org/en/packages/com.trianguloy.urlchecker/ https://f-droid.org/en/packages/com.trianguloy.urlchecker/
- ay 3y agoVery cool read! For the malicious links, did you have a chance to track whether the malware actors verify that their links do not work, e.g. by setting a cookie when they make a link and checking it later ? I wonder if making these malicious links silently work only for the people that submitted them (and to say “no such link” for everyone else) ought to create a degree of confusion and slow them down to some extent at least…
- TimLeland 3y agoI can really relate to this article! I created T.LY URL Shortener in 2018, and I've encountered all these issues and more! I found out the hard way when my hosting company shut down my servers for malicious content about a week into launching the site. Malicious actors will go to all sorts of lengths to achieve their goals. Be careful relying on Stripe to prevent these users. Next they will start using stolen credit cards to create accounts then you will face disputes. If you get too many, Stripe will prevent you from processing payments. About a year ago, I launched a service called Link Shield. It's an API that returns risk scores (0-100) on URLs. It uses AI and other services to score if a URL is malicious. Check it out and let me know if you would be interested in trying it linkshieldapi.com/
- What2159 3y agoAffordable is not a price. I don't want to login before getting pricing.
- elaus 3y agoThis is really one of the worst patterns in the SAAS market. I don't want to provide my data to multiple services just to be able to compare their prices and find out which one I'm actually gonna use. At first this will lead to countless automated mails from all those "founders" asking why I haven't started paying yet, and if I'm unlucky my credentials end up on haveibeenpwned.com…
- hoistbypetard 3y agoThis. And related: I don't want to have to try your system in order to get pricing. I've seen that a couple times, particularly for things that are in beta, where you don't even see pricing until the end of the trial period. Integrating a new system requires some effort. And there are some systems, like the one in question here, where there's a real cap on how much value they could possibly provide for me, even if they're perfect. If I can't see whether the pricing falls in that range before I need to sign up, I'm just not going to seriously consider it for most services.
- stevenicr 3y ago
- JoshTriplett 3y agoWhat's the benefit of a link shortener, these days? It made sense back before Twitter had one of their own. And I know that some people use it to get link analytics. I've also occasionally seen it used for printed materials, to get pretty URLs that are easy to hand-type. People also use it for malicious purposes, such as hiding malware, or disguising referral links, or otherwise trying to obfuscate where a link is going. (Note: I'm not calling referral links malicious, I'm calling disguised referral links malicious.) Other than printed materials (which need pretty URLs and thus often need a dedicated first-party URL shortener) and analytics, what are people using third-party URL shorteners for today?
- donatj 3y agoMy company has one they use to track who clicks on links in emails.
- JoshTriplett 3y agoThat's the link analytics case I mentioned.
- namrog84 3y agoI see most printed things just use qr codes now too. And most phones can go to qr code url pretty easy
- prophesi 3y agoWhich is most unfortunate... QR/Camera apps usually just show the domain anyways, and QR codes can easily fit large URL's. I imagine shorteners are used just so that they can choose a lower QR version and include a pretty logo in the middle.
- aiisjustanif 3y agoBeneficial for shared presentations
- TehShrike 3y ago
- butz 3y agoWhile "honeypot" was mentioned in the title, there seems to be no useful outcome from caught bad actors, like reporting malicious websites, so browsers could block them.
- akpa1 3y agoAmusingly, I thought this website was broken in a myriad of weird ways - I kept getting incomplete response errors and bad SSL errors. As it turns out, my ISP was simply doing a rubbish job at blocking the site. After a few 10s of tries it eventually managed to redirect me to their warning page and prompted me to turn off settings in my account config. Thanks Virgin Media.
- mik3y 3y agoA big problem that came up at the domain level was what I'd call a _trustworthy domain with untrustworthy subdomains_, specifically where those subdomains represent user-generated content. The Public Suffix List (PSL) [1] to the rescue! It can help with this kind of disambiguation. Paraphrasing, it's a list of domains where subdomains should be treated as separate sites (e.g. for cookie purposes). So `blogger.com` on the list means `*.blogger.com` are separate "sites". [1] https://en.wikipedia.org/wiki/Public_Suffix_List https://en.wikipedia.org/wiki/Public_Suffix_List
- ghayes 3y agoBut the cost dynamic would still be different, right? As in, it doesn't cost as much to register a .blogger.com as it does a .com?
- heleninboodler 3y agoHow current is this? It doesn't actually have *.blogger.com in it, nor the other two examples I checked.
- mik3y 3y agoI just made up `blogger.com` as an example. I probably could have picked a better one. `blogspot.com` & its many TLD variations are on the list. It looks like the repo where the list is maintained [1] is pretty active. YMMV, I'm not a maintainer or anything.. [1] https://github.com/publicsuffix/list https://github.com/publicsuffix/list
- heleninboodler 3y agoThe other two I checked were *.wordpress.com and *.bandcamp.com, both well known TLDs with user-generated subdomains. Neither is there.
- david422 3y agoI've dealt with some spammers to various degrees. I think one of the most effective ways of dealing with spammers is to - "shadowban" them. Allow them to use your service, but don't indicate to them that you've identified them as malicious. For instance, when dealing with chat spammers - allow them to chat, but do not show their chats to other users. Another level would be to allow them to chat, but only show their chat to other shadowbanned users. For the author's use case, perhaps something like - if the ip address that created the link shortener accesses the link, they get the real redirect, and if a different ip address accesses it, they get the scam warning page. If the malicious actor doesn't know they've been marked as malicious, they do not know they need to change their behavior. The second most effective thing is making the malicious actor use some sort of resource. Such as a payment (the author uses), or a time commitment (eg new accounts can only create 1 link a day), or some other source of friction. The idea is that for legitimate users the friction is acceptably low, but for consistent spammers the cost becomes too high. The 3rd thing I've found effective is that lots of spam comes from robots - or perhaps robots farming tasks to humans. If you can determine how the traffic is coming in and then filter that traffic effectively without indicating failure, robots can happily spam away and you can happily filter away.
- optimalsolver 3y agoOne step away from this: https://twitter.com/nearcyan/status/1532076277947330561 https://twitter.com/nearcyan/status/1532076277947330561
- LeonenTheDK 3y agoThat's actually a very interesting idea I hadn't seen before. Certainly makes it less obvious that one has been shadowbanned, and probably would help keep (non-bots) happy. I wonder if it'd be worth the investment to implement.
- tim333 3y agoBy the way the NYT article is fake https://news.ycombinator.com/item?id=31588260 https://news.ycombinator.com/item?id=31588260 Still an interesting idea though.
- nerdbert 3y agoI made a link shortener in 2010 and it was such a terrible experience. Constant notices from my hosting company about child porn links, repeated ominous emails from the FBI and their counterparts in other countries, having my server temporarily shut down repeatedly. I abandoned it after 6 months because the amount of time it took to continually adapt countermeasures to all the scummy abusers was too overwhelming. In so doing, I'm sure I contributed to all the link rot out there.
- thimkerbell 3y agoHow can we grow better people?
- AceJohnny2 3y agoTangentially, it's kinda funny how people really don't realize how much websites/companies/social system implement user-unfriendly behavior because of scammers or other bad actors. (Admittedly, it's something that I also did not understand when I was younger and more naive. Hell, I had to explain this to my 70y-old parent just a few weeks ago!) The price of success is you then need to deal with moderation in some form. (and on that note: "it is easier to automate bad behavior than it is to police it") Right now, "enshittification" is (rightly) on many people's minds, but before that the reason any company makes a process difficult is because some assholes ruined it for the rest of us.
- AceJohnny2 3y agoAnd, at the risk of getting too philosophical, this is also the story of life. Parasitism is the reason things are as they are. But we got Sex out of it [1], so that's nice? Maybe? [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5204169/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5204169/
- tpurves 3y agoSemi related. When I worked at Visa, I developed some ideas around making QR codes slightly more resilient to malicious hijacking when used in the context of a payments or commerce usecase. The idea was for the scanning app to look not just for a QR but also look for adjacent payment acceptance marks (e.g. branded Visa, MC, PayPal, or a merchant's brandmark etc.) and then dynamically only resolve URLs to registered domains associate with those marks. The idea was that QR codes not human readable, and URLs are a lot to ask the average person to reliable parse. So instead, have the scanner also see and understand the same contextual cues that the human can see and understand. And for the human, give them the confidence to scan QRs that will take them to a domain they would expect, and not to a Rick Astley video or worse.
- r0s 3y agoI was recently discussing this subject and I have to wonder if some combination of human readable symbols that is also optimized for machine scanning will emerge. Right now any phone should be able to parse a url if it can read the type, and so what is the point of QR besides the ubiquity?
- pants2 3y agoQR codes provide built-in error correction so will stand up to serious wear-and-tear, partially obscured images, etc. - and it won't confuse O with 0 and i with l
- r0s 3y agoAll that is true of regular type as well to some degree, I guess my point is a standard of readable url type could have all those qualities. Also the longer a url is out in physical space the more danger of it being replaced online, longevity may not be desirable.
- tpurves 3y agoYou are raising all the right points. QR code standards come from an era when cheap digital cameras sensors were MUCH less good than they are now, and similarly when OCR/image-recognition resources were much less cheaply available or built-in to mobile devices.
- not2b 3y agoNo disrespect to the folks at y_gy who are clearly doing their best. But link shorteners, even when used by good faith actors, are problematic because they hide the destination of the link, and of course that's an invitation for bad faith actors to exploit, so the battle will be endless. Shorteners got popular on Twitter back in the days when all the characters in the URL counted against a very short limit. But there's less need to use them these days, and I am very reluctant to click on shortened links and don't think that this is unusual.
- fddrdplktrew 3y ago> But link shorteners, even when used by good faith actors, are problematic because they hide the destination of the link In a sense, Google Search is even more evil because they change the destination link on-click. So hovering on a search result link doesn't show you the true destination.
- anonymousDan 3y agoThis is why I love 'Copy clean link' in Brave.
- ksenzee 3y ago“Copy Link Without Site Tracking” is available in Firefox also.
- fddrdplktrew 3y ago"Visit link without site tracking" would be nice, to save a step.
- mid-kid 3y agoWhat worries me the most about things like these is that it makes it seem like it's impossible to make "free for all" products like these anymore if you're not an established player already. You will get blacklisted and you will receive emails from your host telling you to shut it down... Established players like bitly and tinyurl didn't have all the resources to deal with the problem when they started out either, and they arguably still don't, yet they get favored by the antivirus vendors and "safe"search blacklists, since they're well-known services. It doesn't seem fair. Is this really the way it should be? I wonder if they could've explained the situation to the antivirus vendors: The site itself doesn't host malware and doesn't allow the discovery of said malware through its service. It requires a user to receive an exact URL, just like they could've received any other link, and the blocklists should operate on what's hidden behind it instead of the redirect in front. Maybe y.gy could've been hooked into the safesearch API to automatically nuke any URLs blacklisted already by them, or another antivirus vendor.
- gwern 3y agoSee also: https://danluu.com/diseconomies-scale/ https://danluu.com/diseconomies-scale/
- josefresco 3y agoA couple years ago a client asked me for their own URL shortener service. I found YOURLS (https://github.com/YOURLS/YOURLS https://github.com/YOURLS/YOURLS) and reluctantly installed it on a cheap, shared, hosting account. Thankfully after a couple years, I convinced them (it took several tries) to use a 3rd party hosted provider. Bullet dodged.
- deleted 3y ago[deleted]
- 123yawaworht456 3y agothere was a simpler solution - ignoring amazon, ignoring cloudflare, ignoring "antivirus" companies.
- laurent123456 3y agoGetting a chargeback in Stripe is costly. As soon as a dispute is started there's a fixed $25 that won't be refunded even if you win the dispute. So for a service at $4 a month which is likely to get a lot of fraudulent payments I wonder if it's really viable. One thing he should do is immediately cancel accounts and refund subscriptions when there's an early fraud warning. They are usually accurate and help avoiding those fees.
- StayTrue 3y agoThis was my thought as well. I wonder if the author couldn’t achieve similar friction without charging. Require a card for signup but only authorize it in the case of “free tier” users. PayPal will let me do that for free, not sure about Stripe.
- LuciBb 3y ago[dead]
- pquki4 3y agoMy first thought after reading the title: I would never create a link shortener service, too complex and too much responsibility -- can it handle the traffic? what analytics can I provide? should it be a paid service (or rather, can it survive without being a paid service)? how to fight off scammers? what if some day the site goes down or permanently stops running, does that mean all those links are now useless? My thoughts after reading the article: I was so right.
- urbandw311er 3y agoKudos for the great site and ethos. You still seem pretty buoyed by the experience. Ultimately, I found the article pretty depressing. Your initial free offering with a good UX and relatively little ongoing maintenance was destroyed by an army of criminals. It ended up wiping out weeks of your time developing increasingly complex cat and mouse techniques. Ultimately resulting in you abandoning most of the free plan altogether. Kinda sad that this is what the online world has become. And we just put up with it. Imagine if walking down the road each day was like this – people lining up ready to swindle you or manhandle you in order to steal your things. There would be outrage. But online we have just sort of reached a weird state of acceptance I guess.