8 ms·
Email obfuscation: What still works in 2023?
- sdflhasjd 3y agoGiven that the percentages of blocked spam are just 17, 50, 67 & 83, it appears that the sample was likely just 6 spam emails which isn't a lot. Also doesn't really explain the methodology. I assume it's against a control of no obfuscation which was the 6 emails.
- mbauman 3y agoSo is the real story here that putting your email on the open web simply isn't a significant "vulnerability" anymore? Honestly, why would you spider and scrape sites when you can just grab a leaked dataset? It would be slightly more interesting to put the canaries in a more commonly scanned location, like a plain-text bio field on a popular site (like this one... or FB... or LinkedIn...).
- NeoTar 3y agoSpidering and scraping is 'more' legal?
- marginalia_nu 3y agoScraping is pretty dangerous if you're running a spam business, since there are a lot of honeypots out there, and you'll pretty rapidly find yourself blacklisted. Email spammers generally want verified email addresses. Since it's far safer to buy email lists from some broker, you tend to get a lot more spam for signing up for things than posting your email publicly.
- aendruk 3y agoTo offer one corroborating point, my email address is posted in cleartext on the front page of my website and it receives spam at a rate of less than one message per day.
- capableweb 3y agoAnother: Used the same email for ~2 decades, never hidden it, giving it out freely and available on lots of public websites including my own. Get around ~30 spam emails per day, all automatically marked as spam. Around 2-3 gets through my filter each month.
- listenallyall 3y agoDifficult to extract any useful info from this post without specifying what or where your spam filter is. Does it run locally? On your own mail server? An ISP's default spam filter? Gmail? CPanel? Have you tweaked any settings?
- aendruk 3y agoWe’ve been discussing observed rates of attempted spam, not the effectiveness of downstream filters.
- listenallyall 3y ago> not the effectiveness of downstream filters. But he chose to mention that all spam were marked as such and that only 1 or 2 get through. Readers will naturally be curious what methodology and tools are in use.
- aendruk 3y ago“Difficult to extract any useful info from this post” reads as undue criticism, as if preemptive satisfaction of your curiosity about the tangent was compulsory.
- listenallyall 3y agoI disagree that the criticism is "undue"... a post that waves off spam as if it's a non-issue ("2-3 each month"), when in fact many people do struggle with it, helps nobody. A post actually explaining how you built or configured an effective spam filter, however, benefits the HN community.
- jjav 3y ago> So is the real story here that putting your email on the open web simply isn't a significant "vulnerability" anymore? Yes. The whole obfuscate my email thing is silly. I have the same email since the mid 90s, I post it without any care to wherever.
- Ringz 3y agoYou should have posted it right here to prove your point. /s
- nvy 3y agoSpam isn't a problem for you, I take it? What are you using to filter spam out?
- jjav 3y agoSpam is no longer significant. Back around year 2000 I was getting multiple thousands of spam messages per day (to the very same email address I still use today). Those were the dark and difficult days. Today I basically don't get any spam. I don't keep precise stats on the spam that makes it to my inbox but it's something like 2 or 3 per month. Compared to a few thousand per day twenty+ years ago! I want to highlight again that it is the exact same email address I use today. I block misconfigured connections at the SMTP level in postfix (I run my own mail server), that takes care of pretty much all of it. After that I run bayesian filtering (spamprobe) to score spam, but there isn't much to catch. Over the last 30 days spamprobe has caught 37 messages, all true positives. Unlike gmail, I also don't get any false positives (zero so far this year). I know how bad it was couple decades ago and a lot of people are still traumatized by those days and act like it's still 2001, but the reality in 2023 is that spam is a very minor issue.
- tmcdos 3y agoI also run postfix on my own domain but instead of bayesian filtering I also run PostFwd and PostGrey. The combination of these 3 is enough in my case to reach zero SPAM emails. I can share my config if someone is interested.
- NeoTar 3y ago> Given that the percentages of blocked spam are just 17, 50, 67 & 83, it appears that the sample was likely just 6 spam emails which isn't a lot. This is mentioned, but a little hidden (in the description for URL encoding): > This is based on a small sample size: just six bots that were observed over a one-year period. I think the number of spam emails would be a bad measure, since a single scan could result in many hundreds of spam emails.
- PinguTS 3y agoWhat about accessibility for people with visual impairments? The author has warnings for the last three version, because of usability. I see lots of red flags also for the other versions in terms of accessibility.
- RugnirViking 3y agoultimately I'm not sure that is a tractible problem; any method a screen reader would need to use to decode to speech would require getting the address in clear text via programatic means, which is something that one specifically needs to avoid if we want to stop bots scraping addresses
- iudqnolq 3y agoYou could have a button that triggers a captcha that then authenticated a request for the plaintext email. Or you could just use a spam filter.
- deleted 3y ago[deleted]
- RugnirViking 3y agoyou know, you're right, an optional audio capcha alongside these could work quite well. I stand corrected !
- runnerup 3y agoNot really a block. I’ve worked on CAPTCHA-solving bots. But it does increase the cost. I’d probably not scrape for emails anyways though, using leaked datasets is likely far more economical.
- stereo 3y agoWhat I'm missing from the article is an explanation of what the control is. The email with no protection, which you'd expect to be the control, blocked 17% of spam - 17% of what amount of spam? How does the author quantify the spam that's out there?
- RugnirViking 3y agoit says in the page that its because all the mails were on the same page (and presumably then scaped by the same bots), and yet a couple mails were recieved by other addresses and not by the unprotected one. So it would seem that the control is the set of unique mails recieved by all addresses. "Surprisingly, the unprotected email address appears to have blocked a spam email. Either that message wasn’t received, or an extra message was sent to one of the protected email addresses."
- progval 3y ago<a href="mailto:email@example.com">email</a> blocked 17% but email@example.com blocked 0%
- OJFord 3y agoReally? Seems like you'd have to actively try to be blocked by that, i.e. to extract 'email' but not the mailto. I suppose if you didn't actually scan the HTML, but instead all the contained text.. maybe it just tests an implementation detail basically.
- debugnik 3y agoMaybe their scrapper is so bad it actually captured "mailto:example@..."
- apitman 3y agoThe problem with email is that it fills so many roles. It's both the only universal chat program between trusted entities and the only universal way to allow people to cold call you. If you're really worried about spam, I recommend just keeping a different email address for each separate purpose.
- thdc 3y agoHere's an older submission about using ChatGPT to de-obfuscate the more basic methods: https://news.ycombinator.com/item?id=38150096 https://news.ycombinator.com/item?id=38150096 Some comments claim that it can break some of the more complex techniques presented in the article. I've tried it a few times myself with varying results that tend towards not working.
- throw555chip 3y agoFeels like ChatGPT has to show up in every Hacker News thread now.
- boznz 3y agoSeriously if you are still running email with HTML and scripting enabled you need to turn it off. Plain text is fine for everything use a secure messaging program like signal for anything else.
- earthboundkid 3y agoI’m surprised URL encoding works so well. I’ve been base64 encoding and using JS to decode, but if URL encoding is that reliably, it’s probably not worth it.
- notatoad 3y agoi'd be curious to know, of the spam emails received, what percentage were blocked by basic spam filtering? in 2023 email spam seems like mostly a solved problem - i very rarely get any that actually makes it through to my inbox. trying to solve the spam problem by protecting your email address from becoming public might have seemed like a valid strategy 20 years ago, but we have better tools now.
- gspencley 3y agoThe limitation of spam filters is false positives. For most people it's probably not a big deal to have one or two messages land in the spam filter .. then someone follows up and says "Hey did you get that legit email I tried to send you? I haven't heard back." But for certain business accounts, the amount of spam + false positives can get to unmanageable levels, where important emails are flagged as spam and left undiscovered because sifting through the spam folder regularly is as time consuming and annoying as if the spam just went straight to inbox.
- notatoad 3y agookay, but no amount of email obfuscation is going to allow me to disable my spam filter, so this isn't solving the false positive problem.
- gspencley 3y agoMy point is that it's still valuable to try and reduce the overall volume of spam. Spam filters are another angle of attack. Spam is a tough enough problem that it's always worth throwing multiple solutions at it, each good at solving a separate slice of the overall problem.
- pizzafeelsright 3y agoWhat service? I have gmail and O365 and some other web hosting provider. My work email is unknown, probably filtered 5x, plus on O365 so limited to known contacts.
- ape4 3y agoI wonder how much mail the example.com domain gets!
- theblazehen 3y agoGiven the MX record resolves to `.` I suspect not any (successfully)
- rpigab 3y agoNone, I always use those in tests, because they're owned by the IANA, so unless random domains that have no mx records now, but might in the future, the example org, net and com are safe. I do this because sometimes in companies, people will put DB dumps in the wrong environment that has an actual SMTP going to WAN, then shit happens. I also make sure the environments have a dummy smtp or mailcatcher, but it's better to be safe than sorry. See RFC2606: https://www.rfc-editor.org/rfc/rfc2606.html https://www.rfc-editor.org/rfc/rfc2606.html
- tyingq 3y ago> 1.3 No protection Blocked 17% of spam I read the explanation, but it just sounds like something is a bit off in the methodology and metrics. Or at least in my understanding of them :)
- gwbas1c 3y agoMight want to add an intro paragraph. I initially thought that "Email Obfuscation" meant something different, like trying to trick a mail client.
- prajaybasu 3y agoTip: Almost none of the scrapers and bots run WebAssembly :)
- davidy123 3y agoI have used a keyed email address scheme since the mid 1990s (me_context@domain.org). 99% of the email I get now is sent to the same 5 addresses, in particular me_boingboing, me_linkedin. Those addresses were never on a web page. The keyed addresses I used for my company occasionally get spam, but it's generally on topic (offering relevant services, though still imo spam). I also have a gmail address that is a common first/last name but I don't send from that address, I get a lot of presumably guessed address spam there. This suggests quite a bit of organization in how spam works these days.
- hunter2_ 3y agoIt's an interesting claim that defeating xor requires a JS interpreter but defeating the others (e.g., concatenation, rot18, etc.) does not. I mean, if a bot author wanted to scrape a site or sites where all obfuscation used the same xor routine, the author (not the bot!) could just read (not execute) the JS and customize the bot accordingly. Granted it restricts things to more targeted attacks rather than a bot that just crawls arbitrarily across the web, which is noteworthy.
- switch007 3y agoOn the topic of spam..."Spam" to me these days includes email from companies who abuse the soft opt-in, service emails and "existing commercial relationship" rules. Cart abandonment, tips on how to stay safe online, requests to go paperless, newsletters, claim your free subscription that came with your purchase, thank you for your purchase (separate to the order confirmation), requests to leave a review, continue your application, offering support, "you haven't logged in for a while", new login from $device, thank you for completing step X ... it's endless. An endless torrent of excuses to get the their company name in front of your eyeballs.
- brightball 3y ago"new login from $device" That's a security precaution, not spam.
- encom 3y agoExcept all my logins are from "a new device" because I use Cookie AutoDelete, there's no way to opt out of this spam, and I don't give a shit if anyone DID actually hack my Google account because it's only a way to keep track of my Youtube watch history and subs. I filter these as spam.
- brightball 3y agoAs long as you know the reason. For everybody else, it’s not spam.
- deleted 3y ago[deleted]
- switch007 3y agoIt's not a precaution. A security precaution would be blocking the login because of unusual activity I get so many, I tune out. I bet I'm not alone
- Sephr 3y agoI've discovered that the most effective trick is to use an email address that is considered invalid by most email scrapers but valid by most mail servers. I have an unobfuscated mailto: link on my blog and I receive barely any spam to my public email address, ~@eligrey.com
- brightball 3y agoSame. I took over a programming conference earlier this year and it uses a .codes domain.
- jonchurch_ 3y agoOP was saying that their email is literally ~@eligrey.com He’s saying this works bc scrapers might not match a left hand side of an @ that contains no alphanumerics
- brightball 3y agoI get that. I think the less common domain extensions are having a similar effect right now though.
- IshKebab 3y agoI guess you have a different email address that you use for website signups, shopping etc?
- Sephr 3y agoIndeed! I also maintain a list of entities that have leaked the unique email addresses that I have shared with them.[1] 1. https://gist.github.com/eligrey/5084991 https://gist.github.com/eligrey/5084991
- justinhj 3y ago~@eligrey.com implies it's not you, I would add an alias for ~~@eligrey.com for the comfort of any programmer friends.
- deadbabe 3y ago[dead]
- dom96 3y agoI'm using CSS-based obfuscation on my website[1] and it works quite well. 1 - https://picheta.me https://picheta.me
- pizzafeelsright 3y agoI have successfully abandoned email. Sites that require email for logins get my common email that I do not check unless required to reset a password or links for authentication. Long form communication is done via texting or shared chat.
- mock-possum 3y agoHow do you review old communication from 8 years ago? Or find texts with particular criteria - “from x, subject like y, received within a week of date z, has attachment”
- pizzafeelsright 3y agoWhat is there to review? We are living in strange times where the digital stuff just does not exist unless it is consumed in the present. I have loving emails written to me in pen and email. I read neither yet for some reason the penned documents are held onto with an emotional tie. My father, who use to painstakingly capture photos, create slide shows using a rotating projector, laments the days where people would review photos as a time of bonding. That no longer exists and now, even with 500x more photos I still long for the days of having the context of a few photos that are forever lost. GPS and timestamps are helpful I guess but I will never know the reason for that captured moment.
- Ringz 3y agoHe gave up on that too and leaves it to his parents.
- BeetleB 3y ago> Long form communication is done via texting or shared chat. That's a nightmare. In reality, long form just doesn't exist in chat/text. For me, long form is at least 500 words. Years go by before I get "long form" text/chat of that order.
- jowea 3y agoYou're getting downvoted quite hard for doing the same as what I suspect is the majority of Internet users right now.
- hcurtiss 3y agoTotal aside, but I recently purchased a .us domain name for the first time. I was on my phone and signing up with the Namecheap app. I thought it odd that it didn't ask for privacy protection, but I figured I'd quickly turn it on after the fact and didn't get around to it until later that night. I used our home phone as the phone number for the site. The next day we got no fewer than fifty spam phone calls, and the email address I offered was quickly inundated with over a hundred spam messages. It was absolutely nuts. Fortunately, I used an email alias and just turned it off. Likewise, I have a Callcentric VOIP number for the home, so I was able to add a "press six to continue" prompt, but I can't change that number without a lot of pain and I now no longer get the automated phone calls from the school (we still get them through the app). Since then, the call volume has dropped off, but I can see in the logs that we still get several per day. After all of that I learned that privacy protection is not available for .us domains. No wonder they're so cheap. A long way of saying, if you sign up for a .us domain, definitely do not give them your real information.
- deleted 3y ago[deleted]
- ulrischa 3y agoMost scrapers search for anchor tags. So a span with the aria role=link could work too. This will also keep accessibility. Because I'm not shure what a screenreader will read for the encoded E-Mail. https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Roles/link_role#examples https://developer.mozilla.org/en-US/docs/Web/Accessibility/A...
- acylia 3y agoI use JavaScript obfuscation[1] on my website and get very low amount of spam. However I am confident that some modern scrapers are using headless browsers and wait for the DOM to be fully loaded before extracting the email from the text data. [1]: https://ianisbernard.com/contact/ https://ianisbernard.com/contact/
- aragonite 3y agoA similar but CSS based method: put each character of the email address in a separate div, place the divs in the DOM in random order, but use CSS absolute positioning to make them appear in the right order to the user. Probably not straightforwardly scrapable by reading the DOM, would mess up text selection though.
- hurtuvac78 3y agoIt may not be the clerverness of the obfuscation that beats the spammer's bot. If you go to such lengths to obfuscate your email, then you may simply not be a bad target for my bot: response rates tiny. No good ROI for the spammer. Does not matter, it works then!! But not because of technical cleverness. No idea if what I am saying is true.
- djha-skin 3y agoTurn your email into a PNG. It's absolutely easy to read, and My brother who worked for a scraping company said they haven't cracked that yet (as of 5 years ago). Even today, you would need a neural net to do it which makes it more expensive. They would also need another neural net to figure out whether or not it was an email address! It's just an image after all.
- kmoser 3y agoThe article covers this method of using an image: https://spencermortensen.com/articles/email-obfuscation/#text-image https://spencermortensen.com/articles/email-obfuscation/#tex... The problem is that humans who want to cut and paste your email into a different client have to retype it, which is annoying and error prone.
- latchkey 3y agoI recently created a new company domain with email on it and I'm amazed at how quickly I'm getting spam on it. To be fair, most of it is directly from LinkedIn, but still...
- OmarAssadi 3y agoWhile it doesn't stop spam, I have been using a catch-all email system for a while now. The benefit is that I know where someone got my email from, and I can then try to figure out whether the place has been compromised, or whether they're selling my email, etc. And I can just blacklist that particular address forever as well. Previously, I just did whatever@mydomain.tld, but I've switched to something similar to blame.email [1]. This makes my emails look a little weirder, but it has stopped the weird looks I'd get when walking into a physical place, like my doctor, and telling them "Yeah, email me at <doctor's name>@<first><last>.com". It also makes it less obvious that its effectively a throwaway email, particularly combined with my domain; it looks fitting. And since each address is salted and hashed, it pretty much eliminates the risk of someone successfullying trying to phish me by sending me an email to something like `paypal@<first><last>.com`. Lastly, on my HN profile and elsewhere, I've got my "email", but despite them being unique, I still don't want to have to rotate it if it gets picked up by a spambot, so I've tried to do some plaintext simple "obfuscation" like in the article. I went for <address> ~АТ~ <domain>.<tld> -- with the "AT" being Cyrillic rather than Latin - I figure at least some will get tripped up by not being able to use purely English regex. So far, I have yet to receive any spam with that strategy. Maybe I'm lucky or just not getting indexed, or maybe it's working a little. Still torn about how to handle Git or copyright/license headers, though; those addresses need to last a long time, in case anyone needs to reach out and ask for re-licensing/etc, and I figure it'd be annoying doing different emails for each repo. [1]: https://news.ycombinator.com/item?id=31820502 https://news.ycombinator.com/item?id=31820502 / https://blame.email/ https://blame.email/
- shhsshs 3y agoI have been using a catch-all for about 2 years now with great results. I hadn't thought of hashing/obfuscating the emails though. I think since I use a password manager anyway, I could just generate a random 6-8 character prefix when signing up for a new account, and since it's saved in my password manager it's easy to look up again later (no need for a true hash).
- OmarAssadi 3y ago> I think since I use a password manager anyway, I could just generate a random 6-8 character prefix when signing up for a new account, and since it's saved in my password manager it's easy to look up again later (no need for a true hash). Yeah, same. I store all the addresses in KeePass. The main reason I don't just totally randomize them is just that there have been a few moments where I do have my salt somehow, but for whatever reason, it is either inconvenient or impossible to immediately open up the password manager and add a new entry. In those moments, being able to deterministically generate the address and then add it at my leisure without having to double-check what I used is nice. It also likely wouldn't happen to me, but should I ever somehow lose/lose access to both my old emails and my password manager, as long as I have my salt, I can still "remember" my email addresses for important services (e.g., PayPal or whatever) to re-generate the addresses and reset my passwords. Whatever route you go, be it randomized addresses or hashed addresses, even though I think I am more vigilant and careful than most, it's still nice having an extra-layer to the catch-all that can't easily be targeted by someone malicious without first either somehow obtaining your salt, compromising the service, etc; it's handy being able to immediately filter and flag anything relating to my bank or whatever else if it isn't sent to the right address.
- teravolts400 3y agoWhat worked really well for me: storing email in a base64 encoding and then decoding it whenever user hovered over the link for my email.
- 0x38B 3y agoWhat about this one? I think I found it in a Stack Overflow answer a year or two ago. I've used it, but haven't tested it for effectiveness in avoiding spam. <a href="#" class="cryptedmail" data-name="david" data-domain="davidlane" data-tld="io" onclick="window.location.href = 'mailto:' + this.dataset.name + '@' + this.dataset.domain + '.' + this.dataset.tld; return false;"></a> Paired with this CSS: .cryptedmail:after { content: attr(data-name) "@" attr(data-domain) "." attr(data-tld); } Which results in a clickable link that opens whatever is set up to handle mailto: links (1). 1: https://nexus.armylane.com/files/email-crypted.png https://nexus.armylane.com/files/email-crypted.png
- m4l1c3 3y agoThe only thing that works is to stop using email in 2023.