17 ms·

Companies may be losing millions due to emails buried in collapsed Gmail threads

244 points by eaguyhn 8y ago

williamscales 8y ago
All Gmail would have to do to fix the issue is consider the link target when diffing to determine which part is quoted from previous messages. I've noticed this myself on the occasion I end up requesting a password reset multiple times. It's annoying at best to have to click open a closed message and then click again to show the quoted text. I'm sure it took me a minute to figure out what was going on the first time I encountered this.
tedeh 8y ago
TLDR: Gmail hid the most recent password reset email with a working token link at the bottom due to grouping similar looking emails together. The fix was to introduce a date in the subject line, thus preventing grouping in the first place. Edit: Well, actually it didn't work and it is still an ongoing bug.
- tyingq 8y ago
  The "still not implemented" fix, apparently. "Actually, the bug is still active. I never had the opportunity to fix it, didn’t stay long."
shereadsthenews 8y ago
Alternate title: Our Assumptions About MUA Presentation Turned Out To Be Wrong.
ggm 8y ago
The issue is impacting all users who forgot their password, use Gmail (not sure about other clients) and don’t notice the hidden messages being at the bottom (it’s really hard to spot). So the "bug" is collapsed message/sort/thread view? Feels like this is about as un-bug-like as you can get. Adding the unique Subject: line feels like the right fix (in you) rather than shout at google "you have a bug"
- pjc50 8y ago
  The argument is that collapsed view should show the most recent message and collapse the older ones, not the other way round.
  - mrlala 8y ago
    But it doesn't really know which individual messages inside you have read. So collapsing everything but the most recent one is dangerous because that is a real risk of missing messages. The current way yeah you might have to see where the conversation left off- but it's the safest to not hide something from the user.
    - yellowapple 8y ago
      Gmail does know which messages in the thread you've already read, though; it'll leave the unread messages uncollapsed, but the ones that were already read will be collapsed. The problem is that the first message is also uncollapsed, so the user has to know to scroll to the bottom.
- mannykannot 8y ago
  I am quite surprised by this response, and even more that there are several saying the same thing. > So the "bug" is collapsed message/sort/thread view? No - the bug is that new messages were being hidden because they were mischaracterized as redundant. I quickly disabled Google's 'conversations' feature (or whatever they call it), when it was first introduced, because it did not seem to be doing a good job, but did not think about its wider implications. I probably should not be surprised about these reactions, as I have had experience with fellow developers who insist that what is manifestly a bug is not one because it is "working as designed", or because there is an undocumented workaround to the undocumented problem.
  - mattmanser 8y ago
    It does a great job, apart from this, it's a really annoying thing that I have no idea why they've done it.
Someone1234 8y ago
Article admits it isn't a bug then continues calling it a "bug." Also the $187M figure is just pulled out of the air. Essentially Gmail (and other mail services) put messages from the same sender with the same subject into a Conversation View. Users are confused by it, and clicking the oldest password reset link (expired) rather than the newest. Claims that users being unable to reset their passwords costs Expedia $187M.
- Brahma111 8y ago
  Oh..and the clickbaity headline got me again.. All while I was looking for how some hacker exploited it and got Expedia's business
- feketegy 8y ago
  LOL, I guess it's anything for clicks these days...
- Grue3 8y ago
  Nah, the author does have a point. There's something fundamentally wrong with how Gmail collapses messages. I have email notifications from a certain web store when some of the items I might be interested in come in stock/for preorder. I always read them. Years after I started using it, I discovered that I actually miss reading about 30% of these messages. How does this happen? Well, it turns out that Google randomly groups some of these messages in a single thread. And yet this thread looks identical to a single message. So I click on the thread, read the first message, see the email footer and go back to Inbox to read the next one. This marks the entire thread read even though I didn't actually read the subsequent messages. So I never come around to read them. I completely believe that most users will not read any email in the thread beyond the first one, because there's absolutely no indication there's more stuff to read.
  - mrlala 8y ago
    >Well, it turns out that Google randomly groups some of these messages in a single thread Yeah you are right, I remember when they implemented that "random grouping" to confuse their users and make email unreadable. In other news, they group messages with the same subject into a single thread (or rather, how the subject changes when someone clicks reply like adding re:). So when someone replies, it stays in the same thread. It's clear as day that there are messages below because they are showing. There's a number next to the email thread you clicked on with how many replies there are. >I completely believe that most users will not read any email in the thread beyond the first one, because there's absolutely no indication there's more stuff to read. So.. you do realize just how many years people have been using gmail, right? And you believe that most users are missing basically every email that is a reply? I mean.. how does your mind work dude?
shay_ker 8y ago
This title is way too clickbaity for what it's talking about: Gmail grouping emails by subject can bury password reset emails
recursion 8y ago
Very interesting read. Author should have submitted the problem and solution as part of a bug bounty perhaps. Something for us all to watch out for.
Nicksil 8y ago
Is the GMail behavior described here accurate? Does GMail really hide the _latest_ email in a collection sharing similar sender and subject?
- pbogle 8y ago
  Actually, GMail expands the most recent email and collapses older messages. (I just double checked.) So the premise of the article seems a bit misguided.
- killaken2000 8y ago
  I know it hides similar parts of a series of emails because it cuts off my email salutation and sometimes people think that I'm being disrespectful. I wish I could stop it from doing that. If true, I wonder how similar these emails were to have that happen.
- williamscales 8y ago
  GMail will hide parts of a message that are similar to previous ones. When the entire message consists of what GMail considers to be quoted text, it doesn't really look like there's an email there. I just tried this myself and in the second password reset email, I only see a button with three little dots that says "show trimmed content" when I mouse over it. It's tiny and I wouldn't expect someone to figure it out if they didn't spend some time looking or have an idea of what happened.
  - fatnoah 8y ago
    OMG, it's not just me. It's super frustrating on mobile. On certain threads, I'll literally spend 5 minutes just trying to find the new content.
    - mschaef 8y ago
      I still miss pine, tbh (If this reference is too old: https://en.wikipedia.org/wiki/Pine_(email_client) https://en.wikipedia.org/wiki/Pine_(email_client) )
paxy 8y ago
While a whimsical writing and storytelling style sometimes works great, in this case it makes the article very confusing, even contradictory in places. I also find their numbers very hard to believe. Expedia's net income last year was ~$400M. They could apparently make a 60-character fix and increase that by $187M but they don't, because...reasons?
- feifan 8y ago
  I'm guessing $187M is referring to total bookings, which is in the $Billions
- user5994461 8y ago
  Where did it have you confused? Changes can be impossibly difficult in large companies for no particular reason.
abtinf 8y ago
The defense of gmail in the other comments is bizarre. Gmail suffers from a number of similar, very serious usability issues. From the perspective of the user, failing to show the correct password reset email is a bug in gmail that affects many sites. A less advanced non-threaded show-most-recent email view would not suffer from this issue. When you add a more advanced feature, make it the default, and inadvertently reduce usability, you are at fault.
- tyingq 8y ago
  I won't defend Gmail, but not testing critical email with the most popular client seems unwise. And, tracking failed resets for popular domains seems like a good idea.
- brianpgordon 8y ago
  Gmail's user experience is terrible in many ways, but I don't think it's fair to excoriate them for adding a feature that's not so idiot proof that not even one in a hundred users will misapprehend how to use it. That school of ultra-spacious user friendly designs for the lowest common denominator is how we ended up with the nightmare that is the current Gmail interface.
- asark 8y ago
  A few years ago, on Nu Gmail (Ajaxy Gmail) but pre-Inbox, I tried to walk my grandma through adding a contact. Their add contact screen, with the form to fill in contact info, had two same-colored buttons, not that far apart, same copy. One added the contact whose info you'd just been putting in. The other helpfully erased it (add a new contact, was what that one meant). That's probably been fixed, but the whole settings area was kinda that way back then—like someone slapped it together without once thinking about how it'd be used. IDK what it's like now now, I use basic HTML gmail. [EDIT] who's to whose. I cannot believe I wrote that.
- SmellyGeekBoy 8y ago
  > The defense of gmail in the other comments is bizarre. This statement is bizarre, I don't really see anyone defending Gmail here - just commenters offering further explanation on what's happening and other people mentioning specific examples of being caught out by this behaviour.
reitzensteinm 8y ago
Setting aside the validity of the dollar amount, the title belies a dangerous view of product development. In the real world, the buck stops with you. Even if it's not fair, if there's an issue your users are having, you need to fix it. At their scale, there is no excuse for not polishing every minute aspect of the UX, and that it includes how it interacts with every email service. Expedia stole 187m from themselves by not having their shit together.
dan_m2k 8y ago
I can’t believe I got hooked in by that deceptive, clickbaity headline. Am I on the ExpressOnline and didn’t realise?!
pmlnr 8y ago
I have a Gmail nightmare story from a couple of years ago. The company I work for was sending confirmation emails doing everything properly: separate IP range mx boxes, warmed up, DMARC, SPF, DKIM, all the bells and whistles. Nothing else was allowed to go through them. Suddenly people weren't getting those confirmation letters. It turned out that soon after Gmail introduced the "Promotions" tab, a silent, new feature was added: anything from noreply@ was put in there. We tried going through the official channels - once we got to the end of the tunnel, it told us "we'll get back to you in 2 weeks". That's when we started digging into internal google connections across the company, and thank god, we found someone who know someone, and in was sorted within a couple of hours. (It might have helped that the company spends a lot of money on google ads.) Gmail is lovely.
- gnu8 8y ago
  You should never have been sending mail from noreply@ in the first place. Every message needs to be send from a real mailbox that can receive a reply that will be read by a person. If the message isn’t important enough for that, then it is spam and Gmail was doing the correct thing with it.
  - pmlnr 8y ago
    I don't believe your approach is correct, for multiple reasons. There are valid scenarios for one way communication; those emails are only verifications of transaction that can't be simply altered via responding to a mail. Email has the notion of Reply-to header, although this case, that was not populated. Making a decision solely noreply@ is why the life of Mr. Null is quite hard.[^1] No, gmail is not right about this. [^1] https://www.wired.com/2015/11/null/ https://www.wired.com/2015/11/null/
    - lazyasciiart 8y ago
      It doesn't matter if the transaction can't be altered if something went wrong. The customer can still have a complaint to register, like "I am not your customer, please stop sending me his receipts".
ChuckMcM 8y ago
tl;dr - Gmail collapses 'threads' of identical messages and you might not see the newest one, and on password reset links (and others) that can be a problem. At Blekko we ran into a variant of this caused all of our regular logging reports to get thrown into a single gmail thread and collapsed (making it hard to find the results) so we changed our logging script to include the date in the subject line. The fix for the author would have been to change the subject to 'change password request received on 14-mar-2019' or something similar (you can include the time too for more uniqueness). Then it always starts a new thread and the messages are always visible.
- __david__ 8y ago
  If only there was a way for email messages to indicate they were part of a chain or not… Sarcasm aside, it stinks to have to workaround email clients that deliberately don't pay attention to convention. Putting the date in the subject should be a clear indicator that Gmail is in the wrong here—there's already a date field in the headers.
kmlx 8y ago
it doesn’t matter if gmail is buggy or whatnot. you have a duty to your clients and shareholders to fix the issue. in most cases this means circumventing an uncontrollable entity. in expedia’s case it means having pseudo-random email formats when requesting a password reset. as a long time so-called “hacker”, this is junior-level stuff.
tzs 8y ago
I worked at a place that had many customers who used a major free email provider that also ran an ad network (but was not Google or Microsoft) that kept deciding that most mail we sent to our customers was spam: receipts, installation instructions, responses from support, and notices that subscriptions were going to re-bill were all blocked. We'd contact them, they would fix things, and it would work for a while...but invariably it would go back to classifying it all as spam, and we'd get customers calling to complain about not getting instructions or support responses. Finally, our guy who managed our ad spending called up our rep in ad sales at the email company, and asked in rather forceful terms just why we should continue spending $large_number/month on ads there to acquire new customers, when we were then going to be blocked from emailing those customers? The ad rep conference called in their head of IT, who conference called in their engineer in charge of spam filtering, and had him right there add us to a whitelist that would prevent anything from us being classified as spam no matter what the rest of their spam system decided. As far as I know we never had another email go into the spam filter. If we hadn't been spending a large amount on ads with them, I doubt the problem would ever have been resolved. If you sell a software product to end users, and have non-marketing messages that you need users to see such as responses to support requests or reminders that subscriptions are going to automatically re-bill, I'd recommend including some kind of messaging system specifically for this in the product itself. Still send them as email, but also send them via this in-product channel, or at least send notices that there is a message with links to a way to view it on the web. Email only is simply not reliable enough.
- pmlnr 8y ago
  > Email only is simply not reliable enough. That is because implementations, especially gmail, ignore the specifications: (EDIT) there are specific error codes and messages should be sent back to the originator. The reasons to ignore it is, in theory, not to give spammers ideas how to avoid it. Email is incredibly reliable, it can tolerate outages, configuration errors, delays, etc, by specification, but ever since "modern" spam filtering, gmail (and those who follow their practices) it became a nightmare, because everyone ignores how it _should_ work. It was done in a time when connections were slow and prone to errors; it had to be flexible. Yes, I know spam was bad I was there. But I'd rather have spam, than actually important mails getting dropped because of false positive, paranoid spamfilters.
erikig 8y ago
We've encountered this a lot and when sending emails to organizations that use Google Suite or Gmail. When we explicitly don't want the messages to be grouped our team has began making the subject of the email unique to the request. e.g instead of: Subject:"Password Reset Notification" (or) Subject:"Website Support Request" we'd use: Subject:"Your Password Reset Request - March 14th 10:19am" (and) Subject:"Website Support Request - Jimmy Davis, Failed Login"
- NelsonMinar 8y ago
  Yes, this is the exact fix the article suggests making. And then goes on to say that he never did it because it was too hard to format dates in Java or some such nonsense.
  - SilasX 8y ago
    In fairness, it's a bad UX that forces you to change your own workflow to keep it from introducing undesired behavior.
    - mywittyname 8y ago
      It's bad UX to have messages that are entirely identical from a user perspective, but perform different functions. The email with the functionality to reset your password was no different than the past ones that have no effect. The emails should have specified the expiration date in the text of the email document from day one.
      - user5994461 8y ago
        >>> The emails should have specified the expiration date in the text of the email document from day one. That doesn't avoid the issue. Similar emails are collapsed, it's not about being identical. For example, two orders listing the (different) products you bought might get collapsed, despite being fairly different.
  - notJim 8y ago
maxxxxx 8y ago
I have had too with some support emails. With Gmail it’s really easy to not see some messages in long threads. Also reading a thread is quite difficult with gmail hiding parts all the time.
theDoug 8y ago
Is this not just a repost of the flagged post from yesterday?
gerash 8y ago
Google should indeed fix every single UX issue, however minor, given its operation scale. That said, such overblown criticisms reminds me of the quote "There are only two kinds of programming languages: those people always bitch about and those nobody uses". It's true for everything.
mrhappyunhappy 8y ago
I have experience this exact issue described in the article so many times. It is so frustrating when trying to reset passwords because other than the time stamp, each email looks identical.
nedwin 8y ago
This happened to me and was incredibly frustrating. Figured I was just a moron, but perhaps not.
kerng 8y ago
I had never used Gmail until a few month ago when I switched jobs and my current company uses GSuites. The Gmail UI is horrible, the amount of confusion it creates and how illogical it does collapse and order things amazes me. Why not just show me the content as it is and let me figure out how to handle it?
- megous 8y ago
  You can disable conversations and view all individual messages in the chronological order without any grouping. But it will not solve other UX issues, like the fact that gmail UI encourages top-posting, even if the sender replied inside the body of the original e-mail. (it skillfully hides the option to reply inline inside the quoted body, even in this case) It also makes a complete mess when you write responses inside the quoted text, if you're not extremely careful and aware of this fact. For example these are identical messages: As gmail user sent it: https://megous.com/dl/tmp/gmail-garbage2.png https://megous.com/dl/tmp/gmail-garbage2.png As I received it (text e-mail message gmail actually generated): https://megous.com/dl/tmp/gmail-garbage4.png https://megous.com/dl/tmp/gmail-garbage4.png Now someone tell me this is not a complete garbage! Honestly, conversing with gmail users is my least favorite thing, as long as they use the web client. Gmail webmail is not a serious e-mail client. It does not even implement threading.
  - haroldp 8y ago
    Not OP, agree that the UI is still horrible in many ways, but still thank you!
  - kerng 8y ago
    Thanks for the pointer, will check changing the conversation view setting out. Yeah, I have noticed the issues with quotations too - it's really messy and hard to parse.
jwr 8y ago
Many of my customers use Gmail (mostly as part of a company subscription to Apps) and I have frequent problems with their mail getting lost, classified as spam, rejected, filed somewhere, or just difficult to see. I wonder at which point people will start realizing that Gmail is not all it's cracked up to be. Apart from the privacy issues (you basically have to assume that Google is reading all your mail and mining data from it), Gmail treats your mail as their mail: they will do anything they like with it, including hiding it from you. If that sounds like a rant, it is — I am worried about the increasingly centralized nature of E-mail.
- Slippery_John 8y ago
  They claim that they don't use your emails on gsuite the same way as they use it on gmail proper, but I find it very difficult to trust them on that.
  - jnwatson 8y ago
    To betray their paying customers’ trust on the pennies they would make would be a truly boneheaded move.
    - dejejeiuerir 8y ago
      Not everything is about money, you know?
    - 0xffff2 8y ago
      Your post implies that companies (or even just Google in particular) don't make a habit of making truly boneheaded moves. I think that's a pretty difficult position to back up.
    - eitland 8y ago
      Didn't Google do exactly this with school accounts a while ago? Researched briefly and this was the best I found but I have memories of something even more juicy: https://www.theguardian.com/technology/2015/dec/02/google-eff-ftc-privacy-chromebook-gmail-spying-students https://www.theguardian.com/technology/2015/dec/02/google-ef...
rgrove 8y ago
The "fix" for this is actually pretty trivial, and it doesn't require using a unique subject or sender. When you send an email that you don't want to be collapsed into any previous thread on Gmail, include an `X-Entity-Ref-ID` header with a random value. I don't remember where I learned this. I can't seem to find any official documentation mentioning it. But it works.
- quantumhobbit 8y ago
  If the company can’t figure out how to add a date to the subject line, they won’t be able to figure this out either. It’s gmails bug, but I’m amazed they couldn’t figure out hiw to implement the fix. Amazed but not too surprised given how not nimble big companies are.
  - rgrove 8y ago
    It's not a bug.
  - user5994461 8y ago
    To be fair, the header should be significantly easier to implement. There is a truckload of issues with formatting a date in 100 languages that don't apply. Incredible how Hacker News figured out the workaround in 6 hours, when they couldn't in 6 years.
- bluetech 8y ago
  We've been using it for some time as well, not even with a random value, just `X-Entity-Ref-ID: null` works too. Found it through some obscure StackOverflow answer.
- davewasthere 8y ago
  Or changing the sender's email address. e.g. myaccount+randomstring@domain.com Otherwise a unique subject is always easy for some people.
delinka 8y ago
Additionally, the search function doesn't easily surface the most recent messages matching a query. "Gooz Frabba" seems to get me a short list (the first ever mention from years ago; the one where I mentioned it to a friend; etc), followed by a random assortment of messages which don't contain the very latest email I received until I scroll down a screen or few.
apl002 8y ago
some friends thought I was insane when I said i preferred outlook to gmail purely for UX reasons. The email threads are so difficult to track.
- scarface74 8y ago
  Outlook (the client not the service) has a feature that could cause similar issues - the focused and “other” tabs. I’ve missed a few responses to AWS support requests because of it. On the other hand, I chose to use the Outlook client instead of the built in mail client for iOS, specifically because there is a grey area between “spam” and automated email that isn’t urgent but I still want to see. Most of the time Outlook gets it right.
totaldude87 8y ago
I've always felt that gmail is more cumbersome and packed esp in mobile! Inbox was nice in few areas however the core gmail app on mobile is almost impossible to get to a correct mail. Enter Outlook mobile, absolutely love it, no wonder its the most rated mail app in app stores!
Izkata 8y ago
...okay, I have a very vague memory of this being straight up announced by Google as a new feature back around 2002-2005. Basically, if I'm remembering right, there was a semi-standard header field in emails that used some sort of hash to identify which email was being replied to. This was how emails were threaded together in other clients, and worked well for the most part. But there were some situations where it didn't work accurately, leaving some emails un-threaded, so Google created its new conversation view to both flatten the reply tree (so you'd see all emails so far before responding, instead of making the same reply as someone else) and group up emails sent from clients that didn't include the hash. And now we're here.
cwkoss 8y ago
Put the time of the request in the subject line of the emails so they are unique and don't get threaded. Wow, I just saved you $100M! /s
deleted 8y ago
[deleted]
spinchange 8y ago
It's odd to me that the phrase, "conversation view" is nowhere in the OP and it's literally the name of the feature they're talking about, which can be turned off.
- kurtisc 8y ago
  That doesn't matter from the perspective of the sender. Most people won't turn it off.
rabboRubble 8y ago
I opted for one email sent/received, display one line option in Gmail eons ago. The collapsed email format is complete bollocks with one small exception... coming back to work after a vacation. Collapsed view allows for reading a series of emails to acquaint with how a particular subject evolved over time while away. So after vacation > turn collapse back on > clean up vacation built up email > turn collapse off again Edit: oh, I also proactively search for "lost" emails with the following search string: has:nouserlabels -label:inbox -label:drafts With my workflow, I apply a label to everything I want to save when I move it out of the inbox, meaning I've finished dealing with whatever the email requires. So that search string finds whatever has fallen out of my workflow.
josefresco 8y ago
"Debugging complete. It’s not a bug, it’s a feature. "The direct impact of this bug" "Actually, the bug is still active" So I thought the author realized it was a feature, not a bug. But then the author goes on to claim it's a bug, several times. It's clearly not, it may be a shitty feature but it's not a bug. It took me one instance of this to realize what was going on. Why only one time? Because I took the time to learn how my tools work.
- kurtisc 8y ago
  Someone else's mailbox isn't your tool.
rconti 8y ago
I just want my colorful threading back. Every time someone 'improves' the Gmail UI, it gets harder and harder to use. Back when each new message in a thread had a different-colored header, it was vastly easier to use by skimming. Now, with everything in various shades of monochrome, threads collapsed or expanded, quotes hidden or not hidden, signatures here and there, it's virtually impossible to tell at a glance where one message begin and the other one ends.
wheresvic1 8y ago
very interesting read - for an alternative interface to email, try mutt[1] [1] https://smalldata.tech/blog/2016/09/10/gmail-with-mutt https://smalldata.tech/blog/2016/09/10/gmail-with-mutt
northerdome 8y ago
I want to know! Did 9,000 people actually book the listing?
jondubois 8y ago
The concept of collapsed email threads is a terrible idea for the typical user. It probably only makes sense for public figures who get a ton of unsolicited emails (whoever designed Gmail must have been a public figure). It also would explain why Marissa Mayer redesigned Yahoo mail to have collapsed email threads by default. The average person doesn't get so many emails that they need to have them collapsed and sorted based on the sender. Most people don't have a problem reading every email in their inbox each day - In fact, that's what they want to do.
NoPicklez 8y ago
I switched off collapsing Gmail threads, or having them as conversations I believe Google call it. It's quite difficult as I would find myself struggling to find certain emails.
fukbezos 8y ago
I've been using Gmail for nearly two decades. I have not lost a single email...

244 points by eaguyhn 8y ago

All Gmail would have to do to fix the issue is consider the link target when diffing to determine which part is quoted from previous messages. I've noticed this myself on the occasion I end up requesting a password reset multiple times. It's annoying at best to have to click open a closed message and then click again to show the quoted text. I'm sure it took me a minute to figure out what was going on the first time I encountered this.

tedeh 8y ago

TLDR: Gmail hid the most recent password reset email with a working token link at the bottom due to grouping similar looking emails together. The fix was to introduce a date in the subject line, thus preventing grouping in the first place. Edit: Well, actually it didn't work and it is still an ongoing bug.

tyingq 8y ago
The "still not implemented" fix, apparently. "Actually, the bug is still active. I never had the opportunity to fix it, didn’t stay long."

shereadsthenews 8y ago

Alternate title: Our Assumptions About MUA Presentation Turned Out To Be Wrong.

ggm 8y ago

The issue is impacting all users who forgot their password, use Gmail (not sure about other clients) and don’t notice the hidden messages being at the bottom (it’s really hard to spot). So the "bug" is collapsed message/sort/thread view? Feels like this is about as un-bug-like as you can get. Adding the unique Subject: line feels like the right fix (in you) rather than shout at google "you have a bug"

pjc50 8y ago
The argument is that collapsed view should show the most recent message and collapse the older ones, not the other way round.
- mrlala 8y ago
  But it doesn't really know which individual messages inside you have read. So collapsing everything but the most recent one is dangerous because that is a real risk of missing messages. The current way yeah you might have to see where the conversation left off- but it's the safest to not hide something from the user.
  - yellowapple 8y ago
    Gmail does know which messages in the thread you've already read, though; it'll leave the unread messages uncollapsed, but the ones that were already read will be collapsed. The problem is that the first message is also uncollapsed, so the user has to know to scroll to the bottom.
mannykannot 8y ago
I am quite surprised by this response, and even more that there are several saying the same thing. > So the "bug" is collapsed message/sort/thread view? No - the bug is that new messages were being hidden because they were mischaracterized as redundant. I quickly disabled Google's 'conversations' feature (or whatever they call it), when it was first introduced, because it did not seem to be doing a good job, but did not think about its wider implications. I probably should not be surprised about these reactions, as I have had experience with fellow developers who insist that what is manifestly a bug is not one because it is "working as designed", or because there is an undocumented workaround to the undocumented problem.
- mattmanser 8y ago
  It does a great job, apart from this, it's a really annoying thing that I have no idea why they've done it.

Someone1234 8y ago

Article admits it isn't a bug then continues calling it a "bug." Also the $187M figure is just pulled out of the air. Essentially Gmail (and other mail services) put messages from the same sender with the same subject into a Conversation View. Users are confused by it, and clicking the oldest password reset link (expired) rather than the newest. Claims that users being unable to reset their passwords costs Expedia $187M.

Brahma111 8y ago
Oh..and the clickbaity headline got me again.. All while I was looking for how some hacker exploited it and got Expedia's business
feketegy 8y ago
LOL, I guess it's anything for clicks these days...
Grue3 8y ago
Nah, the author does have a point. There's something fundamentally wrong with how Gmail collapses messages. I have email notifications from a certain web store when some of the items I might be interested in come in stock/for preorder. I always read them. Years after I started using it, I discovered that I actually miss reading about 30% of these messages. How does this happen? Well, it turns out that Google randomly groups some of these messages in a single thread. And yet this thread looks identical to a single message. So I click on the thread, read the first message, see the email footer and go back to Inbox to read the next one. This marks the entire thread read even though I didn't actually read the subsequent messages. So I never come around to read them. I completely believe that most users will not read any email in the thread beyond the first one, because there's absolutely no indication there's more stuff to read.
- mrlala 8y ago
  >Well, it turns out that Google randomly groups some of these messages in a single thread Yeah you are right, I remember when they implemented that "random grouping" to confuse their users and make email unreadable. In other news, they group messages with the same subject into a single thread (or rather, how the subject changes when someone clicks reply like adding re:). So when someone replies, it stays in the same thread. It's clear as day that there are messages below because they are showing. There's a number next to the email thread you clicked on with how many replies there are. >I completely believe that most users will not read any email in the thread beyond the first one, because there's absolutely no indication there's more stuff to read. So.. you do realize just how many years people have been using gmail, right? And you believe that most users are missing basically every email that is a reply? I mean.. how does your mind work dude?

shay_ker 8y ago

This title is way too clickbaity for what it's talking about: Gmail grouping emails by subject can bury password reset emails

recursion 8y ago

Very interesting read. Author should have submitted the problem and solution as part of a bug bounty perhaps. Something for us all to watch out for.

Nicksil 8y ago

Is the GMail behavior described here accurate? Does GMail really hide the _latest_ email in a collection sharing similar sender and subject?

pbogle 8y ago
Actually, GMail expands the most recent email and collapses older messages. (I just double checked.) So the premise of the article seems a bit misguided.
killaken2000 8y ago
I know it hides similar parts of a series of emails because it cuts off my email salutation and sometimes people think that I'm being disrespectful. I wish I could stop it from doing that. If true, I wonder how similar these emails were to have that happen.
williamscales 8y ago
GMail will hide parts of a message that are similar to previous ones. When the entire message consists of what GMail considers to be quoted text, it doesn't really look like there's an email there. I just tried this myself and in the second password reset email, I only see a button with three little dots that says "show trimmed content" when I mouse over it. It's tiny and I wouldn't expect someone to figure it out if they didn't spend some time looking or have an idea of what happened.
- fatnoah 8y ago
  OMG, it's not just me. It's super frustrating on mobile. On certain threads, I'll literally spend 5 minutes just trying to find the new content.
  - mschaef 8y ago
    I still miss pine, tbh (If this reference is too old: https://en.wikipedia.org/wiki/Pine_(email_client) https://en.wikipedia.org/wiki/Pine_(email_client) )

paxy 8y ago

While a whimsical writing and storytelling style sometimes works great, in this case it makes the article very confusing, even contradictory in places. I also find their numbers very hard to believe. Expedia's net income last year was ~$400M. They could apparently make a 60-character fix and increase that by $187M but they don't, because...reasons?

feifan 8y ago
I'm guessing $187M is referring to total bookings, which is in the $Billions
user5994461 8y ago
Where did it have you confused? Changes can be impossibly difficult in large companies for no particular reason.

abtinf 8y ago

The defense of gmail in the other comments is bizarre. Gmail suffers from a number of similar, very serious usability issues. From the perspective of the user, failing to show the correct password reset email is a bug in gmail that affects many sites. A less advanced non-threaded show-most-recent email view would not suffer from this issue. When you add a more advanced feature, make it the default, and inadvertently reduce usability, you are at fault.

tyingq 8y ago
I won't defend Gmail, but not testing critical email with the most popular client seems unwise. And, tracking failed resets for popular domains seems like a good idea.
brianpgordon 8y ago
Gmail's user experience is terrible in many ways, but I don't think it's fair to excoriate them for adding a feature that's not so idiot proof that not even one in a hundred users will misapprehend how to use it. That school of ultra-spacious user friendly designs for the lowest common denominator is how we ended up with the nightmare that is the current Gmail interface.
asark 8y ago
A few years ago, on Nu Gmail (Ajaxy Gmail) but pre-Inbox, I tried to walk my grandma through adding a contact. Their add contact screen, with the form to fill in contact info, had two same-colored buttons, not that far apart, same copy. One added the contact whose info you'd just been putting in. The other helpfully erased it (add a new contact, was what that one meant). That's probably been fixed, but the whole settings area was kinda that way back then—like someone slapped it together without once thinking about how it'd be used. IDK what it's like now now, I use basic HTML gmail. [EDIT] who's to whose. I cannot believe I wrote that.
SmellyGeekBoy 8y ago
> The defense of gmail in the other comments is bizarre. This statement is bizarre, I don't really see anyone defending Gmail here - just commenters offering further explanation on what's happening and other people mentioning specific examples of being caught out by this behaviour.

reitzensteinm 8y ago

Setting aside the validity of the dollar amount, the title belies a dangerous view of product development. In the real world, the buck stops with you. Even if it's not fair, if there's an issue your users are having, you need to fix it. At their scale, there is no excuse for not polishing every minute aspect of the UX, and that it includes how it interacts with every email service. Expedia stole 187m from themselves by not having their shit together.

dan_m2k 8y ago

I can’t believe I got hooked in by that deceptive, clickbaity headline. Am I on the ExpressOnline and didn’t realise?!

pmlnr 8y ago

I have a Gmail nightmare story from a couple of years ago. The company I work for was sending confirmation emails doing everything properly: separate IP range mx boxes, warmed up, DMARC, SPF, DKIM, all the bells and whistles. Nothing else was allowed to go through them. Suddenly people weren't getting those confirmation letters. It turned out that soon after Gmail introduced the "Promotions" tab, a silent, new feature was added: anything from noreply@ was put in there. We tried going through the official channels - once we got to the end of the tunnel, it told us "we'll get back to you in 2 weeks". That's when we started digging into internal google connections across the company, and thank god, we found someone who know someone, and in was sorted within a couple of hours. (It might have helped that the company spends a lot of money on google ads.) Gmail is lovely.

gnu8 8y ago
You should never have been sending mail from noreply@ in the first place. Every message needs to be send from a real mailbox that can receive a reply that will be read by a person. If the message isn’t important enough for that, then it is spam and Gmail was doing the correct thing with it.
- pmlnr 8y ago
  I don't believe your approach is correct, for multiple reasons. There are valid scenarios for one way communication; those emails are only verifications of transaction that can't be simply altered via responding to a mail. Email has the notion of Reply-to header, although this case, that was not populated. Making a decision solely noreply@ is why the life of Mr. Null is quite hard.[^1] No, gmail is not right about this. [^1] https://www.wired.com/2015/11/null/ https://www.wired.com/2015/11/null/
  - lazyasciiart 8y ago
    It doesn't matter if the transaction can't be altered if something went wrong. The customer can still have a complaint to register, like "I am not your customer, please stop sending me his receipts".

ChuckMcM 8y ago

tl;dr - Gmail collapses 'threads' of identical messages and you might not see the newest one, and on password reset links (and others) that can be a problem. At Blekko we ran into a variant of this caused all of our regular logging reports to get thrown into a single gmail thread and collapsed (making it hard to find the results) so we changed our logging script to include the date in the subject line. The fix for the author would have been to change the subject to 'change password request received on 14-mar-2019' or something similar (you can include the time too for more uniqueness). Then it always starts a new thread and the messages are always visible.

__david__ 8y ago
If only there was a way for email messages to indicate they were part of a chain or not… Sarcasm aside, it stinks to have to workaround email clients that deliberately don't pay attention to convention. Putting the date in the subject should be a clear indicator that Gmail is in the wrong here—there's already a date field in the headers.

kmlx 8y ago

it doesn’t matter if gmail is buggy or whatnot. you have a duty to your clients and shareholders to fix the issue. in most cases this means circumventing an uncontrollable entity. in expedia’s case it means having pseudo-random email formats when requesting a password reset. as a long time so-called “hacker”, this is junior-level stuff.

tzs 8y ago

I worked at a place that had many customers who used a major free email provider that also ran an ad network (but was not Google or Microsoft) that kept deciding that most mail we sent to our customers was spam: receipts, installation instructions, responses from support, and notices that subscriptions were going to re-bill were all blocked. We'd contact them, they would fix things, and it would work for a while...but invariably it would go back to classifying it all as spam, and we'd get customers calling to complain about not getting instructions or support responses. Finally, our guy who managed our ad spending called up our rep in ad sales at the email company, and asked in rather forceful terms just why we should continue spending $large_number/month on ads there to acquire new customers, when we were then going to be blocked from emailing those customers? The ad rep conference called in their head of IT, who conference called in their engineer in charge of spam filtering, and had him right there add us to a whitelist that would prevent anything from us being classified as spam no matter what the rest of their spam system decided. As far as I know we never had another email go into the spam filter. If we hadn't been spending a large amount on ads with them, I doubt the problem would ever have been resolved. If you sell a software product to end users, and have non-marketing messages that you need users to see such as responses to support requests or reminders that subscriptions are going to automatically re-bill, I'd recommend including some kind of messaging system specifically for this in the product itself. Still send them as email, but also send them via this in-product channel, or at least send notices that there is a message with links to a way to view it on the web. Email only is simply not reliable enough.

pmlnr 8y ago
> Email only is simply not reliable enough. That is because implementations, especially gmail, ignore the specifications: (EDIT) there are specific error codes and messages should be sent back to the originator. The reasons to ignore it is, in theory, not to give spammers ideas how to avoid it. Email is incredibly reliable, it can tolerate outages, configuration errors, delays, etc, by specification, but ever since "modern" spam filtering, gmail (and those who follow their practices) it became a nightmare, because everyone ignores how it _should_ work. It was done in a time when connections were slow and prone to errors; it had to be flexible. Yes, I know spam was bad I was there. But I'd rather have spam, than actually important mails getting dropped because of false positive, paranoid spamfilters.

erikig 8y ago

We've encountered this a lot and when sending emails to organizations that use Google Suite or Gmail. When we explicitly don't want the messages to be grouped our team has began making the subject of the email unique to the request. e.g instead of: Subject:"Password Reset Notification" (or) Subject:"Website Support Request" we'd use: Subject:"Your Password Reset Request - March 14th 10:19am" (and) Subject:"Website Support Request - Jimmy Davis, Failed Login"

NelsonMinar 8y ago
Yes, this is the exact fix the article suggests making. And then goes on to say that he never did it because it was too hard to format dates in Java or some such nonsense.
- SilasX 8y ago
  In fairness, it's a bad UX that forces you to change your own workflow to keep it from introducing undesired behavior.
  - mywittyname 8y ago
    It's bad UX to have messages that are entirely identical from a user perspective, but perform different functions. The email with the functionality to reset your password was no different than the past ones that have no effect. The emails should have specified the expiration date in the text of the email document from day one.
    - user5994461 8y ago
      >>> The emails should have specified the expiration date in the text of the email document from day one. That doesn't avoid the issue. Similar emails are collapsed, it's not about being identical. For example, two orders listing the (different) products you bought might get collapsed, despite being fairly different.
- notJim 8y ago

maxxxxx 8y ago

I have had too with some support emails. With Gmail it’s really easy to not see some messages in long threads. Also reading a thread is quite difficult with gmail hiding parts all the time.

theDoug 8y ago

Is this not just a repost of the flagged post from yesterday?

gerash 8y ago

Google should indeed fix every single UX issue, however minor, given its operation scale. That said, such overblown criticisms reminds me of the quote "There are only two kinds of programming languages: those people always bitch about and those nobody uses". It's true for everything.

mrhappyunhappy 8y ago

I have experience this exact issue described in the article so many times. It is so frustrating when trying to reset passwords because other than the time stamp, each email looks identical.

nedwin 8y ago

This happened to me and was incredibly frustrating. Figured I was just a moron, but perhaps not.

kerng 8y ago

I had never used Gmail until a few month ago when I switched jobs and my current company uses GSuites. The Gmail UI is horrible, the amount of confusion it creates and how illogical it does collapse and order things amazes me. Why not just show me the content as it is and let me figure out how to handle it?

megous 8y ago
You can disable conversations and view all individual messages in the chronological order without any grouping. But it will not solve other UX issues, like the fact that gmail UI encourages top-posting, even if the sender replied inside the body of the original e-mail. (it skillfully hides the option to reply inline inside the quoted body, even in this case) It also makes a complete mess when you write responses inside the quoted text, if you're not extremely careful and aware of this fact. For example these are identical messages: As gmail user sent it: https://megous.com/dl/tmp/gmail-garbage2.png https://megous.com/dl/tmp/gmail-garbage2.png As I received it (text e-mail message gmail actually generated): https://megous.com/dl/tmp/gmail-garbage4.png https://megous.com/dl/tmp/gmail-garbage4.png Now someone tell me this is not a complete garbage! Honestly, conversing with gmail users is my least favorite thing, as long as they use the web client. Gmail webmail is not a serious e-mail client. It does not even implement threading.
- haroldp 8y ago
  Not OP, agree that the UI is still horrible in many ways, but still thank you!
- kerng 8y ago
  Thanks for the pointer, will check changing the conversation view setting out. Yeah, I have noticed the issues with quotations too - it's really messy and hard to parse.

jwr 8y ago

Many of my customers use Gmail (mostly as part of a company subscription to Apps) and I have frequent problems with their mail getting lost, classified as spam, rejected, filed somewhere, or just difficult to see. I wonder at which point people will start realizing that Gmail is not all it's cracked up to be. Apart from the privacy issues (you basically have to assume that Google is reading all your mail and mining data from it), Gmail treats your mail as their mail: they will do anything they like with it, including hiding it from you. If that sounds like a rant, it is — I am worried about the increasingly centralized nature of E-mail.

Slippery_John 8y ago
They claim that they don't use your emails on gsuite the same way as they use it on gmail proper, but I find it very difficult to trust them on that.
- jnwatson 8y ago
  To betray their paying customers’ trust on the pennies they would make would be a truly boneheaded move.
  - dejejeiuerir 8y ago
    Not everything is about money, you know?
  - 0xffff2 8y ago
    Your post implies that companies (or even just Google in particular) don't make a habit of making truly boneheaded moves. I think that's a pretty difficult position to back up.
  - eitland 8y ago
    Didn't Google do exactly this with school accounts a while ago? Researched briefly and this was the best I found but I have memories of something even more juicy: https://www.theguardian.com/technology/2015/dec/02/google-eff-ftc-privacy-chromebook-gmail-spying-students https://www.theguardian.com/technology/2015/dec/02/google-ef...

rgrove 8y ago

The "fix" for this is actually pretty trivial, and it doesn't require using a unique subject or sender. When you send an email that you don't want to be collapsed into any previous thread on Gmail, include an `X-Entity-Ref-ID` header with a random value. I don't remember where I learned this. I can't seem to find any official documentation mentioning it. But it works.

quantumhobbit 8y ago
If the company can’t figure out how to add a date to the subject line, they won’t be able to figure this out either. It’s gmails bug, but I’m amazed they couldn’t figure out hiw to implement the fix. Amazed but not too surprised given how not nimble big companies are.
- rgrove 8y ago
  It's not a bug.
- user5994461 8y ago
  To be fair, the header should be significantly easier to implement. There is a truckload of issues with formatting a date in 100 languages that don't apply. Incredible how Hacker News figured out the workaround in 6 hours, when they couldn't in 6 years.
bluetech 8y ago
We've been using it for some time as well, not even with a random value, just `X-Entity-Ref-ID: null` works too. Found it through some obscure StackOverflow answer.
davewasthere 8y ago
Or changing the sender's email address. e.g. myaccount+randomstring@domain.com Otherwise a unique subject is always easy for some people.

delinka 8y ago

Additionally, the search function doesn't easily surface the most recent messages matching a query. "Gooz Frabba" seems to get me a short list (the first ever mention from years ago; the one where I mentioned it to a friend; etc), followed by a random assortment of messages which don't contain the very latest email I received until I scroll down a screen or few.

apl002 8y ago

some friends thought I was insane when I said i preferred outlook to gmail purely for UX reasons. The email threads are so difficult to track.

scarface74 8y ago
Outlook (the client not the service) has a feature that could cause similar issues - the focused and “other” tabs. I’ve missed a few responses to AWS support requests because of it. On the other hand, I chose to use the Outlook client instead of the built in mail client for iOS, specifically because there is a grey area between “spam” and automated email that isn’t urgent but I still want to see. Most of the time Outlook gets it right.

totaldude87 8y ago

I've always felt that gmail is more cumbersome and packed esp in mobile! Inbox was nice in few areas however the core gmail app on mobile is almost impossible to get to a correct mail. Enter Outlook mobile, absolutely love it, no wonder its the most rated mail app in app stores!

Izkata 8y ago

...okay, I have a very vague memory of this being straight up announced by Google as a new feature back around 2002-2005. Basically, if I'm remembering right, there was a semi-standard header field in emails that used some sort of hash to identify which email was being replied to. This was how emails were threaded together in other clients, and worked well for the most part. But there were some situations where it didn't work accurately, leaving some emails un-threaded, so Google created its new conversation view to both flatten the reply tree (so you'd see all emails so far before responding, instead of making the same reply as someone else) and group up emails sent from clients that didn't include the hash. And now we're here.

cwkoss 8y ago

Put the time of the request in the subject line of the emails so they are unique and don't get threaded. Wow, I just saved you $100M! /s

deleted 8y ago

[deleted]

spinchange 8y ago

It's odd to me that the phrase, "conversation view" is nowhere in the OP and it's literally the name of the feature they're talking about, which can be turned off.

kurtisc 8y ago
That doesn't matter from the perspective of the sender. Most people won't turn it off.

rabboRubble 8y ago

I opted for one email sent/received, display one line option in Gmail eons ago. The collapsed email format is complete bollocks with one small exception... coming back to work after a vacation. Collapsed view allows for reading a series of emails to acquaint with how a particular subject evolved over time while away. So after vacation > turn collapse back on > clean up vacation built up email > turn collapse off again Edit: oh, I also proactively search for "lost" emails with the following search string: has:nouserlabels -label:inbox -label:drafts With my workflow, I apply a label to everything I want to save when I move it out of the inbox, meaning I've finished dealing with whatever the email requires. So that search string finds whatever has fallen out of my workflow.

josefresco 8y ago

"Debugging complete. It’s not a bug, it’s a feature. "The direct impact of this bug" "Actually, the bug is still active" So I thought the author realized it was a feature, not a bug. But then the author goes on to claim it's a bug, several times. It's clearly not, it may be a shitty feature but it's not a bug. It took me one instance of this to realize what was going on. Why only one time? Because I took the time to learn how my tools work.

kurtisc 8y ago
Someone else's mailbox isn't your tool.

rconti 8y ago

I just want my colorful threading back. Every time someone 'improves' the Gmail UI, it gets harder and harder to use. Back when each new message in a thread had a different-colored header, it was vastly easier to use by skimming. Now, with everything in various shades of monochrome, threads collapsed or expanded, quotes hidden or not hidden, signatures here and there, it's virtually impossible to tell at a glance where one message begin and the other one ends.

wheresvic1 8y ago

very interesting read - for an alternative interface to email, try mutt[1] [1] https://smalldata.tech/blog/2016/09/10/gmail-with-mutt https://smalldata.tech/blog/2016/09/10/gmail-with-mutt

northerdome 8y ago

I want to know! Did 9,000 people actually book the listing?

jondubois 8y ago

The concept of collapsed email threads is a terrible idea for the typical user. It probably only makes sense for public figures who get a ton of unsolicited emails (whoever designed Gmail must have been a public figure). It also would explain why Marissa Mayer redesigned Yahoo mail to have collapsed email threads by default. The average person doesn't get so many emails that they need to have them collapsed and sorted based on the sender. Most people don't have a problem reading every email in their inbox each day - In fact, that's what they want to do.

NoPicklez 8y ago

I switched off collapsing Gmail threads, or having them as conversations I believe Google call it. It's quite difficult as I would find myself struggling to find certain emails.

fukbezos 8y ago

I've been using Gmail for nearly two decades. I have not lost a single email...