11 ms·

I had my 2FA at Singlehop bypassed by social engineering attack. They helpfully changed the entire account contact info without any notice to me, presumably fro

by r1ch 11y ago

I had my 2FA at Singlehop bypassed by social engineering attack. They helpfully changed the entire account contact info without any notice to me, presumably from a phone call. The attacker didn't even have any information to go off other than the IP address. I only found out when I saw the server rebooting into rescue mode and luckily I still had an active management portal cookie (changing the password doesn't log you out of the portal, another big problem) so I immediately knew what was happening. I wish there was a way to disable the "customer support backdoor" in all these kinds of services. I've started to deploy full disk encryption to all my servers now so if the attacker does manage to get into the management account the server itself is still protected from single user / rescue mode / etc.

nxzero 11y ago
Even better, have the option to disable tech support and get an alert if a reset is requested with the metadata related to the party making the request.
- brianwawok 11y ago
  Fairly common for enterprise type apps to have a list of preapproved contact points, not on the list they won't even talk to you. Maybe other places could take this up... not foolproof, but at least adds another layer to the challenge.
  - nxzero 11y ago
    I've personally gotten past lists like that a number of times simply by stating that person is not on staff any more, I'm their replacement; legitimately did replace the old point of contact. Getting people to do stuff on the phone is easy a huge amount of the time.
    - Piskvorrr 11y ago
      Indeed. "Oh, we have this completely impenetrable fortress, immune to any sort of attack." "Let me in, pretty pretty please." "Okay."
arviewer 11y ago
We have had this at Amazon AWS. We had 2FA, one phone call was enough to disable 2FA. The only thing they asked were the last four digits of our credit card.
- ruffrey 11y ago
  That is a little unnerving if true.
  - infinite8s 11y ago
    A reporter at Wired had his digital life destroyed (including personal mac wiped) by social engineering, using little more than info from one account (last 4 digits of credit card from Appstore account) to socially engineer Amazon customer support - http://www.wired.com/2012/08/apple-amazon-mat-honan-hacking/ http://www.wired.com/2012/08/apple-amazon-mat-honan-hacking/ Edit. Had some details wrong. It was a reporter at Gizmodo, and basically a collection of data was gathered from various online services (mailing address, last 4 digits of CC) to ultimately social engineer his Apple account and remote wipe his computer. A number of factors were involved, but ultimately he was done in by not having 2FA on google, and not backing up his machine.
  - pilom 11y ago
    Just confirmed with our CTO. He was able to turn MFA off on our root AWS account over the phone.
    - mr_luc 11y ago
      Huh. I'm going to say thanks to HN, and this comment chain, for bringing this to light. That seems kind of insane.
      - deleted 11y ago
        [deleted]
      - brazzledazzle 11y ago
        Maybe it's time for Amazon to decouple AWS accounts from Amazon shopping accounts.
brianwawok 11y ago
It's tricky because a lot of customers really DO lock themselves out of a service, and forget their password reset code. Fun story time. I use to play MTGO, the online Magic the Gathering game. Played it from beta for a few years say 2002-2004. Wanted to check it out in 2014 to see how it changed. Failed password reset online, had to call in to support. The support guy was like chortle what was your security passcode? I had no idea. He tried giving some hints. I said it has literally been 10 years, I am never going to remember. So he went ahead and rest my password. He told me for the record my security code was "I am the nacho king", but I would be prompted to change it at next login. So could I have social engineered a 10 year old MTGO account? Yes. But without it, I would have been locked out. I was NOT going to remember some stupid passcode kid me set on a game account. And apparently I am the nacho King.
- pilom 11y ago
  The answer to "I don't know my password and I don't know my security question/answer" is, "Sorry, for security reasons we can't help you access this account, you'll need to create a new account." This isn't a problem for banks, why is it a problem for tech companies?
  - bearcobra 11y ago
    Probably because your bank (generally) has a physical location you can visit and complete some other verification process to unlock your account.
  - modoc 11y ago
    With a bank you go into a branch, and show them your driver's license/other official ID, and don't lose access to all your money... It's harder online when you don't have the same ability to interact face to face.
    - williamscales 11y ago
      When you are locked out of AWS you must sign an affidavit and provide photo ID. Seems similar to me.
ryanlm 11y ago
I am not surprised by their negligence. I interviewed there once and they did not seem professional.
1123581321 11y ago
What was SingleHop's response to this when you made them aware? We have servers with them.
- r1ch 11y ago
  I tried to call a few times and unfortunately kept getting disconnected from my VoIP line. I ended up using their live chat and the person I spoke to was very quick in recognizing it as a social engineering attack and worked with me to reset all my information. Following up I had to reset my passwords a few more times to ensure the attacker's session was properly terminated. They identified the IPs accessing my account but they were all proxy IPs from Europe and Israel so not much could be done. I still have the support tickets the attacker made, and I feel their support staff should have recognized it as a compromised account (they changed the name to "John Smith", bad spelling / grammar, asking to reset the root password, accidentally "lost" their private key and need password logins and root SSH enabling, etc). Aside from this incident though I've been pretty happy there, the hardware is competitive, uptime is good and the network is solid which is all I really ask for with a server provider.
russnewcomer 11y ago
I agree on the 'customer support backdoor,' with some caveats. I worked for a small IT Services provider, and we a policy of "don't do dangerous stuff that an unknown party asks for unless a known party verifies it." But sometimes someone gets fired, and there are no known parties, especially with cloud services. I realize there has to be a way to work around 2FA, for situations like loss of device, terminations, etc, but that should have some known policy way of verifying (say, you must send a notarized or local equivalent letter, or appear in person somehow) identity, especially for a cloud service. (Totally incidental and you perhaps won't see this, but, r1ch, Conflict Crusher was instrumental in 15 year old me learning how to mod TA, which led me to learn how write BOS scripts, which led me to realize that I liked this programming thing, which led me to my current career/way to support my family. Thanks! :) )
- r1ch 11y ago
  Conflict Crusher, wow that takes me back! Back to that awful Visual Basic UI with random windows metafiles for backgrounds, what was I thinking.. :). Great to hear that I helped you on your path to become a programmer! Modding games inspired me a lot too, especially Quake and TA.
godzillabrennus 11y ago
That's crazy that customer service was able to turn off 2 factor! Godaddy has gotten really good at preventing social engineering stacks like this. I use 2 factor authentication for my account and customer service can't event talk to me till I give them the code. Don't have that? Need to send them my drivers license and other proof to get the account reset.
- brazzledazzle 11y ago
  The problem I think, is that Godaddy only does hosting so has appropriate controls in place. Since Amazon does shopping and hosting and did the former first they have policies designed for the customers that are shopping.
dvcrn 11y ago
Another vote for gandi.net here. I transferred all my domains to them during the past years and have never been happier. Security is almost bomb proof with IP restrictions, GPG keys, 2FA and one little checkbox in their settings I like a lot: "This setting allows you to authorize or disable password resets from the login screen." I think a good practice is to try to social engineer your own account and see if you end up getting it.

by r1ch 11y ago

nxzero 11y ago

Even better, have the option to disable tech support and get an alert if a reset is requested with the metadata related to the party making the request.

brianwawok 11y ago
Fairly common for enterprise type apps to have a list of preapproved contact points, not on the list they won't even talk to you. Maybe other places could take this up... not foolproof, but at least adds another layer to the challenge.
- nxzero 11y ago
  I've personally gotten past lists like that a number of times simply by stating that person is not on staff any more, I'm their replacement; legitimately did replace the old point of contact. Getting people to do stuff on the phone is easy a huge amount of the time.
  - Piskvorrr 11y ago
    Indeed. "Oh, we have this completely impenetrable fortress, immune to any sort of attack." "Let me in, pretty pretty please." "Okay."

arviewer 11y ago

We have had this at Amazon AWS. We had 2FA, one phone call was enough to disable 2FA. The only thing they asked were the last four digits of our credit card.

ruffrey 11y ago
That is a little unnerving if true.
- infinite8s 11y ago
  A reporter at Wired had his digital life destroyed (including personal mac wiped) by social engineering, using little more than info from one account (last 4 digits of credit card from Appstore account) to socially engineer Amazon customer support - http://www.wired.com/2012/08/apple-amazon-mat-honan-hacking/ http://www.wired.com/2012/08/apple-amazon-mat-honan-hacking/ Edit. Had some details wrong. It was a reporter at Gizmodo, and basically a collection of data was gathered from various online services (mailing address, last 4 digits of CC) to ultimately social engineer his Apple account and remote wipe his computer. A number of factors were involved, but ultimately he was done in by not having 2FA on google, and not backing up his machine.
- pilom 11y ago
  Just confirmed with our CTO. He was able to turn MFA off on our root AWS account over the phone.
  - mr_luc 11y ago
    Huh. I'm going to say thanks to HN, and this comment chain, for bringing this to light. That seems kind of insane.
    - deleted 11y ago
      [deleted]
    - brazzledazzle 11y ago
      Maybe it's time for Amazon to decouple AWS accounts from Amazon shopping accounts.

brianwawok 11y ago

It's tricky because a lot of customers really DO lock themselves out of a service, and forget their password reset code. Fun story time. I use to play MTGO, the online Magic the Gathering game. Played it from beta for a few years say 2002-2004. Wanted to check it out in 2014 to see how it changed. Failed password reset online, had to call in to support. The support guy was like chortle what was your security passcode? I had no idea. He tried giving some hints. I said it has literally been 10 years, I am never going to remember. So he went ahead and rest my password. He told me for the record my security code was "I am the nacho king", but I would be prompted to change it at next login. So could I have social engineered a 10 year old MTGO account? Yes. But without it, I would have been locked out. I was NOT going to remember some stupid passcode kid me set on a game account. And apparently I am the nacho King.

pilom 11y ago
The answer to "I don't know my password and I don't know my security question/answer" is, "Sorry, for security reasons we can't help you access this account, you'll need to create a new account." This isn't a problem for banks, why is it a problem for tech companies?
- bearcobra 11y ago
  Probably because your bank (generally) has a physical location you can visit and complete some other verification process to unlock your account.
- modoc 11y ago
  With a bank you go into a branch, and show them your driver's license/other official ID, and don't lose access to all your money... It's harder online when you don't have the same ability to interact face to face.
  - williamscales 11y ago
    When you are locked out of AWS you must sign an affidavit and provide photo ID. Seems similar to me.

ryanlm 11y ago

I am not surprised by their negligence. I interviewed there once and they did not seem professional.

1123581321 11y ago

What was SingleHop's response to this when you made them aware? We have servers with them.

r1ch 11y ago
I tried to call a few times and unfortunately kept getting disconnected from my VoIP line. I ended up using their live chat and the person I spoke to was very quick in recognizing it as a social engineering attack and worked with me to reset all my information. Following up I had to reset my passwords a few more times to ensure the attacker's session was properly terminated. They identified the IPs accessing my account but they were all proxy IPs from Europe and Israel so not much could be done. I still have the support tickets the attacker made, and I feel their support staff should have recognized it as a compromised account (they changed the name to "John Smith", bad spelling / grammar, asking to reset the root password, accidentally "lost" their private key and need password logins and root SSH enabling, etc). Aside from this incident though I've been pretty happy there, the hardware is competitive, uptime is good and the network is solid which is all I really ask for with a server provider.

russnewcomer 11y ago

I agree on the 'customer support backdoor,' with some caveats. I worked for a small IT Services provider, and we a policy of "don't do dangerous stuff that an unknown party asks for unless a known party verifies it." But sometimes someone gets fired, and there are no known parties, especially with cloud services. I realize there has to be a way to work around 2FA, for situations like loss of device, terminations, etc, but that should have some known policy way of verifying (say, you must send a notarized or local equivalent letter, or appear in person somehow) identity, especially for a cloud service. (Totally incidental and you perhaps won't see this, but, r1ch, Conflict Crusher was instrumental in 15 year old me learning how to mod TA, which led me to learn how write BOS scripts, which led me to realize that I liked this programming thing, which led me to my current career/way to support my family. Thanks! :) )

r1ch 11y ago
Conflict Crusher, wow that takes me back! Back to that awful Visual Basic UI with random windows metafiles for backgrounds, what was I thinking.. :). Great to hear that I helped you on your path to become a programmer! Modding games inspired me a lot too, especially Quake and TA.

godzillabrennus 11y ago

That's crazy that customer service was able to turn off 2 factor! Godaddy has gotten really good at preventing social engineering stacks like this. I use 2 factor authentication for my account and customer service can't event talk to me till I give them the code. Don't have that? Need to send them my drivers license and other proof to get the account reset.

brazzledazzle 11y ago
The problem I think, is that Godaddy only does hosting so has appropriate controls in place. Since Amazon does shopping and hosting and did the former first they have policies designed for the customers that are shopping.

dvcrn 11y ago

Another vote for gandi.net here. I transferred all my domains to them during the past years and have never been happier. Security is almost bomb proof with IP restrictions, GPG keys, 2FA and one little checkbox in their settings I like a lot: "This setting allows you to authorize or disable password resets from the login screen." I think a good practice is to try to social engineer your own account and see if you end up getting it.