11 ms·

Another Victim of the Magecart Assault Emerges: Newegg

79 points by GraemeL 8y ago

chrissnell 8y ago
How did the attackers get the JS onto the cart page? That's the interesting part to me that the article leaves out. They managed to break into a PCI-compliant website that presumably has significant defenses and auditing in place.
- vkjv 8y ago
  Unfortunately, PCI does not put very many restrictions on the parent website. If credit card elements are in an iFrame, the parent site is excluded from most requirements because the iFrame is "secure." Of course, if you own the parent site you can replace the iFrame with anything you want.
  - tyingq 8y ago
    PCI also doesn't necessarily mandate some common sense things either, like monitoring for unexpected changes on the cart page.
  - marak830 8y ago
    I'm still confused as to how they could insert code here. Are we talking about a server intrustion where they modified the actual cart code, or something between Newegg and the payment servers? (Sorry this isn't my domain, I'm just curious)
    - lacker 8y ago
      It looks to me like a server intrusion where they modified static files kept on a webserver (like apache or nginx). But it also seems like we don't have enough evidence to know for sure. (Edit: or they might have been static files kept on a CMS.)
  - vkjv 8y ago
    FWIW, I just checked and they don't use an iFrame approach. This means their entire checkout page must be in scope for SAQ-D.
- lacker 8y ago
  The article on the British Airways hack goes into a bit more detail, but still not enough to slake my curiosity. https://www.riskiq.com/blog/labs/magecart-british-airways-breach/ https://www.riskiq.com/blog/labs/magecart-british-airways-br... It seems like we just don't have enough information to know how exactly the attackers got the JS onto the page. What it looks like, though, is that some machine serving static files for the website got compromised, and the attackers replaced an innocent file with one that had some custom javascript tacked onto the end, that skimmed the credit card numbers. For what it's worth, PCI compliance doesn't really help you keep your webservers secure, or to keep people from breaking into your servers at all, it's more about the way you permanently store credit card numbers.
  - pjc50 8y ago
    It was appended to "Modernizr.js". That suggests a number of further possibilities by which that may have been replaced on a developer's computer, or at the point of download by a package manager (are packages authenticated?) It could well have been included in a legitimate re-deploy if that was the case.
    - lacker 8y ago
      Since that file was served from "/cms/global/scripts/lib", perhaps their CMS got compromised. Anecdotally it seems like CMS's are more frequently prone to compromises than systems like nginx or apache.
- zeveb 8y ago
  Ditto. As an aside, while it's certainly true that if one can compromise a web server (whether serving static HTML or generating dynamic HTML) then it's game over, the modern style of JavaScript-centric development greatly expands the attack surface. An attacker can compromise any of a number of different servers (e.g. analytics providers), via numerous means; moreover, he can serve dynamic malicious code, and quickly respond to changes the targeted site makes (his implant needs to survive for awhile, but his access can be revoked and folks can still be compromised). Websites which served static pages and received credit-card information via simple forms would be far more secure than dynamic web apps stitched together from JavaScript served from multiple servers in countries across the globe, with data inserted into the DOM retrieved from multiple servers across the globe. I contend that a more traditional, static architecture would tend to lead to better architectural isolation. As an example, if one needed to install server-side analytics software rather than simply injecting it into a page at runtime, then one would download and install it once: any compromise of the analytics provider after one had download it would be irrelevant. But with dynamic JavaScript, one's users download the analytics software with every single page load: a compromise instantly affects users. Moreover, with traditional server-side software one tends to have reasonably-well defined protocols for interaction, rather than just saying, 'hey, I'll run all your code in the same process as everything else.' But with client-side JavaScript, any piece of code can see everything. As an engineering community, we've adopted a style which has great costs as well as great benefits. My opinion is that we moved too quickly, and now we're feeling the pain. While a universal portable app framework is pretty awesome, JavaScript layered over a document system is probably not the best way to achieve that. As I wrote years ago, JavaScript delenda est.
- adreamingsoul 8y ago
  PCI compliance isn't that hard to get. And, typically the code is changed after the compliance review. Also, developers and engineers at Newegg are probably changing lines of code every day. Which means, at any point in time someone could have accidentally introduced a vector that allowed the attackers to inject their code. Auditing and security reviews only work if every line of code is tested, reviewed, and verified. However, that be extremely costly and slow, which isn't "good for business". Maybe, we as consumers, engineers, managers, and executives should demand that code safety and security is prioritized over profit. I would rather have a secure website then a new redesign every two years.
  - yspeak 8y ago
    This isn't fort Knox It's credit card info. There's always a cost benefit. You can spend all your time and money on doingsecurity reviews but without profit there ain't no company.
  - saas_sam 8y ago
    Do you have any evidence to suggest "that code safety and security" was NOT "prioritized over profit"? How would you falsify this? Are we to assume that every security breach indicates that profit was prioritized over security? Doubtful. Also, "profit" is the reward for doing things efficiently, it is not some kind of scoreboard for how much a company is ripping off consumers. NewEgg is in such a hyper-competitive & mature space with margins in the low single-digits on average. Incidents like this can absolutely tank them, especially this close to the holiday season. E-commerce companies do not make money on security breaches, they lose it. Lots of it. I think you should be a little more open-minded to how businesses operate, that's all.
gregpilling 8y ago
I am surprised that there is no automated alert to tell the webmaster that his code has changed on his website. Especially on the payments page! With 50,000,000 users a month, surely they have a whole team working on checkout, all the time?
- ryanlol 8y ago
  Do you have such automated alerts set up yourself? Do you know anyone with such alerts set up?
  - freddie_mercury 8y ago
    Whenever someone complains about another company's product, code, features, security....I always wish it was mandatory to include a link to the kind of software the poster is putting into production. I can dream. Glass houses and all that.
  - elorant 8y ago
    I have. I deploy in ASP.NET and get a hash of the uploaded DLL. I check it twice a day. Never had any incidents to this day but as the saying goes, it's better to be safe than sorry.
    - tedunangst 8y ago
      And how does the hash of the DLL on some server relate to the content seen by users?
      - ixwt 8y ago
        I would assume the DLL is responsible for the output of the content seen by users.
      - elorant 8y ago
        In ASP.NET pretty much all output, including html and js is included in the DLL. Only external js files are left out. Sure they could hack them which means my solution isn't bulletproof but I could also produce a hash for them too.
anontechworker 8y ago
For a website with so many visitors and transactions, I’m surprised this API call never threw enough errors for them to see in logging. I will admit that JS logging can be messy because of all the different environments but after some time I would have hoped this would have been caught.
raverbashing 8y ago
So how come is Comodo selling certificates to domain squatters? This seems to be one sore point here.
- ams6110 8y ago
  They (the attackers) owned the domain. What more is Comodo supposed to do? LetsEncrypt would have done the same thing.
zxin 8y ago
This all could have been prevented if they had a Content Security Policy.
- eat_veggies 8y ago
  Presumably, if the attacker has write access to the source files, they also have the power to change the CSP headers.
Usu 8y ago
This breach has reminded me of this pretty great article: https://hackernoon.com/im-harvesting-credit-card-numbers-and-passwords-from-your-site-here-s-how-9a8cb347c5b5 https://hackernoon.com/im-harvesting-credit-card-numbers-and...
alyandon 8y ago
Lovely. I made a purchase recently with NewEgg but at least it was with a previously stored credit card so hopefully I'm not impacted by this. However, I am disappointed that NewEgg hasn't made any sort of official announcement yet.
- adreamingsoul 8y ago
  Agreed, they should have given us a notice. I don't know if I will ever buy something from NewEgg again if this is how they are going to treat security events.
- robbyt 8y ago
  They're probably still talking to their lawyers.
- eikenberry 8y ago
  I also purchased something right in the middle of that time frame, but with a regular credit card. I'm not concerned as I pretty much assume my credit-card is always compromised and I check my charges each month for discrepancies and contest anything bad. Though the credit-card catches the fraudulent transactions most of the time these days, so it usually doesn't come to that.
- alyandon 8y ago
  Replying to myself. I use uMatrix and it just occurred to me that it would have blocked any communication attempts to neweggstats.com from newegg.com.... so yay?
  - Bartweiss 8y ago
    I noticed this too; uMaxtrix should catch this entire class of attack. Privacy Badger probably wouldn't - it watches for this sort of export, but its three-strikes rule targets trackers and misses unique attacks. Killing JS outright would do the trick also, but it looks like it would also have broken Newegg's checkout page. Looks like yet another point for "treat web browsing as adversarial". The price of allowing pages to load freely isn't just high weight, trackers, and ads - all too frequently it's actual security failures. I understand that ad/tracker blocking is a problem for keeping sites profitable, but I can't really imagine bending on that issue as long as monetization techniques and threat vectors overlap so heavily.
adreamingsoul 8y ago
Wow, I also made a purchase within that time window. Except, I used PayPal during Checkout.
BeetleB 8y ago
Damn. I made a purchase in that time period. I rarely buy anything from them, but it had to happen in that interval! I paid with Paypal. I assume I'm not affected?
- adventured 8y ago
  Given the way the code works, I don't see how you could be affected if your transaction went through PayPal (ie you didn't enter any payment information, such as a credit card).
  - BeetleB 8y ago
    I did eventually pay with Paypal. However, I do know that at some point in the last few months, I attempted to make a transaction on some web site and had issues. It was either: 1. I tried to pay with Paypal and failed. Paid with CC instead. 2. I tried to pay with my CC and failed. Paid with Paypal instead. I don't remember for which site this happened, but the paranoid part of me is wondering if it may have been Newegg and item 2 above.
- frickinLasers 8y ago
  Did you happen to receive an unexpected address change notification? I did when I checked out with Paypal. The address didn't change (maybe it's all caps now and wasn't before?) and now I'm wondering too. It took them a month to find this--could there be other code they haven't discovered?
crunchlibrarian 8y ago
I had a conversation two days ago with the CTO of a very large company you've definitely heard of who said "we don't need to worry about our website security, we have a firewall and SSL" I think these types of attacks are vastly underreported, if anything.
mwigdahl 8y ago
They are just now (around 11:10 CDT 9/19) sending out notification emails to customers. At the moment they don't even seem to know what accounts were affected.

79 points by GraemeL 8y ago

How did the attackers get the JS onto the cart page? That's the interesting part to me that the article leaves out. They managed to break into a PCI-compliant website that presumably has significant defenses and auditing in place.

vkjv 8y ago
Unfortunately, PCI does not put very many restrictions on the parent website. If credit card elements are in an iFrame, the parent site is excluded from most requirements because the iFrame is "secure." Of course, if you own the parent site you can replace the iFrame with anything you want.
- tyingq 8y ago
  PCI also doesn't necessarily mandate some common sense things either, like monitoring for unexpected changes on the cart page.
- marak830 8y ago
  I'm still confused as to how they could insert code here. Are we talking about a server intrustion where they modified the actual cart code, or something between Newegg and the payment servers? (Sorry this isn't my domain, I'm just curious)
  - lacker 8y ago
    It looks to me like a server intrusion where they modified static files kept on a webserver (like apache or nginx). But it also seems like we don't have enough evidence to know for sure. (Edit: or they might have been static files kept on a CMS.)
- vkjv 8y ago
  FWIW, I just checked and they don't use an iFrame approach. This means their entire checkout page must be in scope for SAQ-D.
lacker 8y ago
The article on the British Airways hack goes into a bit more detail, but still not enough to slake my curiosity. https://www.riskiq.com/blog/labs/magecart-british-airways-breach/ https://www.riskiq.com/blog/labs/magecart-british-airways-br... It seems like we just don't have enough information to know how exactly the attackers got the JS onto the page. What it looks like, though, is that some machine serving static files for the website got compromised, and the attackers replaced an innocent file with one that had some custom javascript tacked onto the end, that skimmed the credit card numbers. For what it's worth, PCI compliance doesn't really help you keep your webservers secure, or to keep people from breaking into your servers at all, it's more about the way you permanently store credit card numbers.
- pjc50 8y ago
  It was appended to "Modernizr.js". That suggests a number of further possibilities by which that may have been replaced on a developer's computer, or at the point of download by a package manager (are packages authenticated?) It could well have been included in a legitimate re-deploy if that was the case.
  - lacker 8y ago
    Since that file was served from "/cms/global/scripts/lib", perhaps their CMS got compromised. Anecdotally it seems like CMS's are more frequently prone to compromises than systems like nginx or apache.
zeveb 8y ago
Ditto. As an aside, while it's certainly true that if one can compromise a web server (whether serving static HTML or generating dynamic HTML) then it's game over, the modern style of JavaScript-centric development greatly expands the attack surface. An attacker can compromise any of a number of different servers (e.g. analytics providers), via numerous means; moreover, he can serve dynamic malicious code, and quickly respond to changes the targeted site makes (his implant needs to survive for awhile, but his access can be revoked and folks can still be compromised). Websites which served static pages and received credit-card information via simple forms would be far more secure than dynamic web apps stitched together from JavaScript served from multiple servers in countries across the globe, with data inserted into the DOM retrieved from multiple servers across the globe. I contend that a more traditional, static architecture would tend to lead to better architectural isolation. As an example, if one needed to install server-side analytics software rather than simply injecting it into a page at runtime, then one would download and install it once: any compromise of the analytics provider after one had download it would be irrelevant. But with dynamic JavaScript, one's users download the analytics software with every single page load: a compromise instantly affects users. Moreover, with traditional server-side software one tends to have reasonably-well defined protocols for interaction, rather than just saying, 'hey, I'll run all your code in the same process as everything else.' But with client-side JavaScript, any piece of code can see everything. As an engineering community, we've adopted a style which has great costs as well as great benefits. My opinion is that we moved too quickly, and now we're feeling the pain. While a universal portable app framework is pretty awesome, JavaScript layered over a document system is probably not the best way to achieve that. As I wrote years ago, JavaScript delenda est.
adreamingsoul 8y ago
PCI compliance isn't that hard to get. And, typically the code is changed after the compliance review. Also, developers and engineers at Newegg are probably changing lines of code every day. Which means, at any point in time someone could have accidentally introduced a vector that allowed the attackers to inject their code. Auditing and security reviews only work if every line of code is tested, reviewed, and verified. However, that be extremely costly and slow, which isn't "good for business". Maybe, we as consumers, engineers, managers, and executives should demand that code safety and security is prioritized over profit. I would rather have a secure website then a new redesign every two years.
- yspeak 8y ago
  This isn't fort Knox It's credit card info. There's always a cost benefit. You can spend all your time and money on doingsecurity reviews but without profit there ain't no company.
- saas_sam 8y ago
  Do you have any evidence to suggest "that code safety and security" was NOT "prioritized over profit"? How would you falsify this? Are we to assume that every security breach indicates that profit was prioritized over security? Doubtful. Also, "profit" is the reward for doing things efficiently, it is not some kind of scoreboard for how much a company is ripping off consumers. NewEgg is in such a hyper-competitive & mature space with margins in the low single-digits on average. Incidents like this can absolutely tank them, especially this close to the holiday season. E-commerce companies do not make money on security breaches, they lose it. Lots of it. I think you should be a little more open-minded to how businesses operate, that's all.

gregpilling 8y ago

I am surprised that there is no automated alert to tell the webmaster that his code has changed on his website. Especially on the payments page! With 50,000,000 users a month, surely they have a whole team working on checkout, all the time?

ryanlol 8y ago
Do you have such automated alerts set up yourself? Do you know anyone with such alerts set up?
- freddie_mercury 8y ago
  Whenever someone complains about another company's product, code, features, security....I always wish it was mandatory to include a link to the kind of software the poster is putting into production. I can dream. Glass houses and all that.
- elorant 8y ago
  I have. I deploy in ASP.NET and get a hash of the uploaded DLL. I check it twice a day. Never had any incidents to this day but as the saying goes, it's better to be safe than sorry.
  - tedunangst 8y ago
    And how does the hash of the DLL on some server relate to the content seen by users?
    - ixwt 8y ago
      I would assume the DLL is responsible for the output of the content seen by users.
    - elorant 8y ago
      In ASP.NET pretty much all output, including html and js is included in the DLL. Only external js files are left out. Sure they could hack them which means my solution isn't bulletproof but I could also produce a hash for them too.

anontechworker 8y ago

For a website with so many visitors and transactions, I’m surprised this API call never threw enough errors for them to see in logging. I will admit that JS logging can be messy because of all the different environments but after some time I would have hoped this would have been caught.

raverbashing 8y ago

So how come is Comodo selling certificates to domain squatters? This seems to be one sore point here.

ams6110 8y ago
They (the attackers) owned the domain. What more is Comodo supposed to do? LetsEncrypt would have done the same thing.

zxin 8y ago

This all could have been prevented if they had a Content Security Policy.

eat_veggies 8y ago
Presumably, if the attacker has write access to the source files, they also have the power to change the CSP headers.

Usu 8y ago

This breach has reminded me of this pretty great article: https://hackernoon.com/im-harvesting-credit-card-numbers-and-passwords-from-your-site-here-s-how-9a8cb347c5b5 https://hackernoon.com/im-harvesting-credit-card-numbers-and...

alyandon 8y ago

Lovely. I made a purchase recently with NewEgg but at least it was with a previously stored credit card so hopefully I'm not impacted by this. However, I am disappointed that NewEgg hasn't made any sort of official announcement yet.

adreamingsoul 8y ago
Agreed, they should have given us a notice. I don't know if I will ever buy something from NewEgg again if this is how they are going to treat security events.
robbyt 8y ago
They're probably still talking to their lawyers.
eikenberry 8y ago
I also purchased something right in the middle of that time frame, but with a regular credit card. I'm not concerned as I pretty much assume my credit-card is always compromised and I check my charges each month for discrepancies and contest anything bad. Though the credit-card catches the fraudulent transactions most of the time these days, so it usually doesn't come to that.
alyandon 8y ago
Replying to myself. I use uMatrix and it just occurred to me that it would have blocked any communication attempts to neweggstats.com from newegg.com.... so yay?
- Bartweiss 8y ago
  I noticed this too; uMaxtrix should catch this entire class of attack. Privacy Badger probably wouldn't - it watches for this sort of export, but its three-strikes rule targets trackers and misses unique attacks. Killing JS outright would do the trick also, but it looks like it would also have broken Newegg's checkout page. Looks like yet another point for "treat web browsing as adversarial". The price of allowing pages to load freely isn't just high weight, trackers, and ads - all too frequently it's actual security failures. I understand that ad/tracker blocking is a problem for keeping sites profitable, but I can't really imagine bending on that issue as long as monetization techniques and threat vectors overlap so heavily.

adreamingsoul 8y ago

Wow, I also made a purchase within that time window. Except, I used PayPal during Checkout.

BeetleB 8y ago

Damn. I made a purchase in that time period. I rarely buy anything from them, but it had to happen in that interval! I paid with Paypal. I assume I'm not affected?

adventured 8y ago
Given the way the code works, I don't see how you could be affected if your transaction went through PayPal (ie you didn't enter any payment information, such as a credit card).
- BeetleB 8y ago
  I did eventually pay with Paypal. However, I do know that at some point in the last few months, I attempted to make a transaction on some web site and had issues. It was either: 1. I tried to pay with Paypal and failed. Paid with CC instead. 2. I tried to pay with my CC and failed. Paid with Paypal instead. I don't remember for which site this happened, but the paranoid part of me is wondering if it may have been Newegg and item 2 above.
frickinLasers 8y ago
Did you happen to receive an unexpected address change notification? I did when I checked out with Paypal. The address didn't change (maybe it's all caps now and wasn't before?) and now I'm wondering too. It took them a month to find this--could there be other code they haven't discovered?

crunchlibrarian 8y ago

I had a conversation two days ago with the CTO of a very large company you've definitely heard of who said "we don't need to worry about our website security, we have a firewall and SSL" I think these types of attacks are vastly underreported, if anything.

mwigdahl 8y ago

They are just now (around 11:10 CDT 9/19) sending out notification emails to customers. At the moment they don't even seem to know what accounts were affected.