4 ms·

This is entirely IT's fault, though. Why did they allow a rogue server on to the production farm in the first place? Why did the idiot have access to them? W

by cube13 15y ago

This is entirely IT's fault, though. Why did they allow a rogue server on to the production farm in the first place? Why did the idiot have access to them? While the blame should still be entirely on the idiot, it's still IT's job to make sure that the production environment is secure and functioning. They should have as much control of the environment as possible to ensure that they can ensure that. If they are unable to do that, then they are simply not doing their job right, and should be pushing management for more resources or training.

patio11 15y ago
Without addressing whose fault it is, let's just say that in the last ten years one hundred thousand different people have worked for Bank of Not Canada. They run the spectrum from "Could give talks at Black Hat" to "Could possibly be allowed to put on a hat without killing themselves, if carefully supervised at all stages of haberdashery." All it takes is one guy screwing up one decision on his worst day six years ago.
- pacaro 15y ago
  I agree entirely, and I like your spectrum, however... "A haberdasher is a person who sells small articles for sewing, such as buttons, ribbons, zips, and other notions" [1] A hatmaker is a "milliner". I know that these two ancient and noble professions are frequently confused in modern English speech, but it's rather like saying "designer" when you mean "developer"... [1] http://en.wikipedia.org/wiki/Haberdasher http://en.wikipedia.org/wiki/Haberdasher
- joeag 15y ago
  Seems like a good consulting business opportunity to audit large companies web presence to get a list of all publicly accessible sites and then gather current info from IT and management on what is authorized and what's not.
T-hawk 15y ago
In this scenario, the server doesn't start out as rogue. It was an existing production machine, then superseded by an upgrade to a new set of production hardware. You have to understand that in companies the size of "Bank of Canada", everybody is terrified of upsetting production functionality in the slightest. In almost every corporate upgrade, the old machine gets left there "just in case". And this is probably rightly so playing the probabilities of the outcomes; the small fraction of superseded machines that come back into service is still much larger than the fraction that get subverted into blackhattery. "Why did the idiot have access to them"? He's not presumed to be an idiot or bozo. He's trusted as an employee. Corps generally don't have the time or manpower or profitability incentive to spy on every action of their own personnel. You only know if somebody is a clown after the fact. "IT's job to make sure that the production environment is secure and functioning" -- IT isn't some mythical infallible entity. IT comprises human beings too, who can make mistakes or look the other way for expediency or even themselves don a black hat.
- cube13 15y ago
  >"Why did the idiot have access to them"? He's not presumed to be an idiot or bozo. He's trusted as an employee. Corps generally don't have the time or manpower or profitability incentive to spy on every action of their own personnel. You only know if somebody is a clown after the fact. He's implied to be a non-technical business employee. Again, why does he have access to production servers? I would understand access for developers, but business guys? They shouldn't have access beyond a web portal or something like that. So again, why did they have access to production servers? EDIT: Realistically, for a large organization like Bank of Not Canada, pushing out a new site should have several steps: 1. Management comes up with new idea, assigns peon to come up with design. 2. Peon makes content in FrontPage. 3. Peon passes content to marketing and legal(especially important, considering it's a bank). They approve it, pass it to development and IT to get it up. 4. Development and IT get it working, and put it up on production servers. None of the people in step 1, 2 or 3 should have direct access to the production servers. That means that neither the Peoon or legal or marketing department should have direct access to the production servers. If this was pushed out by a developer, I would agree that IT isn't at fault. They need that level of access to do their jobs if something goes wrong. However, this access isn't required by the peon or anyone earlier in the chain, so they should not have it. >"IT's job to make sure that the production environment is secure and functioning" -- IT isn't some mythical infallible entity. IT comprises human beings too, who can make mistakes or look the other way for expediency or even themselves don a black hat. So? That's still their job. Mistakes happen, but that isn't an excuse for making them.
tptacek 15y ago
I find amusing your certainty that any competent large company should know exactly what's running on their production network. Exactly how are they supposed to do that? Security is a cost center; headcount is minimized across the board; no business process is gated on "knowing exactly what's out there"†. There are meanwhile hundreds or thousands of servers, many running multiple applications. Most large companies do not, at any given moment, know exactly what's running on their network. † Unlike for instance app security audits, which gate deployment of new versions of code, and are thus straightforward to integrate into business process.
- illumin8 15y ago
  How about controlling the DNS for corporation.com? Any company with a reasonably competent IT staff should be able to flatly deny a request for a subdomain. "You want to setup myblinkenlightsbox.corporation.com? Has that machine been hardened to corporate standards? Request denied."
  - JoachimSchipper 15y ago
    That's how you end up with the testing box on 10.4.253.2 (no DNS), time tracking at 10.3.53.234, and an Excel sheet full of these and other addresses.
    - deleted 15y ago
      [deleted]
    - xorglorb 15y ago
      Internal DNS server with a separate domain for testing (ie. notbankofcanada.testing)?
rmc 15y ago
The other flaw with the IT dept. is that they make The Right Way™ too hard, time consuming and long winded. If people can get significant time savings by taking shortcuts then you system has a flaw.

by cube13 15y ago

patio11 15y ago

Without addressing whose fault it is, let's just say that in the last ten years one hundred thousand different people have worked for Bank of Not Canada. They run the spectrum from "Could give talks at Black Hat" to "Could possibly be allowed to put on a hat without killing themselves, if carefully supervised at all stages of haberdashery." All it takes is one guy screwing up one decision on his worst day six years ago.

pacaro 15y ago
I agree entirely, and I like your spectrum, however... "A haberdasher is a person who sells small articles for sewing, such as buttons, ribbons, zips, and other notions" [1] A hatmaker is a "milliner". I know that these two ancient and noble professions are frequently confused in modern English speech, but it's rather like saying "designer" when you mean "developer"... [1] http://en.wikipedia.org/wiki/Haberdasher http://en.wikipedia.org/wiki/Haberdasher
joeag 15y ago
Seems like a good consulting business opportunity to audit large companies web presence to get a list of all publicly accessible sites and then gather current info from IT and management on what is authorized and what's not.

T-hawk 15y ago

In this scenario, the server doesn't start out as rogue. It was an existing production machine, then superseded by an upgrade to a new set of production hardware. You have to understand that in companies the size of "Bank of Canada", everybody is terrified of upsetting production functionality in the slightest. In almost every corporate upgrade, the old machine gets left there "just in case". And this is probably rightly so playing the probabilities of the outcomes; the small fraction of superseded machines that come back into service is still much larger than the fraction that get subverted into blackhattery. "Why did the idiot have access to them"? He's not presumed to be an idiot or bozo. He's trusted as an employee. Corps generally don't have the time or manpower or profitability incentive to spy on every action of their own personnel. You only know if somebody is a clown after the fact. "IT's job to make sure that the production environment is secure and functioning" -- IT isn't some mythical infallible entity. IT comprises human beings too, who can make mistakes or look the other way for expediency or even themselves don a black hat.

cube13 15y ago
>"Why did the idiot have access to them"? He's not presumed to be an idiot or bozo. He's trusted as an employee. Corps generally don't have the time or manpower or profitability incentive to spy on every action of their own personnel. You only know if somebody is a clown after the fact. He's implied to be a non-technical business employee. Again, why does he have access to production servers? I would understand access for developers, but business guys? They shouldn't have access beyond a web portal or something like that. So again, why did they have access to production servers? EDIT: Realistically, for a large organization like Bank of Not Canada, pushing out a new site should have several steps: 1. Management comes up with new idea, assigns peon to come up with design. 2. Peon makes content in FrontPage. 3. Peon passes content to marketing and legal(especially important, considering it's a bank). They approve it, pass it to development and IT to get it up. 4. Development and IT get it working, and put it up on production servers. None of the people in step 1, 2 or 3 should have direct access to the production servers. That means that neither the Peoon or legal or marketing department should have direct access to the production servers. If this was pushed out by a developer, I would agree that IT isn't at fault. They need that level of access to do their jobs if something goes wrong. However, this access isn't required by the peon or anyone earlier in the chain, so they should not have it. >"IT's job to make sure that the production environment is secure and functioning" -- IT isn't some mythical infallible entity. IT comprises human beings too, who can make mistakes or look the other way for expediency or even themselves don a black hat. So? That's still their job. Mistakes happen, but that isn't an excuse for making them.

tptacek 15y ago

I find amusing your certainty that any competent large company should know exactly what's running on their production network. Exactly how are they supposed to do that? Security is a cost center; headcount is minimized across the board; no business process is gated on "knowing exactly what's out there"†. There are meanwhile hundreds or thousands of servers, many running multiple applications. Most large companies do not, at any given moment, know exactly what's running on their network. † Unlike for instance app security audits, which gate deployment of new versions of code, and are thus straightforward to integrate into business process.

illumin8 15y ago
How about controlling the DNS for corporation.com? Any company with a reasonably competent IT staff should be able to flatly deny a request for a subdomain. "You want to setup myblinkenlightsbox.corporation.com? Has that machine been hardened to corporate standards? Request denied."
- JoachimSchipper 15y ago
  That's how you end up with the testing box on 10.4.253.2 (no DNS), time tracking at 10.3.53.234, and an Excel sheet full of these and other addresses.
  - deleted 15y ago
    [deleted]
  - xorglorb 15y ago
    Internal DNS server with a separate domain for testing (ie. notbankofcanada.testing)?

rmc 15y ago

The other flaw with the IT dept. is that they make The Right Way™ too hard, time consuming and long winded. If people can get significant time savings by taking shortcuts then you system has a flaw.