4 ms·

Or that AS/400 calls the tech to fix it and s/he just does it while no one is around. Had a call once from IBM asking for directions since the AS/400 RAID cach

Or that AS/400 calls the tech to fix it and s/he just does it while no one is around. Had a call once from IBM asking for directions since the AS/400 RAID cache battery needed replacing. No danger of losing data, the AS/400 was just running slower because it was bypassing the cache.

rashkov 10y ago
So... a tech would just show up at your office, ready to fix the machine? On the one hand, that is some fantastic service when you're running mission critical software. On the other hand, that must cost a whole lot of money and is quite devious of IBM.
- jonknee 10y ago
  > that must cost a whole lot of money and is quite devious of IBM. You have essentially described IBM's business model. You pay more, but if something goes wrong someone shows up.
  - deleted 10y ago
    [deleted]
- seanp2k2 10y ago
  Yeah, that type of service and phone-home is also common with $$$ enterprise storage (SANs). Funny story: in one instance I know of first-hand, the people running a NetApp filer setup with a redundant pair of systems had chosen to disable AutoSupport (NetApp's version of the phone-home support system where a tech would just show up with replacement disks when some failed) and take on support themselves. They did not test the alerting well enough after some network changes, which left the poor SAN unable to get distress signals through. One day, the shared mount on a ton of systems around the company just stopped working. The master node (these have internal redundancy at the network, controller, shelf, and disk level) had failed over to the redundant node, which after time had so many disks fail without being replace that it went down hard. They had to work with NetApp support (at great expense) to get it back up, which took a few days. The system had been attempting to warn the admins as best it could of the impending doom for the better part of a year, and no one noticed. The moral of this story is that you can have big-dollar redundant systems, but if you don't also have some dollars going to the care and feeding of them, it's wasted. The other lesson here is that such systems don't exist in isolation and are only as good as their weakest link, which in this case was a now-unreachable SMTP server. Lastly, although the LCD displays were trying their best to also signal this fault condition, people who would walk by these physical machines were used to this, so they didn't bother to actually read the errors. Do that at the peril of anything dependent on that system.
  - hackuser 10y ago
    > The moral of this story is that you can have big-dollar redundant systems, but if you don't also have some dollars going to the care and feeding of them, it's wasted. Agreed. Another moral: Don't believe anything works until you see it with your own eyes.
    - protomyth 10y ago
      Had a friend unplug the UPS to test it while I was still configuring the new machine. "Can't really be sure unless you test it"
    - throwawayish 10y ago
      Especially true for backups. Make sure those aren't write-only!
  - shubb 10y ago
    Note to self, if need to socially engineer my way into the equipment room at a large company, pretend to be an IBM tech answering a phone home...
    - protomyth 10y ago
      Good luck with that, as we do check these things before we allow folks to get to the servers. IBM has been doing this for a fair bit and we aren't exactly blind idiots. It would be interesting where I am since you skip local crime and go straight to federal offense. Kept me away from booze in high school since its not minor in possessions, its federal bootlegging. I can only imagine what trying to enter a server room nets a person.
  - syntheticnature 10y ago
    "The Fail-Safe Theorem: When a Fail-Safe system fails, it fails by failing to fail safe." --https://en.wikipedia.org/wiki/Systemantics#System_failure https://en.wikipedia.org/wiki/Systemantics#System_failure
- protomyth 10y ago
  It actually isn't that bad on the budget. We don't have the same day service (I think its 24 - 48 hour response time). It sure beats the heck out of waiting for the disaster as the thing is pretty proactive. Much, much cheaper than service and support from Oracle, and Oracle doesn't visit or monitor the machine.
- teh_klev 10y ago
  We just decommissioned a pair of EMC CX's. These things had phone home to EMC (and then eventually a third party maintainer) if something was about to go bad or had failed. We'd usually have an engineer onsite within four hours at the DC with all the right parts to carry out a fix. By the time it was being looked after by the third party maintainer they'd just leave us a few disks and a couple of fibre channel controllers onsite and we'd get the DC remote hands to swap them out. The faulty parts were then boxed up and couriered back to the maintenance company all on their dime. There was not an awful lot of involvement on our end except for authorising access to the DC when an engineer needed to attend. None of this was massively expensive (and this was 24x7x365 cover) in the great scheme of things given their mission critical status.
- mavrc 10y ago
  > So... a tech would just show up at your office, ready to fix the machine? Yes. They are expensive and deviously good at it. Or, at least they used to be many years ago. Many years back I worked in a tiny town (1000 people?) four hours from the closest depot and we even got same day service. This cost a king's ransom. But the AS/400 ran most of the business...
nickpsecurity 10y ago
It's possible. I do want to try to get the uptime info from the people that run it if I ever see them. We regularly get techs come in to improve or fix IT systems. Just not that one.
- protomyth 10y ago
  I have had multiple years of uptime until I needed to do an OS upgrade for some new part of the financial system.
  - nickpsecurity 10y ago
    That's what vast majority of my feedback has said. Stays up until OS upgrade. Whereas, my research on VMS side indicated they do clusters that support rolling upgrades of the individual machines to not affect application. Might be around that remaining limitation of AS/400 uptime if they have software for that. Do they? Or are customers forced to upgrade to mainframe with Sysplex?
    - protomyth 10y ago
      OpenVMS has an amazing cluster system, and yes they did upgrade one machine at a time in the cluster and lose no overall uptime. I think the bigger IBM z machines do allow zero-time upgrades, but the iSeries (new name for AS/400) is basically a reboot the machine affair. I've never heard of Sysplex in regards to the iSeries. They sent me two DVDs with 12 pages of instructions. The unnumbered first page is release notes and a notice that physical media deliveries would be an average of 5 to 9 business days on Jan 31, 2007 and beyond. (odd notice for a 2016 document) Page 1 has my customer information with all sorts of tracking numbers. It also include the number of selected files (1,360), Kilo-bytes (sp) of data (3,963,529), fixes (3,394 fixes with 815 Kilo-bytes of data were superceded), etc. Pre-install instructions finally show up on page 3 and the big warning about f'n it up shows on page 4. Page 6 brings us to the actual install which is written to be typed exactly with exact detailed results of each step. to give an example 2. Load the image catalog into the virtual optical device using the following command: LODIMGCLG IMGCLG(ptfcatalog) DEV(OPTVRTxx) OPTION(*LOAD) 3. Type GO PTF and press the Enter key 4. Take menu option 8 and press Enter key Took about an two hours counting the the 45 minute reboot. We have a very old and slow machine (need to buy a new one in the next two years, we are at a decade of use now) so reboots are a bit slow.

rashkov 10y ago

So... a tech would just show up at your office, ready to fix the machine? On the one hand, that is some fantastic service when you're running mission critical software. On the other hand, that must cost a whole lot of money and is quite devious of IBM.

jonknee 10y ago
> that must cost a whole lot of money and is quite devious of IBM. You have essentially described IBM's business model. You pay more, but if something goes wrong someone shows up.
- deleted 10y ago
  [deleted]
seanp2k2 10y ago
Yeah, that type of service and phone-home is also common with $$$ enterprise storage (SANs). Funny story: in one instance I know of first-hand, the people running a NetApp filer setup with a redundant pair of systems had chosen to disable AutoSupport (NetApp's version of the phone-home support system where a tech would just show up with replacement disks when some failed) and take on support themselves. They did not test the alerting well enough after some network changes, which left the poor SAN unable to get distress signals through. One day, the shared mount on a ton of systems around the company just stopped working. The master node (these have internal redundancy at the network, controller, shelf, and disk level) had failed over to the redundant node, which after time had so many disks fail without being replace that it went down hard. They had to work with NetApp support (at great expense) to get it back up, which took a few days. The system had been attempting to warn the admins as best it could of the impending doom for the better part of a year, and no one noticed. The moral of this story is that you can have big-dollar redundant systems, but if you don't also have some dollars going to the care and feeding of them, it's wasted. The other lesson here is that such systems don't exist in isolation and are only as good as their weakest link, which in this case was a now-unreachable SMTP server. Lastly, although the LCD displays were trying their best to also signal this fault condition, people who would walk by these physical machines were used to this, so they didn't bother to actually read the errors. Do that at the peril of anything dependent on that system.
- hackuser 10y ago
  > The moral of this story is that you can have big-dollar redundant systems, but if you don't also have some dollars going to the care and feeding of them, it's wasted. Agreed. Another moral: Don't believe anything works until you see it with your own eyes.
  - protomyth 10y ago
    Had a friend unplug the UPS to test it while I was still configuring the new machine. "Can't really be sure unless you test it"
  - throwawayish 10y ago
    Especially true for backups. Make sure those aren't write-only!
- shubb 10y ago
  Note to self, if need to socially engineer my way into the equipment room at a large company, pretend to be an IBM tech answering a phone home...
  - protomyth 10y ago
    Good luck with that, as we do check these things before we allow folks to get to the servers. IBM has been doing this for a fair bit and we aren't exactly blind idiots. It would be interesting where I am since you skip local crime and go straight to federal offense. Kept me away from booze in high school since its not minor in possessions, its federal bootlegging. I can only imagine what trying to enter a server room nets a person.
- syntheticnature 10y ago
  "The Fail-Safe Theorem: When a Fail-Safe system fails, it fails by failing to fail safe." --https://en.wikipedia.org/wiki/Systemantics#System_failure https://en.wikipedia.org/wiki/Systemantics#System_failure
protomyth 10y ago
It actually isn't that bad on the budget. We don't have the same day service (I think its 24 - 48 hour response time). It sure beats the heck out of waiting for the disaster as the thing is pretty proactive. Much, much cheaper than service and support from Oracle, and Oracle doesn't visit or monitor the machine.
teh_klev 10y ago
We just decommissioned a pair of EMC CX's. These things had phone home to EMC (and then eventually a third party maintainer) if something was about to go bad or had failed. We'd usually have an engineer onsite within four hours at the DC with all the right parts to carry out a fix. By the time it was being looked after by the third party maintainer they'd just leave us a few disks and a couple of fibre channel controllers onsite and we'd get the DC remote hands to swap them out. The faulty parts were then boxed up and couriered back to the maintenance company all on their dime. There was not an awful lot of involvement on our end except for authorising access to the DC when an engineer needed to attend. None of this was massively expensive (and this was 24x7x365 cover) in the great scheme of things given their mission critical status.
mavrc 10y ago
> So... a tech would just show up at your office, ready to fix the machine? Yes. They are expensive and deviously good at it. Or, at least they used to be many years ago. Many years back I worked in a tiny town (1000 people?) four hours from the closest depot and we even got same day service. This cost a king's ransom. But the AS/400 ran most of the business...

nickpsecurity 10y ago

It's possible. I do want to try to get the uptime info from the people that run it if I ever see them. We regularly get techs come in to improve or fix IT systems. Just not that one.

protomyth 10y ago
I have had multiple years of uptime until I needed to do an OS upgrade for some new part of the financial system.
- nickpsecurity 10y ago
  That's what vast majority of my feedback has said. Stays up until OS upgrade. Whereas, my research on VMS side indicated they do clusters that support rolling upgrades of the individual machines to not affect application. Might be around that remaining limitation of AS/400 uptime if they have software for that. Do they? Or are customers forced to upgrade to mainframe with Sysplex?
  - protomyth 10y ago
    OpenVMS has an amazing cluster system, and yes they did upgrade one machine at a time in the cluster and lose no overall uptime. I think the bigger IBM z machines do allow zero-time upgrades, but the iSeries (new name for AS/400) is basically a reboot the machine affair. I've never heard of Sysplex in regards to the iSeries. They sent me two DVDs with 12 pages of instructions. The unnumbered first page is release notes and a notice that physical media deliveries would be an average of 5 to 9 business days on Jan 31, 2007 and beyond. (odd notice for a 2016 document) Page 1 has my customer information with all sorts of tracking numbers. It also include the number of selected files (1,360), Kilo-bytes (sp) of data (3,963,529), fixes (3,394 fixes with 815 Kilo-bytes of data were superceded), etc. Pre-install instructions finally show up on page 3 and the big warning about f'n it up shows on page 4. Page 6 brings us to the actual install which is written to be typed exactly with exact detailed results of each step. to give an example 2. Load the image catalog into the virtual optical device using the following command: LODIMGCLG IMGCLG(ptfcatalog) DEV(OPTVRTxx) OPTION(*LOAD) 3. Type GO PTF and press the Enter key 4. Take menu option 8 and press Enter key Took about an two hours counting the the 45 minute reboot. We have a very old and slow machine (need to buy a new one in the next two years, we are at a decade of use now) so reboots are a bit slow.