3 ms·

If I had more money than sense, I’d do the same thing. If I was filthy rich (which I’m not and probably never will be), I’d build a data centre in my mansion a

If I had more money than sense, I’d do the same thing. If I was filthy rich (which I’m not and probably never will be), I’d build a data centre in my mansion and buy a brand new IBM mainframe and stick it in there. Because I can. For the same reason other rich people build some huge garage with dozens of cars in it.

remram 4y ago
It sounds like they wanted 2 separate network paths but they still end up with only 1 where they can use BGP. Maybe they can still use their residential connection as failover, with a VPN to a VPS where they can use BGP?
- dboreham 4y ago
  They all likey traverse the same physical path and would all be taken out by the same gravel truck hitting a pole. The two connection thing was I think more about defense against random retail connectivity flaps (someone decides to upgrade a router's firmware and just takes the town offline, meanwhile nobody answers support calls, that sort of thing).
  - bombcar 4y ago
    Yeah true multihoming is expensive and involves things like hiring people to actually inspect the physical lines and Make sure they don’t run down the same street at some point (and be subject to the same backhoe of fiber death)
  - remram 4y ago
    True. I thought they wanted BGP to implement multihoming but upon re-reading they needed it anyway for other reasons.
    - dboreham 4y ago
      BGP would be to allow fail over of some service from their primary colo.
HyperSane 4y ago
I would buy one of every high end enterprise storage array to play with. My former life as a VMWare admin made me appreciate storage.
- skissane 4y ago
  Which means you'd need a IBM mainframe, since IBM mainframes require enterprise storage arrays which come with special software for emulating the old mainframe hard disks (ECKD), and if you are going to buy one of those, you'd need a mainframe to drive it.
  - HyperSane 4y ago
    No, most of them use fiber channel.
    - skissane 4y ago
      Yes, they do use Fibre Channel (FC), but the protocol they run on top of FC is very different from the usual FC protocols. FC has four layers, from FC-0 (the physical layer) through to FC-4 (the application layer). At level FC-4, Linux/Unix/Windows/etc servers mostly speak "Fibre Channel Protocol for SCSI" aka SCSI-FCP, which transmits SCSI commands over FC. However, while IBM mainframes do use SCSI-over-FC, they also use something else called "FICON", which instead of transmitting SCSI commands, uses the same command set as IBM's legacy non-SCSI mainframe hard drives (such as the IBM 3390, which was IBM's last non-SAN mainframe hard disk line from the late 80s / early 90s). Mainstream platforms have what IBM calls FBA (Fixed Block Array) disks, in which the disk is an array of sectors which all have the same size. While the IBM mainframe hardware supports that, most IBM mainframe operating systems don't support those, however; Linux and VSEn are the exceptions, and z/VM is a partial exception. As well as FBA, IBM has ECKD (Extended Count Key Data) disks, like the IBM 3390 was. (ECKD was preceded by CKD, which is conceptually the same – the same sector format, etc – but had a less efficient command set.) With ECKD, sectors can have variable sizes – each disk track can have different sized sectors, you can even mix different sector sizes within the same disk track. Also, sectors optionally have keys, and the disk has commands to search for sectors based on their keys. Originally, these variable sized sectors actually physically existed on disk; nowadays, the SAN only has standard hard disks (or SSDs), but ECKD disks are emulated in the SAN software on top of them. z/OS, the most popular IBM mainframe OS, only supports ECKD disks, not FBA ones. Linux and VSEn support both ECKD and FBA. For ECKD disks, you can't use the standard SCSI command set, you have to use IBM's DASD command set. Many high-end enterprise arrays do support ECKD and FICON, but you usually have to pay significant extra license fees to enable the software that supports that.

remram 4y ago

It sounds like they wanted 2 separate network paths but they still end up with only 1 where they can use BGP. Maybe they can still use their residential connection as failover, with a VPN to a VPS where they can use BGP?

dboreham 4y ago
They all likey traverse the same physical path and would all be taken out by the same gravel truck hitting a pole. The two connection thing was I think more about defense against random retail connectivity flaps (someone decides to upgrade a router's firmware and just takes the town offline, meanwhile nobody answers support calls, that sort of thing).
- bombcar 4y ago
  Yeah true multihoming is expensive and involves things like hiring people to actually inspect the physical lines and Make sure they don’t run down the same street at some point (and be subject to the same backhoe of fiber death)
- remram 4y ago
  True. I thought they wanted BGP to implement multihoming but upon re-reading they needed it anyway for other reasons.
  - dboreham 4y ago
    BGP would be to allow fail over of some service from their primary colo.

HyperSane 4y ago

I would buy one of every high end enterprise storage array to play with. My former life as a VMWare admin made me appreciate storage.

skissane 4y ago
Which means you'd need a IBM mainframe, since IBM mainframes require enterprise storage arrays which come with special software for emulating the old mainframe hard disks (ECKD), and if you are going to buy one of those, you'd need a mainframe to drive it.
- HyperSane 4y ago
  No, most of them use fiber channel.
  - skissane 4y ago
    Yes, they do use Fibre Channel (FC), but the protocol they run on top of FC is very different from the usual FC protocols. FC has four layers, from FC-0 (the physical layer) through to FC-4 (the application layer). At level FC-4, Linux/Unix/Windows/etc servers mostly speak "Fibre Channel Protocol for SCSI" aka SCSI-FCP, which transmits SCSI commands over FC. However, while IBM mainframes do use SCSI-over-FC, they also use something else called "FICON", which instead of transmitting SCSI commands, uses the same command set as IBM's legacy non-SCSI mainframe hard drives (such as the IBM 3390, which was IBM's last non-SAN mainframe hard disk line from the late 80s / early 90s). Mainstream platforms have what IBM calls FBA (Fixed Block Array) disks, in which the disk is an array of sectors which all have the same size. While the IBM mainframe hardware supports that, most IBM mainframe operating systems don't support those, however; Linux and VSEn are the exceptions, and z/VM is a partial exception. As well as FBA, IBM has ECKD (Extended Count Key Data) disks, like the IBM 3390 was. (ECKD was preceded by CKD, which is conceptually the same – the same sector format, etc – but had a less efficient command set.) With ECKD, sectors can have variable sizes – each disk track can have different sized sectors, you can even mix different sector sizes within the same disk track. Also, sectors optionally have keys, and the disk has commands to search for sectors based on their keys. Originally, these variable sized sectors actually physically existed on disk; nowadays, the SAN only has standard hard disks (or SSDs), but ECKD disks are emulated in the SAN software on top of them. z/OS, the most popular IBM mainframe OS, only supports ECKD disks, not FBA ones. Linux and VSEn support both ECKD and FBA. For ECKD disks, you can't use the standard SCSI command set, you have to use IBM's DASD command set. Many high-end enterprise arrays do support ECKD and FICON, but you usually have to pay significant extra license fees to enable the software that supports that.