4 ms·

Fun story from a place I worked which got one of the first alpha 4100 sold in our country - it was having intermittent crashes that we couldn't figure out. DEC

by nikau 3y ago

Fun story from a place I worked which got one of the first alpha 4100 sold in our country - it was having intermittent crashes that we couldn't figure out. DEC sent out a very senior engineer to investigate, he eventually discovered some shielding had somehow been bent behind where a bus slot was soldered and with thermal expansion would short against some of the pins causing errors. Aside from that they were rock solid. DEC also sold us a weird windows cluster using where 2 boxes connected to the same scsi array, it was not so solid.

quercusa 3y ago
Was that doing a primary/secondary model where the primary wrote a timestamp to the shared disk and if the secondary didn't see that after X seconds, it would take over?
- nikau 3y ago
  I think it was a Microsoft feature but you had to have certified hardware for support. There were 2 windows servers both plugged into the same scsi disk pack, and it used a couple of crossover cables I believe for the heartbeat. I remember the unix guys were ribbing the windows guy a bit who confessed he regretted pushing for the cluster as it was so unreliable. It was the typical windows experience of the time with a management console that just hung when doing operations.
- Sylamore 3y ago
  Old versions of NT4 and 2000 supported SCSI disks in the cluster. We had one for a "high availability" file server from Compaq and always regretted it - I'm pretty sure a single box would have been more reliable. There was a cross-over LAN cable between the boxes and there were SCSI heartbeats (using persistent reservations IIRC) over the SCSI bus, but we would almost always have an issue with the SCSI drives deadlocking and the cluster failing BOTH nodes due to a fight over disk ownership. We replaced it with a netapp NAS eventually.
gjvc 3y ago
DEC also sold us a weird windows cluster using where 2 boxes connected to the same scsi array, it was not so solid. Was that some witchcraft with a Y-shaped SCSI cable, perchance?
- nikau 3y ago
  Yep that's the one. We ran exchange and some databases on it.
- bradknowles 3y ago
  For the GNN mail system, that was built on a pair of DEC Alpha 2100s, that were both connected to a pair of drive arrays. We had inherited that from ANS, and they had inherited that from O'Reilly. Global Network Navigator was the first nation-wide ISP in the US, and was a partnership between O'Reilly and AOL. But then it got too big for O'Reilly to run, and AOL wasn't ready to take it over, so we made ANS take it -- at the time, we had recently acquired ANS as our Tier-1 network provider, and they knew a lot more about DEC Alpha hardware than we did. But then it got too big for them to deal with, and AOL finally had to take it over. The mail system part was given to me and my co-worker. We learned a lot about DEC Alpha hardware from GNN, and we loved it. That system worked great, up until the time that DEC sent out a Field Engineer to upgrade the firmware on the drive arrays. Then we got weird errors when we tried to come back up. So, we switched to the backup systems that were connected to the same drives. And it was also hosed. Fortunately, I had started taking user mailbox backups to our Auspex NFS file server that week, so we had backups. We soon realized that over 50% of the POP3/leave-on-server mailboxes will change radically over the span of an day. And some of our mailbox backups were over eighteen hours old. So, most of those backups were actually useless. We did finally get back up and running by mounting the backup mailboxes via NFS, but we lost pretty much all user confidence in the system by that time. The drive arrays had been set up to mirror at the lowest level, then stripe across the mirrors in hardware. We then striped again across those volumes in software, which was a high performance disk striping technique we called "plaiding". We ended up breaking each of the lowest level mirrors and sending the whole bunch to the guy who wrote the AdvFS filesystem for DEC, but he never got better than 80% recovery of the data, which meant that on average every fifth character was wrong. So, pretty much none of those recovered mailboxes were useful. That was the death knell for GNN. If you can't trust the mail system, then what else can you trust?

by nikau 3y ago

quercusa 3y ago

Was that doing a primary/secondary model where the primary wrote a timestamp to the shared disk and if the secondary didn't see that after X seconds, it would take over?

nikau 3y ago
I think it was a Microsoft feature but you had to have certified hardware for support. There were 2 windows servers both plugged into the same scsi disk pack, and it used a couple of crossover cables I believe for the heartbeat. I remember the unix guys were ribbing the windows guy a bit who confessed he regretted pushing for the cluster as it was so unreliable. It was the typical windows experience of the time with a management console that just hung when doing operations.
Sylamore 3y ago
Old versions of NT4 and 2000 supported SCSI disks in the cluster. We had one for a "high availability" file server from Compaq and always regretted it - I'm pretty sure a single box would have been more reliable. There was a cross-over LAN cable between the boxes and there were SCSI heartbeats (using persistent reservations IIRC) over the SCSI bus, but we would almost always have an issue with the SCSI drives deadlocking and the cluster failing BOTH nodes due to a fight over disk ownership. We replaced it with a netapp NAS eventually.

gjvc 3y ago

DEC also sold us a weird windows cluster using where 2 boxes connected to the same scsi array, it was not so solid. Was that some witchcraft with a Y-shaped SCSI cable, perchance?

nikau 3y ago
Yep that's the one. We ran exchange and some databases on it.
bradknowles 3y ago
For the GNN mail system, that was built on a pair of DEC Alpha 2100s, that were both connected to a pair of drive arrays. We had inherited that from ANS, and they had inherited that from O'Reilly. Global Network Navigator was the first nation-wide ISP in the US, and was a partnership between O'Reilly and AOL. But then it got too big for O'Reilly to run, and AOL wasn't ready to take it over, so we made ANS take it -- at the time, we had recently acquired ANS as our Tier-1 network provider, and they knew a lot more about DEC Alpha hardware than we did. But then it got too big for them to deal with, and AOL finally had to take it over. The mail system part was given to me and my co-worker. We learned a lot about DEC Alpha hardware from GNN, and we loved it. That system worked great, up until the time that DEC sent out a Field Engineer to upgrade the firmware on the drive arrays. Then we got weird errors when we tried to come back up. So, we switched to the backup systems that were connected to the same drives. And it was also hosed. Fortunately, I had started taking user mailbox backups to our Auspex NFS file server that week, so we had backups. We soon realized that over 50% of the POP3/leave-on-server mailboxes will change radically over the span of an day. And some of our mailbox backups were over eighteen hours old. So, most of those backups were actually useless. We did finally get back up and running by mounting the backup mailboxes via NFS, but we lost pretty much all user confidence in the system by that time. The drive arrays had been set up to mirror at the lowest level, then stripe across the mirrors in hardware. We then striped again across those volumes in software, which was a high performance disk striping technique we called "plaiding". We ended up breaking each of the lowest level mirrors and sending the whole bunch to the guy who wrote the AdvFS filesystem for DEC, but he never got better than 80% recovery of the data, which meant that on average every fifth character was wrong. So, pretty much none of those recovered mailboxes were useful. That was the death knell for GNN. If you can't trust the mail system, then what else can you trust?