4 ms·

For you oldsters out there- how does GFS/Collossus compare to Andrew FS? https://en.wikipedia.org/wiki/Andrew_File_System https://en.wikipedia.org/wiki/Andrew_

For you oldsters out there- how does GFS/Collossus compare to Andrew FS? https://en.wikipedia.org/wiki/Andrew_File_System https://en.wikipedia.org/wiki/Andrew_File_System

lizknope 4y ago
I was just about to ask if anyone had used AFS. When I went to college from 1993-97 our university used AFS for all the Unix workstations. We had a mix of Solaris, SunOS, HP-UX, DEC Ultrix, SGI IRIX, and then some Linux boxes. When you logged in at any workstation from any vendor in any computer lab your home dir would be mounted and everything was encrypted. We had ACLs where we could give permission by username and create our own ACL groups and give permission to a project directory to just our project/lab partners. It was so much easier than the chmod uid/gid stuff. In theory we could access other universities under /afs/mit or whatever site they were but I don't think we ever had that active.
- ricktdotorg 4y ago
  I spent a couple of years at Morgan Stanley and when we cut over from SunOS to Solaris they included a rollout of AFS worldwide. It was pretty successful. It was pretty mind blowing to be able to login to a Solaris workstation while visiting the NYC office and have your [local-to-London] $HOME and CDE environment just kinda sync automatically (albeit slowly back then). The administration of AFS is ... interesting, but considering this was the mid-to-late 90s, it really was "ahead of its time". FWIW, Morgan Stanley back then had an incredible UNIX Engineering department.
  - lizknope 4y ago
    I finally got this book a couple of years ago. "MIT Project Athena: A Model for Distributed Campus Computing" https://www.amazon.com/gp/product/0135853249/ https://www.amazon.com/gp/product/0135853249/ I really thought AFS, DCE, and SSI (single system image) were going to take over. It was kind of depressing when I got a job and everything was NIS/NFS and we manually logged into a machine and started a job. Then we got clustering software like LFS to distribute jobs but I thought the dynamic process migration using checkpoint / restart kind of stuff would take over but it seems to be more trouble than it is worth. At work we would always run stuff off a powerful 4 to 16 CPU Sun and display to our cheaper SPARCstation 5 or Linux desktop via X11. I will say it is great now to have a virtual desktop session Exceed/NoMachine/X2go and be able to go home and connect to it and have all my windows in the same place just like when I left. I've got X2go set up at home so even when I travel I can get a desktop at home if I need it. https://en.wikipedia.org/wiki/Project_Athena https://en.wikipedia.org/wiki/Project_Athena https://en.wikipedia.org/wiki/Distributed_Computing_Environment https://en.wikipedia.org/wiki/Distributed_Computing_Environm... https://en.wikipedia.org/wiki/Single_system_image https://en.wikipedia.org/wiki/Single_system_image
thequux 4y ago
The two are very different technologies. In AFS, the fundamental unit of sharding is the volume, each volume is controlled by a single volser, and the "master" (i.e. vldb) is responsible for resolving volume IDs to the set of volsers that hold a copy (at most one of which is read/write) In GFS, on the other hand, individual files are sharded between servers on a chunk level (where 1 block==64MiB IIRC). To access a file's contents, you would contact the master, ask it for the chunkservers for a file, and it would tell you which set of chunkservers were responsible for each chunk in the file; you'd then contact the chunkserver to access the data. The impact of this is that the latency of accessing a file was much higher on GFS; you needed to go through the master (which tended to be a bottleneck) to get the metadata, and then establish a new connection with authorization to each chunkserver to access the data. However, because each file was stored by multiple machines and the data location was independent of the position in the hierarchy, there was very little risk of chunkservers limiting the transfer speed. With AFS, on the other hand, the master is much less of a bottleneck as it's only used when mounting volumes. Further, once a volume is mounted, accessing files and data within those files could be done in a single round trip. However, if you have a volume with high write traffic, that will end up bottlenecking on a single server's IO speed.
- lossolo 4y ago
  > In GFS, on the other hand, individual files are sharded between servers on a chunk level (where 1 block==64MiB IIRC). To access a file's contents, you would contact the master, ask it for the chunkservers for a file, and it would tell you which set of chunkservers were responsible for each chunk in the file; you'd then contact the chunkserver to access the data. > The impact of this is that the latency of accessing a file was much higher on GFS; you needed to go through the master (which tended to be a bottleneck) to get the metadata, and then establish a new connection with authorization to each chunkserver to access the data. However, because each file was stored by multiple machines and the data location was independent of the position in the hierarchy, there was very little risk of chunkservers limiting the transfer speed. Sound similar to BitTorrent design.

For you oldsters out there- how does GFS/Collossus compare to Andrew FS? https://en.wikipedia.org/wiki/Andrew_File_System https://en.wikipedia.org/wiki/Andrew_File_System

lizknope 4y ago

I was just about to ask if anyone had used AFS. When I went to college from 1993-97 our university used AFS for all the Unix workstations. We had a mix of Solaris, SunOS, HP-UX, DEC Ultrix, SGI IRIX, and then some Linux boxes. When you logged in at any workstation from any vendor in any computer lab your home dir would be mounted and everything was encrypted. We had ACLs where we could give permission by username and create our own ACL groups and give permission to a project directory to just our project/lab partners. It was so much easier than the chmod uid/gid stuff. In theory we could access other universities under /afs/mit or whatever site they were but I don't think we ever had that active.

ricktdotorg 4y ago
I spent a couple of years at Morgan Stanley and when we cut over from SunOS to Solaris they included a rollout of AFS worldwide. It was pretty successful. It was pretty mind blowing to be able to login to a Solaris workstation while visiting the NYC office and have your [local-to-London] $HOME and CDE environment just kinda sync automatically (albeit slowly back then). The administration of AFS is ... interesting, but considering this was the mid-to-late 90s, it really was "ahead of its time". FWIW, Morgan Stanley back then had an incredible UNIX Engineering department.
- lizknope 4y ago
  I finally got this book a couple of years ago. "MIT Project Athena: A Model for Distributed Campus Computing" https://www.amazon.com/gp/product/0135853249/ https://www.amazon.com/gp/product/0135853249/ I really thought AFS, DCE, and SSI (single system image) were going to take over. It was kind of depressing when I got a job and everything was NIS/NFS and we manually logged into a machine and started a job. Then we got clustering software like LFS to distribute jobs but I thought the dynamic process migration using checkpoint / restart kind of stuff would take over but it seems to be more trouble than it is worth. At work we would always run stuff off a powerful 4 to 16 CPU Sun and display to our cheaper SPARCstation 5 or Linux desktop via X11. I will say it is great now to have a virtual desktop session Exceed/NoMachine/X2go and be able to go home and connect to it and have all my windows in the same place just like when I left. I've got X2go set up at home so even when I travel I can get a desktop at home if I need it. https://en.wikipedia.org/wiki/Project_Athena https://en.wikipedia.org/wiki/Project_Athena https://en.wikipedia.org/wiki/Distributed_Computing_Environment https://en.wikipedia.org/wiki/Distributed_Computing_Environm... https://en.wikipedia.org/wiki/Single_system_image https://en.wikipedia.org/wiki/Single_system_image

thequux 4y ago

The two are very different technologies. In AFS, the fundamental unit of sharding is the volume, each volume is controlled by a single volser, and the "master" (i.e. vldb) is responsible for resolving volume IDs to the set of volsers that hold a copy (at most one of which is read/write) In GFS, on the other hand, individual files are sharded between servers on a chunk level (where 1 block==64MiB IIRC). To access a file's contents, you would contact the master, ask it for the chunkservers for a file, and it would tell you which set of chunkservers were responsible for each chunk in the file; you'd then contact the chunkserver to access the data. The impact of this is that the latency of accessing a file was much higher on GFS; you needed to go through the master (which tended to be a bottleneck) to get the metadata, and then establish a new connection with authorization to each chunkserver to access the data. However, because each file was stored by multiple machines and the data location was independent of the position in the hierarchy, there was very little risk of chunkservers limiting the transfer speed. With AFS, on the other hand, the master is much less of a bottleneck as it's only used when mounting volumes. Further, once a volume is mounted, accessing files and data within those files could be done in a single round trip. However, if you have a volume with high write traffic, that will end up bottlenecking on a single server's IO speed.

lossolo 4y ago
> In GFS, on the other hand, individual files are sharded between servers on a chunk level (where 1 block==64MiB IIRC). To access a file's contents, you would contact the master, ask it for the chunkservers for a file, and it would tell you which set of chunkservers were responsible for each chunk in the file; you'd then contact the chunkserver to access the data. > The impact of this is that the latency of accessing a file was much higher on GFS; you needed to go through the master (which tended to be a bottleneck) to get the metadata, and then establish a new connection with authorization to each chunkserver to access the data. However, because each file was stored by multiple machines and the data location was independent of the position in the hierarchy, there was very little risk of chunkservers limiting the transfer speed. Sound similar to BitTorrent design.