5 ms·

I remember reading this book way back when. It is a great introduction to file system design. HOWEVER, the core thesis of BFS is basically "Let's see what happe

by kobeya 9y ago

I remember reading this book way back when. It is a great introduction to file system design. HOWEVER, the core thesis of BFS is basically "Let's see what happens when we make a filesystem that is more like a database." The TL;DR of the BeOS experience is that this is in fact a terrific idea, if you don't care about backwards (POSIX) compatibility. The book is terribly dated though and I'm not sure I would recommend it to anyone except as a historical reference. It lacks much of the latest file system research that has gone into things like ZFS and Btrfs. If you want to go the BFS-like route, then I recommend just accepting the premise and getting a book or other resource on DBMS design, getting one on modern file system design, and then merging the two yourself.

myrandomcomment 9y ago
The filesystem for an AS/400 box from IBM is basically DB2.
- todd8 9y ago
  The predecessor of the AS/400, the IBM System/38, was a really very radical architecture for 1978/1979 it had capability based addressing and a file system based on Relational Database concepts. Some of the same leadership that ran that project (Glen Henry) was also respononible for IBM's adoption of Unix. This IBM Unix system, AIX (1986), was designed to run on another interesting hardware platform, the earliest generation of IBM's RISC hardware PowerPC. Which in turn evolved from IBM's (1975) 801 system (John Cocke--who won the Turing Award in 1987 for invention of RISC) was responsible for the 801. I remember him coming into my office to talk to me about some ideas he had for a high capacity disk drive around 1987 when I was working on AIX.
adamnemecek 9y ago
WinFS was a similar idea https://en.wikipedia.org/wiki/WinFS https://en.wikipedia.org/wiki/WinFS
- smacktoward 9y ago
  Yeah, "what if the file system was a database?" was a very common line of thought in the late '80s/early '90s -- see BFS, WinFS/Cairo, Apple's "soups" for the Newton PDA (https://en.wikipedia.org/wiki/Soup_(Apple) https://en.wikipedia.org/wiki/Soup_(Apple)), Palm OS' virtual file system (https://www.netmeister.org/palm/PalmMisc/PalmMisc.html#1.7 https://www.netmeister.org/palm/PalmMisc/PalmMisc.html#1.7), et al.
  - bitexploder 9y ago
    You could just feed Oracle DB raw filesystem partitions back in the day. Don't know if that is still possible.
    - kalleboo 9y ago
      InnoDB (MySQL) still supports it
    - yellowapple 9y ago
      I'm curious about whether SQLite could do something similar.
- WalterGR 9y ago
  WinFS wasn't a file-system. Its data store was in files on NTFS. Further, if my memory serves, every part of WinFS ran in user mode.
userbinator 9y ago
if you don't care about backwards (POSIX) compatibility. ...nor overall performance, as the benchmarks in chapter 9 show. Apparently BFS is very good at sequential accesses (but then, so is FAT... which has probably become the most popular filesystem in use, especially in embedded devices) but performs poorly for a lot of other operations. This book also reaffirms a common adage that books with "theoretical" in their title tend to be very practical, and vice-versa... but then again, it was written at a time when a lot of people were betting on BeOS being The Next Big Thing and they never expected the current dominance of Linux and Windows. Ultimately, it seems the "keep the filesystem simple and build further abstractions inside files" design has lead to the greatest success, because it's more flexible and allows more interoperability --- building too much "intelligence" into the FS, proprietary or otherwise, makes it harder to exchange data with other systems, whereas you can e.g. move or copy a file containing a relational database between systems and operate on it more easily. Filesystem drivers are usually implemented in the OS kernel, and so it makes sense to keep them minimal from a reliability and security perspective.
- pvg 9y ago
  A few people might have been betting or at least hoping BeOS becomes popular, not so sure about 'next big thing'. Windows was already dominant when BeOS popped up.
  - twoodfin 9y ago
    IIRC much of the upside for alternative OS's at the time was premised on the not outrageous possibility that Microsoft would be broken up and more or less forced to port Office everywhere.
    - kobeya 9y ago
      Also BeOS was wooing Apple to be the replacement for the pre-X macOS. It would have been too, if Jobs hadn't returned and decided on NeXT instead.
      - 9y ago
bogomipz 9y ago
>"It lacks much of the latest file system research that has gone into things like ZFS and Btrfs." Could you or anyone else recommend a book or other single resource about filesystem design that does include more recent research and concepts?
- pkaye 9y ago
  Not exactly what you need but "File System Forensic Analysis by Brian Carrier" goes over a lot of the internal data structures on common filesystem. It is a bit older book so it mainly covers fat, ext2fs and ntfs.
mamcx 9y ago
This is still true in the era of SSD?
- kobeya 9y ago
  Even SSDs need file systems.
  - josefdlange 9y ago
    Pshaw, just make it a general-purpose K/V store and that definitely will of course solve all problems no need for any kind of datastore. /s
Ice_cream_suit 9y ago
"It lacks much of the latest file system research that has gone into things like ZFS and Btrfs." Could you recommend a recent book on file system design ? thanks.

by kobeya 9y ago

myrandomcomment 9y ago

The filesystem for an AS/400 box from IBM is basically DB2.

todd8 9y ago
The predecessor of the AS/400, the IBM System/38, was a really very radical architecture for 1978/1979 it had capability based addressing and a file system based on Relational Database concepts. Some of the same leadership that ran that project (Glen Henry) was also respononible for IBM's adoption of Unix. This IBM Unix system, AIX (1986), was designed to run on another interesting hardware platform, the earliest generation of IBM's RISC hardware PowerPC. Which in turn evolved from IBM's (1975) 801 system (John Cocke--who won the Turing Award in 1987 for invention of RISC) was responsible for the 801. I remember him coming into my office to talk to me about some ideas he had for a high capacity disk drive around 1987 when I was working on AIX.

adamnemecek 9y ago

WinFS was a similar idea https://en.wikipedia.org/wiki/WinFS https://en.wikipedia.org/wiki/WinFS

smacktoward 9y ago
Yeah, "what if the file system was a database?" was a very common line of thought in the late '80s/early '90s -- see BFS, WinFS/Cairo, Apple's "soups" for the Newton PDA (https://en.wikipedia.org/wiki/Soup_(Apple) https://en.wikipedia.org/wiki/Soup_(Apple)), Palm OS' virtual file system (https://www.netmeister.org/palm/PalmMisc/PalmMisc.html#1.7 https://www.netmeister.org/palm/PalmMisc/PalmMisc.html#1.7), et al.
- bitexploder 9y ago
  You could just feed Oracle DB raw filesystem partitions back in the day. Don't know if that is still possible.
  - kalleboo 9y ago
    InnoDB (MySQL) still supports it
  - yellowapple 9y ago
    I'm curious about whether SQLite could do something similar.
WalterGR 9y ago
WinFS wasn't a file-system. Its data store was in files on NTFS. Further, if my memory serves, every part of WinFS ran in user mode.

userbinator 9y ago

if you don't care about backwards (POSIX) compatibility. ...nor overall performance, as the benchmarks in chapter 9 show. Apparently BFS is very good at sequential accesses (but then, so is FAT... which has probably become the most popular filesystem in use, especially in embedded devices) but performs poorly for a lot of other operations. This book also reaffirms a common adage that books with "theoretical" in their title tend to be very practical, and vice-versa... but then again, it was written at a time when a lot of people were betting on BeOS being The Next Big Thing and they never expected the current dominance of Linux and Windows. Ultimately, it seems the "keep the filesystem simple and build further abstractions inside files" design has lead to the greatest success, because it's more flexible and allows more interoperability --- building too much "intelligence" into the FS, proprietary or otherwise, makes it harder to exchange data with other systems, whereas you can e.g. move or copy a file containing a relational database between systems and operate on it more easily. Filesystem drivers are usually implemented in the OS kernel, and so it makes sense to keep them minimal from a reliability and security perspective.

pvg 9y ago
A few people might have been betting or at least hoping BeOS becomes popular, not so sure about 'next big thing'. Windows was already dominant when BeOS popped up.
- twoodfin 9y ago
  IIRC much of the upside for alternative OS's at the time was premised on the not outrageous possibility that Microsoft would be broken up and more or less forced to port Office everywhere.
  - kobeya 9y ago
    Also BeOS was wooing Apple to be the replacement for the pre-X macOS. It would have been too, if Jobs hadn't returned and decided on NeXT instead.
    - 9y ago

bogomipz 9y ago

>"It lacks much of the latest file system research that has gone into things like ZFS and Btrfs." Could you or anyone else recommend a book or other single resource about filesystem design that does include more recent research and concepts?

pkaye 9y ago
Not exactly what you need but "File System Forensic Analysis by Brian Carrier" goes over a lot of the internal data structures on common filesystem. It is a bit older book so it mainly covers fat, ext2fs and ntfs.

mamcx 9y ago

This is still true in the era of SSD?

kobeya 9y ago
Even SSDs need file systems.
- josefdlange 9y ago
  Pshaw, just make it a general-purpose K/V store and that definitely will of course solve all problems no need for any kind of datastore. /s

Ice_cream_suit 9y ago

"It lacks much of the latest file system research that has gone into things like ZFS and Btrfs." Could you recommend a recent book on file system design ? thanks.