4 ms·

I was just discussing with a colleague how technology accretes and how no one reevaluates high-level design decisions even after every single factor leading to

I was just discussing with a colleague how technology accretes and how no one reevaluates high-level design decisions even after every single factor leading to those decisions has changed. It's weird that basic filesystems today are so out of touch with modern realities that we are universally forced to resort to using complex databases even in cases when the logical model of files and directories fits the storage needs really well. It's weird that hierarchical storage is the only universal model available on all OSes and in all languages. The more I think about it, the more I realize that we live in a bizarro world where software runs everything, yet makes little to no sense from either human, or modern hardware or system design perspectives.

kragen 7y ago
I'm interested to hear what kinds of filesystem semantics you think might strike a better balance!
nostrademons 7y ago
Programming languages are in the same boat - the modern CPU works dramatically different from a PDP-11, but most programming models still assume an accumulator machine; flat memory hierarchy; effectively unbounded LIFO program-control stacks; uniform machine word sizes; sequential in-order execution; and byte streams for I/O. This is despite large register files, multiple levels of caching, coroutines/promises, SIMD, multicore/GPU, and page-level I/O being things. In many cases even assembly language encodes assumptions from the 1970s, and then is internally translated by the processor into how the hardware actually works.
- adfkbuvasdas 7y ago
  It’s not a totally fair analogy though. Designing file systems for newer storage devices is an engineering problem that requires a lot of effort, but not a great amount of new insight. Designing a new programming language that somehow exposes the concepts you mentioned more transparently while still being actually useful would require major, major breakthroughs and insights in PL design.
- h0l0cube 7y ago
  I wonder what your thoughts are on the Mill Architecture? It's throwing out every assumption on how to build a CPU, which mandates that the compiler needs to be rewritten to generate code for it.
  - nostrademons 7y ago
    I hadn't heard of it before. I just looked it up and it looks interesting, but I don't have enough hardware engineering experience to judge its feasibility. I think a larger problem with new architectures is that consumer adoption follows price/performance/power, not any inherent architectural quality. The architecture we're stuck with is the one that most hardware devices get sold with (x86/ARM right now). That gets determined by hardware OEMs, which in turn make their decisions based on what'll help them sell the fastest devices with the lowest power consumption for the least money. So something like RISC V is fascinating and quite elegant, but until there are RISC V chips that are cheaper and faster than Intel ones, it remains an academic curiosity. Then if you're a compiler writer, you gotta work with what you've got for an installed base, and you can't really get adoption for a new language unless it lets startup founders unlock new markets because your combination of development velocity + execution speed lets them do things they wouldn't otherwise be able to.
    - h0l0cube 7y ago
      Ah yeah, I'm just as cynical in all the ways you are about the prospects of a 'new architecture' becoming relevant. Just thought you might have known Mills + had some personal insights on the project.
- pkaye 7y ago
  So how would you envision a modern programming language to look like accounting for all these differences?
  - nostrademons 7y ago
    That's the subject of a research project I don't have time for, but some ideas: 1.) automatically defining SoA/AoS transformations for data types in the language, and 2.) automatic profiling of code execution counts & array lengths to determine when inserting such a transformation would save more time than the transformation costs. 3.) native SIMD primitive types 4.) GPU backend a la Futhark 5.) native syntax for parallel execution, where the compiler enforces no data sharing (or read-only shared data) between executions. 6.) [more speculative] pluggable backend for this syntax so that you could run these concurrent executions sequentially, using SIMD instructions on the same thread, in multiple threads, on the GPU, or spread across multiple machines in a cluster without altering the code. That'd let you scale up from a few K to a few T without altering your code. 7.) I/O mechanisms that default to mmap, 0-copy vectored I/O, or RDMA as appropriate. 8.) An unfold primitive in either the language or standard library that lets you compute multiple values with a single traversal over a list, ideally while still keeping each individual computation isolated and modular. I find this is a really common pattern in my Kotlin code, and the biggest culprit for still writing manual accumulator loops. 9.) Support for multi-pass initialization in data structures, where you can hold some fields as uninitialized for a later computation, but mark the whole structure as immutable once every field has been computed. 10.) A standard library that uses these parallelization techniques under the hood so you get really good speed for common operations but it just works. For example, Boyer-Moore can be done on the GPU. Many common regexps can be reduced to string search on a common left prefix + a DFA on potential matches, where the prefix search can be done with Boyer-Moore. That opens the possibility of blazingly fast search over gigabytes of data for common regexps like /<a href=["']?([^"'\s])["']?>(.+?)<\/a>/. Many times the algorithms you'd use when your haystack is ~1G are very different from the algorithms you'd use when your haystack is ~1K, which goes back to the utility of an integrated profiler. It occurs to me that a lot of these boil down to the inefficiency of doing lots of traversals of arrays in a world where touching memory is usually the critical path. So for example, this is slow but modular: val high = oneBillionFloats.max() val low = oneBillionFloats.min() val total = oneBillionFloats.sum() val average = oneBillionFloats.sum() / oneBillionFloats.size() This is fast but coupled: var high = Float.MIN_VALUE var low = Float.MAX_VALUE var total = 0 var count = 0 oneBillionFloats.forEach { high = maxOf(high, it) low = minOf(low, it) total += it ++count } val average = total / count Wouldn't it be nice if you could write your code like the first example but it would generate a single loop that's parallelized onto the GPU? Even moreso if oneBillionFloats isn't an array of a billion primitives, but rather a gig full of deeply nested structs, where each loop pulls out a particular field and does some arithmetic operations to it?
avar 7y ago
That's really not the case. Plenty of storage vendors will sell you storage exposed as an FS that scales wonderfully, provided you follow some sane best-practices like e.g. sharding your directories. The same goes for storing things in a database and not running into certain pathological cases with indexes.
toolslive 7y ago
it's not just file systems. Take a look at the NVMe specifications and then answer this question: "Do you think it's efficient to use this as a block device?"
sidpatil 7y ago
> I was just discussing with a colleague how technology accretes and how no one reevaluates high-level design decisions even after every single factor leading to those decisions has changed. Like a Ship of Theseus...
pfranz 7y ago
That happens all the time in real life. The whole thing reminds me about the disputed articles, "Are U.S. Railroad Gauges Based on Roman Chariots?" and by extension the Space Shuttle was limited by the width of two horses walking side-by-side. I don't think software is all that different than other standards like switching to the metric system, or Southern California switching their power standard from 50hz in 1948 to match 60hz (Japan never unified their power). It's a huge investment and a lot of it is political appetite for adopting the change. It seems pretty obvious that the average person doesn't grok/like using folders (everything goes on the desktop). I was excited when Microsoft and Apple seemed experimental in the late 90's early 2000's with WinFS and Smart Folders. That kind of fizzled out. I think the reliability prevents adoption and it's a bit of a chicken and the egg with architecture vs presentation. Gmail supporting tags instead of folders seemed to drive everyone nuts for awhile (it still seems to come up when exposing it to other clients). https://www.snopes.com/fact-check/railroad-gauge-chariots/ https://www.snopes.com/fact-check/railroad-gauge-chariots/

kragen 7y ago

I'm interested to hear what kinds of filesystem semantics you think might strike a better balance!

nostrademons 7y ago

Programming languages are in the same boat - the modern CPU works dramatically different from a PDP-11, but most programming models still assume an accumulator machine; flat memory hierarchy; effectively unbounded LIFO program-control stacks; uniform machine word sizes; sequential in-order execution; and byte streams for I/O. This is despite large register files, multiple levels of caching, coroutines/promises, SIMD, multicore/GPU, and page-level I/O being things. In many cases even assembly language encodes assumptions from the 1970s, and then is internally translated by the processor into how the hardware actually works.

adfkbuvasdas 7y ago
It’s not a totally fair analogy though. Designing file systems for newer storage devices is an engineering problem that requires a lot of effort, but not a great amount of new insight. Designing a new programming language that somehow exposes the concepts you mentioned more transparently while still being actually useful would require major, major breakthroughs and insights in PL design.
h0l0cube 7y ago
I wonder what your thoughts are on the Mill Architecture? It's throwing out every assumption on how to build a CPU, which mandates that the compiler needs to be rewritten to generate code for it.
- nostrademons 7y ago
  I hadn't heard of it before. I just looked it up and it looks interesting, but I don't have enough hardware engineering experience to judge its feasibility. I think a larger problem with new architectures is that consumer adoption follows price/performance/power, not any inherent architectural quality. The architecture we're stuck with is the one that most hardware devices get sold with (x86/ARM right now). That gets determined by hardware OEMs, which in turn make their decisions based on what'll help them sell the fastest devices with the lowest power consumption for the least money. So something like RISC V is fascinating and quite elegant, but until there are RISC V chips that are cheaper and faster than Intel ones, it remains an academic curiosity. Then if you're a compiler writer, you gotta work with what you've got for an installed base, and you can't really get adoption for a new language unless it lets startup founders unlock new markets because your combination of development velocity + execution speed lets them do things they wouldn't otherwise be able to.
  - h0l0cube 7y ago
    Ah yeah, I'm just as cynical in all the ways you are about the prospects of a 'new architecture' becoming relevant. Just thought you might have known Mills + had some personal insights on the project.
pkaye 7y ago
So how would you envision a modern programming language to look like accounting for all these differences?
- nostrademons 7y ago
  That's the subject of a research project I don't have time for, but some ideas: 1.) automatically defining SoA/AoS transformations for data types in the language, and 2.) automatic profiling of code execution counts & array lengths to determine when inserting such a transformation would save more time than the transformation costs. 3.) native SIMD primitive types 4.) GPU backend a la Futhark 5.) native syntax for parallel execution, where the compiler enforces no data sharing (or read-only shared data) between executions. 6.) [more speculative] pluggable backend for this syntax so that you could run these concurrent executions sequentially, using SIMD instructions on the same thread, in multiple threads, on the GPU, or spread across multiple machines in a cluster without altering the code. That'd let you scale up from a few K to a few T without altering your code. 7.) I/O mechanisms that default to mmap, 0-copy vectored I/O, or RDMA as appropriate. 8.) An unfold primitive in either the language or standard library that lets you compute multiple values with a single traversal over a list, ideally while still keeping each individual computation isolated and modular. I find this is a really common pattern in my Kotlin code, and the biggest culprit for still writing manual accumulator loops. 9.) Support for multi-pass initialization in data structures, where you can hold some fields as uninitialized for a later computation, but mark the whole structure as immutable once every field has been computed. 10.) A standard library that uses these parallelization techniques under the hood so you get really good speed for common operations but it just works. For example, Boyer-Moore can be done on the GPU. Many common regexps can be reduced to string search on a common left prefix + a DFA on potential matches, where the prefix search can be done with Boyer-Moore. That opens the possibility of blazingly fast search over gigabytes of data for common regexps like /<a href=["']?([^"'\s])["']?>(.+?)<\/a>/. Many times the algorithms you'd use when your haystack is ~1G are very different from the algorithms you'd use when your haystack is ~1K, which goes back to the utility of an integrated profiler. It occurs to me that a lot of these boil down to the inefficiency of doing lots of traversals of arrays in a world where touching memory is usually the critical path. So for example, this is slow but modular: val high = oneBillionFloats.max() val low = oneBillionFloats.min() val total = oneBillionFloats.sum() val average = oneBillionFloats.sum() / oneBillionFloats.size() This is fast but coupled: var high = Float.MIN_VALUE var low = Float.MAX_VALUE var total = 0 var count = 0 oneBillionFloats.forEach { high = maxOf(high, it) low = minOf(low, it) total += it ++count } val average = total / count Wouldn't it be nice if you could write your code like the first example but it would generate a single loop that's parallelized onto the GPU? Even moreso if oneBillionFloats isn't an array of a billion primitives, but rather a gig full of deeply nested structs, where each loop pulls out a particular field and does some arithmetic operations to it?

avar 7y ago

That's really not the case. Plenty of storage vendors will sell you storage exposed as an FS that scales wonderfully, provided you follow some sane best-practices like e.g. sharding your directories. The same goes for storing things in a database and not running into certain pathological cases with indexes.

toolslive 7y ago

it's not just file systems. Take a look at the NVMe specifications and then answer this question: "Do you think it's efficient to use this as a block device?"

sidpatil 7y ago

> I was just discussing with a colleague how technology accretes and how no one reevaluates high-level design decisions even after every single factor leading to those decisions has changed. Like a Ship of Theseus...

pfranz 7y ago

That happens all the time in real life. The whole thing reminds me about the disputed articles, "Are U.S. Railroad Gauges Based on Roman Chariots?" and by extension the Space Shuttle was limited by the width of two horses walking side-by-side. I don't think software is all that different than other standards like switching to the metric system, or Southern California switching their power standard from 50hz in 1948 to match 60hz (Japan never unified their power). It's a huge investment and a lot of it is political appetite for adopting the change. It seems pretty obvious that the average person doesn't grok/like using folders (everything goes on the desktop). I was excited when Microsoft and Apple seemed experimental in the late 90's early 2000's with WinFS and Smart Folders. That kind of fizzled out. I think the reliability prevents adoption and it's a bit of a chicken and the egg with architecture vs presentation. Gmail supporting tags instead of folders seemed to drive everyone nuts for awhile (it still seems to come up when exposing it to other clients). https://www.snopes.com/fact-check/railroad-gauge-chariots/ https://www.snopes.com/fact-check/railroad-gauge-chariots/