4 ms·

Two talks come to mind here: Mike Acton's Data-Oriented Design and C++ [1] and Brian Cantrill's The Complexity of Simplicity [2]. Mike's talk argues that code

Two talks come to mind here: Mike Acton's Data-Oriented Design and C++ [1] and Brian Cantrill's The Complexity of Simplicity [2]. Mike's talk argues that code solutions need not be modelled on the real world, and that different data creates different problems, which need different solutions. I can't do the talk justice, but it's had a big impact on me. Brian's talk is about abstraction generally, and how it's difficult to find the "right" abstraction. 1. https://www.youtube.com/watch?v=rX0ItVEVjHc https://www.youtube.com/watch?v=rX0ItVEVjHc 2. https://www.youtube.com/watch?v=Cum5uN2634o https://www.youtube.com/watch?v=Cum5uN2634o

saghm 4mo ago
> Mike's talk argues that code solutions need not be modelled on the real world, and that different data creates different problems, which need different solutions. I've always found it odd when even fairly smart engineers sometimes prioritize real-world metaphors over the actual needs of the codebase. Years ago when I was only a few years out of school, I was implementing a connection pool in Rust, and the most reasonable way to implement it was to have the connection hold a weak reference to the pool so that it could get checked back in automatically when dropped. My manager (an extremely experienced engineer) didn't like this idea because "a library holds library books, not the other way around". I didn't feel like this was a compelling reason to design things differently, but he refused to engage with the issue in any way other than through the lens of that metaphor. Eventually the impasse was solved by one of the other managers in my department suggested that while library books don't contain libraries, they do have the name of the library stamped in the back as a reference to where they should be returned, and I guess my manager found this to be a reasonable extension of the analogy. If I were more experienced, maybe I would have recognized that I could find a way to engage with the analogy like the other manager did without ceding the point, but even today I still feel that it was completely bizarre to insist on that as the canonical way to frame things rather than just considering the ramifications of the abstraction in the code and the experience of using the library based on it.
- Rendello 4mo ago
  This is somewhat related: I mention this a lot, but in researching Data-Oriented Design (what Mike was talking about), I came across Richard Fabian's DoD book [1] which talks a lot about database normalization and the like. I found that odd, because the low-level high-performance game code he was talking about certainly wasn't going to marshal data into a DB to run SQL queries on it. It turns out the relational model has a lot of advantages though. Programmers use trees all the time, in OO, in structs containing structs, in objects pointing to other objects. It's easy to forget that trees are just a special case of graphs (ie. networks), and that there are many ways to represent networks that don't rely on encoding a tree structure directly. So, I've been doing what Richard Fabian suggested and I lay out my data (on paper) into tables, then attempt to normalize it and see the connections. I really like this way of designing things. My big issue is that doing DB-like operations is hellish in most programming languages, and if you really want to try and marshal your data into a real DB (say, SQLite or DuckDB via a library), then you have a big messy translation layer where you're trying to match things to SQL types and you have giant SQL strings everywhere. I see C# has LINQ, which is a query languages embedded in the language. I wonder if that approach is best, and why hasn't it been adopted more broadly? It seems like there's a lot for programming language designers to explore in this dimension, though I wonder if it even matters now with the superintelligence tidal wave. 1. https://www.dataorienteddesign.com/dodmain/ https://www.dataorienteddesign.com/dodmain/
  - throwaway2037 4mo ago
    > My big issue is that doing DB-like operations is hellish in most programming languages, and if you really want to try and marshal your data into a real DB (say, SQLite or DuckDB via a library), then you have a big messy translation layer where you're trying to match things to SQL types and you have giant SQL strings everywhere. Have heard of the JOOQ library for Java? It is a godsend because you can write guaranteed type-safe SQL using pure Java -- no syntax sugar. I expect that LINQ can do the same in C#.
    - Rendello 4mo ago
      I read about it just a few days ago! I don't use Java, either, but it looked awesome. Right now I'm using Rust, which feels limited in this capacity outside of ORMs.
lucas_t_a 4mo ago
this one comes to mind https://youtu.be/17KCHwOwgms https://youtu.be/17KCHwOwgms
so-cal-schemer 4mo ago
See also: Data-Oriented Programming: Reduce software complexity by Yehonathan Sharvit https://www.manning.com/books/data-oriented-programming https://www.manning.com/books/data-oriented-programming and from SICP: 2.4.3 Data-Directed Programming and Additivity https://sarabander.github.io/sicp/html/2_002e4.xhtml#g_t2_002e4_002e3 https://sarabander.github.io/sicp/html/2_002e4.xhtml#g_t2_00...
- Rendello 4mo ago
  There's a reason why "naming things" is one of the two hard problems in Computer Science. Data-Oriented Design (DOD) and Data-Oriented Programming (DOP) are two different things which has caused a fair amount of confusion on HN before. Data-Directed Programming (DDP) appears to be a third, different thing. In searching those threads, I came across a post from the author of the DOP book describing the difference between DOD, DOP, and DDP (a different DDP! Data Driven Programing; a fourth thing!) [2], and I see he also made the "naming things" joke in the first paragraph, so I guess my humour isn't that unique! There's been quite a bit of discussion about DOD and some about DOP (and much conflation between the two) on HN, it can be interesting to read [3]. 1. https://martinfowler.com/bliki/TwoHardThings.html https://martinfowler.com/bliki/TwoHardThings.html 2. https://blog.klipse.tech/visualization/2021/02/16/data-related-paradigms.html https://blog.klipse.tech/visualization/2021/02/16/data-relat... 3. https://hn.algolia.com/?q=Data+Oriented https://hn.algolia.com/?q=Data+Oriented

saghm 4mo ago

> Mike's talk argues that code solutions need not be modelled on the real world, and that different data creates different problems, which need different solutions. I've always found it odd when even fairly smart engineers sometimes prioritize real-world metaphors over the actual needs of the codebase. Years ago when I was only a few years out of school, I was implementing a connection pool in Rust, and the most reasonable way to implement it was to have the connection hold a weak reference to the pool so that it could get checked back in automatically when dropped. My manager (an extremely experienced engineer) didn't like this idea because "a library holds library books, not the other way around". I didn't feel like this was a compelling reason to design things differently, but he refused to engage with the issue in any way other than through the lens of that metaphor. Eventually the impasse was solved by one of the other managers in my department suggested that while library books don't contain libraries, they do have the name of the library stamped in the back as a reference to where they should be returned, and I guess my manager found this to be a reasonable extension of the analogy. If I were more experienced, maybe I would have recognized that I could find a way to engage with the analogy like the other manager did without ceding the point, but even today I still feel that it was completely bizarre to insist on that as the canonical way to frame things rather than just considering the ramifications of the abstraction in the code and the experience of using the library based on it.

Rendello 4mo ago
This is somewhat related: I mention this a lot, but in researching Data-Oriented Design (what Mike was talking about), I came across Richard Fabian's DoD book [1] which talks a lot about database normalization and the like. I found that odd, because the low-level high-performance game code he was talking about certainly wasn't going to marshal data into a DB to run SQL queries on it. It turns out the relational model has a lot of advantages though. Programmers use trees all the time, in OO, in structs containing structs, in objects pointing to other objects. It's easy to forget that trees are just a special case of graphs (ie. networks), and that there are many ways to represent networks that don't rely on encoding a tree structure directly. So, I've been doing what Richard Fabian suggested and I lay out my data (on paper) into tables, then attempt to normalize it and see the connections. I really like this way of designing things. My big issue is that doing DB-like operations is hellish in most programming languages, and if you really want to try and marshal your data into a real DB (say, SQLite or DuckDB via a library), then you have a big messy translation layer where you're trying to match things to SQL types and you have giant SQL strings everywhere. I see C# has LINQ, which is a query languages embedded in the language. I wonder if that approach is best, and why hasn't it been adopted more broadly? It seems like there's a lot for programming language designers to explore in this dimension, though I wonder if it even matters now with the superintelligence tidal wave. 1. https://www.dataorienteddesign.com/dodmain/ https://www.dataorienteddesign.com/dodmain/
- throwaway2037 4mo ago
  > My big issue is that doing DB-like operations is hellish in most programming languages, and if you really want to try and marshal your data into a real DB (say, SQLite or DuckDB via a library), then you have a big messy translation layer where you're trying to match things to SQL types and you have giant SQL strings everywhere. Have heard of the JOOQ library for Java? It is a godsend because you can write guaranteed type-safe SQL using pure Java -- no syntax sugar. I expect that LINQ can do the same in C#.
  - Rendello 4mo ago
    I read about it just a few days ago! I don't use Java, either, but it looked awesome. Right now I'm using Rust, which feels limited in this capacity outside of ORMs.

lucas_t_a 4mo ago

this one comes to mind https://youtu.be/17KCHwOwgms https://youtu.be/17KCHwOwgms

so-cal-schemer 4mo ago

See also: Data-Oriented Programming: Reduce software complexity by Yehonathan Sharvit https://www.manning.com/books/data-oriented-programming https://www.manning.com/books/data-oriented-programming and from SICP: 2.4.3 Data-Directed Programming and Additivity https://sarabander.github.io/sicp/html/2_002e4.xhtml#g_t2_002e4_002e3 https://sarabander.github.io/sicp/html/2_002e4.xhtml#g_t2_00...

Rendello 4mo ago
There's a reason why "naming things" is one of the two hard problems in Computer Science. Data-Oriented Design (DOD) and Data-Oriented Programming (DOP) are two different things which has caused a fair amount of confusion on HN before. Data-Directed Programming (DDP) appears to be a third, different thing. In searching those threads, I came across a post from the author of the DOP book describing the difference between DOD, DOP, and DDP (a different DDP! Data Driven Programing; a fourth thing!) [2], and I see he also made the "naming things" joke in the first paragraph, so I guess my humour isn't that unique! There's been quite a bit of discussion about DOD and some about DOP (and much conflation between the two) on HN, it can be interesting to read [3]. 1. https://martinfowler.com/bliki/TwoHardThings.html https://martinfowler.com/bliki/TwoHardThings.html 2. https://blog.klipse.tech/visualization/2021/02/16/data-related-paradigms.html https://blog.klipse.tech/visualization/2021/02/16/data-relat... 3. https://hn.algolia.com/?q=Data+Oriented https://hn.algolia.com/?q=Data+Oriented