4 ms·

Data-Oriented Design (2018)

55 points by dvfjsdhgfv 3y ago

Chiba-City 3y ago
[dead]
LesZedCB 3y ago
Yehonathan Sharvit wrote a post describing the difference between Data-Oriented Programming, Data-Oriented Design, and Data-Driven Design here. it's a 2 minute read or 15 second skim, very brief. https://blog.klipse.tech/visualization/2021/02/16/data-related-paradigms.html https://blog.klipse.tech/visualization/2021/02/16/data-relat... it includes a nice summary of all of the above which may help when reading the OP.
- morelisp 3y ago
  "Data-oriented programming" is a made-up thing by a few B- tier thought leaders. Biggest scam since "hexagonal architecture".
  - wredue 3y ago
    Instead of addressing the major performance gaps of functional programming, they instead just adopted “data oriented programming” in order to intentionally confuse people and try to drown out the clear and obvious performance pitfalls with data oriented programming.
  - newlisp 3y ago
    Data-oriented programming is a paradigm that treats data of the system as a first-class citizen. Data is represented by generic immutable data structures (like maps and vectors) that are manipulated by general purpose functions (like map, filter, select, group, sort …). CompSci 101, how to represent information as data. Like every industry, lots of bullshitiers.
  - BirdieNZ 3y ago
    What's the issue with hexagonal architecture?
    - morelisp 3y ago
      There's no there there. Maybe even less than DOP.
      - Cthulhu_ 3y ago
        That doesn't explain anything to be honest; what is "there"? why is there no "there" there? Can you explain it like I'm 5?
  - deleted 3y ago
    [deleted]
- michaelteter 3y ago
  This clarification (just definitions) is quite helpful, because probably many readers (myself included) would have assumed that Data Oriented Design would be closely related to Data Oriented Programming (the latter effectively being about the design of software systems).
switchbak 3y ago
This is an important set of concepts, but I must say the tone here really makes it hard to tune in. Within the first few paragraphs I've already heard how OOP is bad, not in-line with a high performing architecture, that design patterns were a mistake, etc - ok got it! I think this coming from a game developer makes sense, especially with the obsessive focus on performance. I really appreciate that point of view, especially when I have to optimize something that wasn't considering performance at all. One really does need to architect software with an eye to how it performs from the very beginning, and during development if you want to have something that truly performs well (sometimes you get lucky / you're working on simple software). This reminds me of the recent podcast on Software Unscripted: Things Web Devs Can Learn from Game Devs with Casey Muratori. Similar obsessive focus, similarly off-putting for me (with a dash more arrogance). That said, there are a lot of situations where a single pointed obsession on performance will not result in a successful outcome. Business software is full of these instances. I'd LOVE a world where we elevated the concern for performance higher than it is now, but there's a range of other considerations that include: - readable, simple code and architecture - expressivity and elegance (related to ^) - the ability to find and hire someone to can work on what you've made - flexibility, being able to respond to change - working around political obstacles - working with less capable / motivated team members So: combining the various tradeoffs into an approach that includes performance as a core concern but also doesn't sacrifice too many of the other concerns is key. Doing so with a data oriented mindset is probably a good way to accomplish this, but it's just one angle from which to view the overall solution. Edit: added podcast mention
- morelisp 3y ago
  But data-oriented design usually does result in much simpler designs, readable code, smaller architectures, and easier composition. What it doesn't do is let some bootcamper pick up some A-B-C-D-E pipeline (whether that's literally some report generation pipeline, or an HTTP ingress to a CRUD DB, or whatever) and plan their whole career around (some subset of) just B. Or let some serial entrepreneur cycle through a cadre of these guys as he burns them out in turn, because to pick the right choices for B you need to understand (and be able to influence the design esp. re. data representations in) A and E. Nothing actually lets you do that and still produce good software, and we should stop pretending otherwise.
dang 3y ago
Reposts count as dupes when the topic has had significant attention in the last year or so (see https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html), which this one did: Data-Oriented Design (2018) - https://news.ycombinator.com/item?id=36571110 https://news.ycombinator.com/item?id=36571110 - July 2023 (133 comments) and in any case it looks like there's a related ongoing thread: Data-Oriented Design (2018) - https://news.ycombinator.com/item?id=38547737 https://news.ycombinator.com/item?id=38547737 - Dec 2023 (11 comments) -- Other past threads, if anyone's curious: The data-oriented design process for game development - https://news.ycombinator.com/item?id=31510432 https://news.ycombinator.com/item?id=31510432 - May 2022 (33 comments) Data Oriented Design – first chapter (2018) - https://news.ycombinator.com/item?id=29292868 https://news.ycombinator.com/item?id=29292868 - Nov 2021 (19 comments) Data-Oriented Design (2018) - https://news.ycombinator.com/item?id=20380397 https://news.ycombinator.com/item?id=20380397 - July 2019 (37 comments) Data-Oriented Design - https://news.ycombinator.com/item?id=12168828 https://news.ycombinator.com/item?id=12168828 - July 2016 (5 comments) Data-Oriented Design (2013) - https://news.ycombinator.com/item?id=11064762 https://news.ycombinator.com/item?id=11064762 - Feb 2016 (8 comments) Data-Oriented Design - https://news.ycombinator.com/item?id=6672854 https://news.ycombinator.com/item?id=6672854 - Nov 2013 (2 comments)
crabmusket 3y ago
This book helped me understand the similarity between the entity-component-system architecture popular in game engines and the relational database model. Lots of ECS libraries are building fast in-memory relational databases with code-based instead of text-based query DSLs.

55 points by dvfjsdhgfv 3y ago

[dead]

Yehonathan Sharvit wrote a post describing the difference between Data-Oriented Programming, Data-Oriented Design, and Data-Driven Design here. it's a 2 minute read or 15 second skim, very brief. https://blog.klipse.tech/visualization/2021/02/16/data-related-paradigms.html https://blog.klipse.tech/visualization/2021/02/16/data-relat... it includes a nice summary of all of the above which may help when reading the OP.

morelisp 3y ago
"Data-oriented programming" is a made-up thing by a few B- tier thought leaders. Biggest scam since "hexagonal architecture".
- wredue 3y ago
  Instead of addressing the major performance gaps of functional programming, they instead just adopted “data oriented programming” in order to intentionally confuse people and try to drown out the clear and obvious performance pitfalls with data oriented programming.
- newlisp 3y ago
  Data-oriented programming is a paradigm that treats data of the system as a first-class citizen. Data is represented by generic immutable data structures (like maps and vectors) that are manipulated by general purpose functions (like map, filter, select, group, sort …). CompSci 101, how to represent information as data. Like every industry, lots of bullshitiers.
- BirdieNZ 3y ago
  What's the issue with hexagonal architecture?
  - morelisp 3y ago
    There's no there there. Maybe even less than DOP.
    - Cthulhu_ 3y ago
      That doesn't explain anything to be honest; what is "there"? why is there no "there" there? Can you explain it like I'm 5?
- deleted 3y ago
  [deleted]
michaelteter 3y ago
This clarification (just definitions) is quite helpful, because probably many readers (myself included) would have assumed that Data Oriented Design would be closely related to Data Oriented Programming (the latter effectively being about the design of software systems).

switchbak 3y ago

This is an important set of concepts, but I must say the tone here really makes it hard to tune in. Within the first few paragraphs I've already heard how OOP is bad, not in-line with a high performing architecture, that design patterns were a mistake, etc - ok got it! I think this coming from a game developer makes sense, especially with the obsessive focus on performance. I really appreciate that point of view, especially when I have to optimize something that wasn't considering performance at all. One really does need to architect software with an eye to how it performs from the very beginning, and during development if you want to have something that truly performs well (sometimes you get lucky / you're working on simple software). This reminds me of the recent podcast on Software Unscripted: Things Web Devs Can Learn from Game Devs with Casey Muratori. Similar obsessive focus, similarly off-putting for me (with a dash more arrogance). That said, there are a lot of situations where a single pointed obsession on performance will not result in a successful outcome. Business software is full of these instances. I'd LOVE a world where we elevated the concern for performance higher than it is now, but there's a range of other considerations that include: - readable, simple code and architecture - expressivity and elegance (related to ^) - the ability to find and hire someone to can work on what you've made - flexibility, being able to respond to change - working around political obstacles - working with less capable / motivated team members So: combining the various tradeoffs into an approach that includes performance as a core concern but also doesn't sacrifice too many of the other concerns is key. Doing so with a data oriented mindset is probably a good way to accomplish this, but it's just one angle from which to view the overall solution. Edit: added podcast mention

morelisp 3y ago
But data-oriented design usually does result in much simpler designs, readable code, smaller architectures, and easier composition. What it doesn't do is let some bootcamper pick up some A-B-C-D-E pipeline (whether that's literally some report generation pipeline, or an HTTP ingress to a CRUD DB, or whatever) and plan their whole career around (some subset of) just B. Or let some serial entrepreneur cycle through a cadre of these guys as he burns them out in turn, because to pick the right choices for B you need to understand (and be able to influence the design esp. re. data representations in) A and E. Nothing actually lets you do that and still produce good software, and we should stop pretending otherwise.

dang 3y ago

Reposts count as dupes when the topic has had significant attention in the last year or so (see https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html), which this one did: Data-Oriented Design (2018) - https://news.ycombinator.com/item?id=36571110 https://news.ycombinator.com/item?id=36571110 - July 2023 (133 comments) and in any case it looks like there's a related ongoing thread: Data-Oriented Design (2018) - https://news.ycombinator.com/item?id=38547737 https://news.ycombinator.com/item?id=38547737 - Dec 2023 (11 comments) -- Other past threads, if anyone's curious: The data-oriented design process for game development - https://news.ycombinator.com/item?id=31510432 https://news.ycombinator.com/item?id=31510432 - May 2022 (33 comments) Data Oriented Design – first chapter (2018) - https://news.ycombinator.com/item?id=29292868 https://news.ycombinator.com/item?id=29292868 - Nov 2021 (19 comments) Data-Oriented Design (2018) - https://news.ycombinator.com/item?id=20380397 https://news.ycombinator.com/item?id=20380397 - July 2019 (37 comments) Data-Oriented Design - https://news.ycombinator.com/item?id=12168828 https://news.ycombinator.com/item?id=12168828 - July 2016 (5 comments) Data-Oriented Design (2013) - https://news.ycombinator.com/item?id=11064762 https://news.ycombinator.com/item?id=11064762 - Feb 2016 (8 comments) Data-Oriented Design - https://news.ycombinator.com/item?id=6672854 https://news.ycombinator.com/item?id=6672854 - Nov 2013 (2 comments)

crabmusket 3y ago

This book helped me understand the similarity between the entity-component-system architecture popular in game engines and the relational database model. Lots of ECS libraries are building fast in-memory relational databases with code-based instead of text-based query DSLs.