11 ms·

Databases are the endgame for data-oriented design

238 points by cloutiertyler 3y ago

jitl 3y ago
If the site designers are on here by chance, take a look on iPhone. The blog post is quite squished there, pretty much unreadable. EDIT: nevermind, fixed after a refresh ¯\_(ツ)_/¯ still maybe worth a look
- cloutiertyler 3y ago
  I am on here. Will definitely take a look!
  - vagrantJin 3y ago
    another thing. firefox issue. firefox on android no longer supports mobile view?
    - cloutiertyler 3y ago
      Will fix. Writing all these down.
  - marshray 3y ago
    While you're here... where's the scrollbar? I'm on desktop (Edge) and don't see a scrollbar. Very distracting.
    - cloutiertyler 3y ago
      Will fix. Writing all these down.
samsquire 3y ago
Thank you, I enjoyed this post. I am thinking that iteration is just traversal and traversal is just execution. Take iterators in C++ or standard library "algorithms" library in C++. Or iterators in Rust or in Java. You just want to traverse and collect values (and calls on functions in some interleaving), which is like joining tables as this article says. I'm thinking that the definition of the traversal (such as Kafka pipeline, clojure threading syntax, Clojure data driven design, Javascript lodash pipeline) can be mapped to tuples and then the computer can optimise arbitrary chains of traversals based on the number of potential tuples are available and what traversals are equivalent paths or routes to the same traversal. In other words, every program is a compiler pipeline or database query pipeline. Maybe Prolog and Datalog can help here. Optimisation of arbitrary traversals and determination of identical traversals. Or monads are just ordered traversals and OCaml compilers are just traversals of execution (function application) and relationship following, which are joins. The relational model is everywhere.
- BoiledCabbage 3y ago
  > In other words, every program is a compiler pipeline or database query pipeline. > Or monads are just ordered traversals and OCaml compilers are just traversals of execution (function application) and relationship following, which are joins. > The relational model is everywhere. This is interesting - could you elaborate a bit more?
- deleted 3y ago
  [deleted]
fifilura 3y ago
My colleagues hate me, but I also found that SQL is The way to write business logic. Lots of caveats about difficulty to test and weird syntax. But it is just that SQL is the most terse and standard way so express logic. And that in itself is the most important factor to avoid bugs. Not what testing strategy you choose.
- Etheryte 3y ago
  I feel like you've come to the right conclusion, but partly for the wrong reasons. Terseness in and of itself is not useful per se. Code golfing languages are the tersest there is, but we don't write code that way.
  - ndriscoll 3y ago
    Or they just used the wrong word. I'd agree with them, but clarify that SQL is not just terse, but concise. A join or a group by is going to be much clearer than writing the code the query plan is going to generate, creating temporary hash maps, doing nested loops, etc.
    - ttfkam 3y ago
      Yep, because unlike general purpose programming languages, you describe WHAT data you want from SQL, not HOW to get that data as in most application languages.
      - fifilura 3y ago
        Exactly. And yes, maybe concise was the word I meant to use (not native English speaker).
- randomdata 3y ago
  > And that in itself is the most important factor to avoid bugs. I'm not sure about that. SQL was the first language I learned and the language that has always been there throughout the decades, and is also where I make the most mistakes.
paulddraper 3y ago
The relational model is one of those eternal diamonds that will be just as relevant in 2090 as in 1990.
- riku_iki 3y ago
  I kinda think adding some logical/semantic level on top of it (like datalog) will make it even more robust.
  - vacuity 3y ago
    Datalog (intentionally) isn't Turing-complete, though. That might hurt applicability for a generalized ECS model.
- slooonz 3y ago
  The biggest problem with the relational model is that it’s buried under the dirt that is SQL.
hitpointdrew 3y ago
Nope, nope and nope. Went to the github page for spacetimedb, it does everything that is terrible. >Instead of deploying a web or game server that sits in between your clients and your database, your clients connect directly to the database and execute your application logic inside the database itself. You can write all of your permission and authorization logic right inside your module just as you would in a normal server. Why? Databases should never, ever, ever, be used to perform logic, they are datastores, that is it. Your logic goes elsewhere. Stored procedures are the worst "feature" of any database, you are just asking for a hard time debugging, troubleshooting, and increasing the chance that you will fuck up the most valuable part of your system, your data. > This means that you can write your entire application in a single language, Rust, and deploy it as a single binary. It also means you have a single point of failure, no read-replicas or redundancy. Hate everything about this.
- riku_iki 3y ago
  > Why? less moving parts, diversity and complexity in your stack (different languages, servers, build/deployment systems, etc). > you are just asking for a hard time debugging, troubleshooting what specifically hard about this in your opinion?..
- deadbabe 3y ago
  Can you keep an open mind? We’ve used stored procedures for years. It has worked wonderfully for creating a single source and producer of truth for business data. Instead of potentially having business logic across multiple repos and deployments, everything exists in one place, with absolute unquestionable authority. It’s not difficult to debug at all, you might just be unskilled.
  - acdha 3y ago
    > It’s not difficult to debug at all, you might just be unskilled. I agree with your larger point but this seems too harsh: it’s definitely harder to debug simply because, as with microservices, understanding how the app is functioning now requires you to understand different code in multiple languages and locations, you’re highly likely to hit non-portable behavior across databases for authoring and debugging, and you’re never going to get a debugger with the whole flow in context. That doesn’t mean there aren’t benefits as well and it could be especially useful as a way to force distinctions about contracts for common operations, but I wouldn’t say it’s right for all or even most projects. The sweet spot is going to vary widely.
munificent 3y ago
As an ex-game developer and software architecture nerd, I'm very excited about data-oriented design and ECS. It really is a cool pattern, and it's a very common one in shipping games today. It's not just architecture astronaut stuff. At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s. Can ECS be a better way to structure your game entities and make your game loop faster? Yes. Will it make your teeth whiter and your partner love you more? No. (There are ECS frameworks in JavaScript, which gives you absolutely no control over memory layout and thus completely defeats one of the primary purposes of the pattern.) Like any pattern, it exists to solve concrete problems. It shouldn't be the One True Way To Think About All Programming Henceforth and Forever. When the author says things like: > For example, how would you model chat messages in your game? I suppose you’d have to represent that as an entity in your game. How would you represent a constraint that would prevent a health component from being added to your chat message erroneously? In ECS it’s straightforward to create a system which operates on chat messages individually, but how would you query your chat messages so that you can display them in order? To me, that just means "Don't use ECS for those." I have a really nice coffee mug that is just perfect for holding coffee. It does its job very well. That does not mean I feel any need to use that coffee mug for digging holes in my garden. Databases and the relational model are great. ECS is great. Object-oriented programming is great. Functional programming is great. But treat them all as tools that should be used for the right job.
- jerf 3y ago
  "At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s." I appreciate that most of the ECS hype has been around specific use cases, though. OOP was claimed as not a specific useful tool, but the answer to all programming, a billing it has not lived up to. It achieved "useful tool", no question, especially as some of the very rough bits were sanded off the original proposals (particularly the idea that objects should only and exactly match real world objects, an idea which I believe in hindsight was accidentally carried over from a simulation worldview in Simula where maybe it worked into the general programming world where it didn't), but it certainly has failed to be the one true programming methdology. I haven't seen anyone claiming everything should be rewritten in ECS.
lijok 3y ago
SpacetimeDB looks great for game dev. I've been hearing chitter chatter in the FaaS world about connecting clients directly to the DB as a means of reducing number of components and complexity in simple-ish CRUD apps. Not sure how well that design holds up there, time will tell. For those seeing this (SpacetimeDB) and immediatelly conjuring images of nightmares to be, consider the following: if you had an extremely latency sensitive usecase and had the opportunity to host your database and business logic on the same machine, why wouldn't you?
- Nezteb 3y ago
  I agree! I first learned about SpacetimeDB from a HN thread posted three months ago: https://news.ycombinator.com/item?id=37146952 https://news.ycombinator.com/item?id=37146952
slifin 3y ago
There's a good interview with Nathan Marz on his Rama project interviewed by Juxt He makes the point that databases are today prescriptive on the structure of data you provide to them I think databases as we know them today are not yet "end game"
inopinatus 3y ago
Author feedback: the large print, and the static elements (screen-thieving banner + “AI assistant” button), made attempting to read this blog as presented a vile experience on my phone. The AI button can’t be dismissed, if you ask it how to dismiss itself it doesn't know, and it won’t even accept negative feedback due to obscured submit buttons. Fortunately there’s Reader mode.
- cloutiertyler 3y ago
  Thanks for the feedback, we've definitely got to fix the mobile scaling stuff.
cgh 3y ago
Andrew Kelley of Zig fame did an excellent talk on this very subject: https://vimeo.com/649009599 https://vimeo.com/649009599 Strongly recommended viewing if you are interested in understanding why DoD is a big deal for performant applications (hint: it's about the cache).
- cloutiertyler 3y ago
  This is an excellent talk for sure.
rendall 3y ago
Hi spacetimedb.com. You apparently want people to read about your product, but have a non-GDPR-compliant popup that requires people to uncheck multiple "vendors" if your readers do not want to be tracked. Edit: it actually does not allow any unchecking. Just "you agree to these marketing and tracking cookies by using this site" Nope. Baffling. It does not lend confidence in your core product. If you're not respectful of your casual reader's data, how can we expect you to be careful with your user's data? I recommend zero tracking, or if you must, have it be genuinely opt-in.
- cloutiertyler 3y ago
  Will fix!
  - rendall 3y ago
    7 days later, it's still there, unchanged.
stuhood 3y ago
The fundamental difference between an ECS and a struct/object layout is that an ECS is column-oriented (aka columnar), while a struct/object layout is row-oriented. Everything else about how you might query these layouts is more superficial... you can provide the same API with either layout, the same way you can in relational database systems (both layouts can be queried with SQL, but with different performance characteristics.)
- gefjon 3y ago
  Are you familiar with any existing ECS implementations which expose a SQL interface?
gorgoiler 3y ago
Has anyone here seen a database where your users are users in the db itself? Not just a user(id, name, email, password) table but actual db users with GRANTed permissions and ACLs etc set appropriately, and open access to the DB for these users. It seems like it would solve a lot of problems by eliminating the need for data broker apps / endpoints that simply put POST or GET parameters into different SQL queries based on which endpoint you hit. I have no idea if it would scale but I don’t see why it couldn’t. This sort of idea came up earlier this month in another post as well: https://news.ycombinator.com/item?id=38489307 https://news.ycombinator.com/item?id=38489307
- hahn-kev 3y ago
  Yes I have, it was a nightmare. The biggest scaling issue was that on a server you must open a new db connection for each request. There's also little benefit because you still need to track application user data, so you still need a user table.
- lgas 3y ago
  > Has anyone here seen a database where your users are users in the db itself? PostgREST does this and it was a pretty nice experience when I used it. https://postgrest.org/en/stable/references/auth.html https://postgrest.org/en/stable/references/auth.html
- sarreph 3y ago
  Yes, Firebase / Firestore has this capability. You grant access to collections (and even fields) using rulesets[0]. Firestore is a NoSQL db so it's a quite different paradigmatically though. [0] - https://firebase.google.com/docs/firestore/security/get-started https://firebase.google.com/docs/firestore/security/get-star...
- cloutiertyler 3y ago
  SpacetimeDB is one such database. It has the concept of an `Identity` which is just a 256 bit unique identifier for a connected user.
zubairq 3y ago
I really resonated with this article and ended up at the same conclusion myself, even going so far as to replace a homegrown ECS system with a relational database in the browser
slooonz 3y ago
> I think that people perceive databases to be slow, not because they are slow, but because they often interact with them in the context of persisting data to disk, from across a network, by passing strings back and forth which must be parsed, compiled and executed. If you’ve ever optimized a program, you know that I just listed pretty much the slowest operations you can do on a computer. I’m sorry, but the difference between "databases are slow" and "the only way I can interact with databases is slow" is only pedantics. I understand that, as a database developer working hard to squeeze every bit of performance possible this is frustrating, but as a developer I couldn’t care less about the difference.
masfoobar 3y ago
I started learning programming in the mid-to-late 90's.. a teenager learning with Turbo C, Turbo Pascal, and VB6... eventually to Visual C++... to then attempting to jump on the OOP bandwagon with Java, I began to dislike coding. I was questioning whether this was the career but, after a few years, decided to give it a go. Job interviews, particulary then, were about "OOP this" and "OOP that" and I would purposely be agreeable but was unahppy with the code I was writing. Eventually I would come out of my shell with views to other devs. Roll on to 2014 watching Mike Actons "Data Oriented Design" - and I immediately felt at home, not because I consider myself an 'expert' or on the same league as seasoned game programmers but I felt I had the right mindset all along! In my 20 years, primarily a C# developer to pay my bills but at home my personal projects are written in C, and tried D.. and now Odin. I just prefer the control they give. (Scheme as well, I do like Scheme) Whenever someone asks me (a developer) what Data Oriented Design is, I try to explain to them it is like building a decent database. You are not thinking about the code (so much) but the representation of data. With databases, you create tables that have relations with other tables, with keys and indexes. You are setting up "lookups" etc. Once they grasp the idea of building a database (which most developers can easily understand) - it is a case of transferring that energy not to tables in a database, but to data structures in your programming language. Of course, this idea is easier to understand if you have experience in languages like C. For those coming from Python or Java or C#, etc, can be a little trickier but only if they think purely in the OOP mindset. However, if they struggle to grasp this, then ECS is a great way for understanding in that OOP-ish way. The penny drops when I discuss Entities and Components. You are not building classes and inheritence, you are building entities and components. It is more flexible creating an Entity and "attaching" a Component for it to do something. Then you don't have some kind of Update method in a class, you just have one function which passes in all of it's type. Anyway - it all comes back full circle now reading this article. Great reading! Now I can just refer people to this link instead.
- cloutiertyler 3y ago
  Wow, this really means a lot to me. I'm so glad you enjoyed the article!
LarsDu88 3y ago
I think I'm still a bit on the fence after reading this post on whether building out relational queries on top of ECS will ultimately nuke the cache locality and multithreading performance benefits of ECS in the long run. The great promise of ECS is that once we all have 128-512 core CPUs, that we will actually be making great use of them in the videogames of the future rather than throwing away the vast majority of performance which is the current world with the vast majority of Unity and Unreal games. Once you hit the real world and start making queries of everything, everywhere, all at once, maybe it'll be lock, lock, lock on your "database queries" And also, it seem like this DB really demands a game engine like Bevy actually get finished!
workfromspace 3y ago
https://archive.is/8uFyy https://archive.is/8uFyy (original website requires JS)
deleted 3y ago
[deleted]

238 points by cloutiertyler 3y ago

jitl 3y ago

If the site designers are on here by chance, take a look on iPhone. The blog post is quite squished there, pretty much unreadable. EDIT: nevermind, fixed after a refresh ¯\_(ツ)_/¯ still maybe worth a look

cloutiertyler 3y ago
I am on here. Will definitely take a look!
- vagrantJin 3y ago
  another thing. firefox issue. firefox on android no longer supports mobile view?
  - cloutiertyler 3y ago
    Will fix. Writing all these down.
- marshray 3y ago
  While you're here... where's the scrollbar? I'm on desktop (Edge) and don't see a scrollbar. Very distracting.
  - cloutiertyler 3y ago
    Will fix. Writing all these down.

samsquire 3y ago

Thank you, I enjoyed this post. I am thinking that iteration is just traversal and traversal is just execution. Take iterators in C++ or standard library "algorithms" library in C++. Or iterators in Rust or in Java. You just want to traverse and collect values (and calls on functions in some interleaving), which is like joining tables as this article says. I'm thinking that the definition of the traversal (such as Kafka pipeline, clojure threading syntax, Clojure data driven design, Javascript lodash pipeline) can be mapped to tuples and then the computer can optimise arbitrary chains of traversals based on the number of potential tuples are available and what traversals are equivalent paths or routes to the same traversal. In other words, every program is a compiler pipeline or database query pipeline. Maybe Prolog and Datalog can help here. Optimisation of arbitrary traversals and determination of identical traversals. Or monads are just ordered traversals and OCaml compilers are just traversals of execution (function application) and relationship following, which are joins. The relational model is everywhere.

BoiledCabbage 3y ago
> In other words, every program is a compiler pipeline or database query pipeline. > Or monads are just ordered traversals and OCaml compilers are just traversals of execution (function application) and relationship following, which are joins. > The relational model is everywhere. This is interesting - could you elaborate a bit more?
deleted 3y ago
[deleted]

fifilura 3y ago

My colleagues hate me, but I also found that SQL is The way to write business logic. Lots of caveats about difficulty to test and weird syntax. But it is just that SQL is the most terse and standard way so express logic. And that in itself is the most important factor to avoid bugs. Not what testing strategy you choose.

Etheryte 3y ago
I feel like you've come to the right conclusion, but partly for the wrong reasons. Terseness in and of itself is not useful per se. Code golfing languages are the tersest there is, but we don't write code that way.
- ndriscoll 3y ago
  Or they just used the wrong word. I'd agree with them, but clarify that SQL is not just terse, but concise. A join or a group by is going to be much clearer than writing the code the query plan is going to generate, creating temporary hash maps, doing nested loops, etc.
  - ttfkam 3y ago
    Yep, because unlike general purpose programming languages, you describe WHAT data you want from SQL, not HOW to get that data as in most application languages.
    - fifilura 3y ago
      Exactly. And yes, maybe concise was the word I meant to use (not native English speaker).
randomdata 3y ago
> And that in itself is the most important factor to avoid bugs. I'm not sure about that. SQL was the first language I learned and the language that has always been there throughout the decades, and is also where I make the most mistakes.

paulddraper 3y ago

The relational model is one of those eternal diamonds that will be just as relevant in 2090 as in 1990.

riku_iki 3y ago
I kinda think adding some logical/semantic level on top of it (like datalog) will make it even more robust.
- vacuity 3y ago
  Datalog (intentionally) isn't Turing-complete, though. That might hurt applicability for a generalized ECS model.
slooonz 3y ago
The biggest problem with the relational model is that it’s buried under the dirt that is SQL.

hitpointdrew 3y ago

Nope, nope and nope. Went to the github page for spacetimedb, it does everything that is terrible. >Instead of deploying a web or game server that sits in between your clients and your database, your clients connect directly to the database and execute your application logic inside the database itself. You can write all of your permission and authorization logic right inside your module just as you would in a normal server. Why? Databases should never, ever, ever, be used to perform logic, they are datastores, that is it. Your logic goes elsewhere. Stored procedures are the worst "feature" of any database, you are just asking for a hard time debugging, troubleshooting, and increasing the chance that you will fuck up the most valuable part of your system, your data. > This means that you can write your entire application in a single language, Rust, and deploy it as a single binary. It also means you have a single point of failure, no read-replicas or redundancy. Hate everything about this.

riku_iki 3y ago
> Why? less moving parts, diversity and complexity in your stack (different languages, servers, build/deployment systems, etc). > you are just asking for a hard time debugging, troubleshooting what specifically hard about this in your opinion?..
deadbabe 3y ago
Can you keep an open mind? We’ve used stored procedures for years. It has worked wonderfully for creating a single source and producer of truth for business data. Instead of potentially having business logic across multiple repos and deployments, everything exists in one place, with absolute unquestionable authority. It’s not difficult to debug at all, you might just be unskilled.
- acdha 3y ago
  > It’s not difficult to debug at all, you might just be unskilled. I agree with your larger point but this seems too harsh: it’s definitely harder to debug simply because, as with microservices, understanding how the app is functioning now requires you to understand different code in multiple languages and locations, you’re highly likely to hit non-portable behavior across databases for authoring and debugging, and you’re never going to get a debugger with the whole flow in context. That doesn’t mean there aren’t benefits as well and it could be especially useful as a way to force distinctions about contracts for common operations, but I wouldn’t say it’s right for all or even most projects. The sweet spot is going to vary widely.

munificent 3y ago

As an ex-game developer and software architecture nerd, I'm very excited about data-oriented design and ECS. It really is a cool pattern, and it's a very common one in shipping games today. It's not just architecture astronaut stuff. At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s. Can ECS be a better way to structure your game entities and make your game loop faster? Yes. Will it make your teeth whiter and your partner love you more? No. (There are ECS frameworks in JavaScript, which gives you absolutely no control over memory layout and thus completely defeats one of the primary purposes of the pattern.) Like any pattern, it exists to solve concrete problems. It shouldn't be the One True Way To Think About All Programming Henceforth and Forever. When the author says things like: > For example, how would you model chat messages in your game? I suppose you’d have to represent that as an entity in your game. How would you represent a constraint that would prevent a health component from being added to your chat message erroneously? In ECS it’s straightforward to create a system which operates on chat messages individually, but how would you query your chat messages so that you can display them in order? To me, that just means "Don't use ECS for those." I have a really nice coffee mug that is just perfect for holding coffee. It does its job very well. That does not mean I feel any need to use that coffee mug for digging holes in my garden. Databases and the relational model are great. ECS is great. Object-oriented programming is great. Functional programming is great. But treat them all as tools that should be used for the right job.

jerf 3y ago
"At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s." I appreciate that most of the ECS hype has been around specific use cases, though. OOP was claimed as not a specific useful tool, but the answer to all programming, a billing it has not lived up to. It achieved "useful tool", no question, especially as some of the very rough bits were sanded off the original proposals (particularly the idea that objects should only and exactly match real world objects, an idea which I believe in hindsight was accidentally carried over from a simulation worldview in Simula where maybe it worked into the general programming world where it didn't), but it certainly has failed to be the one true programming methdology. I haven't seen anyone claiming everything should be rewritten in ECS.

lijok 3y ago

SpacetimeDB looks great for game dev. I've been hearing chitter chatter in the FaaS world about connecting clients directly to the DB as a means of reducing number of components and complexity in simple-ish CRUD apps. Not sure how well that design holds up there, time will tell. For those seeing this (SpacetimeDB) and immediatelly conjuring images of nightmares to be, consider the following: if you had an extremely latency sensitive usecase and had the opportunity to host your database and business logic on the same machine, why wouldn't you?

Nezteb 3y ago
I agree! I first learned about SpacetimeDB from a HN thread posted three months ago: https://news.ycombinator.com/item?id=37146952 https://news.ycombinator.com/item?id=37146952

slifin 3y ago

There's a good interview with Nathan Marz on his Rama project interviewed by Juxt He makes the point that databases are today prescriptive on the structure of data you provide to them I think databases as we know them today are not yet "end game"

inopinatus 3y ago

Author feedback: the large print, and the static elements (screen-thieving banner + “AI assistant” button), made attempting to read this blog as presented a vile experience on my phone. The AI button can’t be dismissed, if you ask it how to dismiss itself it doesn't know, and it won’t even accept negative feedback due to obscured submit buttons. Fortunately there’s Reader mode.

cloutiertyler 3y ago
Thanks for the feedback, we've definitely got to fix the mobile scaling stuff.

cgh 3y ago

Andrew Kelley of Zig fame did an excellent talk on this very subject: https://vimeo.com/649009599 https://vimeo.com/649009599 Strongly recommended viewing if you are interested in understanding why DoD is a big deal for performant applications (hint: it's about the cache).

cloutiertyler 3y ago
This is an excellent talk for sure.

rendall 3y ago

Hi spacetimedb.com. You apparently want people to read about your product, but have a non-GDPR-compliant popup that requires people to uncheck multiple "vendors" if your readers do not want to be tracked. Edit: it actually does not allow any unchecking. Just "you agree to these marketing and tracking cookies by using this site" Nope. Baffling. It does not lend confidence in your core product. If you're not respectful of your casual reader's data, how can we expect you to be careful with your user's data? I recommend zero tracking, or if you must, have it be genuinely opt-in.

cloutiertyler 3y ago
Will fix!
- rendall 3y ago
  7 days later, it's still there, unchanged.

stuhood 3y ago

The fundamental difference between an ECS and a struct/object layout is that an ECS is column-oriented (aka columnar), while a struct/object layout is row-oriented. Everything else about how you might query these layouts is more superficial... you can provide the same API with either layout, the same way you can in relational database systems (both layouts can be queried with SQL, but with different performance characteristics.)

gefjon 3y ago
Are you familiar with any existing ECS implementations which expose a SQL interface?

gorgoiler 3y ago

Has anyone here seen a database where your users are users in the db itself? Not just a user(id, name, email, password) table but actual db users with GRANTed permissions and ACLs etc set appropriately, and open access to the DB for these users. It seems like it would solve a lot of problems by eliminating the need for data broker apps / endpoints that simply put POST or GET parameters into different SQL queries based on which endpoint you hit. I have no idea if it would scale but I don’t see why it couldn’t. This sort of idea came up earlier this month in another post as well: https://news.ycombinator.com/item?id=38489307 https://news.ycombinator.com/item?id=38489307

hahn-kev 3y ago
Yes I have, it was a nightmare. The biggest scaling issue was that on a server you must open a new db connection for each request. There's also little benefit because you still need to track application user data, so you still need a user table.
lgas 3y ago
> Has anyone here seen a database where your users are users in the db itself? PostgREST does this and it was a pretty nice experience when I used it. https://postgrest.org/en/stable/references/auth.html https://postgrest.org/en/stable/references/auth.html
sarreph 3y ago
Yes, Firebase / Firestore has this capability. You grant access to collections (and even fields) using rulesets[0]. Firestore is a NoSQL db so it's a quite different paradigmatically though. [0] - https://firebase.google.com/docs/firestore/security/get-started https://firebase.google.com/docs/firestore/security/get-star...
cloutiertyler 3y ago
SpacetimeDB is one such database. It has the concept of an `Identity` which is just a 256 bit unique identifier for a connected user.

zubairq 3y ago

I really resonated with this article and ended up at the same conclusion myself, even going so far as to replace a homegrown ECS system with a relational database in the browser

slooonz 3y ago

> I think that people perceive databases to be slow, not because they are slow, but because they often interact with them in the context of persisting data to disk, from across a network, by passing strings back and forth which must be parsed, compiled and executed. If you’ve ever optimized a program, you know that I just listed pretty much the slowest operations you can do on a computer. I’m sorry, but the difference between "databases are slow" and "the only way I can interact with databases is slow" is only pedantics. I understand that, as a database developer working hard to squeeze every bit of performance possible this is frustrating, but as a developer I couldn’t care less about the difference.

masfoobar 3y ago

I started learning programming in the mid-to-late 90's.. a teenager learning with Turbo C, Turbo Pascal, and VB6... eventually to Visual C++... to then attempting to jump on the OOP bandwagon with Java, I began to dislike coding. I was questioning whether this was the career but, after a few years, decided to give it a go. Job interviews, particulary then, were about "OOP this" and "OOP that" and I would purposely be agreeable but was unahppy with the code I was writing. Eventually I would come out of my shell with views to other devs. Roll on to 2014 watching Mike Actons "Data Oriented Design" - and I immediately felt at home, not because I consider myself an 'expert' or on the same league as seasoned game programmers but I felt I had the right mindset all along! In my 20 years, primarily a C# developer to pay my bills but at home my personal projects are written in C, and tried D.. and now Odin. I just prefer the control they give. (Scheme as well, I do like Scheme) Whenever someone asks me (a developer) what Data Oriented Design is, I try to explain to them it is like building a decent database. You are not thinking about the code (so much) but the representation of data. With databases, you create tables that have relations with other tables, with keys and indexes. You are setting up "lookups" etc. Once they grasp the idea of building a database (which most developers can easily understand) - it is a case of transferring that energy not to tables in a database, but to data structures in your programming language. Of course, this idea is easier to understand if you have experience in languages like C. For those coming from Python or Java or C#, etc, can be a little trickier but only if they think purely in the OOP mindset. However, if they struggle to grasp this, then ECS is a great way for understanding in that OOP-ish way. The penny drops when I discuss Entities and Components. You are not building classes and inheritence, you are building entities and components. It is more flexible creating an Entity and "attaching" a Component for it to do something. Then you don't have some kind of Update method in a class, you just have one function which passes in all of it's type. Anyway - it all comes back full circle now reading this article. Great reading! Now I can just refer people to this link instead.

cloutiertyler 3y ago
Wow, this really means a lot to me. I'm so glad you enjoyed the article!

LarsDu88 3y ago

I think I'm still a bit on the fence after reading this post on whether building out relational queries on top of ECS will ultimately nuke the cache locality and multithreading performance benefits of ECS in the long run. The great promise of ECS is that once we all have 128-512 core CPUs, that we will actually be making great use of them in the videogames of the future rather than throwing away the vast majority of performance which is the current world with the vast majority of Unity and Unreal games. Once you hit the real world and start making queries of everything, everywhere, all at once, maybe it'll be lock, lock, lock on your "database queries" And also, it seem like this DB really demands a game engine like Bevy actually get finished!

workfromspace 3y ago

https://archive.is/8uFyy https://archive.is/8uFyy (original website requires JS)

deleted 3y ago

[deleted]