3 ms·

> Why is a router a thing that needs to be continually rebuilt and tinkered with. Did we not learn ages ago how routers should work? Nyes. The biggest innovati

by Swizec 1y ago

> Why is a router a thing that needs to be continually rebuilt and tinkered with. Did we not learn ages ago how routers should work? Nyes. The biggest innovation in the past 5 years has been routers that can coordinate loading data because they’re perfectly positioned to know what you’re about to access. This is a hard problem that we’ve been solving forever. It feels like super tedious formulaic work to write an optimized SQL query or series of API requests that fetches all the necessary data with the fewest possible lookups. So we try to automate it with a sufficiently smart compiler of some sort. Query planners inside a database, ORMs, graphql, routers, memory managing compilers, it’s all chasing the same dream – what if the computer Just Knew the quickest way to grab just the right amount of data.

BoorishBears 1y ago
I've re-read this comment at least 5 times and feel like I'm having a stroke reading it each time. And something similar happens really often when I enter the hype-driven side of React these days.. I do wish I had a more useful critique, and I'm not even trying to be mean (or boorish as it were) but you're rolling so many things up into each other that there's no useful way to interpret the statement. At least, not without ending up giving you a great chance to just say "no no no you completely missed what I'm saying" and then coming up with a new equally dizzying statement. How you manage to drag query planners into routers into compilers, how are these chasing one dream or fungible or even barely correlated, I don't even know. - It's awful and sad how tech is one of the few fields that just insists on making things worse for itself. You can walk into McDonalds and ask how the process can be improved, and I guarantee every suggestion is targeted at making their jobs easier in a way that at least superficially aligns with making the service quicker, which is something the company does care about. In tech you ask and someone goes on a poetic journey about how the paper cups, the drive-thru speaker, and the coffee machine are all chasing the same dream, and also we need a $100,000 espresso machine that takes 10 minutes of intense labor to brew a shot because then I'll be qualified to operate a $100,000 espresso machine at my next job which pays better than McDonalds. We did not figure out how to brew coffee before, that was all wrong and we needed to make the process at least 10x more complicated.
- pas 1y ago
  the niche that Next dominates rightfully is these latency optimized but highly dynamic and parallelizable sites, minimal "per user" state lives close to the user, it's vertically integrated, and gives tools to frontend-oriented devs. React expanded into server-side, it's messy now, but not that complicated. Things look like the SpaceX rocket motor with all the diagnostic stuff still on, but in a few revisions it'll likely look (and feel) more sleek. ... And it's true that the end-to-end experience matters at restaurants, be they fine dining 3 thousand Gault Milleu stars or a worldwide franchise with so perverse incentives built in that it should be 18+ anyway (and not just because of the hyper-palatable sauces). ... And the argument is that these things matter especially when you have a lot of users. ... That said the people responsible for docs at NextJS should be sent to the fullstack mines.
  - BoorishBears 1y ago
    The niche Next dominated before they started chasing the RSC dragon was scaling SPAs with complex frontend state. React Query and atom based state management offered nice alternatives to Redux/Context hell, and Next could have just hung up major architectural refactors for QoL updates. They want to dominate the headless Shopify/CMS niche, but that doesn't have to mean dragging Next.js along for the ride. It was a business/branding decision, not one that actually benefits most users of what they had already had.
- Swizec 1y ago
  > How you manage to drag query planners into routers into compilers, how are these chasing one dream or fungible or even barely correlated, I don't even know. Let me expand. In component-based UIs you have components that fetch their own data. Declare their data dependencies if you will. This makes them independent and easy to move around. You can blindly take a component that renders, say, the user’s name and avatar and move it anywhere in your app. It’s going to know how to recognize the current user, fetch their data, and render what it needs. This is great for reusability and flexibility. Now what happens if you have a bunch of these components on a page? You get lots of uncoordinated data fetches. Each component is independent so it doesn’t know about the others so they can’t coordinate. But who does know about them? That’s right, the router knows. So the router can make a more coordinated data fetch and get data for all those components. This is the same job as a controller in the classical MVC approach where you write data fetching (sql queries) in the controller so data is ready when you access it in the template (jinja or whatever). Exact same problem, different naming conventions. And just like with ORMs you have the same N+1 issue where your template pokes an attribute that was not prefetched and triggers a bunch of data fetches (sql queries). So you end up doing lots of tedious work to always hoist up the joins into your controller to trick the ORM into having that data preloaded. Wouldn’t it be great if somehow your ORM could understand all the render logic and write its own single query that gets all the data? Yes it would. Just as wouldn’t it be great if your router could understand all the fetches your UI components will do and make one big request in the beginning? Ofc. Now here’s how this relates to query planners: Wouldn’t it be great if you could just declare what data you want and the computer figured out how to read just the right amount from disk, join together the relevant bits, and gave you the output? Without you having to write a bunch of manual disk reading logic, loops to clean and correlate the data, and while also minimizing the amount of disk access manually? Of course it would.

by Swizec 1y ago

BoorishBears 1y ago

I've re-read this comment at least 5 times and feel like I'm having a stroke reading it each time. And something similar happens really often when I enter the hype-driven side of React these days.. I do wish I had a more useful critique, and I'm not even trying to be mean (or boorish as it were) but you're rolling so many things up into each other that there's no useful way to interpret the statement. At least, not without ending up giving you a great chance to just say "no no no you completely missed what I'm saying" and then coming up with a new equally dizzying statement. How you manage to drag query planners into routers into compilers, how are these chasing one dream or fungible or even barely correlated, I don't even know. - It's awful and sad how tech is one of the few fields that just insists on making things worse for itself. You can walk into McDonalds and ask how the process can be improved, and I guarantee every suggestion is targeted at making their jobs easier in a way that at least superficially aligns with making the service quicker, which is something the company does care about. In tech you ask and someone goes on a poetic journey about how the paper cups, the drive-thru speaker, and the coffee machine are all chasing the same dream, and also we need a $100,000 espresso machine that takes 10 minutes of intense labor to brew a shot because then I'll be qualified to operate a $100,000 espresso machine at my next job which pays better than McDonalds. We did not figure out how to brew coffee before, that was all wrong and we needed to make the process at least 10x more complicated.

pas 1y ago
the niche that Next dominates rightfully is these latency optimized but highly dynamic and parallelizable sites, minimal "per user" state lives close to the user, it's vertically integrated, and gives tools to frontend-oriented devs. React expanded into server-side, it's messy now, but not that complicated. Things look like the SpaceX rocket motor with all the diagnostic stuff still on, but in a few revisions it'll likely look (and feel) more sleek. ... And it's true that the end-to-end experience matters at restaurants, be they fine dining 3 thousand Gault Milleu stars or a worldwide franchise with so perverse incentives built in that it should be 18+ anyway (and not just because of the hyper-palatable sauces). ... And the argument is that these things matter especially when you have a lot of users. ... That said the people responsible for docs at NextJS should be sent to the fullstack mines.
- BoorishBears 1y ago
  The niche Next dominated before they started chasing the RSC dragon was scaling SPAs with complex frontend state. React Query and atom based state management offered nice alternatives to Redux/Context hell, and Next could have just hung up major architectural refactors for QoL updates. They want to dominate the headless Shopify/CMS niche, but that doesn't have to mean dragging Next.js along for the ride. It was a business/branding decision, not one that actually benefits most users of what they had already had.
Swizec 1y ago
> How you manage to drag query planners into routers into compilers, how are these chasing one dream or fungible or even barely correlated, I don't even know. Let me expand. In component-based UIs you have components that fetch their own data. Declare their data dependencies if you will. This makes them independent and easy to move around. You can blindly take a component that renders, say, the user’s name and avatar and move it anywhere in your app. It’s going to know how to recognize the current user, fetch their data, and render what it needs. This is great for reusability and flexibility. Now what happens if you have a bunch of these components on a page? You get lots of uncoordinated data fetches. Each component is independent so it doesn’t know about the others so they can’t coordinate. But who does know about them? That’s right, the router knows. So the router can make a more coordinated data fetch and get data for all those components. This is the same job as a controller in the classical MVC approach where you write data fetching (sql queries) in the controller so data is ready when you access it in the template (jinja or whatever). Exact same problem, different naming conventions. And just like with ORMs you have the same N+1 issue where your template pokes an attribute that was not prefetched and triggers a bunch of data fetches (sql queries). So you end up doing lots of tedious work to always hoist up the joins into your controller to trick the ORM into having that data preloaded. Wouldn’t it be great if somehow your ORM could understand all the render logic and write its own single query that gets all the data? Yes it would. Just as wouldn’t it be great if your router could understand all the fetches your UI components will do and make one big request in the beginning? Ofc. Now here’s how this relates to query planners: Wouldn’t it be great if you could just declare what data you want and the computer figured out how to read just the right amount from disk, join together the relevant bits, and gave you the output? Without you having to write a bunch of manual disk reading logic, loops to clean and correlate the data, and while also minimizing the amount of disk access manually? Of course it would.