5 ms·

This is a great comment and an amazing insight. What's particularly interesting is that people have attempted to collapse (almost?) every stage of that abstract

This is a great comment and an amazing insight. What's particularly interesting is that people have attempted to collapse (almost?) every stage of that abstraction hierarchy individually, but none of them have been so successful as to take over the world. If you were writing a desktop application, you would still have at least three of the layers (serialized data on disk, in-memory data, and the rendering of the objects), but without the dramatic impedance mismatch that the web platform introduces everywhere.

adrianratnapala 8y ago
Even on the desktop those three layers involve impedance mismatch and much the same pathologies as meredydd describes. But I guess three layers of it are better than 6.
- pas 8y ago
  It's data. There's a fix format for serializing it. In a lossless way. I don't know what meredydd talks about, there's no mismatch with regards to data. You can get the same bits in your JS objects as you have in the DB. If not, that means your system is shit. The problem with frameworks is not the hardship of funneling data up and down the stack. The problem is that they are optimizing for different things. React optimizes for simplicity of making components. Angular optimizes for providing a full toolkit. And new versions then focus on different things. Server Side Rendering was hot, but now that Google just executes some JS and penalizes large downloads, it's the quest for less bytes on the wire. And tree-shake-ability. And faster time to first paint. And as browsers and the web changes, so do frameworks. And frameworks try to target, at the same time, both the future, and the very present problems, they try to provide instant gratification, yet try to optimize for the future. So they usually look half-assed useless pieces of autogenerated-by-MS-Word code all the time. But they work, nevertheless, and power a lot of sites.
  - randomdata 8y ago
    > I don't know what meredydd talks about, there's no mismatch with regards to data. He's talking about different services each having their own preferred way to structure the data. When the layout differs, it cannot simply be a memcpy, and so you get tools to try to ease the tedium of translating one structure's layout into to another. They get the job done most of the time, but run into edge cases that return the developer back to manual tedium. Since developers do not like tedious work, some set out to find a new solution that solves for those edge cases, but they end up leaving many more on the table for the next intrepid developer.
    - pas 8y ago
      > own preferred way to structure the data. Absolutely. But user/business data? That doesn't matter. When you design the system/stack you pick the right components/tools (right data structures) that can losslessly represent the input/output of the neighboring/adjacent layers. If you want to store 500 byte long fields, then make your DB column 500 byte wide, make sure the backend accepts 500 byte long input, but rejects longer ones, make sure your HTML input has a maxlen=500 (and account for Unicode code point surrogate / multibyte fuckery if applicable) There's mismatch, of course, but as I've detailed in a sibling comment [0], it's because of difference in purpose and function. A DB is different from a HTML/CSS layout rendering engine, because they have a very (set) of purpose(s), hence different interfaces, and so on. And frameworks are glue between these functions (and the layers as we allocate them to). > Since developers do not like tedious work, some set out to find a new solution that solves for those edge cases [...] Yes, perfectly agreed. And since we concentrate on different edge-cases each time, we move from trade-off to trade-off with each new framework, and browsing trend/fad (mobile, tablet, SSR, ultra-tree-shakable-gzip-able, "native" [mobile] compilable, etc). [0] https://news.ycombinator.com/item?id=17209305 https://news.ycombinator.com/item?id=17209305
  - fauigerzigerk 8y ago
    >You can get the same bits in your JS objects as you have in the DB. If not, that means your system is shit. I'm not sure what exactly you mean by that. But one thing is absolutely clear. You cannot automatically derive a logical layer from the layer above or below. If you could, there would be no reason to have seperate (logical) layers in the first place. That's why you get an "impedance mismatch" that has to be bridged by providing some additional information, which often has consequences for performance, debuggabilty and clarity. Maybe I misunderstand the gist of your comment though.
    - pas 8y ago
      Maybe I misunderstood the original comment, but the claim was that frameworks are leaky (this I wholeheartedly agree with, and this of course leads to impedance mismatches, after all, pixels on the screen are very different from an SQL DB, but that's why we have the libraries and frameworks, to help us do this translation from one layer to the other, to glue together very different functional components of systems). But then follows it up with talk about data. How JSON and SQL is not a great match. Which is nonsense. You can losslessly represent the same data in both JSON and SQL, you can engineer a perfect system for handling data (you use the same field and column types, lengths, constraints, validation, and so on on both the front- and the backend, and it works), the mismatch is not around data. The problems are about development trade offs (TypeScript vs JS, small library - few features, complexity - code modularization + chunked lazy loading, optimization - script load time vs development time, and throw in cross browser compatibility; supported features vs complexity - HTTP/2 is nice, and fast, but it's more complex plus you need HTTP1.1 too for old clients, and maybe your API somewhere doesn't support prefetch, or you can't hint your backend to push that to the client, or you can't access the raw request after the framework extracted the request attributes, blablabla), visual communication (current/modern components vs old jQuery sprinkled DOM result in different sites; mobile first, mobile browsers, React Native and Ionic). And these trade offs are different over time. So we get different frameworks over time. And since the change in browsing is very fast, and the effort to start a new framework is small, we get a lot of new unstable frameworks. (And since those frameworks rarely mature really, we get a lot of new ones, because there's not really a "sunk cost" for developers when abandoning the old ones. And it's easy and hip to pick up new skills, and try them out on a new project, etc.) Also, you can put the "business logic" into one place, and represent it and then push that representation to the client. (GWT, Scala.js, or crude autogenerated forms, point and click website/workflow builders, and so on). Of course, if you want to change the system, then it might be a big pain in the ass to represent something very different than what it was designed for, so these kinds of entombed vertical complexity barriers lead to a metastable state - when you hack something quick on the layer most accessible for you with respect to the task, instead of properly implement it in the whole vertical stack, and these hacks grow and the elegant single source representation of business logic goes out of the window. (Or if you implement everything in one place, you are destined to implement a very powerful - or verbose - DSL to describe the "front end logic" - which should be CSS, and the DB optimization logic - which should be SQL, and so on.)
meredydd 8y ago
Thanks! And funny you should mention that. We're challenging every layer of that heirarchy simultaneously, by building a development environment for the web and making it as integrated as Delphi or VB were on the desktop: https://anvil.works https://anvil.works
- winkeltripel 8y ago
  How does one build/distribute modules for it? Do you have a package manager?

adrianratnapala 8y ago

Even on the desktop those three layers involve impedance mismatch and much the same pathologies as meredydd describes. But I guess three layers of it are better than 6.

pas 8y ago
It's data. There's a fix format for serializing it. In a lossless way. I don't know what meredydd talks about, there's no mismatch with regards to data. You can get the same bits in your JS objects as you have in the DB. If not, that means your system is shit. The problem with frameworks is not the hardship of funneling data up and down the stack. The problem is that they are optimizing for different things. React optimizes for simplicity of making components. Angular optimizes for providing a full toolkit. And new versions then focus on different things. Server Side Rendering was hot, but now that Google just executes some JS and penalizes large downloads, it's the quest for less bytes on the wire. And tree-shake-ability. And faster time to first paint. And as browsers and the web changes, so do frameworks. And frameworks try to target, at the same time, both the future, and the very present problems, they try to provide instant gratification, yet try to optimize for the future. So they usually look half-assed useless pieces of autogenerated-by-MS-Word code all the time. But they work, nevertheless, and power a lot of sites.
- randomdata 8y ago
  > I don't know what meredydd talks about, there's no mismatch with regards to data. He's talking about different services each having their own preferred way to structure the data. When the layout differs, it cannot simply be a memcpy, and so you get tools to try to ease the tedium of translating one structure's layout into to another. They get the job done most of the time, but run into edge cases that return the developer back to manual tedium. Since developers do not like tedious work, some set out to find a new solution that solves for those edge cases, but they end up leaving many more on the table for the next intrepid developer.
  - pas 8y ago
    > own preferred way to structure the data. Absolutely. But user/business data? That doesn't matter. When you design the system/stack you pick the right components/tools (right data structures) that can losslessly represent the input/output of the neighboring/adjacent layers. If you want to store 500 byte long fields, then make your DB column 500 byte wide, make sure the backend accepts 500 byte long input, but rejects longer ones, make sure your HTML input has a maxlen=500 (and account for Unicode code point surrogate / multibyte fuckery if applicable) There's mismatch, of course, but as I've detailed in a sibling comment [0], it's because of difference in purpose and function. A DB is different from a HTML/CSS layout rendering engine, because they have a very (set) of purpose(s), hence different interfaces, and so on. And frameworks are glue between these functions (and the layers as we allocate them to). > Since developers do not like tedious work, some set out to find a new solution that solves for those edge cases [...] Yes, perfectly agreed. And since we concentrate on different edge-cases each time, we move from trade-off to trade-off with each new framework, and browsing trend/fad (mobile, tablet, SSR, ultra-tree-shakable-gzip-able, "native" [mobile] compilable, etc). [0] https://news.ycombinator.com/item?id=17209305 https://news.ycombinator.com/item?id=17209305
- fauigerzigerk 8y ago
  >You can get the same bits in your JS objects as you have in the DB. If not, that means your system is shit. I'm not sure what exactly you mean by that. But one thing is absolutely clear. You cannot automatically derive a logical layer from the layer above or below. If you could, there would be no reason to have seperate (logical) layers in the first place. That's why you get an "impedance mismatch" that has to be bridged by providing some additional information, which often has consequences for performance, debuggabilty and clarity. Maybe I misunderstand the gist of your comment though.
  - pas 8y ago
    Maybe I misunderstood the original comment, but the claim was that frameworks are leaky (this I wholeheartedly agree with, and this of course leads to impedance mismatches, after all, pixels on the screen are very different from an SQL DB, but that's why we have the libraries and frameworks, to help us do this translation from one layer to the other, to glue together very different functional components of systems). But then follows it up with talk about data. How JSON and SQL is not a great match. Which is nonsense. You can losslessly represent the same data in both JSON and SQL, you can engineer a perfect system for handling data (you use the same field and column types, lengths, constraints, validation, and so on on both the front- and the backend, and it works), the mismatch is not around data. The problems are about development trade offs (TypeScript vs JS, small library - few features, complexity - code modularization + chunked lazy loading, optimization - script load time vs development time, and throw in cross browser compatibility; supported features vs complexity - HTTP/2 is nice, and fast, but it's more complex plus you need HTTP1.1 too for old clients, and maybe your API somewhere doesn't support prefetch, or you can't hint your backend to push that to the client, or you can't access the raw request after the framework extracted the request attributes, blablabla), visual communication (current/modern components vs old jQuery sprinkled DOM result in different sites; mobile first, mobile browsers, React Native and Ionic). And these trade offs are different over time. So we get different frameworks over time. And since the change in browsing is very fast, and the effort to start a new framework is small, we get a lot of new unstable frameworks. (And since those frameworks rarely mature really, we get a lot of new ones, because there's not really a "sunk cost" for developers when abandoning the old ones. And it's easy and hip to pick up new skills, and try them out on a new project, etc.) Also, you can put the "business logic" into one place, and represent it and then push that representation to the client. (GWT, Scala.js, or crude autogenerated forms, point and click website/workflow builders, and so on). Of course, if you want to change the system, then it might be a big pain in the ass to represent something very different than what it was designed for, so these kinds of entombed vertical complexity barriers lead to a metastable state - when you hack something quick on the layer most accessible for you with respect to the task, instead of properly implement it in the whole vertical stack, and these hacks grow and the elegant single source representation of business logic goes out of the window. (Or if you implement everything in one place, you are destined to implement a very powerful - or verbose - DSL to describe the "front end logic" - which should be CSS, and the DB optimization logic - which should be SQL, and so on.)

meredydd 8y ago

Thanks! And funny you should mention that. We're challenging every layer of that heirarchy simultaneously, by building a development environment for the web and making it as integrated as Delphi or VB were on the desktop: https://anvil.works https://anvil.works

winkeltripel 8y ago
How does one build/distribute modules for it? Do you have a package manager?