9 ms·

Web frameworks are churn-y because they are incredibly leaky abstractions covering really awkward impedance mismatches. This means that they are never quite sat

Web frameworks are churn-y because they are incredibly leaky abstractions covering really awkward impedance mismatches. This means that they are never quite satisfactory - and that just to use one, you need to be capable of building a new one yourself. Think of a typical web app. Your data exists: 1. As rows in a database, accessed via SQL 2. As model objects on the server, accessed via method calls and attributes 3. As JSON, accessed via many HTTP endpoints with a limited set of verbs (GET/PUT/POST/DELETE) 4. As Javascript objects, accessed via (a different set of) method calls and attributes 5. As HTML tags, accessed via the DOM API 6. As pixels, styled by CSS. -- Each time you translate from one layer to the next, there's a nasty impedance mismatch. This, in turn, attracts "magic": ORMs (DB<->Object); Angular Resources (REST<->JS Object); templating engines (JS Object<->DOM); etc. Each of these translation layers shares two characteristics: (A) It is "magic": It abuses the semantics of one layer (eg DB model objects) in an attempt to interface with another (eg SQL). (B) It's a terribly leaky abstraction. This means that (a) every translation layer is prone to unintuitive failures, and (b) every advanced user of it needs to know enough to build one themselves. So when the impedance mismatch bites you on the ass, some fraction of users are going to flip the table, swear they could do better, and write their own. Which, of course, can't solve the underlying mismatch, and therefore won't be satisfactory...and so the cycle continues. Of these nasty transitions, 4/5 are associated with the front end, so the front end gets the rap. (I gave a lightning talk at PyCon two weeks ago, about exactly this - stacking this up against the "Zen of Python" and talking about some of the ways we avoid this in Anvil: https://anvil.works/blog/pycon18-making-the-web-more-pythonic https://anvil.works/blog/pycon18-making-the-web-more-pythoni...)

chrisweekly 8y ago
Some good points... but I think you're too pessimistic. Some tiny subset of those fraction of users you mention manage to do something different / better. React is one example. GraphQL is another.
bjterry 8y ago
This is a great comment and an amazing insight. What's particularly interesting is that people have attempted to collapse (almost?) every stage of that abstraction hierarchy individually, but none of them have been so successful as to take over the world. If you were writing a desktop application, you would still have at least three of the layers (serialized data on disk, in-memory data, and the rendering of the objects), but without the dramatic impedance mismatch that the web platform introduces everywhere.
- adrianratnapala 8y ago
  Even on the desktop those three layers involve impedance mismatch and much the same pathologies as meredydd describes. But I guess three layers of it are better than 6.
  - pas 8y ago
    It's data. There's a fix format for serializing it. In a lossless way. I don't know what meredydd talks about, there's no mismatch with regards to data. You can get the same bits in your JS objects as you have in the DB. If not, that means your system is shit. The problem with frameworks is not the hardship of funneling data up and down the stack. The problem is that they are optimizing for different things. React optimizes for simplicity of making components. Angular optimizes for providing a full toolkit. And new versions then focus on different things. Server Side Rendering was hot, but now that Google just executes some JS and penalizes large downloads, it's the quest for less bytes on the wire. And tree-shake-ability. And faster time to first paint. And as browsers and the web changes, so do frameworks. And frameworks try to target, at the same time, both the future, and the very present problems, they try to provide instant gratification, yet try to optimize for the future. So they usually look half-assed useless pieces of autogenerated-by-MS-Word code all the time. But they work, nevertheless, and power a lot of sites.
    - randomdata 8y ago
      > I don't know what meredydd talks about, there's no mismatch with regards to data. He's talking about different services each having their own preferred way to structure the data. When the layout differs, it cannot simply be a memcpy, and so you get tools to try to ease the tedium of translating one structure's layout into to another. They get the job done most of the time, but run into edge cases that return the developer back to manual tedium. Since developers do not like tedious work, some set out to find a new solution that solves for those edge cases, but they end up leaving many more on the table for the next intrepid developer.
      - 8y ago
zebraflask 8y ago
(B) Nails it. I'd guess that a large portion of new frameworks start simply because everyone who spent enough time with the old one gets sick of the bad or non-existent documentation for edge cases, etc. Rinse and repeat. Vue is no better, after reading the rest of the comments. In many ways worse.
Chyzwar 8y ago
That's good, in each cycle, sooner or later you get improvement: webpack over grunt, react over jQuery and npm over vendoring your jQuery plugins. OP forgot how life looked when your web app project was handcrafted HTML page with manually inserted scripts tags. When your form submission was multi-level backend API in PHP. jQuery plugins with 20+ options published randomly on the internet.
- aaronbrethorst 8y ago
  I feel like this is a false dichotomy. The choice doesn't have to be roasting squirrels over an open flame/handcrafting PHP pages vs. shiny futurism/Node+React. I have been getting along just fine with Rails, HTML, and a sprinkling of JS for over 10 years.
  - ehnto 8y ago
    There is an immense productivity gain to be found in mastering a set of tools. After a while of dogmatic tooling changes you begin to analyze more critically whether the new shiny thing is going to provide any real value. I would argue that for the huge majority of websites, tried and tested tooling is perfectly fine and definitely more robust and supported. Every tool you introduce is hours of troubleshooting just waiting to happen.
  - emodendroket 8y ago
    That stuff works fine if your client-side needs are simple, but if you actually want a single-page application, or just an application with a lot of rich JS functionality, it quickly becomes unwieldy.
    - Chris_Newton 8y ago
      That stuff works fine if your client-side needs are simple To be fair, just about anything works fine if your client-side needs are simple. However, I have reached the opposite conclusion to you: the more customised and complicated and large-scale and long-lived the software becomes, the less value I see in a lot of the popular but ever-changing web technologies and the more I am likely to favour building on the standard foundations with minimal dependencies in between and usually a relatively small but high-value set of libraries. The benefits of quickly fetching many tiny packages with a package manager or of building on top of all-encompassing frameworks or automation tools are mostly found in two situations, in my experience: getting started quickly (including rapid prototyping exercises) and ongoing development if (and only if) you are staying almost entirely within the bounds of what your chosen technologies already do well. However, if your requirements start to evolve and diversify in a longer-lasting project, it’s all too easy for those numerous tiny dependencies to become a liability or for that framework or tool you built everything around to become a straitjacket. The relatively short lifetimes of many of these technologies can also become an expensive problem if the community drifts away and the security and compatibility work slows down or stops entirely but your project still depends on them as much as ever.
tracker1 8y ago
All of this is why, in the end, more and more... I do things "the hard way". If I'm using a dynamic language WTF do I need an ORM for, if I understand enough to write an SQL command, and use a library for that DB that does parameterized queries? On the front end, I tend to lean towards abstractions that work together... I really like React and the material-ui library's switch to JSS. It's relatively clean, and useful. Even then, it's only a mild syntax adjustment, not a full on abstraction. React is more of an abstraction, but that comes with functional paradigms that aid in testing, and predictive behaviors. It really depends though. One can always do just JS/HTML/CSS, and there's something to be said for that. There are lighter tools that are similar to JQuery to smooth over a few of the rough edges. There's really an ala cart of available options. The problem is that people assume that the PFM (pure fucking magic) will solve it all for them. You can use the cleanest or simplest abstractions, and then still write layers of incomprehensible spaghetti in between.
- bonesss 8y ago
  > I do things "the hard way". If I'm using a dynamic language WTF do I need an ORM for I think it's mostly premature optimization. People think writing DTOs is challenging, so they want an ORM. But since you end up needing DTOs anwyays, removing SQL capabilities from the app means writing SQL in not SQL, and things like joins suddenly become slow and problematic and result in really heavy systems that are harder to change. For the joy of a quicker startup the entire project moves slower. ORMs have their place, but in the majority of the systems I've seen they were unnecessary, and in broad terms don't provide any particular productivity advantage over using "dumb" SQL-based mapping solutions (a la Dapper [https://github.com/StackExchange/Dapper] https://github.com/StackExchange/Dapper]), that preserve the power of SQL.
  - unclebucknasty 8y ago
    About 17 years ago, I wrote a GUI-based code generator that allows me to generate the boilerplate JDBC cruft from SQL statements, with several options for common scenarios. The code it generates is extensible and provides helpers for extending. To this day, I haven't found anything (including ORMs, Spring support, etc.) easier to use, more flexible, or more sensible.
lkrubner 8y ago
What is curious is that the tech industry has been very conservative about rethinking these 6 things that you just listed. Why rows in a relational database? Why objects? Why Javascript? And why HTML? We'd surely be in a better place if we got rid of these things and rethought our approach from first principles. I've written about these issues many times before. Regarding the problem with objects, I wrote "Object Oriented Programming Is An Expensive Disaster Which Must End": http://www.smashcompany.com/technology/object-oriented-programming-is-an-expensive-disaster-which-must-end http://www.smashcompany.com/technology/object-oriented-progr... Regarding the problems of HTML, I wrote "The problem with HTML": http://www.smashcompany.com/technology/the-problem-with-html http://www.smashcompany.com/technology/the-problem-with-html To answer the question "Why Is Front-End Development So Unstable?" the answer is surely, in part, the fact that we refuse to build technologies that are designed to be great front-end technologies.
- tigershark 8y ago
  There have been decades of attempts at databases with a different model than relational and we have seen what happened. The reality is that a relational model is very well suited for general purpose databases. For specific needs you can use timeseries databases or key/value store, but at this point I seriously doubt that the Nth attempt of killing the relational model will succeed. And honestly I prefer an enforced relational schema rather than an ungodly mess of schemaless documents.
- Cthulhu_ 8y ago
  There have been attempts at rethinking it though; noSQL was the buzzword of a couple of years ago, and even nowdays there's mature tools like Firebase that allow you to store and retrieve data much more directly than e.g. SQL. The challenge in nosql storage is of course data migrations and whatnot. But yeah, in theory you can just open up a MongoDB instance to your front-end and not have to bother with SQL or much of a back-end. And your last comment doesn't make much sense tbf; the big frameworks, most notably Angular and React (and its ecosystem) were both designed to be great front-end technologies.
  - lkrubner 8y ago
gutnor 8y ago
You miss the last layer: user and browser. Even with all the improvement is web technologies in recent past, the browser is still not close to native widget. And finally the users themselves. No matter how neat you managed to be in the underlying layers, the UI is messy. It can change fast, meaning either massive cost rebuilding an entire application, or breaking those neat layer. Users also wants everything connected to everything. It does not matter if those connections are explicitely done via ugly spagetthi code or implicitely through clever abstractions, they effectively exist and that's how the requirements, user experience feedback, bugs and testing will be based on.
emodendroket 8y ago
I don't see what's so "leaky" about going from 2-4.
brylie 8y ago
Are there plans to publish Anvil as open source?
- arunix 8y ago
  And are there any open source projects similar to Anvil?

chrisweekly 8y ago

Some good points... but I think you're too pessimistic. Some tiny subset of those fraction of users you mention manage to do something different / better. React is one example. GraphQL is another.

bjterry 8y ago

This is a great comment and an amazing insight. What's particularly interesting is that people have attempted to collapse (almost?) every stage of that abstraction hierarchy individually, but none of them have been so successful as to take over the world. If you were writing a desktop application, you would still have at least three of the layers (serialized data on disk, in-memory data, and the rendering of the objects), but without the dramatic impedance mismatch that the web platform introduces everywhere.

adrianratnapala 8y ago
Even on the desktop those three layers involve impedance mismatch and much the same pathologies as meredydd describes. But I guess three layers of it are better than 6.
- pas 8y ago
  It's data. There's a fix format for serializing it. In a lossless way. I don't know what meredydd talks about, there's no mismatch with regards to data. You can get the same bits in your JS objects as you have in the DB. If not, that means your system is shit. The problem with frameworks is not the hardship of funneling data up and down the stack. The problem is that they are optimizing for different things. React optimizes for simplicity of making components. Angular optimizes for providing a full toolkit. And new versions then focus on different things. Server Side Rendering was hot, but now that Google just executes some JS and penalizes large downloads, it's the quest for less bytes on the wire. And tree-shake-ability. And faster time to first paint. And as browsers and the web changes, so do frameworks. And frameworks try to target, at the same time, both the future, and the very present problems, they try to provide instant gratification, yet try to optimize for the future. So they usually look half-assed useless pieces of autogenerated-by-MS-Word code all the time. But they work, nevertheless, and power a lot of sites.
  - randomdata 8y ago
    > I don't know what meredydd talks about, there's no mismatch with regards to data. He's talking about different services each having their own preferred way to structure the data. When the layout differs, it cannot simply be a memcpy, and so you get tools to try to ease the tedium of translating one structure's layout into to another. They get the job done most of the time, but run into edge cases that return the developer back to manual tedium. Since developers do not like tedious work, some set out to find a new solution that solves for those edge cases, but they end up leaving many more on the table for the next intrepid developer.
    - 8y ago

zebraflask 8y ago

(B) Nails it. I'd guess that a large portion of new frameworks start simply because everyone who spent enough time with the old one gets sick of the bad or non-existent documentation for edge cases, etc. Rinse and repeat. Vue is no better, after reading the rest of the comments. In many ways worse.

Chyzwar 8y ago

That's good, in each cycle, sooner or later you get improvement: webpack over grunt, react over jQuery and npm over vendoring your jQuery plugins. OP forgot how life looked when your web app project was handcrafted HTML page with manually inserted scripts tags. When your form submission was multi-level backend API in PHP. jQuery plugins with 20+ options published randomly on the internet.

aaronbrethorst 8y ago
I feel like this is a false dichotomy. The choice doesn't have to be roasting squirrels over an open flame/handcrafting PHP pages vs. shiny futurism/Node+React. I have been getting along just fine with Rails, HTML, and a sprinkling of JS for over 10 years.
- ehnto 8y ago
  There is an immense productivity gain to be found in mastering a set of tools. After a while of dogmatic tooling changes you begin to analyze more critically whether the new shiny thing is going to provide any real value. I would argue that for the huge majority of websites, tried and tested tooling is perfectly fine and definitely more robust and supported. Every tool you introduce is hours of troubleshooting just waiting to happen.
- emodendroket 8y ago
  That stuff works fine if your client-side needs are simple, but if you actually want a single-page application, or just an application with a lot of rich JS functionality, it quickly becomes unwieldy.
  - Chris_Newton 8y ago
    That stuff works fine if your client-side needs are simple To be fair, just about anything works fine if your client-side needs are simple. However, I have reached the opposite conclusion to you: the more customised and complicated and large-scale and long-lived the software becomes, the less value I see in a lot of the popular but ever-changing web technologies and the more I am likely to favour building on the standard foundations with minimal dependencies in between and usually a relatively small but high-value set of libraries. The benefits of quickly fetching many tiny packages with a package manager or of building on top of all-encompassing frameworks or automation tools are mostly found in two situations, in my experience: getting started quickly (including rapid prototyping exercises) and ongoing development if (and only if) you are staying almost entirely within the bounds of what your chosen technologies already do well. However, if your requirements start to evolve and diversify in a longer-lasting project, it’s all too easy for those numerous tiny dependencies to become a liability or for that framework or tool you built everything around to become a straitjacket. The relatively short lifetimes of many of these technologies can also become an expensive problem if the community drifts away and the security and compatibility work slows down or stops entirely but your project still depends on them as much as ever.

tracker1 8y ago

All of this is why, in the end, more and more... I do things "the hard way". If I'm using a dynamic language WTF do I need an ORM for, if I understand enough to write an SQL command, and use a library for that DB that does parameterized queries? On the front end, I tend to lean towards abstractions that work together... I really like React and the material-ui library's switch to JSS. It's relatively clean, and useful. Even then, it's only a mild syntax adjustment, not a full on abstraction. React is more of an abstraction, but that comes with functional paradigms that aid in testing, and predictive behaviors. It really depends though. One can always do just JS/HTML/CSS, and there's something to be said for that. There are lighter tools that are similar to JQuery to smooth over a few of the rough edges. There's really an ala cart of available options. The problem is that people assume that the PFM (pure fucking magic) will solve it all for them. You can use the cleanest or simplest abstractions, and then still write layers of incomprehensible spaghetti in between.

bonesss 8y ago
> I do things "the hard way". If I'm using a dynamic language WTF do I need an ORM for I think it's mostly premature optimization. People think writing DTOs is challenging, so they want an ORM. But since you end up needing DTOs anwyays, removing SQL capabilities from the app means writing SQL in not SQL, and things like joins suddenly become slow and problematic and result in really heavy systems that are harder to change. For the joy of a quicker startup the entire project moves slower. ORMs have their place, but in the majority of the systems I've seen they were unnecessary, and in broad terms don't provide any particular productivity advantage over using "dumb" SQL-based mapping solutions (a la Dapper [https://github.com/StackExchange/Dapper] https://github.com/StackExchange/Dapper]), that preserve the power of SQL.
- unclebucknasty 8y ago
  About 17 years ago, I wrote a GUI-based code generator that allows me to generate the boilerplate JDBC cruft from SQL statements, with several options for common scenarios. The code it generates is extensible and provides helpers for extending. To this day, I haven't found anything (including ORMs, Spring support, etc.) easier to use, more flexible, or more sensible.

lkrubner 8y ago

What is curious is that the tech industry has been very conservative about rethinking these 6 things that you just listed. Why rows in a relational database? Why objects? Why Javascript? And why HTML? We'd surely be in a better place if we got rid of these things and rethought our approach from first principles. I've written about these issues many times before. Regarding the problem with objects, I wrote "Object Oriented Programming Is An Expensive Disaster Which Must End": http://www.smashcompany.com/technology/object-oriented-programming-is-an-expensive-disaster-which-must-end http://www.smashcompany.com/technology/object-oriented-progr... Regarding the problems of HTML, I wrote "The problem with HTML": http://www.smashcompany.com/technology/the-problem-with-html http://www.smashcompany.com/technology/the-problem-with-html To answer the question "Why Is Front-End Development So Unstable?" the answer is surely, in part, the fact that we refuse to build technologies that are designed to be great front-end technologies.

tigershark 8y ago
There have been decades of attempts at databases with a different model than relational and we have seen what happened. The reality is that a relational model is very well suited for general purpose databases. For specific needs you can use timeseries databases or key/value store, but at this point I seriously doubt that the Nth attempt of killing the relational model will succeed. And honestly I prefer an enforced relational schema rather than an ungodly mess of schemaless documents.
Cthulhu_ 8y ago
There have been attempts at rethinking it though; noSQL was the buzzword of a couple of years ago, and even nowdays there's mature tools like Firebase that allow you to store and retrieve data much more directly than e.g. SQL. The challenge in nosql storage is of course data migrations and whatnot. But yeah, in theory you can just open up a MongoDB instance to your front-end and not have to bother with SQL or much of a back-end. And your last comment doesn't make much sense tbf; the big frameworks, most notably Angular and React (and its ecosystem) were both designed to be great front-end technologies.
- lkrubner 8y ago

gutnor 8y ago

You miss the last layer: user and browser. Even with all the improvement is web technologies in recent past, the browser is still not close to native widget. And finally the users themselves. No matter how neat you managed to be in the underlying layers, the UI is messy. It can change fast, meaning either massive cost rebuilding an entire application, or breaking those neat layer. Users also wants everything connected to everything. It does not matter if those connections are explicitely done via ugly spagetthi code or implicitely through clever abstractions, they effectively exist and that's how the requirements, user experience feedback, bugs and testing will be based on.

emodendroket 8y ago

I don't see what's so "leaky" about going from 2-4.

brylie 8y ago

Are there plans to publish Anvil as open source?

arunix 8y ago
And are there any open source projects similar to Anvil?