7 ms·

This is a very good question because it gets into a deeper issue that I find in all corners of real world programming and results in large complicated applicati

This is a very good question because it gets into a deeper issue that I find in all corners of real world programming and results in large complicated applications even when they use powerful frameworks. Real-world programming is absolutely choc full of things that look like simple applications of a well known solutions, but which turn out to need lots of special-snowflake code to actually implement. Part of this is unforced errors, but usually I can't look at other people's work and say in detail how I would have done it substantially better. So I think there is something fundamental going on. One guess is that although we can write tools that automate the perfectly common case, we are not good at writing things that smooth the inevitable app-specific corner case.

memebox3a 9y ago
I agree. I think its best to think about it in terms of high dimensional spaces and manifolds. The manifold of solutions to business problems is currently not well approximated by the way that we decompose business problems. And by this I mean that if you were to look at the manifold of problems solvable using current techniques in amount of time x, you would find that in general that manifold is a long way away from the manifold of useful business applications. I'm not saying that there is a better decomposition - a better decomposition that leads to a manifold that lies closer to the target manifold - it may be that the manifold is complex in a way that makes it impossible to approximate efficiently in terms of time to develop. And we may be butting up against the fact that most objects in mathematical space do not have effecient ways of representing them. I'm thinking of kolmorogrov complexity and the pigeon hole principle. Most outputs have no short programs to produce them.
- sshine 9y ago
  Being John Manifold.
  - rlkf 9y ago
    Although a bit subtle, I think the line of exchange from top and down to here is amongst the best I've seen.
binocarlos 9y ago
100% this - I've spent many years feeling like I'm one abstraction away from having a killer library that would get me the initial crud based app done in a single line of code... I've never actually managed it - the edge cases mean I end up chasing my tail around instead of "just coding the app". Now I use small libraries for individual parts (like menus and forms) but even then 50% of the app becomes special edge case forms anyway. Accepting that the job is to just implement the spec and not invent a rube goldberg machine is an important lesson to absorb.
- davidjnelson 9y ago
  The goal is to have something that makes "just coding the app" simpler and faster without sacrificing 100% customizability or user experience quality.
xg15 9y ago
I think even with edge-cases, the question remains valid why you can't automate the common cases. I wonder how often it's actually non-standard requirements and how often it's a mix of processes and poor communication/changing requirements. I think in many projects, in hindsight it's perfectly obvious it is just a standard CRUD app - but it might need a few iterations to get the understanding of all stakeholders to that point... Also, seems to me that such tools clash with the organisational process of how webapps are created today: It almost seems to be required that a web app has some kind of custom-written "business layer" in between, even if that business layer doesn't do anything than pass requests around.
- adrianratnapala 9y ago
  I think heavyweight frameworks often make the edge cases harder, because when you need to do something "unusual" you end up fighting the framework, and now you end up with a complex program where the framework as well as the buisness needs count as the problem domain. But you are likely onto something that some things are just standard apps even if you can't see that at the time. And my mind might not be well tuned to seeing it even in hindsight. Which makes me think that banging out simple apps fast is a very specific skill, because you need to see how to meet business needs within the slot provided to your by framework.
danenania 9y ago
I think it mostly comes down to user experience. The best user experience is usually going to be specifically tailored to an individual application. It won't be very generalizable. Apps created with ui builders (or even with just a constrained set of ui components like bootstrap) tend not to have good ux since every workflow has to be shoehorned into an existing abstraction. It's convenient for the programmer (and the budget), but not for the user.
buro9 9y ago
A really good example of this could be validation. Almost every CRUD app builder includes the ability to validate fields and properties, and this would appear to be sufficient for nearly all scenarios except it is not. I can not name a single CRUD app that I have built that didn't require multi-field validation... i.e. start date prior to end date, time must be completed if date is filled in, if this part of the form is filled in then that part must be blank. Without this capability to really express the application business logic, the technical simplification fails short.
- artpar 9y ago
  The reason for that is because we tend to "stop thinking after automating CRUD". I think CRUD is only the basic tools on top of which "Actions" and "Streams" need to be built. Thinking about scenarios, example "Sign up" is not really just "Create a user row in db" but probably assign them a group also, and more. Similarly, we rarely would use a "R" in CRUD (except administrators). Read is a basic operation, more frequently we need transformations/filters/aggregates on this Read, hence streams. I have lately been trying to formulate the 90% of the 10% remaining. Checkout my other comment if interested.
- adrianratnapala 9y ago
  The existence of multi-field validation is a sign that your app can't really get a way with an "anemic" domain model -- i.e. the app is not just a front-end to the DB. What's really going on here is that you have a domain object with some nontrivial invariant. It's a mistake for the UI or anything that is aware of the UI library to try to model these domain invariants. Instead a call should go the model layer which then returns (or throws) an error providing enough info for the UI to gracefully help the user. The UI framework can play an important role in this: but that role is to help programmers provide a UI for the validator that lives in a model layer. It is not to do the validation itself. (Although frameworks should provoide an arsenal of simple, single field validators, of the "this field has data-type phone-number" kind).
  - buro9 9y ago
    Sure, I don't disagree. But for the current state of CRUD application frameworks that do not do this, it implies some enforcement at the data layer and probably some hint at this at the ORM level. This really does not exist in any sufficient way. It reminds me of the idea that a graph/diagram generator that used some generic data source could be a common analytics tool, and that with a few well-defined defaults no-one would need to run lots of different analytics tools, or they could be generated automatically and would be good enough. In reality they never are good enough. There is no one-size-fits-all analytics system because the KPIs differ from company to company, project to project, and these differences go all the way down. Sometimes we see something being sold as a "this is all you need", i.e. a SAP solution or some generic tooling, even Stripe, etc. But the level of customisation typically required to make it sufficient for a mature organisation to use it is typically quite high. A strategy to resolve this usually is: Change the org to fit the tech, rather than the tech to fit the org. But I've seen this fail consistently as typically the tech is too simplistic. Then there's the "customise it to fit your needs", but going back to custom multi-field validation, or analytics... the number of permutations of logic is immense and the customisation tools themselves only allow simple expression of that customisation. The end conclusion is that we need extreme levels of customisation, but when provided through tooling the effort to perform and manage the customisation (with auditing of who changed what, when, etc) is such that actually it is simpler and cheaper to write the customisation in code.

memebox3a 9y ago

I agree. I think its best to think about it in terms of high dimensional spaces and manifolds. The manifold of solutions to business problems is currently not well approximated by the way that we decompose business problems. And by this I mean that if you were to look at the manifold of problems solvable using current techniques in amount of time x, you would find that in general that manifold is a long way away from the manifold of useful business applications. I'm not saying that there is a better decomposition - a better decomposition that leads to a manifold that lies closer to the target manifold - it may be that the manifold is complex in a way that makes it impossible to approximate efficiently in terms of time to develop. And we may be butting up against the fact that most objects in mathematical space do not have effecient ways of representing them. I'm thinking of kolmorogrov complexity and the pigeon hole principle. Most outputs have no short programs to produce them.

sshine 9y ago
Being John Manifold.
- rlkf 9y ago
  Although a bit subtle, I think the line of exchange from top and down to here is amongst the best I've seen.

binocarlos 9y ago

100% this - I've spent many years feeling like I'm one abstraction away from having a killer library that would get me the initial crud based app done in a single line of code... I've never actually managed it - the edge cases mean I end up chasing my tail around instead of "just coding the app". Now I use small libraries for individual parts (like menus and forms) but even then 50% of the app becomes special edge case forms anyway. Accepting that the job is to just implement the spec and not invent a rube goldberg machine is an important lesson to absorb.

davidjnelson 9y ago
The goal is to have something that makes "just coding the app" simpler and faster without sacrificing 100% customizability or user experience quality.

xg15 9y ago

I think even with edge-cases, the question remains valid why you can't automate the common cases. I wonder how often it's actually non-standard requirements and how often it's a mix of processes and poor communication/changing requirements. I think in many projects, in hindsight it's perfectly obvious it is just a standard CRUD app - but it might need a few iterations to get the understanding of all stakeholders to that point... Also, seems to me that such tools clash with the organisational process of how webapps are created today: It almost seems to be required that a web app has some kind of custom-written "business layer" in between, even if that business layer doesn't do anything than pass requests around.

adrianratnapala 9y ago
I think heavyweight frameworks often make the edge cases harder, because when you need to do something "unusual" you end up fighting the framework, and now you end up with a complex program where the framework as well as the buisness needs count as the problem domain. But you are likely onto something that some things are just standard apps even if you can't see that at the time. And my mind might not be well tuned to seeing it even in hindsight. Which makes me think that banging out simple apps fast is a very specific skill, because you need to see how to meet business needs within the slot provided to your by framework.

danenania 9y ago

I think it mostly comes down to user experience. The best user experience is usually going to be specifically tailored to an individual application. It won't be very generalizable. Apps created with ui builders (or even with just a constrained set of ui components like bootstrap) tend not to have good ux since every workflow has to be shoehorned into an existing abstraction. It's convenient for the programmer (and the budget), but not for the user.

buro9 9y ago

A really good example of this could be validation. Almost every CRUD app builder includes the ability to validate fields and properties, and this would appear to be sufficient for nearly all scenarios except it is not. I can not name a single CRUD app that I have built that didn't require multi-field validation... i.e. start date prior to end date, time must be completed if date is filled in, if this part of the form is filled in then that part must be blank. Without this capability to really express the application business logic, the technical simplification fails short.

artpar 9y ago
The reason for that is because we tend to "stop thinking after automating CRUD". I think CRUD is only the basic tools on top of which "Actions" and "Streams" need to be built. Thinking about scenarios, example "Sign up" is not really just "Create a user row in db" but probably assign them a group also, and more. Similarly, we rarely would use a "R" in CRUD (except administrators). Read is a basic operation, more frequently we need transformations/filters/aggregates on this Read, hence streams. I have lately been trying to formulate the 90% of the 10% remaining. Checkout my other comment if interested.
adrianratnapala 9y ago
The existence of multi-field validation is a sign that your app can't really get a way with an "anemic" domain model -- i.e. the app is not just a front-end to the DB. What's really going on here is that you have a domain object with some nontrivial invariant. It's a mistake for the UI or anything that is aware of the UI library to try to model these domain invariants. Instead a call should go the model layer which then returns (or throws) an error providing enough info for the UI to gracefully help the user. The UI framework can play an important role in this: but that role is to help programmers provide a UI for the validator that lives in a model layer. It is not to do the validation itself. (Although frameworks should provoide an arsenal of simple, single field validators, of the "this field has data-type phone-number" kind).
- buro9 9y ago
  Sure, I don't disagree. But for the current state of CRUD application frameworks that do not do this, it implies some enforcement at the data layer and probably some hint at this at the ORM level. This really does not exist in any sufficient way. It reminds me of the idea that a graph/diagram generator that used some generic data source could be a common analytics tool, and that with a few well-defined defaults no-one would need to run lots of different analytics tools, or they could be generated automatically and would be good enough. In reality they never are good enough. There is no one-size-fits-all analytics system because the KPIs differ from company to company, project to project, and these differences go all the way down. Sometimes we see something being sold as a "this is all you need", i.e. a SAP solution or some generic tooling, even Stripe, etc. But the level of customisation typically required to make it sufficient for a mature organisation to use it is typically quite high. A strategy to resolve this usually is: Change the org to fit the tech, rather than the tech to fit the org. But I've seen this fail consistently as typically the tech is too simplistic. Then there's the "customise it to fit your needs", but going back to custom multi-field validation, or analytics... the number of permutations of logic is immense and the customisation tools themselves only allow simple expression of that customisation. The end conclusion is that we need extreme levels of customisation, but when provided through tooling the effort to perform and manage the customisation (with auditing of who changed what, when, etc) is such that actually it is simpler and cheaper to write the customisation in code.