9 ms·

If this is true, then why does it take so many engineers to make web sites today? People keep claiming these things increase productivity, yet there seems to b

by jblow 7y ago

If this is true, then why does it take so many engineers to make web sites today? People keep claiming these things increase productivity, yet there seems to be very little evidence of productivity increase.

JimmyRuska 7y ago
There's a lot of shops that are pure django, drupal, laravel, wordpress experts, who would never be able to put something together from scratch, but can probably glue you a customized website in not a lot of time from relatively well tested components. I remember going to a meetup for level 10 in Dallas, TX. Huge beautiful building they had, all from django and wordpress revenue. I guess it's like anything else, what are you trying to optimize for. There's still a lot of charm in PHP being a very simple wrapper to a lot of very efficient C functions.
- ScottFree 7y ago
  > There's a lot of shops that are pure django, drupal, laravel, wordpress experts, who would never be able to put something together from scratch, but can probably glue you a customized website in not a lot of time from relatively well tested components. Nobody denies that these frameworks allow people who would never be able to program in C/C++ to create websites. Nobody denies that it takes more of these quasi-programmers to create and maintain websites than it would take more knowledgeable programmers writing in more efficient languages. So, here's the killer question: At which point do these frameworks and the large number of quasi-programmers required to maintain them cost more than just hiring somebody who can create a native app in C/C++?
  - taurath 7y ago
    Hire a C/C++ programmer and see how that goes for you. If what you’re saying is true they can revolutionize the industry and make a billion dollars with their superior code.
    - 0xEFF 7y ago
      I’ve seen it happen. The code is faster, but it doesn’t matter because users don’t care. It also took two years to rewrite, doesn’t have the features of the original, and all of the operational experience had to be relearned by the engineering and ops teams. So, millions of dollars, years of time for no actual benefit, and now the business is left with an app they can’t hire talent to maintain because it’s so custom.
  - jpmoral 7y ago
    'Quasi-programmer'? I don't think elitism helps your point.
    - deleted 7y ago
      [deleted]
  - Daishiman 7y ago
    The C/C++ engineers are going to spend their time dealing with memory management issues that have no relevance to the final delivery of product features. And if industry experience says somethings, those engineers who spent that time honing their skills at wrangling the C++ stdlib will have _appalingly_ bad design and UI skills, which are very relevant for today's web and are a much more important driver of revenue.
taurath 7y ago
The ones that have a lot of engineers have 50x more features today. Businesses and brands want a beautiful website with analytics and A/B testing out of the box, skinnable, changable via a CMS so that non technical designers can drag and drop in new components at will, and also get graphs of their efficacy. You have built in support chat systems to spend less on phone support. Your IT department to support, not develop, this site was 30+ with colo space in 2008 but now its like 3 dedicated ops people and the rest are devs. OTOH, so many more business websites are built for a few hundred or thousand dollars by someone on upwork to host a bootstrap site with a shopify integration, and then you have an online store that would've been 10x the cost 5 years ago.
csande17 7y ago
+1. For example, according to one of their job listings[1], Twitter currently has 25 people working on the main twitter.com experience. Twenty-five people! For a Twitter client! And they're still trying to hire more! (Important to keep in mind: this doesn't include any of the people working on the database, or the advertising algorithm, or the server architecture, or any of the other hard problems Twitter faces. This is just the team that writes the JavaScript code that takes the Tweets from the JSON API response and puts them onto the screen.) [1] https://careers.twitter.com/en/work-for-twitter/201910/twitter-com-senior-javascript-engineer.html https://careers.twitter.com/en/work-for-twitter/201910/twitt...
- taurath 7y ago
  The amount of ignorance of the complexity required in a globally reachable frontend web app is astounding in this thread.
  - csande17 7y ago
    I understand why it's hard to make a popular, globally-reachable web service -- you have to run a bunch of servers in different locations, and distribute the database, and solve concurrency problems and stuff. So I get why Twitter might have large back-end and ops teams. But client-side code doesn't have scaling issues like that. If I put some JavaScript code in a static file, it'll work exactly the same when 1 person runs it as when a million people run it. What "complexity" is Twitter's front-end team facing as a result of their global reach? Like, I guess they have to localize their app, but the hard part there is getting the text translated; the technical implementation is usually as easy as putting all your UI strings in a dictionary that you swap out based on the user's language. (Maybe if they're extra fancy, they make a version of their stylesheet that flips the layout horizontally for Arabic users.)
    - x0x0 7y ago
      I doubt you've ever done a translation in a large webapp. Wait until German overflows your text boxes. Or you have to deal with date formats and number formats and ... And rtl languages are just a stylesheet away. Hah, but no. And how many devices across how many OSes do you test on? Particularly all the devices that functionally no one in the global north purchases? What is your accessibility test plan and how often do you run it?
deleted 7y ago
[deleted]
topkai22 7y ago
A couple reasons- 1) As the cost of feature creation goes down, the demand for features goes up- as others have mentioned, a lot of sites now have an enormous number of interactive features or cool rendering that just wasn't there in the good old days of nothing but text, images, and marquee tags. 2) Different frameworks solve different problems, not all of which are related to "productivity." JQuery solved a browser compatibility issues and made interactivity way more approachable, AngularJS did a great job dealing with input and output mechanics, and React is pretty awesome at managing compomentization for large projects. And a lot of frameworks give a "feel" users are looking for. Frameworks improve productivity, but the difference between frameworks is not that great. 3) one thing frameworks do well is allow team scaling, and scaling teams leads to a lowering of "individual" productivity. What works on a team of 1 doesn't on a team of 5. The larger team will need more "overhead." A team of 5 won't be able to do a task an individual could do 5x faster, it'll be more like 2.5x or something. That's not why you scale a team though- you scale a team to take on new problems that can't be solved (or solved fast enough) at the scale you are at currently. I've run a decent size team using angular JS, I shudder to think of our ability to execute using JQuery, even though I successfully accomplished scaled down projects in those frameworks before.
tr1coder 7y ago
Jonathan Blow has a good presentation containing some interesting facts about the companies that employ thousands of engineers and can't get simple things to work. It's the first part of https://www.youtube.com/watch?v=k56wra39lwA https://www.youtube.com/watch?v=k56wra39lwA EDIT: OMG it is you! xD
nprateem 7y ago
Part of the productivity "gain" is increasing agility, making it easier/faster to scale/pivot if you need to. Whether you actually do or not is up to you, and you pay a time penalty as an overhead for those abilities when using a framework. It's similar with choosing e.g. Hadoop instead of running entirely locally. You're mitigating the risk that in future your workload might exponentially increase. Yeah, it probably won't but if it does, whoever decided to on that architecture will look like a forward-planning genius. But nothing comes free.
rkangel 7y ago
My intuition (that I have no data to support) is because the average level of skill of software engineers is dropping. As writing software has gained a reputation for being a lucrative and desirable profession, it has attracted more and more people. If you were someone who took up programming in the 80s it was probably because you had a good aptitude for it and found it interesting. Now, there are more people joining the profession who don't have the same level of aptitude. So now instead of a small team of highly skilled people, you often end up with large teams of less skilled people. Interestingly there doesn't seem to be much of a halfway house - your project is either staffed by 6 people or 30+.
- wruza 7y ago
  Frameworks also effectively hide “bare metal” from developer, so they develop magical thinking. I’m really tired at my job of debunking magical and simply wrong assumptions and claims about how something works. Being unable to understand and implement something yourself, shadowed by Framework Authority, dooms one to remain in a pit of false incompetence.
  - thfuran 7y ago
    Is false incompetence where true genius lies?
- collyw 7y ago
  > My intuition (that I have no data to support) is because the average level of skill of software engineers is dropping. As someone that has been in the industry close to 20 years, my guess is that the constant churn is getting in the way. Around 2000 when I started, we had 3 tier apps, people knew how to optimise relational databases to get their application to perform. These days we need to add Kubernetes, Docker, Nosql to cloud platforms in order for it to scale - despite never needing this stuff 15 years ago. Instead of mastering our craft we are encouraged to forget everything and use this new technology instead, as it will be the silver bullet to all our problems. It's closer to a fashion show than engineering.
mindfulplay 7y ago
Why would it take a single engineer 3+ years to build a Mario-like game with a tweak around time-mechanics? I am discounting the artwork and asking primarily around the efficiency of the coding and development. Each developer has a focus area. You might focus on millisecond level optimization; a web-developer might focus on content provisioning and rapid development; a data engineer might care more about sanity and robustness of data. It's not clear how anyone can judge other engineers' ability based on arbitrary metrics without considering their engineering system's goals and requirements. As an example, both Braid and The Witness have very limited accessibility; perhaps that wasn't a focus area of yours and again each developer to his/her own.

by jblow 7y ago

JimmyRuska 7y ago

There's a lot of shops that are pure django, drupal, laravel, wordpress experts, who would never be able to put something together from scratch, but can probably glue you a customized website in not a lot of time from relatively well tested components. I remember going to a meetup for level 10 in Dallas, TX. Huge beautiful building they had, all from django and wordpress revenue. I guess it's like anything else, what are you trying to optimize for. There's still a lot of charm in PHP being a very simple wrapper to a lot of very efficient C functions.

ScottFree 7y ago
> There's a lot of shops that are pure django, drupal, laravel, wordpress experts, who would never be able to put something together from scratch, but can probably glue you a customized website in not a lot of time from relatively well tested components. Nobody denies that these frameworks allow people who would never be able to program in C/C++ to create websites. Nobody denies that it takes more of these quasi-programmers to create and maintain websites than it would take more knowledgeable programmers writing in more efficient languages. So, here's the killer question: At which point do these frameworks and the large number of quasi-programmers required to maintain them cost more than just hiring somebody who can create a native app in C/C++?
- taurath 7y ago
  Hire a C/C++ programmer and see how that goes for you. If what you’re saying is true they can revolutionize the industry and make a billion dollars with their superior code.
  - 0xEFF 7y ago
    I’ve seen it happen. The code is faster, but it doesn’t matter because users don’t care. It also took two years to rewrite, doesn’t have the features of the original, and all of the operational experience had to be relearned by the engineering and ops teams. So, millions of dollars, years of time for no actual benefit, and now the business is left with an app they can’t hire talent to maintain because it’s so custom.
- jpmoral 7y ago
  'Quasi-programmer'? I don't think elitism helps your point.
  - deleted 7y ago
    [deleted]
- Daishiman 7y ago
  The C/C++ engineers are going to spend their time dealing with memory management issues that have no relevance to the final delivery of product features. And if industry experience says somethings, those engineers who spent that time honing their skills at wrangling the C++ stdlib will have _appalingly_ bad design and UI skills, which are very relevant for today's web and are a much more important driver of revenue.

taurath 7y ago

The ones that have a lot of engineers have 50x more features today. Businesses and brands want a beautiful website with analytics and A/B testing out of the box, skinnable, changable via a CMS so that non technical designers can drag and drop in new components at will, and also get graphs of their efficacy. You have built in support chat systems to spend less on phone support. Your IT department to support, not develop, this site was 30+ with colo space in 2008 but now its like 3 dedicated ops people and the rest are devs. OTOH, so many more business websites are built for a few hundred or thousand dollars by someone on upwork to host a bootstrap site with a shopify integration, and then you have an online store that would've been 10x the cost 5 years ago.

csande17 7y ago

+1. For example, according to one of their job listings[1], Twitter currently has 25 people working on the main twitter.com experience. Twenty-five people! For a Twitter client! And they're still trying to hire more! (Important to keep in mind: this doesn't include any of the people working on the database, or the advertising algorithm, or the server architecture, or any of the other hard problems Twitter faces. This is just the team that writes the JavaScript code that takes the Tweets from the JSON API response and puts them onto the screen.) [1] https://careers.twitter.com/en/work-for-twitter/201910/twitter-com-senior-javascript-engineer.html https://careers.twitter.com/en/work-for-twitter/201910/twitt...

taurath 7y ago
The amount of ignorance of the complexity required in a globally reachable frontend web app is astounding in this thread.
- csande17 7y ago
  I understand why it's hard to make a popular, globally-reachable web service -- you have to run a bunch of servers in different locations, and distribute the database, and solve concurrency problems and stuff. So I get why Twitter might have large back-end and ops teams. But client-side code doesn't have scaling issues like that. If I put some JavaScript code in a static file, it'll work exactly the same when 1 person runs it as when a million people run it. What "complexity" is Twitter's front-end team facing as a result of their global reach? Like, I guess they have to localize their app, but the hard part there is getting the text translated; the technical implementation is usually as easy as putting all your UI strings in a dictionary that you swap out based on the user's language. (Maybe if they're extra fancy, they make a version of their stylesheet that flips the layout horizontally for Arabic users.)
  - x0x0 7y ago
    I doubt you've ever done a translation in a large webapp. Wait until German overflows your text boxes. Or you have to deal with date formats and number formats and ... And rtl languages are just a stylesheet away. Hah, but no. And how many devices across how many OSes do you test on? Particularly all the devices that functionally no one in the global north purchases? What is your accessibility test plan and how often do you run it?

deleted 7y ago

[deleted]

topkai22 7y ago

A couple reasons- 1) As the cost of feature creation goes down, the demand for features goes up- as others have mentioned, a lot of sites now have an enormous number of interactive features or cool rendering that just wasn't there in the good old days of nothing but text, images, and marquee tags. 2) Different frameworks solve different problems, not all of which are related to "productivity." JQuery solved a browser compatibility issues and made interactivity way more approachable, AngularJS did a great job dealing with input and output mechanics, and React is pretty awesome at managing compomentization for large projects. And a lot of frameworks give a "feel" users are looking for. Frameworks improve productivity, but the difference between frameworks is not that great. 3) one thing frameworks do well is allow team scaling, and scaling teams leads to a lowering of "individual" productivity. What works on a team of 1 doesn't on a team of 5. The larger team will need more "overhead." A team of 5 won't be able to do a task an individual could do 5x faster, it'll be more like 2.5x or something. That's not why you scale a team though- you scale a team to take on new problems that can't be solved (or solved fast enough) at the scale you are at currently. I've run a decent size team using angular JS, I shudder to think of our ability to execute using JQuery, even though I successfully accomplished scaled down projects in those frameworks before.

tr1coder 7y ago

Jonathan Blow has a good presentation containing some interesting facts about the companies that employ thousands of engineers and can't get simple things to work. It's the first part of https://www.youtube.com/watch?v=k56wra39lwA https://www.youtube.com/watch?v=k56wra39lwA EDIT: OMG it is you! xD

nprateem 7y ago

Part of the productivity "gain" is increasing agility, making it easier/faster to scale/pivot if you need to. Whether you actually do or not is up to you, and you pay a time penalty as an overhead for those abilities when using a framework. It's similar with choosing e.g. Hadoop instead of running entirely locally. You're mitigating the risk that in future your workload might exponentially increase. Yeah, it probably won't but if it does, whoever decided to on that architecture will look like a forward-planning genius. But nothing comes free.

rkangel 7y ago

My intuition (that I have no data to support) is because the average level of skill of software engineers is dropping. As writing software has gained a reputation for being a lucrative and desirable profession, it has attracted more and more people. If you were someone who took up programming in the 80s it was probably because you had a good aptitude for it and found it interesting. Now, there are more people joining the profession who don't have the same level of aptitude. So now instead of a small team of highly skilled people, you often end up with large teams of less skilled people. Interestingly there doesn't seem to be much of a halfway house - your project is either staffed by 6 people or 30+.

wruza 7y ago
Frameworks also effectively hide “bare metal” from developer, so they develop magical thinking. I’m really tired at my job of debunking magical and simply wrong assumptions and claims about how something works. Being unable to understand and implement something yourself, shadowed by Framework Authority, dooms one to remain in a pit of false incompetence.
- thfuran 7y ago
  Is false incompetence where true genius lies?
collyw 7y ago
> My intuition (that I have no data to support) is because the average level of skill of software engineers is dropping. As someone that has been in the industry close to 20 years, my guess is that the constant churn is getting in the way. Around 2000 when I started, we had 3 tier apps, people knew how to optimise relational databases to get their application to perform. These days we need to add Kubernetes, Docker, Nosql to cloud platforms in order for it to scale - despite never needing this stuff 15 years ago. Instead of mastering our craft we are encouraged to forget everything and use this new technology instead, as it will be the silver bullet to all our problems. It's closer to a fashion show than engineering.

mindfulplay 7y ago

Why would it take a single engineer 3+ years to build a Mario-like game with a tweak around time-mechanics? I am discounting the artwork and asking primarily around the efficiency of the coding and development. Each developer has a focus area. You might focus on millisecond level optimization; a web-developer might focus on content provisioning and rapid development; a data engineer might care more about sanity and robustness of data. It's not clear how anyone can judge other engineers' ability based on arbitrary metrics without considering their engineering system's goals and requirements. As an example, both Braid and The Witness have very limited accessibility; perhaps that wasn't a focus area of yours and again each developer to his/her own.