9 ms·

I think pg understates the problem. It isn't just fashionable. Think about how the solutions we use today evolved. Someone had an idea originally and certain d

I think pg understates the problem. It isn't just fashionable. Think about how the solutions we use today evolved. Someone had an idea originally and certain decisions were made about that approach.. those that best adapted to the conditions of the time were successful... rinse and repeat over several decades. Those ideas that preserved the past were more likely to succeed because they preserved the ecosystem that already existed. Ideas that diverged too much from the existing successful tech had a huge obstacle in their way: they had to recreate all of the existing solutions in their new model. So even if they would lead to a better solution over time, they may never get past that initial roadblock. And the longer we stay on the path, the bigger that roadblock becomes. If we are all thinking of ideas around the existing model and assuming all of the existing assumptions, then we end up with similar solutions (fashionable solutions). We've reduced the solution space, leading to a limited number of solutions. Perhaps groundbreaking/valuable tech can be found by questioning those existing assumptions; identifying where tech is still built on assumptions that are no longer true; or reexamining past solutions to see if they can be solved in better ways today with what we've learned since then...

cjblomqvist 7y ago
Many insights to be made by this comment. One that is easily missed is how hard it is to fight against the existing ecosystem/assumptions. I've found that a lot of people either don't, or make a point of challenging everything. A strategy they've worked fairly well for me so far is to challenge one(!) assumption after a lot of careful analysis, and then try to adapt to the ecosystem as best as possible. Most revolutionary things are not made in one big bang, but in many incremental steps. They only look like big bangs in hindsight. Even the nuclear bomb was fairly incremental, with truly a big bang in the end.... (pun intended ;)
- jbverschoor 7y ago
  The problem is that multi-layered problem solving is superhard to execute. It's not that it's hard to find a solution that would be orders better, but it's just that it's risky, as it's an all-or-nothing path. It at least requires a lot of confidence for the existing market. With waterfall-kind of planning, that would work. Unfortunately it got a bad name in the '90s and early '00s. The wild-west of inexperienced developers, who needed to solve business problems they didn't understand, combined with clients who didn't understand technology. I believe this is a great period to do bigger things, and you see this happening with for example SpaceX, Tesla and Apple: space is happening again, companies are creating super specialized and complex ICs, there's actual business value and consumer value being produced, and projects are reasonably on time with these companies. This in constract with companies who, what it seems, do more of an iterative approach: Facebook, Google, Amazon. No huge innovation going there.
  - cjblomqvist 7y ago
    Aren't those companies great examples of this though? They have a great vision, but really try to play inside the existing system except for one thing. Tesla for example haven't yet done much innovation that the other car companies don't, except for betting hard on electricity (or more specifically, batteries). A more radical approach challenging the whole system on all fronts would be to re-think personal transportation (perhaps cars VS horses is a better example of that). Sure, they realize that in order to make electrical cars viable, they have to put up chargers. But that's really just a means to an end (batteries/electricity as fuel). Same goes with how they sell cars - they have to in order to break through the market, it's not something that is necessarily core to their business or why they exist. They exist to replace ICEs with battery driven mirrors. The rest can stay. (autonomous driving might be more revolutionary if they succeed and create something different than what cars are today, but that's a lot of ifs and buts) Apple is also very much in that corner. The smartphone existed long before the iPhone. They just packaged it and polished the concept so it made sense for a broader market (and they are/were really great at that), but how radical were they really? Revolutionary after a while perhaps (if you can be a slow revolutionary - seems a little bit contradictory), but technologically they just put a lot of existing techs together and packaged it differently than the existing players for a different market/crowd and had tremendous timing (I bet the smartphone revolution was coming either way, they just helped define the concept and bridge the gap from early adopters to mass market).
    - trhway 7y ago
      You, like many others, me sometime ago including, are missing the point that can be illustrated with Apple - Jobs broke the carrier control over software that is running on the device which, until Jobs, had been a given unquestionable fact of life like the sky is blue. That is what unleashed the revolution, not the nice piece of hardware which would be just a brick under ATT control like many before.
    - erikpukinskis 7y ago
      Consider that Tesla is still an abject failure on their goal of converting the world’s transport to sustainable energy. I don’t think the strategy you outlined will be enough. In addition to “advance batteries” I believe “full self-driving” as well as “mobility as a service”, “radically production-driven design”, and possibly “car bundled with energy production and storage devices” will be required for Tesla to hit their goal. If you take those 4-5 things together I do believe that constitutes a total reinvention of the automobile.
stephenamills 7y ago
"Ideas that diverged too much from the existing successful tech had a huge obstacle in their way: they had to recreate all of the existing solutions in their new model.” You’re so right! There are great examples of companies that preserved the existing ecosystem of solutions (Facebook with PHP, Microsoft with C++, Google with Java and C++, maybe also Amazon with Perl 5 and C++). There are also great examples of successful companies that in a certain sense did or had to do things in an entirely new model (WhatsApp with the Open Telecom Platform, F5 Networks with their data center FPGA-powered hardware load balancers, Tesla and SpaceX, and so on). There are also great examples of companies that did both, where they preserved an existing ecosystem but also added another ecosystem on top to fix flaws (NeXT Computer with Smalltalk and C, Stripe with the modern version of that as Ruby and Go, Twitter with Ruby and Java, so on). Microsoft is the best example of a company that can succeed even with what seems like a crazy choice. Almost everything being written in C++ sounds insane to the point of being business suicidal, but they worked hard enough to make it work. If Microsoft can succeed in the ways that they have with C++, and if Facebook could succeed for as long as they did with PHP, and Instagram can allegedly succeed with Django[1], then anyone can succeed with anything as long as you can endure the stress that a seemingly peculiar decision might cause you. And those standard ecosystems that they chose all had immense benefits, they just also had some pretty immense weaknesses as well (Powering 500 million daily users primarily with Python scripts?? That just seems pretty crazy to me, if it’s true). [1]Still very unclear about how much Python actually powers the web services behind Instagram. <https://instagram-engineering.com/tagged/python> https://instagram-engineering.com/tagged/python>.
- bawolff 7y ago
  I don't see why writing an operating system and surounding tools in C++ is a crazy decision (I'm assuming you meant windows & desktop apps by saying microsoft). Certainly not in the early 90's, but if you have a big codebase of that already, sticking with it doesn't seem crazy to me. People make too much of a fuss over the differences between programming language. There are some important differences (whether or not memory safe is a big one, but it is hardly critical to a succful bussiness) but by and large the difference between using say php, python, etc for your program is superficial in the extreme.
- 7y ago
darksaints 7y ago
An interesting theory as to why we never evolved with wheels is the idea that an evolutionary improvement requires a viable intermediate state...something that works well enough in the transition from inferior to superior. It is worth noting that human inventions can evolve in much faster timelines than biological evolution, but no matter how fast they come, there does need to be a way to fulfill intermediate needs, or else there is no path forward. What follows is my own personal views on current technological evolution, which may be wrong. Regardless: Electric vehicles have been evolving extremely rapidly, and now they are viable enough for most privately owned passenger vehicle use cases. Through continuous evolutionary investment, they may make their way to some commercial vehicles, maybe even small planes. But the chasm is so extremely large for large commercial vehicles like airliners and cargo ships, that we would need massive (several orders of magnitude) technological improvements in energy density in order to even start considering them. SOFCs are inferior to batteries and supercapacitors from an efficiency standpoint. They may never be the power system of choice for passenger cars or other small scale and lightly used systems. But the thing they have going for them is their ability to evolve as the ecosystem evolves. They can run on diesel fuel, JP8, even crude. They can run on biodiesel, or renewable ethanol or methanol. And they can run on pure hydrogen. They can even run on a mix of all of those fuels. At every path in the transition to hydrogen, there is a viable intermediate state. For this reason, I think you'll see fuel cell powered cargo ships and airliners long before you see battery powered.
- thaumasiotes 7y ago
  > An interesting theory as to why we never evolved with wheels is the idea that an evolutionary improvement requires a viable intermediate state Not really; wheels aren't an improvement in the first place so there is no need to explain why they didn't develop. Note that it's easy to make robots that use wheels, and difficult to make robots that use legs, but we make legged robots anyway so that they'll be able to handle environments other than dedicated roads. (More recently, we make flying robots, sidestepping the issue that we don't really know how to do legs well.)
  - darksaints 7y ago
jandrewrogers 7y ago
This is called "first principles thinking" and is one of the most reliable methods for producing novel insights into a problem space. It is uncommon in practice because it has a high cost both economically, because you are re-deriving everything you think you know from scratch instead of "standing on the shoulders of giants", and socially, because you are deviating from orthodoxy promoted by high-status individuals. This kind of relentless indifference to conformity is a rare quality in people.
- jacques_chester 7y ago
  There's also the cost that you are likely to just spin your wheels retreading old ground. Examples to the contrary are notable, but notable because they are rare.
  - warent 7y ago
    This here is the real danger. Many people choose to reinvent the wheel and end up finding out that their approach was actually horribly flawed compared to what already existed, or they turned out not to have the resources to see it through. It takes a brilliant person with a lot of luck to make this work in an advantageous way. Two great examples come to mind right away, both from Elon Musk. SpaceX was founded on first principles and now he's been able to undercut competitors by an order of magnitude. Conversely, his attempt approaching car manufacturing from first principles in the gigafactory has proven to be a complete disaster.
    - Qwertious 7y ago
      >Conversely, his attempt approaching car manufacturing from first principles in the gigafactory has proven to be a complete disaster. Nonsense. What is that based on?
    - Aloha 7y ago
      I think the older and more competitive a space is, the less likely a first principles approach will bring success.
bumby 7y ago
Another hurdle to examining alternate models is “theory induced blindness” where new approaches are either unseen or written off because they don’t fit well within a more traditional framework. It’s especially hard to convince those who have staked a career on the theory that blinds them.
ssivark 7y ago
Which means that this is what the exploration pattern ends up looking like in practice: https://www.smbc-comics.com/index.php?db=comics&id=2866#comic https://www.smbc-comics.com/index.php?db=comics&id=2866#comi... Aside: This SMBC shall be long etched in my memory: "All the best work has been done over here! [...] The funding is here too!" --- Here's a teaser challenge for someone who has access to HN comments data for analytics: How often does an early comment come to dominate the discussion, rather than a better comment which comes later? Assume that there is some IID stochastic process which generates comments of varying quality (play with your distribution of choice), and plot what the time distribution of "best" comments looks like under that model -- compared to the time distribution of most upvoted/discussed comments on HN (can try different composite metrics here too). --- Provocative meta question for PG: Are (venture-funded) startups the "fashionable" idea that is over-subscribed? How would one know? (suppose we restrict the discussion to Silicon Valley)
rdiddly 7y ago
Inertia seems a good metaphor too, although that too lacks a bit of the specificity you're getting at.
hinkley 7y ago
Isn't this just evolution versus revolution? Most of the time new products evolve from the current state. Sometimes there's pent-up demand/frustration/hostility that allows for a clean break to a new solution (even if that solution has just as many problems, at least they're not the same problems). Probably nine times out of ten people are looking for a better buggy whip. Without any other information, improving what you have is statistically the better bet. You have a very strong predictor, but no guarantees.
jacques_chester 7y ago
You're describing what economists and systems theorists call "path dependency". It is common that someone has questioned the assumptions, but that the cost to overcome the path dependency is higher than the gained efficiency or capability. Meaning: the rareness of disruption is to be expected, because most disruptions fail in stifling silence.

cjblomqvist 7y ago

Many insights to be made by this comment. One that is easily missed is how hard it is to fight against the existing ecosystem/assumptions. I've found that a lot of people either don't, or make a point of challenging everything. A strategy they've worked fairly well for me so far is to challenge one(!) assumption after a lot of careful analysis, and then try to adapt to the ecosystem as best as possible. Most revolutionary things are not made in one big bang, but in many incremental steps. They only look like big bangs in hindsight. Even the nuclear bomb was fairly incremental, with truly a big bang in the end.... (pun intended ;)

jbverschoor 7y ago
The problem is that multi-layered problem solving is superhard to execute. It's not that it's hard to find a solution that would be orders better, but it's just that it's risky, as it's an all-or-nothing path. It at least requires a lot of confidence for the existing market. With waterfall-kind of planning, that would work. Unfortunately it got a bad name in the '90s and early '00s. The wild-west of inexperienced developers, who needed to solve business problems they didn't understand, combined with clients who didn't understand technology. I believe this is a great period to do bigger things, and you see this happening with for example SpaceX, Tesla and Apple: space is happening again, companies are creating super specialized and complex ICs, there's actual business value and consumer value being produced, and projects are reasonably on time with these companies. This in constract with companies who, what it seems, do more of an iterative approach: Facebook, Google, Amazon. No huge innovation going there.
- cjblomqvist 7y ago
  Aren't those companies great examples of this though? They have a great vision, but really try to play inside the existing system except for one thing. Tesla for example haven't yet done much innovation that the other car companies don't, except for betting hard on electricity (or more specifically, batteries). A more radical approach challenging the whole system on all fronts would be to re-think personal transportation (perhaps cars VS horses is a better example of that). Sure, they realize that in order to make electrical cars viable, they have to put up chargers. But that's really just a means to an end (batteries/electricity as fuel). Same goes with how they sell cars - they have to in order to break through the market, it's not something that is necessarily core to their business or why they exist. They exist to replace ICEs with battery driven mirrors. The rest can stay. (autonomous driving might be more revolutionary if they succeed and create something different than what cars are today, but that's a lot of ifs and buts) Apple is also very much in that corner. The smartphone existed long before the iPhone. They just packaged it and polished the concept so it made sense for a broader market (and they are/were really great at that), but how radical were they really? Revolutionary after a while perhaps (if you can be a slow revolutionary - seems a little bit contradictory), but technologically they just put a lot of existing techs together and packaged it differently than the existing players for a different market/crowd and had tremendous timing (I bet the smartphone revolution was coming either way, they just helped define the concept and bridge the gap from early adopters to mass market).
  - trhway 7y ago
    You, like many others, me sometime ago including, are missing the point that can be illustrated with Apple - Jobs broke the carrier control over software that is running on the device which, until Jobs, had been a given unquestionable fact of life like the sky is blue. That is what unleashed the revolution, not the nice piece of hardware which would be just a brick under ATT control like many before.
  - erikpukinskis 7y ago
    Consider that Tesla is still an abject failure on their goal of converting the world’s transport to sustainable energy. I don’t think the strategy you outlined will be enough. In addition to “advance batteries” I believe “full self-driving” as well as “mobility as a service”, “radically production-driven design”, and possibly “car bundled with energy production and storage devices” will be required for Tesla to hit their goal. If you take those 4-5 things together I do believe that constitutes a total reinvention of the automobile.

stephenamills 7y ago

"Ideas that diverged too much from the existing successful tech had a huge obstacle in their way: they had to recreate all of the existing solutions in their new model.” You’re so right! There are great examples of companies that preserved the existing ecosystem of solutions (Facebook with PHP, Microsoft with C++, Google with Java and C++, maybe also Amazon with Perl 5 and C++). There are also great examples of successful companies that in a certain sense did or had to do things in an entirely new model (WhatsApp with the Open Telecom Platform, F5 Networks with their data center FPGA-powered hardware load balancers, Tesla and SpaceX, and so on). There are also great examples of companies that did both, where they preserved an existing ecosystem but also added another ecosystem on top to fix flaws (NeXT Computer with Smalltalk and C, Stripe with the modern version of that as Ruby and Go, Twitter with Ruby and Java, so on). Microsoft is the best example of a company that can succeed even with what seems like a crazy choice. Almost everything being written in C++ sounds insane to the point of being business suicidal, but they worked hard enough to make it work. If Microsoft can succeed in the ways that they have with C++, and if Facebook could succeed for as long as they did with PHP, and Instagram can allegedly succeed with Django[1], then anyone can succeed with anything as long as you can endure the stress that a seemingly peculiar decision might cause you. And those standard ecosystems that they chose all had immense benefits, they just also had some pretty immense weaknesses as well (Powering 500 million daily users primarily with Python scripts?? That just seems pretty crazy to me, if it’s true). [1]Still very unclear about how much Python actually powers the web services behind Instagram. <https://instagram-engineering.com/tagged/python> https://instagram-engineering.com/tagged/python>.

bawolff 7y ago
I don't see why writing an operating system and surounding tools in C++ is a crazy decision (I'm assuming you meant windows & desktop apps by saying microsoft). Certainly not in the early 90's, but if you have a big codebase of that already, sticking with it doesn't seem crazy to me. People make too much of a fuss over the differences between programming language. There are some important differences (whether or not memory safe is a big one, but it is hardly critical to a succful bussiness) but by and large the difference between using say php, python, etc for your program is superficial in the extreme.
7y ago

darksaints 7y ago

An interesting theory as to why we never evolved with wheels is the idea that an evolutionary improvement requires a viable intermediate state...something that works well enough in the transition from inferior to superior. It is worth noting that human inventions can evolve in much faster timelines than biological evolution, but no matter how fast they come, there does need to be a way to fulfill intermediate needs, or else there is no path forward. What follows is my own personal views on current technological evolution, which may be wrong. Regardless: Electric vehicles have been evolving extremely rapidly, and now they are viable enough for most privately owned passenger vehicle use cases. Through continuous evolutionary investment, they may make their way to some commercial vehicles, maybe even small planes. But the chasm is so extremely large for large commercial vehicles like airliners and cargo ships, that we would need massive (several orders of magnitude) technological improvements in energy density in order to even start considering them. SOFCs are inferior to batteries and supercapacitors from an efficiency standpoint. They may never be the power system of choice for passenger cars or other small scale and lightly used systems. But the thing they have going for them is their ability to evolve as the ecosystem evolves. They can run on diesel fuel, JP8, even crude. They can run on biodiesel, or renewable ethanol or methanol. And they can run on pure hydrogen. They can even run on a mix of all of those fuels. At every path in the transition to hydrogen, there is a viable intermediate state. For this reason, I think you'll see fuel cell powered cargo ships and airliners long before you see battery powered.

thaumasiotes 7y ago
> An interesting theory as to why we never evolved with wheels is the idea that an evolutionary improvement requires a viable intermediate state Not really; wheels aren't an improvement in the first place so there is no need to explain why they didn't develop. Note that it's easy to make robots that use wheels, and difficult to make robots that use legs, but we make legged robots anyway so that they'll be able to handle environments other than dedicated roads. (More recently, we make flying robots, sidestepping the issue that we don't really know how to do legs well.)
- darksaints 7y ago

jandrewrogers 7y ago

This is called "first principles thinking" and is one of the most reliable methods for producing novel insights into a problem space. It is uncommon in practice because it has a high cost both economically, because you are re-deriving everything you think you know from scratch instead of "standing on the shoulders of giants", and socially, because you are deviating from orthodoxy promoted by high-status individuals. This kind of relentless indifference to conformity is a rare quality in people.

jacques_chester 7y ago
There's also the cost that you are likely to just spin your wheels retreading old ground. Examples to the contrary are notable, but notable because they are rare.
- warent 7y ago
  This here is the real danger. Many people choose to reinvent the wheel and end up finding out that their approach was actually horribly flawed compared to what already existed, or they turned out not to have the resources to see it through. It takes a brilliant person with a lot of luck to make this work in an advantageous way. Two great examples come to mind right away, both from Elon Musk. SpaceX was founded on first principles and now he's been able to undercut competitors by an order of magnitude. Conversely, his attempt approaching car manufacturing from first principles in the gigafactory has proven to be a complete disaster.
  - Qwertious 7y ago
    >Conversely, his attempt approaching car manufacturing from first principles in the gigafactory has proven to be a complete disaster. Nonsense. What is that based on?
  - Aloha 7y ago
    I think the older and more competitive a space is, the less likely a first principles approach will bring success.

bumby 7y ago

Another hurdle to examining alternate models is “theory induced blindness” where new approaches are either unseen or written off because they don’t fit well within a more traditional framework. It’s especially hard to convince those who have staked a career on the theory that blinds them.

ssivark 7y ago

Which means that this is what the exploration pattern ends up looking like in practice: https://www.smbc-comics.com/index.php?db=comics&id=2866#comic https://www.smbc-comics.com/index.php?db=comics&id=2866#comi... Aside: This SMBC shall be long etched in my memory: "All the best work has been done over here! [...] The funding is here too!" --- Here's a teaser challenge for someone who has access to HN comments data for analytics: How often does an early comment come to dominate the discussion, rather than a better comment which comes later? Assume that there is some IID stochastic process which generates comments of varying quality (play with your distribution of choice), and plot what the time distribution of "best" comments looks like under that model -- compared to the time distribution of most upvoted/discussed comments on HN (can try different composite metrics here too). --- Provocative meta question for PG: Are (venture-funded) startups the "fashionable" idea that is over-subscribed? How would one know? (suppose we restrict the discussion to Silicon Valley)

rdiddly 7y ago

Inertia seems a good metaphor too, although that too lacks a bit of the specificity you're getting at.

hinkley 7y ago

Isn't this just evolution versus revolution? Most of the time new products evolve from the current state. Sometimes there's pent-up demand/frustration/hostility that allows for a clean break to a new solution (even if that solution has just as many problems, at least they're not the same problems). Probably nine times out of ten people are looking for a better buggy whip. Without any other information, improving what you have is statistically the better bet. You have a very strong predictor, but no guarantees.

jacques_chester 7y ago

You're describing what economists and systems theorists call "path dependency". It is common that someone has questioned the assumptions, but that the cost to overcome the path dependency is higher than the gained efficiency or capability. Meaning: the rareness of disruption is to be expected, because most disruptions fail in stifling silence.