5 ms·

> Signs of realization that pay is only weakly correlated with skill This is of course the big problem. Facebook is admitting that developers elsewhere in the

> Signs of realization that pay is only weakly correlated with skill This is of course the big problem. Facebook is admitting that developers elsewhere in the world are just as good as developers in Silicon Valley. Or if you want to put it another way: if Silicon Valley is special, it's NOT because the developers.

paxys 6y ago
It is special not because of the quality of developers but because there are so many of them together in close proximity. It still remains to be seen whether the 100% remote experiment will work out or not as an industry standard, so let's not write the Silicon Valley obituaries just yet.
zaptheimpaler 6y ago
Personally, i think there was a time where we genuinely needed to solve hard engineering problems - around 2010-2015 when a ton of companies figured out concurrency, distributed systems, scaling, way better web tech, smartphones etc.. in those times companies needed a LOT of good devs, so the network effect Silicon Valley had was valuable. And the VCs saw huge empires to be built and so much money on the table, so high salaries were easy to justify. Now the market is crowded, scaling is a simple economic problem, the network effect is turning on its head and there isn't any revolutionary software to build on the horizon. Times have changed but the narrative hasn't caught up yet.. its going to suck when it does honestly. Software is just so commoditized and cookie cutter at this point i can't see it being a great career for much longer.
- packetlost 6y ago
  There are still hard engineering problems to solve, they just aren't related to scaling out services as much (there's still optimizations and improvements to be made in that space, but not nearly as much). You can't seriously think all the hard engineering problems have been solved, right?
  - zaptheimpaler 6y ago
    Give me some then! I see maybe some cool stuff in the move to ARM processors. I also don't have the resume or fame or background to get offered the chance to solve them if they're out there - in 2015 there were so many and the average level of competence was so low that people gave me a shot anyways. I've been trying to learn more about the dark corners of the linux kernel and graphics drivers for now. But its not 2015 and I don't have 3 years of experience in embedded/systems/C++ so no ones gonna hire me for that stuff because my resume just says "generic backend dev lol".
    - packetlost 6y ago
      > in 2015 there were so many and the average level of competence was so low The average competence is still low, the technologies are just no longer new and shiny (they aren't mature either), they're just better understood throughout the industry. 2010-2015 saw huge cross-industry revolutions in terms of web adoption, and until the next "revolution" happens, we'll probably be in a lull in terms of innovation in the web app space. Of course JS/Node will continue to "innovate", but overall I think that portion of the industry has been successfully "commoditized." Hardware engineering (drivers, embedded, IoT, etc.) will always have tough engineering problems, but as you said, it's kind of hard to break into that (though maybe not, I haven't tried). Quantum Computing and other experimental and uncharted spaces are good sources of problems looking for well engineered solution (source: I work in quantum computing, on the engineering side), but is a much smaller and niche industry. > I see maybe some cool stuff in the move to ARM processors I also agree on ARM, I think things will gradually shift towards ARM-based processor architecture in the server and desktop spaces. From the software side, most modern programming can be run on those processors without issue, though there's some work to be done on optimizing compilers (hard, not very rewarding). > I've been trying to learn more about the dark corners of the linux kernel and graphics drivers for now This is an area I'm also particularly interested in. If I wasn't getting close to burning out, I'd probably focus on it on the side a little more. AMD is behind, in particular, with regards to graphics driver optimization and toolkits for machine learning (nVidia is beating them out there, thanks CUDA). I'm not sure what their hiring is like, but there's been huge industry shakeups over the last few years from them in the processor space and they're swimming in money. Machine learning is also a hot topic, and probably will be for awhile. Security engineering (especially in the context of Kubernetes) is an ongoing issue that needs more attention. There's definitely others. I'd eventually like to find a problem that interests me and I see a market for so I could start my own company, but alas, I am not good at ideas.
- Hermitian909 6y ago
  There are still a lot of hard problems to solve, they're just cross-disciplinary. Software is the infrastructure of the 21st century and if you step out of the tech bubble to things like material science, biotech, etc. you'll see that their infrastructure is garbage despite there being really big returns available. Developers who can marry modern software to those domains should remain in high demand for decades.
  - zaptheimpaler 6y ago
    Do you have any companies or spaces you would recommend where theres hard problems but low signalling/barriers to entry? I'm autistic AF (no joke) and learn new domains in a frenzy of 3 months of joy and digging but its not easy to prove/signal that.
    - dkn775 6y ago
      The best one is state or local government. You’ll be able to make a lot of change and get tons of experience with a low barrier to entry. Anything related to public safety (including transportation) probably needs the most tech talent.
  - Tehchops 6y ago
    So much this. Spend a month with a tech stack of any passing complexity, even ones at $FAANGco... We're nowhere close to solving all the hard problems.
- willberman 6y ago
  Do you mind being specific on how there are no longer hard engineering problems to be solved?
  - zaptheimpaler 6y ago
    Well beginning in 2008ish, we saw websites and applications that served the entire world, literally billions of users. That simply did not exist before then. The internet as a whole was a tiny part of peoples lives. All the stuff that went into solving that was genuinely new ideas in servers, databases, ops, data centre's etc. Eventual consistency went from being a joke to the only game in town. Distributed computing on large data sets went from a Google paper in 2005 on Hadoop to a widely used idea. Smartphones went from zero to iPhone to literally more than half the world has one. Devops became standard. Google went from just managing 1 or 2 datacenters to running AI to optimize the power efficiency of each one. When you already have architecture that is capable of serving the entire world, when everyone has a smartphone all that's left in that space is efficiency/cost optimization. Whatever shiny new programming language or framework comes out, the bedrock of computing is solid (and the shiny new programming language is really mashing together decades old ideas which is cool if you're new to them but boring if you're not) . Every company is a mess on the inside of course, but they do their core functions well enough. AWS did not exist 10 years ago, now it's big internet drama when it goes down for a few hours twice a YEAR.A detailed postmortem comes out (not standard before), revealing the problem was a configuration change (because everything has long since been automated, so it's driven purely by configuration) . I know not everyone is interested in those same problems, but I thought they were interesting and what's left now in the guts of software is an adoption phase, people rediscovering lessons that others long since figured out. Perhaps people who work on higher layers of the stack think differently, I'm only interested in the deep tech below, and it's pretty solid at this point.
    - willberman 6y ago
      There's still lots of relevant work to do deeper in the stack. It might just not be at large internet scale companies (I doubt this but lets assume it's true). Most every university has a systems research lab doing interesting things. I think I paged through 5 different professor's websites in CMU's systems department the other month and found 9-10 interesting projects.
minhazm 6y ago
You make it sound like this is some big secret. It's not like Facebook exclusively hires from people in the area only. Facebook and nearly every company regularly hire people from outside the area. Most developers in the bay area are from outside the area. Out of my team of 30 people there's maybe 1 person that's from the bay area. Everyone else was re-located.
dmode 6y ago
I mean, if you extrapolate that, developers in US should really earn min wage as you can get someone in a developing country with a lower wage no matter what
gilbetron 6y ago
Having done a recent stint at a Silicon Valley startup, I was honestly shocked to discover most of the engineers I was working with were at the bottom half of those I've worked with in my career. I think there are far more great engineers in SV than many places, but the truly great ones are still relatively rare and gobbled up by wealthier companies rather quickly. Even at 10x the normal amount of top notch engineers, that still only means a few thousand in the SV area.

paxys 6y ago

It is special not because of the quality of developers but because there are so many of them together in close proximity. It still remains to be seen whether the 100% remote experiment will work out or not as an industry standard, so let's not write the Silicon Valley obituaries just yet.

zaptheimpaler 6y ago

Personally, i think there was a time where we genuinely needed to solve hard engineering problems - around 2010-2015 when a ton of companies figured out concurrency, distributed systems, scaling, way better web tech, smartphones etc.. in those times companies needed a LOT of good devs, so the network effect Silicon Valley had was valuable. And the VCs saw huge empires to be built and so much money on the table, so high salaries were easy to justify. Now the market is crowded, scaling is a simple economic problem, the network effect is turning on its head and there isn't any revolutionary software to build on the horizon. Times have changed but the narrative hasn't caught up yet.. its going to suck when it does honestly. Software is just so commoditized and cookie cutter at this point i can't see it being a great career for much longer.

packetlost 6y ago
There are still hard engineering problems to solve, they just aren't related to scaling out services as much (there's still optimizations and improvements to be made in that space, but not nearly as much). You can't seriously think all the hard engineering problems have been solved, right?
- zaptheimpaler 6y ago
  Give me some then! I see maybe some cool stuff in the move to ARM processors. I also don't have the resume or fame or background to get offered the chance to solve them if they're out there - in 2015 there were so many and the average level of competence was so low that people gave me a shot anyways. I've been trying to learn more about the dark corners of the linux kernel and graphics drivers for now. But its not 2015 and I don't have 3 years of experience in embedded/systems/C++ so no ones gonna hire me for that stuff because my resume just says "generic backend dev lol".
  - packetlost 6y ago
    > in 2015 there were so many and the average level of competence was so low The average competence is still low, the technologies are just no longer new and shiny (they aren't mature either), they're just better understood throughout the industry. 2010-2015 saw huge cross-industry revolutions in terms of web adoption, and until the next "revolution" happens, we'll probably be in a lull in terms of innovation in the web app space. Of course JS/Node will continue to "innovate", but overall I think that portion of the industry has been successfully "commoditized." Hardware engineering (drivers, embedded, IoT, etc.) will always have tough engineering problems, but as you said, it's kind of hard to break into that (though maybe not, I haven't tried). Quantum Computing and other experimental and uncharted spaces are good sources of problems looking for well engineered solution (source: I work in quantum computing, on the engineering side), but is a much smaller and niche industry. > I see maybe some cool stuff in the move to ARM processors I also agree on ARM, I think things will gradually shift towards ARM-based processor architecture in the server and desktop spaces. From the software side, most modern programming can be run on those processors without issue, though there's some work to be done on optimizing compilers (hard, not very rewarding). > I've been trying to learn more about the dark corners of the linux kernel and graphics drivers for now This is an area I'm also particularly interested in. If I wasn't getting close to burning out, I'd probably focus on it on the side a little more. AMD is behind, in particular, with regards to graphics driver optimization and toolkits for machine learning (nVidia is beating them out there, thanks CUDA). I'm not sure what their hiring is like, but there's been huge industry shakeups over the last few years from them in the processor space and they're swimming in money. Machine learning is also a hot topic, and probably will be for awhile. Security engineering (especially in the context of Kubernetes) is an ongoing issue that needs more attention. There's definitely others. I'd eventually like to find a problem that interests me and I see a market for so I could start my own company, but alas, I am not good at ideas.
Hermitian909 6y ago
There are still a lot of hard problems to solve, they're just cross-disciplinary. Software is the infrastructure of the 21st century and if you step out of the tech bubble to things like material science, biotech, etc. you'll see that their infrastructure is garbage despite there being really big returns available. Developers who can marry modern software to those domains should remain in high demand for decades.
- zaptheimpaler 6y ago
  Do you have any companies or spaces you would recommend where theres hard problems but low signalling/barriers to entry? I'm autistic AF (no joke) and learn new domains in a frenzy of 3 months of joy and digging but its not easy to prove/signal that.
  - dkn775 6y ago
    The best one is state or local government. You’ll be able to make a lot of change and get tons of experience with a low barrier to entry. Anything related to public safety (including transportation) probably needs the most tech talent.
- Tehchops 6y ago
  So much this. Spend a month with a tech stack of any passing complexity, even ones at $FAANGco... We're nowhere close to solving all the hard problems.
willberman 6y ago
Do you mind being specific on how there are no longer hard engineering problems to be solved?
- zaptheimpaler 6y ago
  Well beginning in 2008ish, we saw websites and applications that served the entire world, literally billions of users. That simply did not exist before then. The internet as a whole was a tiny part of peoples lives. All the stuff that went into solving that was genuinely new ideas in servers, databases, ops, data centre's etc. Eventual consistency went from being a joke to the only game in town. Distributed computing on large data sets went from a Google paper in 2005 on Hadoop to a widely used idea. Smartphones went from zero to iPhone to literally more than half the world has one. Devops became standard. Google went from just managing 1 or 2 datacenters to running AI to optimize the power efficiency of each one. When you already have architecture that is capable of serving the entire world, when everyone has a smartphone all that's left in that space is efficiency/cost optimization. Whatever shiny new programming language or framework comes out, the bedrock of computing is solid (and the shiny new programming language is really mashing together decades old ideas which is cool if you're new to them but boring if you're not) . Every company is a mess on the inside of course, but they do their core functions well enough. AWS did not exist 10 years ago, now it's big internet drama when it goes down for a few hours twice a YEAR.A detailed postmortem comes out (not standard before), revealing the problem was a configuration change (because everything has long since been automated, so it's driven purely by configuration) . I know not everyone is interested in those same problems, but I thought they were interesting and what's left now in the guts of software is an adoption phase, people rediscovering lessons that others long since figured out. Perhaps people who work on higher layers of the stack think differently, I'm only interested in the deep tech below, and it's pretty solid at this point.
  - willberman 6y ago
    There's still lots of relevant work to do deeper in the stack. It might just not be at large internet scale companies (I doubt this but lets assume it's true). Most every university has a systems research lab doing interesting things. I think I paged through 5 different professor's websites in CMU's systems department the other month and found 9-10 interesting projects.

minhazm 6y ago

You make it sound like this is some big secret. It's not like Facebook exclusively hires from people in the area only. Facebook and nearly every company regularly hire people from outside the area. Most developers in the bay area are from outside the area. Out of my team of 30 people there's maybe 1 person that's from the bay area. Everyone else was re-located.

dmode 6y ago

I mean, if you extrapolate that, developers in US should really earn min wage as you can get someone in a developing country with a lower wage no matter what

gilbetron 6y ago

Having done a recent stint at a Silicon Valley startup, I was honestly shocked to discover most of the engineers I was working with were at the bottom half of those I've worked with in my career. I think there are far more great engineers in SV than many places, but the truly great ones are still relatively rare and gobbled up by wealthier companies rather quickly. Even at 10x the normal amount of top notch engineers, that still only means a few thousand in the SV area.