6 ms·

Honestly, I think the more important takeaway isn’t the decline of NoSQL. As the article says, it’s the conclusion that’s what’s good for FAANG isn’t necessaril

by shrumm 6y ago

Honestly, I think the more important takeaway isn’t the decline of NoSQL. As the article says, it’s the conclusion that’s what’s good for FAANG isn’t necessarily what’s good for your average project. I know this sounds obvious but it’s been a constant source of frustration for me with the hype train. You could apply the same logic to a whole bunch of tech like Kubernetes, service mesh etc etc and arrive at the same result. Every tech has a trade off, understanding it is critical. Don’t pick tech spot SOLELY based on FAANG use.

hooloovoo_zoo 6y ago
Sometimes engineers have perverse incentives. If you want a FAANG job, it probably helps to have experience with FAANG tech, even if your current company doesn't need it.
- ehnto 6y ago
  It is a very frustrating industry to spend a long time in. A hamster wheel of constantly shifting goal posts for mastery. I wouldn't mind if they were fundamental advances in software development, but it's mostly the same stuff with small advances but massive learning curves of arbitrary, non-transferrable minutiae. The side effect of this ever shifting tooling set, is almost no one masters anything. All software is varying levels of crap written by newbies, because when the tools change every project you are always a newbie in that tool set.
  - llbbdd 6y ago
    As someone at a FAANG company right now, your second paragraph neatly identifies a core frustration I have with my job that I haven't been able to articulate before now. Might be time for a change...
  - kqr 6y ago
    > I wouldn't mind if they were fundamental advances in software development, but it's mostly the same stuff with small advances but massive learning curves of arbitrary, non-transferrable minutiae. I think this is pinpointing the key exploitability in the market. If you have seen enough tech come and go, you can figure out which 98% of this year's idea are the same as the year before, and 40 years ago. At that point you can start cutting through the bullshit and design things using brand new tech as if you had used it for 20 years already. At that point you're way ahead of the rest of the pack. (And you can choose not to use the brand new thing, and argue convincingly for why the almost-exactly-the-same 30 year old, more mature and stable, tech is better.)
    - krab 6y ago
      I mostly agree. Though you may still develop blind spots and miss a crucial difference in this year's iteration of the same idea.
  - vinay_ys 6y ago
    Firstly software development is not one monolithic job. There are many different jobs within software development. You can be a systems engineer – understand the computers at a fundamental level – with focus on systems software – building low-level high-performance components/kernels – for storage systems, databases, in-memory systems etc (each of these have a lot of technical domain knowledge specialisation within them). You can spend 10 years in this field and continuously develop/enhance your expertise incrementally – it is quite stable and rewarding experience. A branch of this specialization is distributed systems engineering – where emphasis is on software operating in a distributed cluster over network – which has additional challenges unique to the aspect of being distributed. You can be an applications engineer – understand application software engineering methodology – with focus on user/business facing application feature development in a scalable software team setting. The key focus here is not so much about low-level computer systems but more about software engineering discipline. It is about the inter/intra-team sport that is developing and continuously evolving a very large application software code base that is alive with changing/evolving user/business features. It is about modeling the functional domain of the user/business/real world. It is about the modern consumer Internet software development techniques – experimentation, live incremental safe feature releases etc. A branch of this specialization is developing application frameworks and tooling (rather than user/business functional features) to make the life of a feature engineer more productive. You can spend 10 years in this field and become an expert at software engineering discipline of churning out quality features on time and on budget. The experience and learning in the above job families transcend any particular technology stack. The learnings are transferable from one tech-stack/functional-domain to another with relatively minimal effort. This effort is part of the work itself and doesn't turn you into a newbie for having to do it. p.s: A generalist full stack engineer is usually an applications engineer who is somewhat good at systems engineering and is able to glue together systems to achieve the application features well. These engineers can take a startup from start and through initial growth phase and up to start of hyper growth phase. But there's a scale/performance threshold – where the scale of application deployment grows, performance starts to hurt your users, you need strong systems engineering specialists to fix those deeper systems/distributed-systems problems. Public cloud systems have been continuously raising that threshold since the beginning.
  - theshadowknows 6y ago
    I just learned last week that we're bringing Pega on for some projects. Never even heard of Pega before then lol. Time to sit down with a free account and learn something else that we'll throw away in five years.
    - treeman79 6y ago
      5 years is sadly decent run. Was in a big Javascript project, where any article over 3 months old was useless or wrong. The internals of the app constantly being rewritten to flavor of the month.
  - jrochkind1 6y ago
    This rings so true to me. I really like mastering things, so I've tried to stick with a technology stack (that I have kind of mastered), but I don't think it's been good for my career.
- tjpnz 6y ago
  It's not just engineers either. I've seen new EMs copy and paste ill fitting FAANG development practices into organizations and then bounce off to which ever company championed it not long after.
- whatsmyusername 6y ago
  Yeah we've run into this. Before my time they gave a full application to one developer, who proceeded to build it in a language we barely use on tech no one else in the company understands (and, while not bad, isn't even close to something we'd choose to use in any other circumstance). Once it was done...ish, he ducked out to be a consultant, leaving others to clean up the mess (we ended up just selling it for not what we put into it).
FartyMcFarter 6y ago
> Don’t pick tech spot SOLELY based on FAANG use. Agreed, and I'd go even further than that: FAANG use should be quite far down the list of criteria, since the use cases and engineering culture are so different to most people's.
- spyspy 6y ago
  I’d go even a step further and say FAANG companies, despite what their tech blogs might argue, aren’t at all immune from making very dumb decisions or adopting very silly practices. To follow those same ideas blindly could cripple your company.
  - bombcar 6y ago
    In fact - FAANG companies have the resources to make horribly inefficient processes work (in human time or computer time) - I suspect many of the AI powered things are examples of this - and a smaller company will die trying to get it to work.
    - strken 6y ago
      I don't think ML-powered systems are inefficient[0], so much as they are incremental gains over the pre-existing system that are only justified by huge sales volumes. If you can get a 4% lift in sales by using a complicated ML model with 100k features instead of a handful of basic heuristics, then whether it's worth spending engineering effort on depends on what your sales are. [0] Inefficient in the sense that they're wasting money which could be easily reclaimed - they're probably going to be improved over time as the state of the art improves, but that relies on the whole field moving forward.
      - QuesnayJr 6y ago
        I do some consulting, and one odd consequence of the rise of ML is that surprisingly many companies weren't even doing simple models.
zyang 6y ago
From inside FAANG, I have seen many projects over engineered to death. I think the problem is lack of experience combined with the selfish need to make the job more interesting.
- dward 6y ago
  Spanner and Dremel/BigQuery are both SQL database in that you interact with them by sending them SQL. Maybe I don't understand the terminology.
  - xyzzyz 6y ago
    Dremel is not a database.
  - alisonkisk 6y ago
    For non-analytics use cases, only in the loosest sense. Much of the "schema" is key-value is binary blobs.
- 14u2c 6y ago
  > selfish need to make the job more interesting. I'll admit I have been tempted to stray down this path in the past. I think for me it was the fact that the company refused to provide any time for training or improvement, leading to the desire to chose tech in projects that would build skills rather than being the most expedient.
- ikiris 6y ago
  Its usually less about more interesting, and more about "show meaningful contributions for performance reviews"
- yibg 6y ago
  Resume based development.
  - bergie 6y ago
    RDD - Resume-Driven Development You can find this in lots of places not just FAANGs
xorcist 6y ago
One problem is that technology is sold with stories about wildly successful companies. This storytelling doesn't concern itself with real world use cases. I can't count the number of times I've heard Docker pitched with "it's what Google uses!", while the truth was that they didn't. Before that it was MongoDB which was "just like Spanner which was key to Google's success", while in reality it wasn't even similar. And you should organize in Spotify-like tribes, even though that idea is one person's idea about how they wished things worked, filtered though an eco chamber of conference talks. As an engineer it is easy to dismiss these ideas when you have enough behind the scenes knowledge. But the point of storytelling isn't to build technology, but to pitch it. It does a good job at that. So use it, but wisely.
quetzthecoatl 6y ago
Major reason why senior engineers/architects go for whatever technology is the hottest in the market (ie used at FAANG) is because it's the safe choice. For every new project, or for a fresh rewrite, one can either go for the incremental improvement over whatever was in use, or for the revolutionary ones that worked for FAANG and open sourced by them. If one were to go with the former, their solution will always be compared against the hypothetical much better one using the latest shiny framework/componenets used by FAANG. It doesn't matter if you made the right choice, because for you to prove you did, you have to again rebuild the application using the FAANG framework and compare the profiling/scalability numbers. Much easier to just go with kubernets microservice service mesh on nosql.
- AnIdiotOnTheNet 6y ago
  I can't help but wonder if one of the big differences between real engineers and silicon valley software "engineers" is that real engineers don't make decisions this way.
  - sidlls 6y ago
    I don't wonder at all: it's (more or less) true. Other differences that are important: - Real engineering interviews test skills that are pertinent to the day to day work of the engineers; FAANG (and copycat) interviews tend not to - Real engineering isn't plagued by fad-following or driven by personalities: it's backed by research and established empirical practice That said, the profession of "real engineering" has its share of problems: advancement is as political as it is in any other industry and it's got relatively low pay compared to the true value of the output are two of the biggest.
Bombthecat 6y ago
I would argue that kubernetes is often overblown for small firms. But service mesh is getting easier and easier by the day. And brings quite a lot of benefits!
whatsmyusername 6y ago
> As the article says, it’s the conclusion that’s what’s good for FAANG isn’t necessarily what’s good for your average project. This, stop doing things just because google does them. You don't have google problems.
tamrix 6y ago
Even within FAANG companies, developers still don't know how to write Sql.

by shrumm 6y ago

hooloovoo_zoo 6y ago

Sometimes engineers have perverse incentives. If you want a FAANG job, it probably helps to have experience with FAANG tech, even if your current company doesn't need it.

ehnto 6y ago
It is a very frustrating industry to spend a long time in. A hamster wheel of constantly shifting goal posts for mastery. I wouldn't mind if they were fundamental advances in software development, but it's mostly the same stuff with small advances but massive learning curves of arbitrary, non-transferrable minutiae. The side effect of this ever shifting tooling set, is almost no one masters anything. All software is varying levels of crap written by newbies, because when the tools change every project you are always a newbie in that tool set.
- llbbdd 6y ago
  As someone at a FAANG company right now, your second paragraph neatly identifies a core frustration I have with my job that I haven't been able to articulate before now. Might be time for a change...
- kqr 6y ago
  > I wouldn't mind if they were fundamental advances in software development, but it's mostly the same stuff with small advances but massive learning curves of arbitrary, non-transferrable minutiae. I think this is pinpointing the key exploitability in the market. If you have seen enough tech come and go, you can figure out which 98% of this year's idea are the same as the year before, and 40 years ago. At that point you can start cutting through the bullshit and design things using brand new tech as if you had used it for 20 years already. At that point you're way ahead of the rest of the pack. (And you can choose not to use the brand new thing, and argue convincingly for why the almost-exactly-the-same 30 year old, more mature and stable, tech is better.)
  - krab 6y ago
    I mostly agree. Though you may still develop blind spots and miss a crucial difference in this year's iteration of the same idea.
- vinay_ys 6y ago
  Firstly software development is not one monolithic job. There are many different jobs within software development. You can be a systems engineer – understand the computers at a fundamental level – with focus on systems software – building low-level high-performance components/kernels – for storage systems, databases, in-memory systems etc (each of these have a lot of technical domain knowledge specialisation within them). You can spend 10 years in this field and continuously develop/enhance your expertise incrementally – it is quite stable and rewarding experience. A branch of this specialization is distributed systems engineering – where emphasis is on software operating in a distributed cluster over network – which has additional challenges unique to the aspect of being distributed. You can be an applications engineer – understand application software engineering methodology – with focus on user/business facing application feature development in a scalable software team setting. The key focus here is not so much about low-level computer systems but more about software engineering discipline. It is about the inter/intra-team sport that is developing and continuously evolving a very large application software code base that is alive with changing/evolving user/business features. It is about modeling the functional domain of the user/business/real world. It is about the modern consumer Internet software development techniques – experimentation, live incremental safe feature releases etc. A branch of this specialization is developing application frameworks and tooling (rather than user/business functional features) to make the life of a feature engineer more productive. You can spend 10 years in this field and become an expert at software engineering discipline of churning out quality features on time and on budget. The experience and learning in the above job families transcend any particular technology stack. The learnings are transferable from one tech-stack/functional-domain to another with relatively minimal effort. This effort is part of the work itself and doesn't turn you into a newbie for having to do it. p.s: A generalist full stack engineer is usually an applications engineer who is somewhat good at systems engineering and is able to glue together systems to achieve the application features well. These engineers can take a startup from start and through initial growth phase and up to start of hyper growth phase. But there's a scale/performance threshold – where the scale of application deployment grows, performance starts to hurt your users, you need strong systems engineering specialists to fix those deeper systems/distributed-systems problems. Public cloud systems have been continuously raising that threshold since the beginning.
- theshadowknows 6y ago
  I just learned last week that we're bringing Pega on for some projects. Never even heard of Pega before then lol. Time to sit down with a free account and learn something else that we'll throw away in five years.
  - treeman79 6y ago
    5 years is sadly decent run. Was in a big Javascript project, where any article over 3 months old was useless or wrong. The internals of the app constantly being rewritten to flavor of the month.
- jrochkind1 6y ago
  This rings so true to me. I really like mastering things, so I've tried to stick with a technology stack (that I have kind of mastered), but I don't think it's been good for my career.
tjpnz 6y ago
It's not just engineers either. I've seen new EMs copy and paste ill fitting FAANG development practices into organizations and then bounce off to which ever company championed it not long after.
whatsmyusername 6y ago
Yeah we've run into this. Before my time they gave a full application to one developer, who proceeded to build it in a language we barely use on tech no one else in the company understands (and, while not bad, isn't even close to something we'd choose to use in any other circumstance). Once it was done...ish, he ducked out to be a consultant, leaving others to clean up the mess (we ended up just selling it for not what we put into it).

FartyMcFarter 6y ago

> Don’t pick tech spot SOLELY based on FAANG use. Agreed, and I'd go even further than that: FAANG use should be quite far down the list of criteria, since the use cases and engineering culture are so different to most people's.

spyspy 6y ago
I’d go even a step further and say FAANG companies, despite what their tech blogs might argue, aren’t at all immune from making very dumb decisions or adopting very silly practices. To follow those same ideas blindly could cripple your company.
- bombcar 6y ago
  In fact - FAANG companies have the resources to make horribly inefficient processes work (in human time or computer time) - I suspect many of the AI powered things are examples of this - and a smaller company will die trying to get it to work.
  - strken 6y ago
    I don't think ML-powered systems are inefficient[0], so much as they are incremental gains over the pre-existing system that are only justified by huge sales volumes. If you can get a 4% lift in sales by using a complicated ML model with 100k features instead of a handful of basic heuristics, then whether it's worth spending engineering effort on depends on what your sales are. [0] Inefficient in the sense that they're wasting money which could be easily reclaimed - they're probably going to be improved over time as the state of the art improves, but that relies on the whole field moving forward.
    - QuesnayJr 6y ago
      I do some consulting, and one odd consequence of the rise of ML is that surprisingly many companies weren't even doing simple models.

zyang 6y ago

From inside FAANG, I have seen many projects over engineered to death. I think the problem is lack of experience combined with the selfish need to make the job more interesting.

dward 6y ago
Spanner and Dremel/BigQuery are both SQL database in that you interact with them by sending them SQL. Maybe I don't understand the terminology.
- xyzzyz 6y ago
  Dremel is not a database.
- alisonkisk 6y ago
  For non-analytics use cases, only in the loosest sense. Much of the "schema" is key-value is binary blobs.
14u2c 6y ago
> selfish need to make the job more interesting. I'll admit I have been tempted to stray down this path in the past. I think for me it was the fact that the company refused to provide any time for training or improvement, leading to the desire to chose tech in projects that would build skills rather than being the most expedient.
ikiris 6y ago
Its usually less about more interesting, and more about "show meaningful contributions for performance reviews"
yibg 6y ago
Resume based development.
- bergie 6y ago
  RDD - Resume-Driven Development You can find this in lots of places not just FAANGs

xorcist 6y ago

One problem is that technology is sold with stories about wildly successful companies. This storytelling doesn't concern itself with real world use cases. I can't count the number of times I've heard Docker pitched with "it's what Google uses!", while the truth was that they didn't. Before that it was MongoDB which was "just like Spanner which was key to Google's success", while in reality it wasn't even similar. And you should organize in Spotify-like tribes, even though that idea is one person's idea about how they wished things worked, filtered though an eco chamber of conference talks. As an engineer it is easy to dismiss these ideas when you have enough behind the scenes knowledge. But the point of storytelling isn't to build technology, but to pitch it. It does a good job at that. So use it, but wisely.

quetzthecoatl 6y ago

Major reason why senior engineers/architects go for whatever technology is the hottest in the market (ie used at FAANG) is because it's the safe choice. For every new project, or for a fresh rewrite, one can either go for the incremental improvement over whatever was in use, or for the revolutionary ones that worked for FAANG and open sourced by them. If one were to go with the former, their solution will always be compared against the hypothetical much better one using the latest shiny framework/componenets used by FAANG. It doesn't matter if you made the right choice, because for you to prove you did, you have to again rebuild the application using the FAANG framework and compare the profiling/scalability numbers. Much easier to just go with kubernets microservice service mesh on nosql.

AnIdiotOnTheNet 6y ago
I can't help but wonder if one of the big differences between real engineers and silicon valley software "engineers" is that real engineers don't make decisions this way.
- sidlls 6y ago
  I don't wonder at all: it's (more or less) true. Other differences that are important: - Real engineering interviews test skills that are pertinent to the day to day work of the engineers; FAANG (and copycat) interviews tend not to - Real engineering isn't plagued by fad-following or driven by personalities: it's backed by research and established empirical practice That said, the profession of "real engineering" has its share of problems: advancement is as political as it is in any other industry and it's got relatively low pay compared to the true value of the output are two of the biggest.

Bombthecat 6y ago

I would argue that kubernetes is often overblown for small firms. But service mesh is getting easier and easier by the day. And brings quite a lot of benefits!

whatsmyusername 6y ago

> As the article says, it’s the conclusion that’s what’s good for FAANG isn’t necessarily what’s good for your average project. This, stop doing things just because google does them. You don't have google problems.

tamrix 6y ago

Even within FAANG companies, developers still don't know how to write Sql.