4 ms·

I worked there a few years. The thing is, some systems are so complex it might take 6-9 months to even understand the value add within the org. At which point i

I worked there a few years. The thing is, some systems are so complex it might take 6-9 months to even understand the value add within the org. At which point it comes to light that there is very little value add. And all the meetings you were kept from attending were political gatekeeping. An example would be a recommendation system who’s recommendations are ignored by downstream systems. And yet, 30 people in an org work on that engine.

killingtime74 4y ago
Did people working on that engine know that their work is being ignored?
- greatpostman 4y ago
  Yes…but the thing is downstream systems are dynamic and real time, so “could” use the recommendations, but never actually do. Hypothetically… This type of thing goes on all over the company. Another example, a complex “data provider platform”. Which is actually just spinning up spark clusters, joining data, and making it available to downstream consumers. There’s a lot of bells and whistles, notifications built on top. Strip away the details, it’s just doing a join. Ten engineers, two senior engineers making 500k, long hours. That’s Amazon.
  - operatingthetan 4y ago
    I encountered the same sort of problems at amazon, I felt like it was because leadership didn't incentivize those optimizations to be made across the org. We'd identify some team that was the only one that could help us with something and nobody would lift a finger for us. Teams who were on the hook would ship us incomplete and untested APIs claiming they were done in order to work on other stuff. Managers didn't want to bring it up with other teams because of political ramifications.
mattnewton 4y ago
That’s insane, how do the people in the org justify their work for promotion? There isn’t any need to map it back to observed top line metric shifts even for a 30 person effort?
- greatpostman 4y ago
  “Person xyz built a complex model deployment system, capable of updating the recommendation engine in real time, with automated rollbacks and A/B testing. He leveraged 4 junior engineers and collaborated with multiple data science teams”
- ethbr0 4y ago
  Indeed, this is why corporate KPIs exist. Requiring everything to move KPIs is psychotic, but if your effort should have moved a KPI, and didn't, then you'd better have a damn good story about how it delivered value elsewhere instead.
raz32dust 4y ago
I think you are describing all big tech companies. But what's the solution? I think it's just a result of having a large org. 20% of the org delivers 80% of the value. But when risk appetite is high and the company can afford to throw money at experiments, it makes sense to still pay for the rest of the 80%. When times are tough, it doesn't.
- greatpostman 4y ago
  Definitely it makes sense to take bets on new tech. But a lot of higher up managers are not running these projects in good faith. At some level they know the projects are futile, and many around them are playing a game acting otherwise
  - deleted 4y ago
    [deleted]
  - raz32dust 4y ago
    How do you know that? I don't disagree that many projects are futile. But it is usually only apparent in hindsight, and humans are very good at justifying their own jobs and by extension, their team's jobs.
- deleted 4y ago
  [deleted]
- imiric 4y ago
  > But what's the solution? I think it's just a result of having a large org. Right. Maybe Elon's approach is correct, and all these layoffs are a chance for companies to trim the fat and focus on what matters. We're only now realizing the bubble the tech industry was in, and that pursuing endless growth is not sustainable. > But when risk appetite is high and the company can afford to throw money at experiments, it makes sense to still pay for the rest of the 80%. Does it, though? You're paying for full-time engineers to work on bets that might be completely misaligned even if they succeed, which also might be unlikely. And this makes up the majority of your company? It sounds ludicrous in any market.
harryVic 4y ago
How does a team of 30 not measure the usage of their system?
- pas 4y ago
  req/sec can be still high, it's just that the downstream service decides not to use the data
- binhqx 4y ago
  When the data your team serves is copied from service to service to provide high availability and no centralized failure point.

killingtime74 4y ago

Did people working on that engine know that their work is being ignored?

greatpostman 4y ago
Yes…but the thing is downstream systems are dynamic and real time, so “could” use the recommendations, but never actually do. Hypothetically… This type of thing goes on all over the company. Another example, a complex “data provider platform”. Which is actually just spinning up spark clusters, joining data, and making it available to downstream consumers. There’s a lot of bells and whistles, notifications built on top. Strip away the details, it’s just doing a join. Ten engineers, two senior engineers making 500k, long hours. That’s Amazon.
- operatingthetan 4y ago
  I encountered the same sort of problems at amazon, I felt like it was because leadership didn't incentivize those optimizations to be made across the org. We'd identify some team that was the only one that could help us with something and nobody would lift a finger for us. Teams who were on the hook would ship us incomplete and untested APIs claiming they were done in order to work on other stuff. Managers didn't want to bring it up with other teams because of political ramifications.

mattnewton 4y ago

That’s insane, how do the people in the org justify their work for promotion? There isn’t any need to map it back to observed top line metric shifts even for a 30 person effort?

greatpostman 4y ago
“Person xyz built a complex model deployment system, capable of updating the recommendation engine in real time, with automated rollbacks and A/B testing. He leveraged 4 junior engineers and collaborated with multiple data science teams”
ethbr0 4y ago
Indeed, this is why corporate KPIs exist. Requiring everything to move KPIs is psychotic, but if your effort should have moved a KPI, and didn't, then you'd better have a damn good story about how it delivered value elsewhere instead.

raz32dust 4y ago

I think you are describing all big tech companies. But what's the solution? I think it's just a result of having a large org. 20% of the org delivers 80% of the value. But when risk appetite is high and the company can afford to throw money at experiments, it makes sense to still pay for the rest of the 80%. When times are tough, it doesn't.

greatpostman 4y ago
Definitely it makes sense to take bets on new tech. But a lot of higher up managers are not running these projects in good faith. At some level they know the projects are futile, and many around them are playing a game acting otherwise
- deleted 4y ago
  [deleted]
- raz32dust 4y ago
  How do you know that? I don't disagree that many projects are futile. But it is usually only apparent in hindsight, and humans are very good at justifying their own jobs and by extension, their team's jobs.
deleted 4y ago
[deleted]
imiric 4y ago
> But what's the solution? I think it's just a result of having a large org. Right. Maybe Elon's approach is correct, and all these layoffs are a chance for companies to trim the fat and focus on what matters. We're only now realizing the bubble the tech industry was in, and that pursuing endless growth is not sustainable. > But when risk appetite is high and the company can afford to throw money at experiments, it makes sense to still pay for the rest of the 80%. Does it, though? You're paying for full-time engineers to work on bets that might be completely misaligned even if they succeed, which also might be unlikely. And this makes up the majority of your company? It sounds ludicrous in any market.

harryVic 4y ago

How does a team of 30 not measure the usage of their system?

pas 4y ago
req/sec can be still high, it's just that the downstream service decides not to use the data
binhqx 4y ago
When the data your team serves is copied from service to service to provide high availability and no centralized failure point.