3 ms·

I would wager that its because ex-Uber folks work there and carried their ways with them.

by hcnews 7y ago

pianoben 7y ago
That's rather pointed, but also matches just about every system-design interview I've ever given to an Uber engineer. Next time, I want to ask them how they keep track of all those services! edit: replaced an incorrectly-used idiom
- jordanbeiber 7y ago
  Tip - use a database. Computers are awesome at automating things, that goes for dev tooling as well. If you’ve touched the ITSM space you’re used to managing and maintaining many thousand of assets. A few hundred microservices is nothing, really. My team use what you could call a simplified CMDB (configuration management database) which is cross referenced against the service discovery. The cmdb keep info about every service, such as persistent data-sources, vm’s etc, but most important - relationships: domain, team, services and resources. A microservice is basically a ”ci” (configuration item) with a managed lifecycle.
  - anton_gogolev 7y ago
    Keeping track is one thing. Actually _running_ the Lernaean Hydra of an application in production is a whole another story. The amount of "housekeeping" you have to do to keep the thing afloat is astounding: cascading failures, distributing tracing, logging and diagnostics, metrics. Even operational side of things require a lot of attention. Presumably, each microservice would require at least a minimal level of admin-level tooling around it.
    - jordanbeiber 7y ago
      Logging and monitoring is part of the lifecycle. Use strict automated conventions to aid developer teams. Always opt for convention before configuration is our tooling motto! :) Log shipping is what we do from thousands of servers already (you should at least!), adding a shipper for a few 100 containers on a set of hosts is no big deal. Fluent(d/bit) -> some kind of elastic? There are a few resonable patterns available that works and scales pretty well. Failures and issues with the actual code - well I might have been lucky... DDD with somewhat senior devs where no spaghetti action takes place. The tooling we keep usually seem to pinpoint issues fairly well. We’re on the scale of roughly 40 devs and my team of 3 support them with tooling that handles service lifecycle and operational stuff. It let’s us be pretty fluent with what and how teams build and iterate stuff. I guess it requires a certain scale and experience though.

I would wager that its because ex-Uber folks work there and carried their ways with them.

by hcnews 7y ago

I would wager that its because ex-Uber folks work there and carried their ways with them.

That's rather pointed, but also matches just about every system-design interview I've ever given to an Uber engineer. Next time, I want to ask them how they keep track of all those services! edit: replaced an incorrectly-used idiom

jordanbeiber 7y ago
Tip - use a database. Computers are awesome at automating things, that goes for dev tooling as well. If you’ve touched the ITSM space you’re used to managing and maintaining many thousand of assets. A few hundred microservices is nothing, really. My team use what you could call a simplified CMDB (configuration management database) which is cross referenced against the service discovery. The cmdb keep info about every service, such as persistent data-sources, vm’s etc, but most important - relationships: domain, team, services and resources. A microservice is basically a ”ci” (configuration item) with a managed lifecycle.
- anton_gogolev 7y ago
  Keeping track is one thing. Actually _running_ the Lernaean Hydra of an application in production is a whole another story. The amount of "housekeeping" you have to do to keep the thing afloat is astounding: cascading failures, distributing tracing, logging and diagnostics, metrics. Even operational side of things require a lot of attention. Presumably, each microservice would require at least a minimal level of admin-level tooling around it.
  - jordanbeiber 7y ago
    Logging and monitoring is part of the lifecycle. Use strict automated conventions to aid developer teams. Always opt for convention before configuration is our tooling motto! :) Log shipping is what we do from thousands of servers already (you should at least!), adding a shipper for a few 100 containers on a set of hosts is no big deal. Fluent(d/bit) -> some kind of elastic? There are a few resonable patterns available that works and scales pretty well. Failures and issues with the actual code - well I might have been lucky... DDD with somewhat senior devs where no spaghetti action takes place. The tooling we keep usually seem to pinpoint issues fairly well. We’re on the scale of roughly 40 devs and my team of 3 support them with tooling that handles service lifecycle and operational stuff. It let’s us be pretty fluent with what and how teams build and iterate stuff. I guess it requires a certain scale and experience though.