4 ms·

I'd love to know more about your webapp methodology. It sounds wonderful.

lobster_johnson 10y ago
The current implementation is nothing special. However, I'm quite happy with our methodology itself. We use a simple microservice design: All our microservices talk using JSON over HTTP. They share nothing. Everything is done by communicating. There's also an event bus so they can listen to each other's events. All our microservices are multitenant by design, so that they can support any product with no data spillover. We have a bunch of general-purpose microservices built out to service our particular needs. User-facing webapps are built entirely out of these building blocks. They don't need to deal with login or OAuth; there's a microservice for that. Similarly, almost all data is stored in a single document-oriented data store (which we've heavily rearchitecting in Go and will soon release as open source — email me if you want to be notified) on top Postgres and Elasticsearch. For anything that we need from a webapp, we build a microservice. (We have a lot of older Ruby and Node stuff, but all our new microservices are written in Go, which has been a very pleasant experience.) One key component of our development workflow is the use of a Linux virtual machine that each developer runs locally. It hosts the exact same stack (and uses the same deployment system) as our real production and staging environments. This lets the developer focus on work and not on how the microservices are run. It's not pain-free, but we have made it quite comfortable, among other things through a seamless system of mounting, where the code running on the VM for a particular app is using the developer's own project folder on the host machine. Hot code reloading is also enabled where possible. So a React app will automatically bundle with Babel and so on when you reload a page. There are challenges to using microservices in this way. One is deployment and hosting. We currently have our own homegrown system for this that works fine, but hope to migrate to Kubernetes soon. Another challenge is documentation. In the end, the code (and project readmes) is the authority on how a particular microservice works and what APIs it implements; we did automatically generate API docs at one point, but it hasn't kept up with our Go projects. These days we just refer to the code. The most painful aspect of this, and onboarding, is documenting the cross-cutting, "big picture" concerns. I'd be happy to answer any other questions you might have.
- colinramsay 10y ago
  That's much appreciated - lots to think about! Edit: how do you scale? For example, do you just have lots of nodes running your authentication microservice for all of your active apps?
  - lobster_johnson 10y ago
    Yes, we run every microservice on multiple nodes. How many depends on how much traffic a microservice needs. Some receive tens of thousands of reqs/sec through frontends, whereas some are only invoked for daily processing jobs etc.

The current implementation is nothing special. However, I'm quite happy with our methodology itself. We use a simple microservice design: All our microservices talk using JSON over HTTP. They share nothing. Everything is done by communicating. There's also an event bus so they can listen to each other's events. All our microservices are multitenant by design, so that they can support any product with no data spillover. We have a bunch of general-purpose microservices built out to service our particular needs. User-facing webapps are built entirely out of these building blocks. They don't need to deal with login or OAuth; there's a microservice for that. Similarly, almost all data is stored in a single document-oriented data store (which we've heavily rearchitecting in Go and will soon release as open source — email me if you want to be notified) on top Postgres and Elasticsearch. For anything that we need from a webapp, we build a microservice. (We have a lot of older Ruby and Node stuff, but all our new microservices are written in Go, which has been a very pleasant experience.) One key component of our development workflow is the use of a Linux virtual machine that each developer runs locally. It hosts the exact same stack (and uses the same deployment system) as our real production and staging environments. This lets the developer focus on work and not on how the microservices are run. It's not pain-free, but we have made it quite comfortable, among other things through a seamless system of mounting, where the code running on the VM for a particular app is using the developer's own project folder on the host machine. Hot code reloading is also enabled where possible. So a React app will automatically bundle with Babel and so on when you reload a page. There are challenges to using microservices in this way. One is deployment and hosting. We currently have our own homegrown system for this that works fine, but hope to migrate to Kubernetes soon. Another challenge is documentation. In the end, the code (and project readmes) is the authority on how a particular microservice works and what APIs it implements; we did automatically generate API docs at one point, but it hasn't kept up with our Go projects. These days we just refer to the code. The most painful aspect of this, and onboarding, is documenting the cross-cutting, "big picture" concerns. I'd be happy to answer any other questions you might have.

colinramsay 10y ago
That's much appreciated - lots to think about! Edit: how do you scale? For example, do you just have lots of nodes running your authentication microservice for all of your active apps?
- lobster_johnson 10y ago
  Yes, we run every microservice on multiple nodes. How many depends on how much traffic a microservice needs. Some receive tens of thousands of reqs/sec through frontends, whereas some are only invoked for daily processing jobs etc.