3 ms·

This advice sounds rad! Do you know of any open-source codebase that does this? > Use dynamodb for storage. So far I've done the opposite, i.e. use a relation

This advice sounds rad! Do you know of any open-source codebase that does this? > Use dynamodb for storage. So far I've done the opposite, i.e. use a relational database at the beginning, and, if the access pattern is clear and there are some parts of the application that would not scale well with SQL, move those parts to DynamoDB.

anonyfox 3y ago
can not give you a codebase for permission reasons, but ie in nodejs land you rather easily can do: 1. write a bog standard express.js backend (might or might not use a SPA/SSR frontend) 2. make the normal (dev mode) server start with an index.js (which boots the express app on a port) and ontop of that an additional lambda.js next to it, which wraps the app (not booting it) with `@vendia/serverless-express`. 3. use AWS CDK (which is typescript anyways!) and ship it as a "NodejsFunction", which also will use esbuild under the hood to bundle your app into one JS blob (no need or a ZIP container with node_modules!) on deploy. The entry file is the "lambda.js" from step 2. 4. in addition use the CDK to configure some traffic origin for the lambda, the cheapest one being a CloudFront function URL - which also acts as a quite good cache in general when you send the appropriate HTTP response headers from your express app! 5. Point a domain to the Cloudfront. 6. Profit! ... I've used this excessivbely in the past years for lots of microservices/microfrontends and it basically works as promised: scale to zero, scale horizontally on demand automatically, and being rather cheap (esp when in the free tier then its basically free to run small-mid loads). Having said that, this WILL become a maintenance trainwreck when scaled up, telling from my devops experience. All the libraries/apis/infrastructure (especially in nodejs land) tend to have breaking updates all the time, so better have at least one FTE dedicated to maintenance of all the moving parts. And tbh: all this pain goes away for like <100$/mo with Elixir/Phoenix/LiveView on fly.io. a single phoenix app on a single server can deal with _suprising_ amounts of traffic, and even if you get to serious load, scaling horizontally is outright trivial with Elixir+Fly. And there is no need for additional infra like Redis or Message Queues, since these things have natively built-in equivalents in Elixir (or: the BEAM itself). So only an appserver and a database you need here in terms of infrastructure/maintenance.
- zxt_tzx 3y ago
  > Having said that, this WILL become a maintenance trainwreck when scaled up, telling from my devops experience. All the libraries/apis/infrastructure (especially in nodejs land) tend to have breaking updates all the time, so better have at least one FTE dedicated to maintenance of all the moving parts. Oh man, this sounds like a dealbreaker, but thanks for sharing your approach nonetheless!
  - anonyfox 3y ago
    It depends solely on company/team culture. If people are churning after like 1-2 years, they never get to experience the long term pain themselves, and new hires will lobby for rewrites anyways. Since individual apps are rather small, rewrites service-by-service should be straightforward and manageable. Ancient knowledge would be helpful here, but most teams simply don’t have „real“ seniors around. Therefore simple advice from my end: 1. keep the database stupid and sane. Postgres usually has everything needed for nearly every usecase in a good enough and battleground way. Don’t overdo complex join queries in hot code paths. 2. use a mature, slow-changing (as in: no breaking changes yearly) tech stack, that is powerful out of the box. When you need to use several infrastructure crudges, the tech isn’t powerful enough. I recommend Elixir, Go or even JVM-flavors, these will carry you far. 3. Monolith it is, fullstack. Every service/frontend you split apart might become easier in isolation, but you pile up significant overhead and complexity in terms of global monitoring/orchestration/dependencies/testability/debugging. Elixir/Phoenix, PHP/Laravel, … are fine choices that also have straightforward scaling options and cover everything you need probably. 4. minimize/eliminate usage of JS wherever possible. Aside from infrastructure providers/external APIs, this is the biggest source of maintenance issues. Also SPAs are rarely worth it, there still is a boring way of progressive enhancement that just works.

anonyfox 3y ago

can not give you a codebase for permission reasons, but ie in nodejs land you rather easily can do: 1. write a bog standard express.js backend (might or might not use a SPA/SSR frontend) 2. make the normal (dev mode) server start with an index.js (which boots the express app on a port) and ontop of that an additional lambda.js next to it, which wraps the app (not booting it) with `@vendia/serverless-express`. 3. use AWS CDK (which is typescript anyways!) and ship it as a "NodejsFunction", which also will use esbuild under the hood to bundle your app into one JS blob (no need or a ZIP container with node_modules!) on deploy. The entry file is the "lambda.js" from step 2. 4. in addition use the CDK to configure some traffic origin for the lambda, the cheapest one being a CloudFront function URL - which also acts as a quite good cache in general when you send the appropriate HTTP response headers from your express app! 5. Point a domain to the Cloudfront. 6. Profit! ... I've used this excessivbely in the past years for lots of microservices/microfrontends and it basically works as promised: scale to zero, scale horizontally on demand automatically, and being rather cheap (esp when in the free tier then its basically free to run small-mid loads). Having said that, this WILL become a maintenance trainwreck when scaled up, telling from my devops experience. All the libraries/apis/infrastructure (especially in nodejs land) tend to have breaking updates all the time, so better have at least one FTE dedicated to maintenance of all the moving parts. And tbh: all this pain goes away for like <100$/mo with Elixir/Phoenix/LiveView on fly.io. a single phoenix app on a single server can deal with _suprising_ amounts of traffic, and even if you get to serious load, scaling horizontally is outright trivial with Elixir+Fly. And there is no need for additional infra like Redis or Message Queues, since these things have natively built-in equivalents in Elixir (or: the BEAM itself). So only an appserver and a database you need here in terms of infrastructure/maintenance.

zxt_tzx 3y ago
> Having said that, this WILL become a maintenance trainwreck when scaled up, telling from my devops experience. All the libraries/apis/infrastructure (especially in nodejs land) tend to have breaking updates all the time, so better have at least one FTE dedicated to maintenance of all the moving parts. Oh man, this sounds like a dealbreaker, but thanks for sharing your approach nonetheless!
- anonyfox 3y ago
  It depends solely on company/team culture. If people are churning after like 1-2 years, they never get to experience the long term pain themselves, and new hires will lobby for rewrites anyways. Since individual apps are rather small, rewrites service-by-service should be straightforward and manageable. Ancient knowledge would be helpful here, but most teams simply don’t have „real“ seniors around. Therefore simple advice from my end: 1. keep the database stupid and sane. Postgres usually has everything needed for nearly every usecase in a good enough and battleground way. Don’t overdo complex join queries in hot code paths. 2. use a mature, slow-changing (as in: no breaking changes yearly) tech stack, that is powerful out of the box. When you need to use several infrastructure crudges, the tech isn’t powerful enough. I recommend Elixir, Go or even JVM-flavors, these will carry you far. 3. Monolith it is, fullstack. Every service/frontend you split apart might become easier in isolation, but you pile up significant overhead and complexity in terms of global monitoring/orchestration/dependencies/testability/debugging. Elixir/Phoenix, PHP/Laravel, … are fine choices that also have straightforward scaling options and cover everything you need probably. 4. minimize/eliminate usage of JS wherever possible. Aside from infrastructure providers/external APIs, this is the biggest source of maintenance issues. Also SPAs are rarely worth it, there still is a boring way of progressive enhancement that just works.