8 ms·
If you replace NodeJS with Django and Redis with Postgres it pretty much maps 1:1 with how I deal with it. Once/if postgres becomes a bottleneck I simply add r
by aprdm 7y ago
If you replace NodeJS with Django and Redis with Postgres it pretty much maps 1:1 with how I deal with it.
Once/if postgres becomes a bottleneck I simply add redis to cache, the next step is doing some stuff asynch with RabbitMQ + Celery and you can scale really really really far...
- StavrosK 7y agoTo many people's surprise, I can crank out a fully-functional side project in two to five days (latest ones were https://imgz.org https://imgz.org and https://nobsgames.stavros.io https://nobsgames.stavros.io) by having already completely automated devops. Much like you, I have a Django template project and use Dokku for deployment, so I can use the template to create a new project, commit, push, and it's already up on production. Here's mine: https://github.com/skorokithakis/django-project-template https://github.com/skorokithakis/django-project-template Having all the best practices and boilerplate already sorted helps immensely.
- williamdclt 7y agoI've worked a lot on this sort of thing in my company, as we spawn new projects every few weeks. We went from 4 days to deploy a fully-functioning Django+React+CI on AWS to literally 30min (much of it just waiting) using similar approaches. And now we have the same thing for API Platform, NestJS, Vue, NextJS... Even React-Native with Fastlane and CodePush :) It's been invaluable, and a very strong commercial argument
- say_it_as_it_is 7y agoThe situation isn't clear to me. What changed that lead to more productive deployments?
- williamdclt 7y agoUsing templates, boilerplates and generators. Creating a new project is simply "make generate" from our generator, then answering a bunch of questions about what tech stack you need. This works mostly with templating (using Plop) and a sprinkle of automation for shell tasks (eg `pipenv install`)
- say_it_as_it_is 7y agoThanks for clarifying. What used to take 4 days to set up a new server can now be done in <1hr. How often is that being taken advantage of? "Any time we need a new server" is determined by a policy determining whether to bolt onto additional servers or not. Curious about that policy..
- jwdunne 7y agoIs your generator hand rolled or do you use something in particular? We’re working towards a similar solution where we want to layer/compose different tech choices based on what’s mandated by project requirements!
- edelans 7y agoWhat kind of company that spawns new projects every few weeks are you working for ? This sounds cool =)
- weavie 7y agoI love the pricing for imgz.org! :-)
- StavrosK 7y agoThanks, I love how the high price just overflows out of the box :P
- cercatrova 7y agoI saw your no BS games Show HN recently, good work. That imgz website and copy, what an incredible piece of art! You're totally right about what imgur has become, first starting out as a reddit image host, then taking VC money, and inevitably becoming "web scale".
- StavrosK 7y agoThank you! I can promise you imgz will never become "web scale" because nobody's using it and that's the way it should be.
- notyourday 7y ago> To many people's surprise, I can crank out a fully-functional side project in two to five days by having already completely automated devops. This is something that did not occur to me until I have done a lot of consulting for small and medium size companies. All of them have the same basic challenges - they don't have tooling to do basic crap for example, they can blast some notification to everyone that is registered in their service but sending an email from a email-message-to-specific-set-of-people requires a workflow deploy. Same goes for provisioning a new domain and wiring it up to nodejs backend, etc, etc, etc. Automate basic devops first, from DNS management to email sending to deploys to workflows, etc. Think of every project you are going to to later being a "client" of your devops "infrastructure".
- richardARPANET 7y agoDjango + Postgres + Redis + Celery deployed to AWS Beanstalk covers 99% of application needs, in my personal experience.
- 0dana 7y agoHey, I’m pretty new to this process. I have Been working on Django app with Postgres, but I’m struggling with the Redis/Celery piece. Would you happen to know of any resources or do you have any open source projects of which I could study the config that might help me out?
- _tkzm 7y agonext time try cockroachdb instead of postgres.
- stevewodil 7y agoPerhaps you could elaborate further on this to add to the discussion
- _tkzm 7y agowhen you hit scaling issues with standard postgres. cdb has the same wire format so no code change would be needed. for mysql alternative, there is tidb but that one is not a single binary so i never used it.
- williamdclt 7y agoAn explanation of why would be nice
- zahrc 7y agoI’ve never heard of cockroachdb, what are the advantages of it and why is it preferable to Postgres?
- vvern 7y agoA primary advantage is horizontal scalability. If you need more capacity, just add more nodes. Additionally it offers better fault tolerance and zero-downtime online upgrades. For more advanced use cases it can be deployed in a geo-replicated topology and configured with partitioning for low latency transactions when confined to a single region while proving strong consistency and a single logical database globally. Disclaimer: I work on cockroach.
- fizixer 7y agoThanks. This brings up the point that, if OP clarifies in his (already great) document, would really help. By using node.js, OP has implicitly made a statement of javascript as the server-side language of choice. Even though there is some mention of one system doing multiple things, it would really help if OP adds a section about justification of an all-js approach. And especially, what OP's opinion is about programming in 'JS The Good Parts' (a-la Crockford) vs 'anything goes'. I believe OP may have good opinions about how to approach JS programming. It would be great if that is added to the document. In any case, I highly appreciate OP sharing this work. Thank you. edit: OP address part of this in his comment [1] [1] https://news.ycombinator.com/item?id=22116437 https://news.ycombinator.com/item?id=22116437
- fpereiro 7y agoHi there! - I choose js/node as my server-side language of choice. This is because I love both js the language and node itself, and so far haven't seen the need to try something else. That doesn't mean, however, that it should be the solution for everyone. Particularly if you don't like js - you're probably better off programming in a language you enjoy! - I have rambled a bit about js style here: https://github.com/fpereiro/ustack#mes5-a-javascript-subset https://github.com/fpereiro/ustack#mes5-a-javascript-subset . My approach, however, is very minimalistic and backward-looking (or aspiring to timelessness, if you're being very kind), so I recommend it very cautiously. You might be better off using more language features, particularly those that come in ES6-7-8. For all of this, I'd rather leave this content separately in the ustack repo.