5 ms·

Basic infrastructure patterns

97 points by dkarapetyan 11y ago

zappo2938 11y ago
> just as long as it’s not grunt, gulp Well that sucks. What if all I know is PHP and Javascript? To manage a pipeline I'm slowly learning out of necessity bash -- sort of a need to know basis. What do people recommend for this? I'm considering to explore Python as a third language over Ruby.
- djokkataja 11y ago
  You can use whatever you want to build--the author of the post didn't offer any reasons not to use grunt/gulp/broccoli; they just said they don't recommend it. If those tools are currently doing what you want, then it's a waste of your time to switch to another system. That said, if you really want to switch and you're using npm (since you mentioned JavaScript), you can set up a dead simple build system just with bash/npm/JavaScript: http://blog.keithcirkel.co.uk/how-to-use-npm-as-a-build-tool/ http://blog.keithcirkel.co.uk/how-to-use-npm-as-a-build-tool...
- dkarapetyan 11y ago
  Then use jake. It's a port of rake to js: http://jakejs.com/ http://jakejs.com/
- adamors 11y ago
  Grunt and gulp are fine as long as you don't use them for everything, ie. fine for asset management but not for copying files/folders or in cronjobs etc. Bash is much better for that.
tair 11y ago
Thanks for the good write-up! I was surprised to find out how many of these practices were actually baked into https://dockbit.com https://dockbit.com design (I am the founder). We've even open-sourced Hacher https://github.com/Dockbit/hacher https://github.com/Dockbit/hacher, which we use internally to detect the changes to dependency files and cache them for faster builds. Can't wait to read about deployment patterns you're having ;)
- dkarapetyan 11y ago
  dockbit looks pretty interesting. The one thing that is always unclear about services like yours is how well it scales to hundreds of servers and how it integrates with other orchestration mechanisms/tools like fabric, salt, awscli, and other custom services and tools. I'd be interested in knowing more about how you're approaching that aspect since most places re-invent the same things over and over again internally. The other issue is that what happens when you have downtime? Is my deploy train stopped because of it? How do I design things so that even though I use your service I still have fallback mechanisms?
  - tair 11y ago
    We're not PaaS and not hosting anything, only orchestrating, so scaling is not our biggest concern right now. All the integrations are run in one-off Docker containers, which we're managing in our internal Docker cluster. Up until now we've been adding integrations ourselves, but the plan is to open-source our integration framework and let anyone add support for their tools. It's going to be quite easy, we will most probably require only Dockerfile and a metadata file for us to build the pipeline UI. Answering your downtime question, sure we will provide plans with SLAs and multi-AZs, but basically it's a similar situation when repository hosting provider, DNS, container registry, or another infrastructure service is unavailable. That being said, we'll definitely think of a way to provide a fallback mechanism. Thank you!
- nikolay 11y ago
  When are we gonna get some invites? It's been a while since i signed up for the beta!
  - tair 11y ago
    Oh, we've sent them out. Please email me tair at dockbit.com and you'll get it right away. I also want to check our mail logs why you haven't got it. Thank you!
    - nikolay 11y ago

97 points by dkarapetyan 11y ago

> just as long as it’s not grunt, gulp Well that sucks. What if all I know is PHP and Javascript? To manage a pipeline I'm slowly learning out of necessity bash -- sort of a need to know basis. What do people recommend for this? I'm considering to explore Python as a third language over Ruby.

djokkataja 11y ago
You can use whatever you want to build--the author of the post didn't offer any reasons not to use grunt/gulp/broccoli; they just said they don't recommend it. If those tools are currently doing what you want, then it's a waste of your time to switch to another system. That said, if you really want to switch and you're using npm (since you mentioned JavaScript), you can set up a dead simple build system just with bash/npm/JavaScript: http://blog.keithcirkel.co.uk/how-to-use-npm-as-a-build-tool/ http://blog.keithcirkel.co.uk/how-to-use-npm-as-a-build-tool...
dkarapetyan 11y ago
Then use jake. It's a port of rake to js: http://jakejs.com/ http://jakejs.com/
adamors 11y ago
Grunt and gulp are fine as long as you don't use them for everything, ie. fine for asset management but not for copying files/folders or in cronjobs etc. Bash is much better for that.

tair 11y ago

Thanks for the good write-up! I was surprised to find out how many of these practices were actually baked into https://dockbit.com https://dockbit.com design (I am the founder). We've even open-sourced Hacher https://github.com/Dockbit/hacher https://github.com/Dockbit/hacher, which we use internally to detect the changes to dependency files and cache them for faster builds. Can't wait to read about deployment patterns you're having ;)

dkarapetyan 11y ago
dockbit looks pretty interesting. The one thing that is always unclear about services like yours is how well it scales to hundreds of servers and how it integrates with other orchestration mechanisms/tools like fabric, salt, awscli, and other custom services and tools. I'd be interested in knowing more about how you're approaching that aspect since most places re-invent the same things over and over again internally. The other issue is that what happens when you have downtime? Is my deploy train stopped because of it? How do I design things so that even though I use your service I still have fallback mechanisms?
- tair 11y ago
  We're not PaaS and not hosting anything, only orchestrating, so scaling is not our biggest concern right now. All the integrations are run in one-off Docker containers, which we're managing in our internal Docker cluster. Up until now we've been adding integrations ourselves, but the plan is to open-source our integration framework and let anyone add support for their tools. It's going to be quite easy, we will most probably require only Dockerfile and a metadata file for us to build the pipeline UI. Answering your downtime question, sure we will provide plans with SLAs and multi-AZs, but basically it's a similar situation when repository hosting provider, DNS, container registry, or another infrastructure service is unavailable. That being said, we'll definitely think of a way to provide a fallback mechanism. Thank you!
nikolay 11y ago
When are we gonna get some invites? It's been a while since i signed up for the beta!
- tair 11y ago
  Oh, we've sent them out. Please email me tair at dockbit.com and you'll get it right away. I also want to check our mail logs why you haven't got it. Thank you!
  - nikolay 11y ago