3 ms·

We can't go NIH for everything. If we do that we're back to baremetal in our own datacenters and that's expensive and (comparatively) low velocity. We have to p

We can't go NIH for everything. If we do that we're back to baremetal in our own datacenters and that's expensive and (comparatively) low velocity. We have to pick and choose our dependencies and take the trade off of risk for velocity. This is the tradeoff we made with the move to cloud. We run our workloads on AWS, GCP or Azure, use DataDog or New Relic for monitoring, use Github or GitLab for repos and pipelines, and so forth. Each speeds us up but is a risk. We hope they are relatively low risks and we work to ameliorate those risks as we can. An organization like Docker should have been low risk. Clearly, it's not. So now it's a strong candidate for replacement with a local solution rather than a vendor to rely on.

tetrep 4y ago
It's less NIH and more "cache your dependencies." Details will very greatly depending on what your tech stack looks like, if you're lucky you can just inline a cache. I know Artifactory is a relatively general commercial solution although I can't speak personally about it. If you can't easily use an existing caching solution, then the only NIH you need to do is copying files that your build system downloads. I know many build systems are "just a bunch of scripts" so those would probably be pretty amenable to this, I don't know if more opaque systems exist that wouldn't give you any access like that. If so, I suppose you could try to just copy the disk the build system writes everything to, but then you're getting into pretty hacky stuff and that's not ideal. Copying the files doesn't give you the nice UX of a cache, but it does mean that in the worst case scenario you at least have the all the dependencies you've used in recent builds, so you'll be able to keep building your things.
- friendzis 4y ago
  > I don't know if more opaque systems exist that wouldn't give you any access like that As long as there is there is "server reimplementation", i.e. private registries available, one can always hack together a solution out of self signed CA, DNS and routing to replace "the server" with local registry.
MikePlacid 4y ago
> that's expensive and (comparatively) low velocity The problem with this approach begins when many people your build depends upon start to share it.
theamk 4y ago
"Free service which requires $$ to maintain" and "low risk" are not compatible. We moved to cloud as well, and we use AWS ECR for caching. We have a script for "docker login to ECR" and a list of images to auto-mirror periodically. There is a bit of friction when adding new, never-seen-before image, but in general this does not slow developers much. And we never hit any rate-limits, too! We pay for those ECR accesses, so I am pretty confident they are not going to go away. Unlike free docker images.
GoblinSlayer 4y ago
You can prototype without NIH and later go NIH when you have stuff to lose.
jjav 4y ago
> We can't go NIH for everything. If we do that we're back to baremetal in our own datacenters[...] It's a bit of a leap from keeping copies of dependencies to building your own datacenter. Even the smallest startup can easily do the former. > This is the tradeoff we made with the move to cloud. To clarify, when I say keep local copies I meant copies which are under local control (i.e. control of your organization). They may well still physically be in AWS somewhere. The key is that they can't be modified/deleted by some third party who doesn't report to your organization. Yes, this assumes AWS is too big to fail, but for the typical startup whose entire existence is already dependent on their AWS account being available, this would not increase risk beyond what it already is. Whereas each additional hard dependency on third-party repos do increase risk.

tetrep 4y ago

It's less NIH and more "cache your dependencies." Details will very greatly depending on what your tech stack looks like, if you're lucky you can just inline a cache. I know Artifactory is a relatively general commercial solution although I can't speak personally about it. If you can't easily use an existing caching solution, then the only NIH you need to do is copying files that your build system downloads. I know many build systems are "just a bunch of scripts" so those would probably be pretty amenable to this, I don't know if more opaque systems exist that wouldn't give you any access like that. If so, I suppose you could try to just copy the disk the build system writes everything to, but then you're getting into pretty hacky stuff and that's not ideal. Copying the files doesn't give you the nice UX of a cache, but it does mean that in the worst case scenario you at least have the all the dependencies you've used in recent builds, so you'll be able to keep building your things.

friendzis 4y ago
> I don't know if more opaque systems exist that wouldn't give you any access like that As long as there is there is "server reimplementation", i.e. private registries available, one can always hack together a solution out of self signed CA, DNS and routing to replace "the server" with local registry.

MikePlacid 4y ago

> that's expensive and (comparatively) low velocity The problem with this approach begins when many people your build depends upon start to share it.

theamk 4y ago

"Free service which requires $$ to maintain" and "low risk" are not compatible. We moved to cloud as well, and we use AWS ECR for caching. We have a script for "docker login to ECR" and a list of images to auto-mirror periodically. There is a bit of friction when adding new, never-seen-before image, but in general this does not slow developers much. And we never hit any rate-limits, too! We pay for those ECR accesses, so I am pretty confident they are not going to go away. Unlike free docker images.

GoblinSlayer 4y ago

You can prototype without NIH and later go NIH when you have stuff to lose.

jjav 4y ago

> We can't go NIH for everything. If we do that we're back to baremetal in our own datacenters[...] It's a bit of a leap from keeping copies of dependencies to building your own datacenter. Even the smallest startup can easily do the former. > This is the tradeoff we made with the move to cloud. To clarify, when I say keep local copies I meant copies which are under local control (i.e. control of your organization). They may well still physically be in AWS somewhere. The key is that they can't be modified/deleted by some third party who doesn't report to your organization. Yes, this assumes AWS is too big to fail, but for the typical startup whose entire existence is already dependent on their AWS account being available, this would not increase risk beyond what it already is. Whereas each additional hard dependency on third-party repos do increase risk.