5 ms·

This is certainly true, but it strikes me that the solution must be a move towards simpler primitives so that managing a cloud-based piece of software doesn't r

by eandre 7y ago

This is certainly true, but it strikes me that the solution must be a move towards simpler primitives so that managing a cloud-based piece of software doesn't require so many moving parts.

AlchemistCamp 7y ago
On the other hand Alan Kay has made powerful arguments that systems can be made much simpler if only their building blocks were slightly less simple! It's a tough question. https://youtu.be/NdSD07U5uBs?t=319 https://youtu.be/NdSD07U5uBs?t=319
- eandre 7y ago
  Good point, I didn't necessarily mean simpler primitives but rather "higher-order" primitives. We need to move to a higher level of abstraction on top of a new layer of primitives (even if those primitives themselves are more complex than the ones we have today).
bastawhiz 7y ago
You still need to consider the functional aspects of the service, as well. Perhaps you're using some critical software that suddenly has a very bad CVE. It turns out that the only developer working on it was hit by a bus, and now the magical connector between your git server and your LDAP server has no functional replacement (or the replacement isn't compatible with your current setup). No amount of better abstractions will prevent a painful midnight page. There will always be a meaningful amount of effort involved in supporting your own systems, and that cost (be it in ongoing maintenance time, or in lost time because your own system failed) _can_ always be higher than paying to outsource the work to a trusted third party.
- eandre 7y ago
  I agree! But isn't that the beauty in abstractions? If you are using an unmaintained file system with a CVE, the beauty of the POSIX interface means you can just migrate to another file systems and basically all applications would just continue working. I'm not advocating for having nobody at the wheel to notice and address security vulnerabilities, just that those concerns should be isolated from the application code to a much greater extent than they are today.
jerf 7y ago
I've mentally tried to spec out what that would look like. It's weird and very unlike how we operate today. It is not clear to me that anything I've mentally sketched out is a win even if it were magically manifested for me with no effort. I don't think this is accidental complexity, it's essential complexity. To the extent that it seemed easier in the past, it's because we ignored some of that complexity and paid the price. Putting up a truly production-grade service is fundamentally hard. Now, I think it probably will get easier over time as we grapple with these problems. Integrating with auth, for instance, should be easier, and that can be solved with some more code, and formal and informal standards. It's not all essential complexity. But I think a good deal of it is, or at least it is from anything remotely resembling our current perspective.
- avmich 7y ago
  > I don't think this is accidental complexity, it's essential complexity. I think this is the kind of essential complexity which we faced when developing operating systems. OSes are fundamentally helpers, which don't solve the application problem, but make the solving easier (a good OS makes it easier by much). So we can and should use the results which we got from OS development.
- eandre 7y ago
  I agree that the parts are needed and the complexity essential. What I mean is that we need higher-order primitives that abstract away and shield us from the conplexity. Just like we don't need to spend time crafting TCP packets by hand, I don't think we should have to mess with transport protocols, writing RPC calls by hand, juggling yaml files to stand up Kubernetes clusters, and so on. I've been working on building something like this for a year now [1] and it's certainly quite different from how we do backend development normally,but that's also the point :). The tricky part is to find abstractions that are general and non-leaky so they can be leveraged to build a wide variety of software. Feedback appreciated! [1] https://encore.dev https://encore.dev

by eandre 7y ago

This is certainly true, but it strikes me that the solution must be a move towards simpler primitives so that managing a cloud-based piece of software doesn't require so many moving parts.

AlchemistCamp 7y ago

On the other hand Alan Kay has made powerful arguments that systems can be made much simpler if only their building blocks were slightly less simple! It's a tough question. https://youtu.be/NdSD07U5uBs?t=319 https://youtu.be/NdSD07U5uBs?t=319

eandre 7y ago
Good point, I didn't necessarily mean simpler primitives but rather "higher-order" primitives. We need to move to a higher level of abstraction on top of a new layer of primitives (even if those primitives themselves are more complex than the ones we have today).

bastawhiz 7y ago

You still need to consider the functional aspects of the service, as well. Perhaps you're using some critical software that suddenly has a very bad CVE. It turns out that the only developer working on it was hit by a bus, and now the magical connector between your git server and your LDAP server has no functional replacement (or the replacement isn't compatible with your current setup). No amount of better abstractions will prevent a painful midnight page. There will always be a meaningful amount of effort involved in supporting your own systems, and that cost (be it in ongoing maintenance time, or in lost time because your own system failed) _can_ always be higher than paying to outsource the work to a trusted third party.

eandre 7y ago
I agree! But isn't that the beauty in abstractions? If you are using an unmaintained file system with a CVE, the beauty of the POSIX interface means you can just migrate to another file systems and basically all applications would just continue working. I'm not advocating for having nobody at the wheel to notice and address security vulnerabilities, just that those concerns should be isolated from the application code to a much greater extent than they are today.

jerf 7y ago

I've mentally tried to spec out what that would look like. It's weird and very unlike how we operate today. It is not clear to me that anything I've mentally sketched out is a win even if it were magically manifested for me with no effort. I don't think this is accidental complexity, it's essential complexity. To the extent that it seemed easier in the past, it's because we ignored some of that complexity and paid the price. Putting up a truly production-grade service is fundamentally hard. Now, I think it probably will get easier over time as we grapple with these problems. Integrating with auth, for instance, should be easier, and that can be solved with some more code, and formal and informal standards. It's not all essential complexity. But I think a good deal of it is, or at least it is from anything remotely resembling our current perspective.

avmich 7y ago
> I don't think this is accidental complexity, it's essential complexity. I think this is the kind of essential complexity which we faced when developing operating systems. OSes are fundamentally helpers, which don't solve the application problem, but make the solving easier (a good OS makes it easier by much). So we can and should use the results which we got from OS development.
eandre 7y ago
I agree that the parts are needed and the complexity essential. What I mean is that we need higher-order primitives that abstract away and shield us from the conplexity. Just like we don't need to spend time crafting TCP packets by hand, I don't think we should have to mess with transport protocols, writing RPC calls by hand, juggling yaml files to stand up Kubernetes clusters, and so on. I've been working on building something like this for a year now [1] and it's certainly quite different from how we do backend development normally,but that's also the point :). The tricky part is to find abstractions that are general and non-leaky so they can be leveraged to build a wide variety of software. Feedback appreciated! [1] https://encore.dev https://encore.dev