4 ms·

I think what is needed is a statically typed systems-integration language. Problems are caused when there is nothing to "type-check" that the values you write i

I think what is needed is a statically typed systems-integration language. Problems are caused when there is nothing to "type-check" that the values you write into config-files are references of objects of correct type. Static typing would also support editors that let you choose from allowed values. Maybe such a language exists? If not, why not?

avmich 7y ago
System integration is a programming, or a kind of programming. With its own features (like limited expressive power in some places - see e.g. dhall language), traditions, history etc. but still - you ought to be able to use the classical programming toolkit to solve system integration problems. So you should ask, can you use the common languages - and if not, why?
- galaxyLogic 7y ago
  > can you use the common languages - and if not, why? Good question. I do not know the answer to that. But I think there must be some answer since there are all these different configuration languages, Ant, Gradle etc. They must have been created for a reason.
corndoge 7y ago
Like visudo, for everything! That would be cool.
Qasaur 7y ago
This already sort-of exists in the form of Nix/NixOS/NixOps. Entire system configuration is deterministic and is specified in a configuration file, and it has its own little functional DSL called Nix that is used to specify system configs and how to build software packages. Just started using it and I think it's probably one of the best software ensembles I've ever used in my career. Completely knocks Docker, K8S etc. out of the water.
- nh2 7y ago
  Yes, the Nix ecosystem makes running both servers and services much easier than any other tools I have used. I describe the deployment I run with it here: https://news.ycombinator.com/item?id=21468506 https://news.ycombinator.com/item?id=21468506 Nix is not typed (it would be slightly better if it were), but everything is evaluated before it hits your servers which allows for lots of static checks. I started writing a tutorial on NixOps here if you want to learn it: https://github.com/nh2/nixops-tutorial https://github.com/nh2/nixops-tutorial (only has 1 part so far, I'd like to show how to bootstrap a Consul cluster and distributed file system next).
jpolitz 7y ago
You might be interested in Rehearsal: https://people.cs.umass.edu/~arjun/main/papers/2016-rehearsal.html https://people.cs.umass.edu/~arjun/main/papers/2016-rehearsa...
gtycomb 7y ago
Isn't it Go?
- nothrabannosir 7y ago
  This is a fair question if you’re lucky enough not to be familiar with devops tooling. No, it’s not Go. Go is a general purpose programming language. What parent poster wants is more like Terraform, but better (I assume). With semantics that reflect devops actions. You might write that tool in Go :) but the users wouldn’t necessarily know. Just like a gamer doesn’t need to know about C++.
  - throwaway894345 7y ago
    You could use Go to generate Terraform configs. I’m using Python to generate CloudFormation and it’s an order of magnitude easier and more maintainable than vanilla CloudFormation, especially for large integrations.
weberc2 7y ago
Any statically types, general purpose programming language would do the trick—just generate YAML. I’ve been playing with typed-Python and it works really well. The only issues are that Python’s type system is pretty immature (can’t represent JSON, can’t constrain generic types to a particular interface/protocol a la Rust traits, type annotations in a class share the same namespace as member variables, etc). I would like to try Go for this, but it would be a fair bit more imperative.
vetrom 7y ago
Chef at least provides a partial approach here, where components essentially define an api that you can define mock and/or integration tests for. Ansible is not as 'data-structure' oriented, so testing that is more difficult, but each element (modules, libraries, roles, playbooks) can be written to be fairly environment-neutral, which you could then use a test a fixture like BATS to easily check for regressions. To answer your 'type-checking' expectation, its generally on the developer to implement those tests. In my experience, its rare that those are implemented in publicly available devops libraries, even though we have most of the tools to do so. It's definitely an area where the practice can improve.
Joker_vD 7y ago
I have a sinking feeling that the whole business of writing huge config files (semi-)manually (templating, #include, and YAML-references that reference things from god knows where) is... too low-level? Too imperative? It feels like if one was manually laying out the vtables and the unwinding info in assembly language: sure, the macros will go a long way helping you, but maybe the better way would be to have an actual high-level declarative description that would get translated down to metal. And "high-level" doesn't mean, let's use an analogy, M4, it means Gradle.

avmich 7y ago

System integration is a programming, or a kind of programming. With its own features (like limited expressive power in some places - see e.g. dhall language), traditions, history etc. but still - you ought to be able to use the classical programming toolkit to solve system integration problems. So you should ask, can you use the common languages - and if not, why?

galaxyLogic 7y ago
> can you use the common languages - and if not, why? Good question. I do not know the answer to that. But I think there must be some answer since there are all these different configuration languages, Ant, Gradle etc. They must have been created for a reason.

corndoge 7y ago

Like visudo, for everything! That would be cool.

Qasaur 7y ago

This already sort-of exists in the form of Nix/NixOS/NixOps. Entire system configuration is deterministic and is specified in a configuration file, and it has its own little functional DSL called Nix that is used to specify system configs and how to build software packages. Just started using it and I think it's probably one of the best software ensembles I've ever used in my career. Completely knocks Docker, K8S etc. out of the water.

nh2 7y ago
Yes, the Nix ecosystem makes running both servers and services much easier than any other tools I have used. I describe the deployment I run with it here: https://news.ycombinator.com/item?id=21468506 https://news.ycombinator.com/item?id=21468506 Nix is not typed (it would be slightly better if it were), but everything is evaluated before it hits your servers which allows for lots of static checks. I started writing a tutorial on NixOps here if you want to learn it: https://github.com/nh2/nixops-tutorial https://github.com/nh2/nixops-tutorial (only has 1 part so far, I'd like to show how to bootstrap a Consul cluster and distributed file system next).

jpolitz 7y ago

You might be interested in Rehearsal: https://people.cs.umass.edu/~arjun/main/papers/2016-rehearsal.html https://people.cs.umass.edu/~arjun/main/papers/2016-rehearsa...

gtycomb 7y ago

Isn't it Go?

nothrabannosir 7y ago
This is a fair question if you’re lucky enough not to be familiar with devops tooling. No, it’s not Go. Go is a general purpose programming language. What parent poster wants is more like Terraform, but better (I assume). With semantics that reflect devops actions. You might write that tool in Go :) but the users wouldn’t necessarily know. Just like a gamer doesn’t need to know about C++.
- throwaway894345 7y ago
  You could use Go to generate Terraform configs. I’m using Python to generate CloudFormation and it’s an order of magnitude easier and more maintainable than vanilla CloudFormation, especially for large integrations.

weberc2 7y ago

Any statically types, general purpose programming language would do the trick—just generate YAML. I’ve been playing with typed-Python and it works really well. The only issues are that Python’s type system is pretty immature (can’t represent JSON, can’t constrain generic types to a particular interface/protocol a la Rust traits, type annotations in a class share the same namespace as member variables, etc). I would like to try Go for this, but it would be a fair bit more imperative.

vetrom 7y ago

Chef at least provides a partial approach here, where components essentially define an api that you can define mock and/or integration tests for. Ansible is not as 'data-structure' oriented, so testing that is more difficult, but each element (modules, libraries, roles, playbooks) can be written to be fairly environment-neutral, which you could then use a test a fixture like BATS to easily check for regressions. To answer your 'type-checking' expectation, its generally on the developer to implement those tests. In my experience, its rare that those are implemented in publicly available devops libraries, even though we have most of the tools to do so. It's definitely an area where the practice can improve.

Joker_vD 7y ago

I have a sinking feeling that the whole business of writing huge config files (semi-)manually (templating, #include, and YAML-references that reference things from god knows where) is... too low-level? Too imperative? It feels like if one was manually laying out the vtables and the unwinding info in assembly language: sure, the macros will go a long way helping you, but maybe the better way would be to have an actual high-level declarative description that would get translated down to metal. And "high-level" doesn't mean, let's use an analogy, M4, it means Gradle.