3 ms·

If anybody wants to chat about how to harden production systems, let's talk. That's a huge focus for what I have in my head to fix right now. (Founder of a YC s

If anybody wants to chat about how to harden production systems, let's talk. That's a huge focus for what I have in my head to fix right now. (Founder of a YC startup) Lambda helps but is limited. Dealing with existing code is hard. If this is a problem that keeps you awake at night, I want to talk with you. I want to figure out what can be done to remove single points of failure/compromise in production software systems. Email is free at refinery dot io

js8 6y ago
Not an expert, but if you like lambda, I suggest looking at its predecessors, in particular, CICS and OTP. My current take on this. The "business logic" should not be written in Turing complete languages, but rather composable languages based on simple type theory. So the "business logic" would be limited in what it can do to compromise the system. And then it should be deployed on the execution platform, which is a different concern altogether. I think using TC (like Java or Javascript) languages for everything is a wrong direction in SW engineering. I think we want to have TC languages for the low level tools (like parsing data, algorithms), and for the execution engine, but we want to compose these in languages that can easily compose, and thus are restricted. IMHO there are at least two different categories of compositional restricted languages. One is data manipulation languages, that describes how to manipulate data in the schema, regardless of the platform or physical data representation (I really like what David Spivak is doing in this area with categorical databases). The other is system architecture languages, which shows how are these data manipulations executed, and how they compose into transactions, and more broadly, software systems. The former category concerns with how the schema of the data changes, and the latter category concerns with how are the data in the same schema exchanged at the boundaries of different SW systems. (There are probably also useful categories of restricted languages to describe UI and visualizations.)
- Lio 6y ago
  > My current take on this. The "business logic" should not be written in Turing complete languages, I think this is an excellent point. I've been coming to this same conclusion for some time. For a lot of use cases a less powerful but more predictable language is probably what we need. State machines and DSLs go some way towards this but I don't think that's the full solution either. Maybe something based on a restricted APL language, either way it sounds like a good PHD project to me.
  - bantunes 6y ago
    This might interest you https://alan-lang.org/alan_overview.html#parallel-computation-and-the-problem-of-turing-completeness https://alan-lang.org/alan_overview.html#parallel-computatio...
    - freeqaz 6y ago
      Going to give this a deep dive. Thank you. Super curious how it works after reading the intro page!
  - Martinsos 6y ago
    I also believe in this as a promising direction! Specifically I have been trying to apply it to full-stack web app development for some time now by building a DSL (https://github.com/wasp-lang/wasp https://github.com/wasp-lang/wasp). Why do you think DSLs are not the full solution? My main thought was that having DSLs for specific areas of programming domains is the way to go. But they should have interop with Turing complete languages to implement specific sub-parts. That way DSLs can capture a lot of knowledge and best practices that we learned through years in that domain while still facilitating innovation being flexible. Such language would be very limited, more like "smart configuration languages". Some of them I have seen around are Terraform, Nextflow, Meson-build. APL still sounds too general and self-sufficient to me to be a solution to this (but I don't know much about it so maybe I am wrong, I just checked it out on the wikipedia).
- freeqaz 6y ago
  Super awesome write up -- thank you! Honestly I have been thinking about this problem for a few years now but not as succinctly as you have described it here. That's really helpful to split it down the line like that. It's very hard to uncouple complex topics cleanly. "Workflows" have been an area of focus for a product we've been building for a while now. A big part of that goal is to remove the "Turing completeness" from the system and keep each "block" of a workflow as a simple blackboxed data transformation. And "conditional transitions" are used to emulate business logic. It's a bit clunky though and, after reading your text here, it makes me want to sit down and think for a few hours. Thank, internet stranger, you have made a material difference in my day over here! It's a nice feeling to hop out of bed early and excited :)

js8 6y ago

Not an expert, but if you like lambda, I suggest looking at its predecessors, in particular, CICS and OTP. My current take on this. The "business logic" should not be written in Turing complete languages, but rather composable languages based on simple type theory. So the "business logic" would be limited in what it can do to compromise the system. And then it should be deployed on the execution platform, which is a different concern altogether. I think using TC (like Java or Javascript) languages for everything is a wrong direction in SW engineering. I think we want to have TC languages for the low level tools (like parsing data, algorithms), and for the execution engine, but we want to compose these in languages that can easily compose, and thus are restricted. IMHO there are at least two different categories of compositional restricted languages. One is data manipulation languages, that describes how to manipulate data in the schema, regardless of the platform or physical data representation (I really like what David Spivak is doing in this area with categorical databases). The other is system architecture languages, which shows how are these data manipulations executed, and how they compose into transactions, and more broadly, software systems. The former category concerns with how the schema of the data changes, and the latter category concerns with how are the data in the same schema exchanged at the boundaries of different SW systems. (There are probably also useful categories of restricted languages to describe UI and visualizations.)

Lio 6y ago
> My current take on this. The "business logic" should not be written in Turing complete languages, I think this is an excellent point. I've been coming to this same conclusion for some time. For a lot of use cases a less powerful but more predictable language is probably what we need. State machines and DSLs go some way towards this but I don't think that's the full solution either. Maybe something based on a restricted APL language, either way it sounds like a good PHD project to me.
- bantunes 6y ago
  This might interest you https://alan-lang.org/alan_overview.html#parallel-computation-and-the-problem-of-turing-completeness https://alan-lang.org/alan_overview.html#parallel-computatio...
  - freeqaz 6y ago
    Going to give this a deep dive. Thank you. Super curious how it works after reading the intro page!
- Martinsos 6y ago
  I also believe in this as a promising direction! Specifically I have been trying to apply it to full-stack web app development for some time now by building a DSL (https://github.com/wasp-lang/wasp https://github.com/wasp-lang/wasp). Why do you think DSLs are not the full solution? My main thought was that having DSLs for specific areas of programming domains is the way to go. But they should have interop with Turing complete languages to implement specific sub-parts. That way DSLs can capture a lot of knowledge and best practices that we learned through years in that domain while still facilitating innovation being flexible. Such language would be very limited, more like "smart configuration languages". Some of them I have seen around are Terraform, Nextflow, Meson-build. APL still sounds too general and self-sufficient to me to be a solution to this (but I don't know much about it so maybe I am wrong, I just checked it out on the wikipedia).
freeqaz 6y ago
Super awesome write up -- thank you! Honestly I have been thinking about this problem for a few years now but not as succinctly as you have described it here. That's really helpful to split it down the line like that. It's very hard to uncouple complex topics cleanly. "Workflows" have been an area of focus for a product we've been building for a while now. A big part of that goal is to remove the "Turing completeness" from the system and keep each "block" of a workflow as a simple blackboxed data transformation. And "conditional transitions" are used to emulate business logic. It's a bit clunky though and, after reading your text here, it makes me want to sit down and think for a few hours. Thank, internet stranger, you have made a material difference in my day over here! It's a nice feeling to hop out of bed early and excited :)