5 ms·

Sounds like a reliability and auditability nightmare. And, in securing software systems, full bill of materials and even reproducible builds are used to lock in

by bdamm 4y ago

Sounds like a reliability and auditability nightmare. And, in securing software systems, full bill of materials and even reproducible builds are used to lock in the software to known good versions. How could software that can change thanks to the whim of a dev in the ops environment ever be considered reproducible? Wouldn't the result be just hugely fragile code state with no known source?

jrockway 4y ago
Clearly some people just don't care about that sort of thing. Look at the popularity of Jupyter notebooks, Julia, etc. There just isn't much forcing people to engage in good software development and operational strategies. Sometimes there are regulations in the field (PCI, HIPAA), sometimes adversaries force care (state sponsored attacks on Google), but that's kind of rare. The usual forcing function is a competitor that delivers features faster than you. In that case, maybe "live programming" is worth a look. I've never seen "less bugs" on a product comparison sheet, for example. (It's a contentious view, and I'm personally not a fan. I like it for software I'm sitting in front of, like Emacs, but hate it for things that need to run unattended on their own, which is like everything except text editors. And "less bugs" is a big selling point for me personally. Every bug I run into consumes time that I'd rather spend elsewhere. But, not everyone sees it that way.)
- fardo 4y ago
  >I've never seen "less bugs" on a product comparison sheet, for example. Those show up, your sales department just uses the more formal names for the actual customer demands that “less bugs” encodes. That will look like some combination of • “We have an SLA for an annual 99.9xx% uptime and average sub x00 latency!” (so if a bug causes significant annual reduction in service, your business gets penalized), • ”We guarantee regulatory compliance!” (so if a bug or business use causes regulatory issues, it’s your ass on the line), • “We guarantee 24/7 same day chat or email support!” (so if a bug causes an outage, they have a warm body they can yell at, demand an explanation for their customers, and ask questions) • “We guarantee backups, data redundancy, and worldwide data replication!” (so if a bug blows up one of your data centers somewhere, or the intern fat-fingers an accidental deletion of the production user database, the customer doesn’t even notice something went wrong) • “We guarantee API backwards compatibility or a service maintenance guarantee until 20xx!” (So you’re forced to fix bugs, and the bug isn’t ‘your PM team might try to #KilledByGoogle’ the service you invested in building infrastructure on.) Teams that believe in the health, care, and security of their design are far better equipped to offer the above valuable terms to customers to gain competitive business advantage, and teams with bad engineering hygiene are likely to be scared off.
- sundarurfriend 4y ago
  > Look at the popularity of Jupyter notebooks, Julia, etc. Neither of these (inherently) have the kind of "live programming" environment that the original comment was talking about. Jupyter notebooks have a different reproducibility problem though, with how easy it is to create hidden state. But they're not intended to be used in production at all anyway, so the problem mainly affects pedagogical and information sharing use cases. Julia with the Revise.jl package installed and loaded comes kinda close to the live programming model - but the state is always preserved in the source file necessarily with Revise (and it's a development tool too, not one loaded in production). Since you mentioned it alongside Jupyter, maybe you meant the Pluto notebook for Julia? There too the state is always preserved, both in terms of packages (with Julia's usual Manifest file) and in terms of your code (no hidden state like Jupyter, since it's a reactive notebook).
cgarvis 4y ago
Smalltalk has a change file and an ability to be source controlled. My understand is that the source controlled output is not human friendly.
- Jtsummers 4y ago
  https://github.com/pharo-project/pharo/blob/Pharo11/src/Announcements-Core/AnnouncementSubscription.class.st https://github.com/pharo-project/pharo/blob/Pharo11/src/Anno... (random class selected) It's not that human unfriendly, if you know Smalltalk (which is not a terribly hard thing to learn). But you also wouldn't interact with it in that form but rather inside of Pharo and using its class browser.
  - mst 4y ago
    I've never actually -written- any Smalltalk and can read that code just fine.
- 0x445442 4y ago
  Actually there are serializable forms like TOML that are very readable.
bcrosby95 4y ago
Production environments aren't the only environment software is run in. Pretty much by definition, more code touches non production environments than is ever ran in a production one: everything that touches production likely passed through at least 1 other environment beforehand. Then you have the code that never made it to production due to it being buggy/etc.
- williamcotton 4y ago
  If all you have is a hammer…
Supermancho 4y ago
> Sounds like a reliability and auditability nightmare. Why? Reliability, auditability, and testing isn't affected. This is about developing methodology and how the language syntax and runtimes can enable that.
skissane 4y ago
Why can't a "live coding" interface have auditing, access control, etc, just like any other interface? In production, disable it completely or limit its access to highly privileged users – but developers can still enjoy it in their local environments. A good implementation of the idea would incorporate version control – when a service starts, it knows which version of the code it is running (commit hash, etc), and it has an introspection interface to report that version. If any "live coding" changes are made, it knows exactly which classes/methods/functions/etc have been changed compared to that initial version, and reports that through that interface too. You can then have a centralised configuration management system, which polls all services to find out which version they are running (and whether there are any additional changes beyond that version), and alerts if any production system is running different code from what it is supposed to be. Since your "live coding" interface is audited, you know the exact timestamp/username/etc of the change, and so anyone making inappropriate changes to production systems can be caught and dealt with. In production, a "live coding" interface can be used to enable live patching – so you can apply a patch to a running service, without even having to restart it. Of course, the patch would be tested first in lower environments, and the interface would be invoked by some patching tool / deployment pipeline, not an individual developer.
- robocat 4y ago
  I assume SQL stored procedures on some database work have some of those features? Seems like the sort of things that database engine developers would consider and the atomic semantics of committing stored procedures to a database lend itself to that model more closely (for databases with atomic DDL). I worked with an Object Oriented database that had some cool features because the code was stored as highly structured data within the database itself, i.e. the code you saved was not text at all (except when dumped/reloaded to external file - similar to exporting/importing data of a database to/from a file as a textual representation).
  - lmm 4y ago
    Stored procedures are very poorly controlled in my experience, and tend to work on a YOLO deployment model, even at e.g. big financial institutions.
peterkelly 4y ago
Excellent question. The way I see it, there's two scenarios where this approach can be useful, which I'll address separately. The first is for local development. When iterating on a solution to a problem, it can be a hassle to restart the program from the beginning and incur the time penalty to get to the part you're modifying in order to test it. This can include not just compile times, but also program startup, loading in necessary data, and going through a series of steps either in a UI or via API calls to get the system into a state where it's about to trigger the functionality you're working on and want to test. For example, I'm currently working on a business workflow that has a bunch of steps that require someone to fill in several forms, and things happen after each of those. When I'm working on stage 5 of that process, testing it involves repeating all the actions necessary to go through steps 1, 2, 3, and 4. Yes, it's possible with extra effort to automate this, or to create mock state that can be used to jump directly to step 5 on launch, but that's just a manual substitute for what a live programming environment gives you automatically. A similar situation occurs in game development; you don't want to play through half the game every time you want to get to a particular part where you've changed how a particular enemy acts. The second scenario is production. You're correct that it's important to only deploy code that's undergone proper testing and review, has a known version, and is reproducible. However this is orthogonal to the use of a live programming environment. With the latter, you wouldn't just have developers interacting with it in an ad-hoc fashion (just as you wouldn't have developers directly modifying python scripts on a production server). Instead, the ability to modify running programs can be used as a deployment mechanism. Once a set of changes has been made in a local development environment, undergone testing, and is deemed ready for deployment, the same mechanisms for updating running code that are used for development purposes can be used as a way of deploying code. And in production, the fact that you don't need to restart the process means you can avoid downtime. As far as deployment and versioning is concerned, you can think of it as being a similar to how you would do those things for a server based on CGI scripts or similar (e.g. PHP) where every time a request is served, the file is read from disk and executed. The difference there is that all state has to live in a database, so if you have long-running processes e.g. business workflows that span days or weeks, all state transitions/control flow has to be managed manually, rather than using language features like sequencing, iteration, and function calls. With a live programming environment that supports persistence (meaning that execution state is stored in a file or database in a manner transparent to the programmer), deployment consists of adding/updating a set of objects in the data store, rather than copying a set of files to a particular location. An example of a system that supports runtime code updates is Erlang, though it is not persistent. An example of a system that is persistent is Ethereum, though that doesn't support runtime code updates. Smalltalk and its variants support both. Auditability can be supported by ensuring all changes to code or runtime state are made through transactions.

by bdamm 4y ago

jrockway 4y ago

Clearly some people just don't care about that sort of thing. Look at the popularity of Jupyter notebooks, Julia, etc. There just isn't much forcing people to engage in good software development and operational strategies. Sometimes there are regulations in the field (PCI, HIPAA), sometimes adversaries force care (state sponsored attacks on Google), but that's kind of rare. The usual forcing function is a competitor that delivers features faster than you. In that case, maybe "live programming" is worth a look. I've never seen "less bugs" on a product comparison sheet, for example. (It's a contentious view, and I'm personally not a fan. I like it for software I'm sitting in front of, like Emacs, but hate it for things that need to run unattended on their own, which is like everything except text editors. And "less bugs" is a big selling point for me personally. Every bug I run into consumes time that I'd rather spend elsewhere. But, not everyone sees it that way.)

fardo 4y ago
>I've never seen "less bugs" on a product comparison sheet, for example. Those show up, your sales department just uses the more formal names for the actual customer demands that “less bugs” encodes. That will look like some combination of • “We have an SLA for an annual 99.9xx% uptime and average sub x00 latency!” (so if a bug causes significant annual reduction in service, your business gets penalized), • ”We guarantee regulatory compliance!” (so if a bug or business use causes regulatory issues, it’s your ass on the line), • “We guarantee 24/7 same day chat or email support!” (so if a bug causes an outage, they have a warm body they can yell at, demand an explanation for their customers, and ask questions) • “We guarantee backups, data redundancy, and worldwide data replication!” (so if a bug blows up one of your data centers somewhere, or the intern fat-fingers an accidental deletion of the production user database, the customer doesn’t even notice something went wrong) • “We guarantee API backwards compatibility or a service maintenance guarantee until 20xx!” (So you’re forced to fix bugs, and the bug isn’t ‘your PM team might try to #KilledByGoogle’ the service you invested in building infrastructure on.) Teams that believe in the health, care, and security of their design are far better equipped to offer the above valuable terms to customers to gain competitive business advantage, and teams with bad engineering hygiene are likely to be scared off.
sundarurfriend 4y ago
> Look at the popularity of Jupyter notebooks, Julia, etc. Neither of these (inherently) have the kind of "live programming" environment that the original comment was talking about. Jupyter notebooks have a different reproducibility problem though, with how easy it is to create hidden state. But they're not intended to be used in production at all anyway, so the problem mainly affects pedagogical and information sharing use cases. Julia with the Revise.jl package installed and loaded comes kinda close to the live programming model - but the state is always preserved in the source file necessarily with Revise (and it's a development tool too, not one loaded in production). Since you mentioned it alongside Jupyter, maybe you meant the Pluto notebook for Julia? There too the state is always preserved, both in terms of packages (with Julia's usual Manifest file) and in terms of your code (no hidden state like Jupyter, since it's a reactive notebook).

cgarvis 4y ago

Smalltalk has a change file and an ability to be source controlled. My understand is that the source controlled output is not human friendly.

Jtsummers 4y ago
https://github.com/pharo-project/pharo/blob/Pharo11/src/Announcements-Core/AnnouncementSubscription.class.st https://github.com/pharo-project/pharo/blob/Pharo11/src/Anno... (random class selected) It's not that human unfriendly, if you know Smalltalk (which is not a terribly hard thing to learn). But you also wouldn't interact with it in that form but rather inside of Pharo and using its class browser.
- mst 4y ago
  I've never actually -written- any Smalltalk and can read that code just fine.
0x445442 4y ago
Actually there are serializable forms like TOML that are very readable.

bcrosby95 4y ago

Production environments aren't the only environment software is run in. Pretty much by definition, more code touches non production environments than is ever ran in a production one: everything that touches production likely passed through at least 1 other environment beforehand. Then you have the code that never made it to production due to it being buggy/etc.

williamcotton 4y ago
If all you have is a hammer…

Supermancho 4y ago

> Sounds like a reliability and auditability nightmare. Why? Reliability, auditability, and testing isn't affected. This is about developing methodology and how the language syntax and runtimes can enable that.

skissane 4y ago

Why can't a "live coding" interface have auditing, access control, etc, just like any other interface? In production, disable it completely or limit its access to highly privileged users – but developers can still enjoy it in their local environments. A good implementation of the idea would incorporate version control – when a service starts, it knows which version of the code it is running (commit hash, etc), and it has an introspection interface to report that version. If any "live coding" changes are made, it knows exactly which classes/methods/functions/etc have been changed compared to that initial version, and reports that through that interface too. You can then have a centralised configuration management system, which polls all services to find out which version they are running (and whether there are any additional changes beyond that version), and alerts if any production system is running different code from what it is supposed to be. Since your "live coding" interface is audited, you know the exact timestamp/username/etc of the change, and so anyone making inappropriate changes to production systems can be caught and dealt with. In production, a "live coding" interface can be used to enable live patching – so you can apply a patch to a running service, without even having to restart it. Of course, the patch would be tested first in lower environments, and the interface would be invoked by some patching tool / deployment pipeline, not an individual developer.

robocat 4y ago
I assume SQL stored procedures on some database work have some of those features? Seems like the sort of things that database engine developers would consider and the atomic semantics of committing stored procedures to a database lend itself to that model more closely (for databases with atomic DDL). I worked with an Object Oriented database that had some cool features because the code was stored as highly structured data within the database itself, i.e. the code you saved was not text at all (except when dumped/reloaded to external file - similar to exporting/importing data of a database to/from a file as a textual representation).
- lmm 4y ago
  Stored procedures are very poorly controlled in my experience, and tend to work on a YOLO deployment model, even at e.g. big financial institutions.

peterkelly 4y ago

Excellent question. The way I see it, there's two scenarios where this approach can be useful, which I'll address separately. The first is for local development. When iterating on a solution to a problem, it can be a hassle to restart the program from the beginning and incur the time penalty to get to the part you're modifying in order to test it. This can include not just compile times, but also program startup, loading in necessary data, and going through a series of steps either in a UI or via API calls to get the system into a state where it's about to trigger the functionality you're working on and want to test. For example, I'm currently working on a business workflow that has a bunch of steps that require someone to fill in several forms, and things happen after each of those. When I'm working on stage 5 of that process, testing it involves repeating all the actions necessary to go through steps 1, 2, 3, and 4. Yes, it's possible with extra effort to automate this, or to create mock state that can be used to jump directly to step 5 on launch, but that's just a manual substitute for what a live programming environment gives you automatically. A similar situation occurs in game development; you don't want to play through half the game every time you want to get to a particular part where you've changed how a particular enemy acts. The second scenario is production. You're correct that it's important to only deploy code that's undergone proper testing and review, has a known version, and is reproducible. However this is orthogonal to the use of a live programming environment. With the latter, you wouldn't just have developers interacting with it in an ad-hoc fashion (just as you wouldn't have developers directly modifying python scripts on a production server). Instead, the ability to modify running programs can be used as a deployment mechanism. Once a set of changes has been made in a local development environment, undergone testing, and is deemed ready for deployment, the same mechanisms for updating running code that are used for development purposes can be used as a way of deploying code. And in production, the fact that you don't need to restart the process means you can avoid downtime. As far as deployment and versioning is concerned, you can think of it as being a similar to how you would do those things for a server based on CGI scripts or similar (e.g. PHP) where every time a request is served, the file is read from disk and executed. The difference there is that all state has to live in a database, so if you have long-running processes e.g. business workflows that span days or weeks, all state transitions/control flow has to be managed manually, rather than using language features like sequencing, iteration, and function calls. With a live programming environment that supports persistence (meaning that execution state is stored in a file or database in a manner transparent to the programmer), deployment consists of adding/updating a set of objects in the data store, rather than copying a set of files to a particular location. An example of a system that supports runtime code updates is Erlang, though it is not persistent. An example of a system that is persistent is Ethereum, though that doesn't support runtime code updates. Smalltalk and its variants support both. Auditability can be supported by ensuring all changes to code or runtime state are made through transactions.