4 ms·

My experience exactly. "Can you automate it?" should always be followed by a discussion of what "it" precisely means. If you can describe in English not just t

My experience exactly. "Can you automate it?" should always be followed by a discussion of what "it" precisely means. If you can describe in English not just the happy path but all possible failure modes, then your problem is automate-able. If there are nodes in the real workflow that are missing from your mental flowchart, you need to put in more work to understand and describe your problem before it can be automated. Codifying these decisions in writing and using it as part of your _manual_ process is a mandatory pre-requisite to automation. Automation is almost entirely limited by the human mental capacity to model the problem precisely. It's not a technical problem at all. This is hard for people to hear - automation means a lot of front-loaded mental work for everyone involved. In a manual system, you can defer that mental work to runtime and make decisions on the fly. In an automated system, you're ceding control to a machine. You must first codify the contents of someone's brain with incredible detail such that a computer can make every decision the same way a human operator would under the same conditions. In an automated system, the criteria cannot be ambiguous in any way (if your flowchart diverges depending on whether a file is valid or not, you need battle-tested `file_is_valid` function). It's useless to talk about automating an entire graph when you can't even describe the nodes. A good model that I use is self-driving cars. We certainly have the technology for 100% self-driving vehicles. But considering the number of edge cases in real driving conditions, most people are very hesitant to cede control to a machine. There's an implicit realization that while automation might be convenient, the potential negative consequences of a machine error are too great and most drivers choose to retain control. The negative consequences of bad automation in IT are usually less severe but still an enormous drain on resources to babysit the failure states of poorly-conceived automation workflows.

epgui 4y ago
I mostly agree, except for the idea that instructions “cannot be ambiguous in any way”. I think this is more of a nuance than anything, however. I think the difficulty of the nuance is that we usually don’t realize when our language is ambiguous. Handwriting recognition is a super ambiguous problem, despite being easily solved with great accuracy today: good luck formalizing how it’s actually done and explaining it to a layperson. The difficulty in most non-trivial AI/ML tasks is in figuring out how to structure the problem (for handwriting recognition, let’s not forget that we have the tremendous benefit of hindsight), which essentially amounts to deciding that some kinds of ambiguity are okay (eg.: things that can be reduced to a statistical or classification problem), and some kinds of ambiguity need to be resolved (eg.: avoiding potholes is probably a good idea, and one way or another this has to be rewarded or encoded, by some mechanism… and great care is required to account for basically everything you’d expect to find in a really good driving manual). There are some types of ambiguities that AI/ML are very good at… But to your point, there is a degree of “relative unambiguity” that is required. The difficulty is compounded by human biases too: what is obviously-unclear to a careful observer might appear to be super clear to an overconfident person.

epgui 4y ago

I mostly agree, except for the idea that instructions “cannot be ambiguous in any way”. I think this is more of a nuance than anything, however. I think the difficulty of the nuance is that we usually don’t realize when our language is ambiguous. Handwriting recognition is a super ambiguous problem, despite being easily solved with great accuracy today: good luck formalizing how it’s actually done and explaining it to a layperson. The difficulty in most non-trivial AI/ML tasks is in figuring out how to structure the problem (for handwriting recognition, let’s not forget that we have the tremendous benefit of hindsight), which essentially amounts to deciding that some kinds of ambiguity are okay (eg.: things that can be reduced to a statistical or classification problem), and some kinds of ambiguity need to be resolved (eg.: avoiding potholes is probably a good idea, and one way or another this has to be rewarded or encoded, by some mechanism… and great care is required to account for basically everything you’d expect to find in a really good driving manual). There are some types of ambiguities that AI/ML are very good at… But to your point, there is a degree of “relative unambiguity” that is required. The difficulty is compounded by human biases too: what is obviously-unclear to a careful observer might appear to be super clear to an overconfident person.