To me the most intuitive and flexible UI would be something like Avidemux where you have a picker of filters/transforms and you can add them to a box on the rig

To me the most intuitive and flexible UI would be something like Avidemux where you have a picker of filters/transforms and you can add them to a box on the right (potentially you can add them multiple times, etc) and then configure them. That is a very powerful "general" UI flow that works reasonably well imo. But yeah you're right, it's all a question of how much capability you want to surface, because the more you surface the more quickly it gets complex and inscrutable to anyone not in the know.