What you're describing would be called "end-to-end learning", and the way things have been progressing is that a lot of systems cobbled together (e.g speech syn

What you're describing would be called "end-to-end learning", and the way things have been progressing is that a lot of systems cobbled together (e.g speech synthesis, translation, image recognition) have been converted to models (mostly neural nets) that are learned end-to-end. Autonomous driving is not an exception, and they might still get there, but you have to put your pragmatic hat on and do whatever works right now. Benefit of this approach include more interpretable results, potentially improved safety guarantees (e.g you can limit the failure to a subsystem, things like that)