It would still be awful, because it would disadvantage other forms of content that have the same bandwidth needs but are not video, and also anything that doesn

It would still be awful, because it would disadvantage other forms of content that have the same bandwidth needs but are not video, and also anything that doesn't have "both ends". And also, obviously, that ideal is completely unrealistic: Either the ISP gets to look into the video stream to verify that it is indeed video, which would be awful, or what you in effect end up with is a flat rate for "medium bit rate connections", which would be fine, but absolutely not what any of those ISPs want.