I always thought it was a matter of economy more than anything else. If all you need to do is produce text it's very "easy" to build a massive branching storyli

I always thought it was a matter of economy more than anything else. If all you need to do is produce text it's very "easy" to build a massive branching storyline with lots of failure conditions and truly modifying developments since text is cheap. When each of these branches also need additional hours of voice acting, more characters to design and animate, more locations to model, etc it becomes increasingly expensive to make a storyline that has strongly branching paths. There was a time where there was a nice sweet spot that allowed for a decent amount of creativity and branching and opportunity for failure, but as games started to have more and more of these details, developers started making choices more minor, making choices that are illusory, and building at most 2-3 endings that only really branch very late game or mostly rely on mechanics (oftentimes how many people you killed) so that there is less that needs to be adjusted in terms of plot. Secondarily, you also likely have pressure to make a game that is enjoyable for as long as possible and caters to as many people as possible, which discourages building/allowing critical failure points like the pie in King's Quest V, and maybe also discourages branching too hard so that all content that can be experienced is mostly easily digestible in one playthrough (the subtext being, if you're only ever going to be paid once per user in the context of a single player game, that any path not explored by most people is essentially a massive waste of money so the experience should be streamlined to a tunnel as far as possible to a cost limit the minimum expected adventure).