Exactly, any algorithm choice that resulted in lower engagement metrics would necessarily be something Twitter (and any social site really) would try and avoid

Exactly, any algorithm choice that resulted in lower engagement metrics would necessarily be something Twitter (and any social site really) would try and avoid at all costs. It's why government regulation is probably the only way to fix the issue, since you need companies to act in a way that's counter to their own interests (but would lead to a better experience for users.)