You don't need a whole new OS, Android is open source and can be forked. The problem is that people's expectation of polish on their phones is through the roof

You don't need a whole new OS, Android is open source and can be forked. The problem is that people's expectation of polish on their phones is through the roof and nobody has figured out how to fund that kind of development effort. CyanogenMod came close but was torn apart, though LineageOS is still there. In my opinion LineageOS should ditch the neverending cycle of trying to keep up with Google's increasingly useless Android releases, just pick a branch point and leave Google behind. It's the only way to implement OS features and be able to maintain them going forward. Then they should focus on features users actually want and that Google would never implement, for example OS level adblock and firewalls. Instead of other existing attempts that focus on nebulous privacy and security, things that users don't really care about, the focus should be on giving users back the control that Google and Apple have taken away from us. Note that user-control and security can often be seen as being at odds. An OS that chooses user-control could be a strong differentiator that would set it apart from Google/Apple, who always frame their removal of user-control as a security/privacy feature. But still, the tricky bit is figuring out a funding model.