IMO a waterfall like process can work for a lot of projects, but you need a lot of time for estimation/derisking, and this work is hard, so often developers don

IMO a waterfall like process can work for a lot of projects, but you need a lot of time for estimation/derisking, and this work is hard, so often developers don't want to do it, and management/customers often don't want to pay for it. Some of the most successful projects I've worked on have started with an estimate that had only a few items with more than 4 hours attached to them. If you get estimates down to that level, that means that most of the significant technical decisions have already been made, and you've determined that your technical approach should work. If you do that, and then add a calibrated amount of padding for mistakes, you can very often hit estimates pretty much dead on (+-10%). How much time it takes to prepare the estimate depends a lot on the nature of the work though. If there's something pretty similar the team has done multiple times, and you have access to estimates/tickets/timesheets, you can often get it done in about 5-10% of the total project time. However, for example, if it's an ML project, there may be a lot of unknowns that have to be clarified first. For example, data quality is almost always not what it is claimed to be at the outset, so this needs to be resolved before the estimate is generated.