One has to be very careful challenging the final authority of the crew. In the times where this was not the case, you could just tell your pilots to be on time

One has to be very careful challenging the final authority of the crew. In the times where this was not the case, you could just tell your pilots to be on time no matter what and end up with crashes due to flying into weather, running out of fuel, etc. At the end of the day, the PIC needs final authority since it's their ass on the line as well. As an aside, the flight deck crew don't really pay much attention to whats happening in the back and have to rely on the FAs to manage the passengers. If the FA complains about someone to the flight deck, the PIC will generally follow their recommendation without pause.