It is intended to disengage upon any physical movement of the throttles. Normal usage of the TOGA buttons would be to push them and not move the throttles - as

It is intended to disengage upon any physical movement of the throttles. Normal usage of the TOGA buttons would be to push them and not move the throttles - assuming the throttle has normal non-worn or adjusted throttle friction. Abnormal usage may well be that the throttle friction has worn or has not been adjusted by maintenance (afaik it's not crew-adjustable on boeing aircraft as it is on smaller light airliners), and thus pressing TOGA may shift the throttle lever enough to disengage A/T.