indeed it tips perpendicular to the direction of applied force. and this is what happens in reality. try spinning a bicycle wheel holding its axle on both ends.

indeed it tips perpendicular to the direction of applied force. and this is what happens in reality. try spinning a bicycle wheel holding its axle on both ends. now try to twist the axle and it curiously wants to move the wheel in a perpendicular direction.