It is not wrong, it is rather correct. Speed of propagation in semiconductor materials is at most a third of speed of light in vacuum. So the distance travelled

It is not wrong, it is rather correct. Speed of propagation in semiconductor materials is at most a third of speed of light in vacuum. So the distance travelled is rather limited for a signal. Also, a signal might have to traverse a few transistors or gates, so frequency in the 3GHz range does really limit processor sizes to the order of millimeters. You already said how to get around it: Pipelines, that limits the area a signal has to propagate. Also, one has to take care to make signals arrive early enough in the longest possible signal path as well as to distribute the clock in a way for it to arrive at aligned times everywhere, so you need a clock distribution net with known delays, etc. Chip timing is black art.