The bigger you make it the slower it'll end up working, basically. The time for the signal to propagate through all those discrete transistors will be quite big

The bigger you make it the slower it'll end up working, basically. The time for the signal to propagate through all those discrete transistors will be quite big and that puts an upper bound to your clock frequency.