Climate is the big one, land and access to water (I know this seems weird for California but they do have it). You could grow in West TX, southern NM and AZ but

Climate is the big one, land and access to water (I know this seems weird for California but they do have it). You could grow in West TX, southern NM and AZ but the water logistics are not there. You see the same crops in FL, but they have less land to support the same kind of agriculture infrastructure as they do in California. The labor situation is a symptom of the above factors, by its very nature its mobile. If you had the farms to support it, the labor would follow.