Another factor is that if your department's area is hiring for a new position that you might be a good fit for, your P.I. or advisor is likely on the hiring com

Another factor is that if your department's area is hiring for a new position that you might be a good fit for, your P.I. or advisor is likely on the hiring committee. Thus the committee then has to be wary of nepotism and conflicts of interest. When everyone is so incredibly specialized, it turns out that the sub-field you are working in is small.