The former is technically correct and much more descriptive. The latter is potentially-misleading. One might think that the rate did not increase at all, when

The former is technically correct and much more descriptive. The latter is potentially-misleading. One might think that the rate did not increase at all, when in fact it did, or that it did not increase "a lot" (the colloquial understanding of significant). "Statistically-significant" is a keyword with technical meaning. The former could mean that the rate increased by 2±2 units. A more-precise study might show that the rate did increase, or it might not. All we know from the former is that the rate increased, but that the rate was not statistically-significant.