The issue lies in the way the signal was actually measured. The first observation was in the digamma decay channel, where it basically amounted to an excess pho

The issue lies in the way the signal was actually measured. The first observation was in the digamma decay channel, where it basically amounted to an excess photon count over the background at a specific energy (i.e. invariant mass). So the idea of a simple "null" hypothesis was not really well defined here, because the excess could have also been at other energies. But the five sigma value was given only for the bump over background at ~125 GeV. In that specific sense, Krauss' statement is very much correct, but the overall hypothesis of whether H0 exists - yes or no - given arbitrary data is a bit more complicated when doing the statistics rigorously. That's also why you really want 5 sigmas, because as you widen the parameter space you're searching in, even 3 sigma results (= less than half a percent chance of statistical error) at specific energies regularly turn out to be flukes.