Exactly - probability (and by extension, statements such as "more likely") only make sense when we have a reference class of similar events for which we have it

Exactly - probability (and by extension, statements such as "more likely") only make sense when we have a reference class of similar events for which we have iterated trials (frequentism), or iterated trials we can reasonably hypothesize about (hypothetical frequentism). Statistics and probability was not designed for one-off events with specious reference classes. The more speculative the reference class, the more we should doubt any claim of "more likely". In the case of the Boltzmann brain, given we have no evidence of this ever occurring thus far, we can at very least claim it does not have a clear reference class that we can identify empirically.