The headline predictions aren't his only predictions. Silver also publishes projected margins of victory (with margins of error), and he does so for every stat

The headline predictions aren't his only predictions. Silver also publishes projected margins of victory (with margins of error), and he does so for every state and every senate race, not just the battleground ones. As such, it's possible to do a fairly substantial review of his model against actual results. Statistics are never perfect, but his predictions are far more transparent and verifiable than anything else on the market.