Even if deterrence had a significant effect, surely it's totally immoral to punish one person for the potential future crimes of others? I don't understand how

Even if deterrence had a significant effect, surely it's totally immoral to punish one person for the potential future crimes of others? I don't understand how anyone can accept the idea of deterrence when deciding sentencing terms. Either the crime is bad enough to warrant a significant punishment, or it's not and therefore unnecessary to deter anyone else from committing the same crime.