The real problem with jails is that they (1) destroy your life outside and, (2) have you spend time surrounded by other criminals. So, they make it more likely

The real problem with jails is that they (1) destroy your life outside and, (2) have you spend time surrounded by other criminals. So, they make it more likely that you will become/remain a criminal once you come out, even after a short time. So, the middle ground doesn't really work well; we either have to try hard to not put someone in prison, or we basically commit to putting them in prison forever. Because we cannot really say the latter part out loud we release them occasionally and then put them back. The argument currently appears to be between these two extremes in isolation, and the real middle ground IMHO would be to do both things, to different people. But moderate punishment might be the worst of both worlds.