Some philosophical background. The purpose of the criminal justice system is to protect the innocent. Incarcerating the guilty is considered an effective way to accomplish that because a criminal (someone who's demonstrated a willingness to harm the innocent) can't commit crimes from inside a prison. This approach is just one of several alternatives, with their own advantages and drawbacks, and unique complications introduced by the notions of political and victimless crimes.
Punishments of various sorts, ranging from simple fines to torture, have been used in the past, with the expectation that would-be criminals would fear the punishment. Incarceration can be treated as a punishment in addition to being simply a means of separating criminals from society; inhumane conditions in prisons have this effect, whether implemented as a matter of universal policy or simple negligence. Execution might be considered the most extreme form of punishment.
Rehabilitation is considered ideal, as it's preferable in a peaceful society that criminals should give up their willingness to transgress rather than their ability. However, it seems that many criminals cannot be induced to change their habits by any means, let alone by means necessarily overseen by a bureaucracy. If nothing else it is demonstrably insufficient by itself.
If execution is the most extreme form of punishment, then banishment is the most extreme form of incarceration. Highly cost-effective and without the moral complications of having to get one's own hands dirty, it nevertheless has the obvious drawback of requiring somebody to willingly receive criminals, which is increasingly uncommon; it also means addressing your problems by making them somebody else's problem, which is wholly inadequate as a solution.
In small societies, it can be possible to declare a criminal outlawed; an individual could do anything to an outlaw and it would be acceptable. There's a certain poetry to the notion of depriving them of the protection of the law they transgressed, and allowing the victims to be the one to decide the punishment, but beyond the social implications of the practice, it only works in environments where identifying criminals is not a major difficulty, which is simply not the case in most urban environments.
It seems to me that the controversy specifically at issue in this thread is: To what extent should prison be a punishment itself rather than simply a removal from society, and what should be done in the case of prisons that routinely exceed that level?