One reason airlines rely on overbooking is that US Department of Transportation regulations encourage it, as pointed out in this HBR online piece. The regs allow bumped passengers to be paid 200-400% of the price of their ticket (one-way, I might add) with an overall cap of $1350. So a passenger who bought a heavily discounted ticket might only receive $400-500 in compensation for being bumped, well below what a true volunteer might demand.
Airlines could manage passenger loads in different ways, such as penalizing no-shows who do not contact the airline in advance and are not on a connecting flight. If airlines insist on overbooking, then the most efficient (in the economics sense of the word) compensation mechanism would be an auction where passengers bid for the right to be bumped. In the case of the infamous Chicago to Louisville flight two weeks ago, the bid price for being bumped would rise until there were four true volunteers. That would no doubt be quite a bit more than United actually spent, but I bet they sure wished in retrospect that they had paid those four passengers enough to get them to exit the plane without assistance.