An interesting possibility would be for, once backing store is full, if any new page should go to swap, the LRU page on the backing store should be "forgotten"

An interesting possibility would be for, once backing store is full, if any new page should go to swap, the LRU page on the backing store should be "forgotten" and freed; if a "forgotten" page was ever used again, such process would receive the SIGBUS signal. That would keep the system running happily even if processes leak arbitrary amounts of memory.