Hah. I did some experimenting with language model tasks here. I re-phrased the sentence as "John took the water bottle out of the backpack in order to make lig

Hah. I did some experimenting with language model tasks here. I re-phrased the sentence as "John took the water bottle out of the backpack in order to make lighter|less heavy|handy the" etc. The only way it would complete with something other than the water bottle was "John took the water bottle out of the backpack in order to lighten the " and it completed with 'load' The most amusing was a task to fill in the blank with "John took the water bottle out of the backpack so that [MASK] would be lighter" The model was 98% confident the blank should be 'it' facepalm Interestingly enough when I change the fill in the blank sentence to: "John took the water bottle out of the backpack so that the [MASK] would be lighter." The result was: "23.9% liquid 13.4% water 7.7% contents 6.5% weight" and the last two are pretty close. I ran these tests on https://demo.allennlp.org/ https://demo.allennlp.org/.