This happens to me as well. I have noticed, however, that while I can't recall facts very well from books I've read they often do change my perspective on thing

This happens to me as well. I have noticed, however, that while I can't recall facts very well from books I've read they often do change my perspective on things and that is much longer lasting. For example, reading the book "The Selfish Gene" had a profound effect on how I thought about the world, even though I can't remember much at all of its contents other than it being about how evolution selects at the level of genes. With more technical books I find it is almost impossible for me to learn anything substantial from them unless I do the exercises, however, the exercises tend to take an enormous amount of time. So what I find myself doing is skimming most technical books and then then going more in depth on the few that seem relevant/interesting to me.