Not a classical bandit because the arms have a finite lifetime (the arm is no longer active once you read the whole book) and you know this lifetime. Definitely

Not a classical bandit because the arms have a finite lifetime (the arm is no longer active once you read the whole book) and you know this lifetime. Definitely can be viewed as some form of RL problem. For me the challenge is how you define rewards and hence what information reading part of a book gives you about the total reward for completing the book. Maybe assume some smoothness criteria, so that the reward for reading part n of a book is highly informative of reading part n+1, but less informative of part n+2, and so on. More formally, each book gives some total reward on completion and that reward is distributed according to some stochastic process. Greedy strategies probably work reasonably well.