Respecting robots.txt is a convention not enforced by anything so yes the bot is certainly free to ignore it. But I’m not sure I understand your distinction. A

Respecting robots.txt is a convention not enforced by anything so yes the bot is certainly free to ignore it. But I’m not sure I understand your distinction. A scraper is a crawler regardless of whether it is “custom”or an off the shelf solution. The author also said the bot identifed itself as a crawler > Mozilla/5.0 (compatible; crawler)