It seems you are mistaking the actual book (a story, an exposition of a subject, etc.) for the printed set of cover and pages, which is only a physical instance

It seems you are mistaking the actual book (a story, an exposition of a subject, etc.) for the printed set of cover and pages, which is only a physical instance of the book itself. Writing your notes in a separate medium is just as much engaging with the book as writing in the margins of a physical copy. It's a matter of taste really, but definitely not an insult to the author!