Generally for usability I'd suggest to take a good look at OneNote, which in my eyes gets the "type and draw everywhere, with minimal friction" concept very rig

Generally for usability I'd suggest to take a good look at OneNote, which in my eyes gets the "type and draw everywhere, with minimal friction" concept very right. And while the underlying data model of such an application may necessitate a collection of objects being scattered on a canvas, that doesn't mean that the interaction model has to follow that directly. So indeed, just let me type anywhere. Another thing that adds friction in my eyes is that when editing text your first click changes the interface into text editing mode where the text neither stays at the position you've seen it before, nor can you actually interact with it with that first click. So when I see a word that I'd like to change and I click there, that will only spawn the edit window for that text object, instead of just placing a cursor there so I can directly edit it.