"Why does going to your parties make someone not incompetent?" I read this as Declan saying that he and Baker were sufficiently acquainted for him to form a ju

"Why does going to your parties make someone not incompetent?" I read this as Declan saying that he and Baker were sufficiently acquainted for him to form a judgement of Baker's competence, with "he'd show up at parties I held" being to clarify the degree of their acquaintance. He's not claiming that they two things are causally connected. And ignoring demagogues is not necessarily a good idea.