Note investor who's blowing up a deal over a board seat

We just signed a term sheet with a top-tier VC for our first priced round, and their only board requirement is a three-person board. So far, we’ve raised money via convertible notes from several investors—one of whom is ultra-wealthy, and has a rather large ego. He isn’t investing in this round, but now he’s demanding a board seat. He’s threatened to call his note or sue us (and the new VC) if we don’t give it to him. As a result, our lead investor is threatening to walk away—they absolutely don’t want him on the board or even a board observer.I spoke with other founders whom this guy has invested in, and he's done this to several of his portfolio companies, so I know he's not bluffing. Any advice on how to handle this?