It was part of the colony of Quebec at the time. Since that colony did not declare independence, one of the declaring colonies or later the new United States,

It was part of the colony of Quebec at the time. Since that colony did not declare independence, one of the declaring colonies or later the new United States, would have had to annex and occupy that territory. That whole independence and self-government argument would start to sound a bit hollow. More practically, the region was pretty well-defended. Upper Canada was strongly loyalist and Lower Canada had come to at least tolerate British rule (which was pretty hands-off about local rule and customs, as far as 18th century empires go). And of course the British had excellent transport by controlling the waters, a major disadvantage for the Americans. Ultimately, see the War of 1812. It's the proposed scenario, more or less. Some have suggested the US fully intended to annex Canada, but there's no clear historical attestation by any leaders to that effect. In any case, at least officially, the Americans declared war following a string of British outrages, and preemptively invade Upper and Lower Canada. Sporadic fighting over two years results in no significant gains for either side and the whole front locked into a stalemale, until the war ended diplomatically.