Most people I've talked to who are concerned about "cancel culture" are concerned about the dynamic where even a small group of people can get someone fired by

Most people I've talked to who are concerned about "cancel culture" are concerned about the dynamic where even a small group of people can get someone fired by being very loud about it. (I do agree with your implicit statement that it's not a very helpful term to use, though - at this point it's been too thoroughly repurposed for weird political battles.)