3 ms·

Lots of models are trained with dropout, which is kinda like bitrot at very high rates...

• clayhacks 2y ago
  But that’s during training, not during inference. Also it’s more structured in where the dropout is happening. I do think that points to them being somewhat resilient but we haven’t had LLMs exist long enough for a good test