I've heard Bard was previously 3B parameters but I could never find a good source for it. I honestly think the end game here is running on consumer devices, 7B

I've heard Bard was previously 3B parameters but I could never find a good source for it. I honestly think the end game here is running on consumer devices, 7B and under need ~4GB of ram to actually run which is likely the max reasonable requirement for consumer devices. That said medium end hardware can do 15B, anything larger then this is currently something only "enthusiasts" can run. If it is small enough to run on consumer devices then they don't have to pay for the inference compute at that point, and presumably the latency will be improved for consumers.