Sigh. My comment gives a measurement of the memory usage of the model in question, Falcon 180B, (in mostly-q8 form, which is how someone would likely use it if

Sigh. My comment gives a measurement of the memory usage of the model in question, Falcon 180B, (in mostly-q8 form, which is how someone would likely use it if not q4). Perhaps it would have been more obvious that I actually ran the model and measured it if people hadn't flagged the self-reply with the model output.