Random sample query ("What shape should a kumquat be?") against a 7B model quantised to 4b running on an i7-9750H (so a good CPU, but also a good laptop CPU fro

Random sample query ("What shape should a kumquat be?") against a 7B model quantised to 4b running on an i7-9750H (so a good CPU, but also a good laptop CPU from 2019) gives: 148 tokens predicted, 159 ms per token, 6.27 tokens per second