This is useful information, but it would even more useful if it was mention which CPUs were used - knowing how long something took with 12 CPUs without knowing

This is useful information, but it would even more useful if it was mention which CPUs were used - knowing how long something took with 12 CPUs without knowing their model, cannot be used to guess how long it would take with any other CPU(s). It is also not clear if cores were confused with CPUs - 12 CPUs with many cores each or a single 12-core CPU is a very big difference. By the way, according to this it is possible to minimize VRAM and RAM usage significantly: https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-vram https://github.com/arrmansa/Basic-UI-for-GPT-J-6B-with-low-v... ("a repository to run GPT-J-6B on low vram systems by using both ram, vram and pinned memory"): I did not tried this myself yet, but here are their results on two different systems: System 1: 16 gb ddr4 ram . 1070 8gb gpu. 23 blocks on ram (ram_blocks = 23) out of which 18 are on shared/pinned memory (max_shared_ram_blocks = 18). timing - single run of the model(inputs) takes 6.5 seconds. 35 seconds to generate 25 tokens at 2000 context. (1.4 seconds/token) System 2: 16 gb ddr4 ram . 1060 6gb gpu. 26 blocks on ram (ram_blocks = 26) out of which 18 are on shared/pinned memory (max_shared_ram_blocks = 18). timing - 40 seconds to generate 25 tokens at 2000 context. (1.6 seconds/token)