From the HuggingFace page: > You will need at least 85-100GB of memory to swiftly run inference with Falcon-40B. So it may be possible with less (swap around

From the HuggingFace page: > You will need at least 85-100GB of memory to swiftly run inference with Falcon-40B. So it may be possible with less (swap around method), though not as efficiently and also slower.