4 ms·

Anyone know the largest possible image size > 512x512? I'm getting the following error when trying 1024x1024 with 64 GB RAM on M1 MAX: /opt/homebrew/Cellar/pyt

Anyone know the largest possible image size > 512x512? I'm getting the following error when trying 1024x1024 with 64 GB RAM on M1 MAX: /opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '

capableweb 4y ago
just 512x require something like 10GB VRAM on your GPU, 1024x would need even more. How much VRAM does the M1 Max GPU have? You're probably running out of memory.
- tgtweak 4y ago
  M architecture is unified memory - so the system memory is shared as GPU memory. There is likely a cap somewhere on how much can be used by a single app (or collectively by all apps).
enduser 4y ago
I have the same problem with anything over 512x512 on my M1 Ultra with 128GB. VRAM must be capped.
- moneycantbuy 4y ago
  Thanks for the Ultra data point. I'm able to get 768x896 to run, but the output image is still white noisy at 50 ddim steps, perhaps related to the phenomena of being trained/windowed on 512x512 images as sibling squeaky-clean described. RAM usage at various sizes: 512x512 14 GB, 768x768 26 GB, 768x896 32 GB
  - squeaky-clean 4y ago
    Those values seem really high compared to my setup, windows/nvidia/lstein repo. For me 512x512 uses 6.1GB. Random guess but I think your pipeline is running with full-precision floats (32bit), while by default the repo should be using autocast() which will try to use half-precision floats wherever possible. I know an optimizedSD repo exists and one of the steps they take is explicitly setting precision to half. (And other changes that reduce memory usage but decrease iteration speed). However I don't know how M1/Metal handles half-precision, hopefully it doesn't just cast them back to 32bit. Also white noisy images at 50 steps seems off to me. At 50 steps in a large image I definitely get a visible product. It's just often non-euclidean or very scattered bits of organization and chaos.
squeaky-clean 4y ago
Don't specifically know sorry, largest I can generate with my (windows pc) vram size is 512x1024. But just wanted to comment that SD is trained on 512x512 images and runs iterations in a 512x512 window. This means anything larger than 512x512 tends to confuse it. For example 1024x1024 will have 4 non-overlapping windows and many overlapping windows. So if your prompt is "a cat wearing sunglasses", you may get 4 separate cats as the 512x512 windows have no knowledge of each other, and each window is trying to fulfill the goal. Even more likely you'll get some sort of eldrtich horror 16-legged cat being as the windows shuffle around. Sometimes it just works perfectly somehow, but 90% of the time the non native resolution really screws it up. I'd suggest generating 512x512 images and using a different AI to upscale them in most cases. However it does lead to some amazing fantasy landscape art as you get weird terrains and mountains shoved up against eachother in fantastic/magical ways.
valley_guy_12 4y ago
Supposedly Stable Diffusion was trained on 512 x 512 images, so it's not clear that it will work well for larger images even if you had the RAM. To generate larger images, it is standard practice to generate 512 x 512 and then use a separate tool to upscale, and maybe a second separate tool to improve the face. The Windows versions of SD environments are starting to incorporate these additional tools, but the Apple Silicon versions of SD environments are lagging behind due to Pytorch metal limitations.... It'll hopefully sort itself out in the next few months.

capableweb 4y ago

just 512x require something like 10GB VRAM on your GPU, 1024x would need even more. How much VRAM does the M1 Max GPU have? You're probably running out of memory.

tgtweak 4y ago
M architecture is unified memory - so the system memory is shared as GPU memory. There is likely a cap somewhere on how much can be used by a single app (or collectively by all apps).

enduser 4y ago

I have the same problem with anything over 512x512 on my M1 Ultra with 128GB. VRAM must be capped.

moneycantbuy 4y ago
Thanks for the Ultra data point. I'm able to get 768x896 to run, but the output image is still white noisy at 50 ddim steps, perhaps related to the phenomena of being trained/windowed on 512x512 images as sibling squeaky-clean described. RAM usage at various sizes: 512x512 14 GB, 768x768 26 GB, 768x896 32 GB
- squeaky-clean 4y ago
  Those values seem really high compared to my setup, windows/nvidia/lstein repo. For me 512x512 uses 6.1GB. Random guess but I think your pipeline is running with full-precision floats (32bit), while by default the repo should be using autocast() which will try to use half-precision floats wherever possible. I know an optimizedSD repo exists and one of the steps they take is explicitly setting precision to half. (And other changes that reduce memory usage but decrease iteration speed). However I don't know how M1/Metal handles half-precision, hopefully it doesn't just cast them back to 32bit. Also white noisy images at 50 steps seems off to me. At 50 steps in a large image I definitely get a visible product. It's just often non-euclidean or very scattered bits of organization and chaos.

squeaky-clean 4y ago

Don't specifically know sorry, largest I can generate with my (windows pc) vram size is 512x1024. But just wanted to comment that SD is trained on 512x512 images and runs iterations in a 512x512 window. This means anything larger than 512x512 tends to confuse it. For example 1024x1024 will have 4 non-overlapping windows and many overlapping windows. So if your prompt is "a cat wearing sunglasses", you may get 4 separate cats as the 512x512 windows have no knowledge of each other, and each window is trying to fulfill the goal. Even more likely you'll get some sort of eldrtich horror 16-legged cat being as the windows shuffle around. Sometimes it just works perfectly somehow, but 90% of the time the non native resolution really screws it up. I'd suggest generating 512x512 images and using a different AI to upscale them in most cases. However it does lead to some amazing fantasy landscape art as you get weird terrains and mountains shoved up against eachother in fantastic/magical ways.

valley_guy_12 4y ago

Supposedly Stable Diffusion was trained on 512 x 512 images, so it's not clear that it will work well for larger images even if you had the RAM. To generate larger images, it is standard practice to generate 512 x 512 and then use a separate tool to upscale, and maybe a second separate tool to improve the face. The Windows versions of SD environments are starting to incorporate these additional tools, but the Apple Silicon versions of SD environments are lagging behind due to Pytorch metal limitations.... It'll hopefully sort itself out in the next few months.