5 ms·

Seeing this kind of image generation limited to M series Macs just goes to show how far ahead Apple is in the notebook GPU game.

minimaxir 3y ago
It's possible to do on non-Apple Silicon Macs, just more annoying. There are a few generative AI implementations which use raw Metal but not sure what the most popular one is.
- deleted 3y ago
  [deleted]
andybak 3y ago
I've got a Windows laptop with an RTX 3080 in it that runs this model no problem. I don't have it to hand or else I'd post some timings. On my Desktop PC with a 4090 in I was getting speeds of 0.2 to 0.3 seconds for reasonably acceptable quality settings so I would expect 0.5s or so on the laptop. What Apple are ahead on is doing this on a fanless laptop that doesn't hit internal temperatures of triple digits.
- traceroute66 3y ago
  > What Apple are ahead on is doing this on a fanless laptop that doesn't hit internal temperatures of triple digits. You also forgot the bit where Apple are ahead of doing it on a laptop that can achieve it without needing to be tethered to a power socket to achieve the performance.
  - alex_duf 3y ago
    It's the same thing, power is heat with talking about chips
  - echelon 3y ago
    > You also forgot the bit where Apple are ahead of doing it on a laptop that can achieve it without needing to be tethered to a power socket to achieve the performance. Kind of sad that a huge anti-competitive, trillion dollar company is the one offering it. Especially given their stances around user freedom. I'd much rather innovation be distributed. The goal posts should be moved to a point everyone is pushing towards the next thing. Having Apple be the only game in town is unhealthy.
    - astrange 3y ago
      Would say that rather than one company being the only one who can do it, there is only one company that can't do it, and it's Intel.
      - echelon 3y ago
        Ouch. But true.
- brucethemoose2 3y ago
  > What Apple are ahead on is doing this on a fanless laptop that doesn't hit internal temperatures of triple digits. I think you could pull this off on a Asus G14 in an ultra power saver mode, with the fans off or running inaudibly. The cooling is so beefy they will actually work fanless if you throttle everything down and mostly keep the GPU asleep. The M chips could certainly sustain image generation better without a fan.
- errnoh 3y ago
  45it/s (0.1~s per image) on 7900XTX here, so it's still one magnitude faster on GPU with a lot higher power draw than the macs. Doing 10x slower with non-tethered is quite nice outcome.
- wasyl 3y ago
  At this point what Apple is ahead with is hype that M Macs are that fast, and the developers targeting them because things just work. Plenty of people should be able to run these models locally but there's close to no nice software that does that out of the box for Windows or Linux
  - astrange 3y ago
    It's because of the unified memory architecture. It's harder/different to do this on x86, because you have to have a large memory GPU and target that.
deleted 3y ago
[deleted]
orbital-decay 3y ago
Not sure why you think it's limited to M series Macs or has to do anything with Apple at all. It's just an instruction on how to run a diffusion model trained in a novel way on particular hardware.
amelius 3y ago
It mostly shows how shitty compatibility is between platforms that share the same roots.
liuliu 3y ago
The implementation is not even optimized for Macs. LCM is just very easy to be fast (batch size = 1 and only 2 to 8 steps, depending on what kind of headline you are trying to make).
filterfiber 3y ago
They also have a decent advantage for LLMs because of their memory bandwidth to system memory vs GPU's with limited VRAM limited over PCIE to the system memory.
novaomnidev 3y ago
Got this working on an intel Mac
- deleted 3y ago
  [deleted]
  - deleted 3y ago
    [deleted]

minimaxir 3y ago

It's possible to do on non-Apple Silicon Macs, just more annoying. There are a few generative AI implementations which use raw Metal but not sure what the most popular one is.

deleted 3y ago
[deleted]

andybak 3y ago

I've got a Windows laptop with an RTX 3080 in it that runs this model no problem. I don't have it to hand or else I'd post some timings. On my Desktop PC with a 4090 in I was getting speeds of 0.2 to 0.3 seconds for reasonably acceptable quality settings so I would expect 0.5s or so on the laptop. What Apple are ahead on is doing this on a fanless laptop that doesn't hit internal temperatures of triple digits.

traceroute66 3y ago
> What Apple are ahead on is doing this on a fanless laptop that doesn't hit internal temperatures of triple digits. You also forgot the bit where Apple are ahead of doing it on a laptop that can achieve it without needing to be tethered to a power socket to achieve the performance.
- alex_duf 3y ago
  It's the same thing, power is heat with talking about chips
- echelon 3y ago
  > You also forgot the bit where Apple are ahead of doing it on a laptop that can achieve it without needing to be tethered to a power socket to achieve the performance. Kind of sad that a huge anti-competitive, trillion dollar company is the one offering it. Especially given their stances around user freedom. I'd much rather innovation be distributed. The goal posts should be moved to a point everyone is pushing towards the next thing. Having Apple be the only game in town is unhealthy.
  - astrange 3y ago
    Would say that rather than one company being the only one who can do it, there is only one company that can't do it, and it's Intel.
    - echelon 3y ago
      Ouch. But true.
brucethemoose2 3y ago
> What Apple are ahead on is doing this on a fanless laptop that doesn't hit internal temperatures of triple digits. I think you could pull this off on a Asus G14 in an ultra power saver mode, with the fans off or running inaudibly. The cooling is so beefy they will actually work fanless if you throttle everything down and mostly keep the GPU asleep. The M chips could certainly sustain image generation better without a fan.
errnoh 3y ago
45it/s (0.1~s per image) on 7900XTX here, so it's still one magnitude faster on GPU with a lot higher power draw than the macs. Doing 10x slower with non-tethered is quite nice outcome.
wasyl 3y ago
At this point what Apple is ahead with is hype that M Macs are that fast, and the developers targeting them because things just work. Plenty of people should be able to run these models locally but there's close to no nice software that does that out of the box for Windows or Linux
- astrange 3y ago
  It's because of the unified memory architecture. It's harder/different to do this on x86, because you have to have a large memory GPU and target that.

deleted 3y ago

[deleted]

orbital-decay 3y ago

Not sure why you think it's limited to M series Macs or has to do anything with Apple at all. It's just an instruction on how to run a diffusion model trained in a novel way on particular hardware.

amelius 3y ago

It mostly shows how shitty compatibility is between platforms that share the same roots.

liuliu 3y ago

The implementation is not even optimized for Macs. LCM is just very easy to be fast (batch size = 1 and only 2 to 8 steps, depending on what kind of headline you are trying to make).

filterfiber 3y ago

They also have a decent advantage for LLMs because of their memory bandwidth to system memory vs GPU's with limited VRAM limited over PCIE to the system memory.

novaomnidev 3y ago

Got this working on an intel Mac

deleted 3y ago
[deleted]
- deleted 3y ago
  [deleted]