5 ms·

Both tif and wav are lossless so this doesn’t really surprise me. It is however nice to see Apple researchers publishing, you don’t see that often at the foref

Both tif and wav are lossless so this doesn’t really surprise me. It is however nice to see Apple researchers publishing, you don’t see that often at the forefront of transformer or generative model research. I hope it means that everyone here on HN is right and they are seriously looking into running the generative models locally on their own silicon

brucethemoose2 3y ago
Oh, and also, lossless does not necessarily mean readable. Hence they note zlib compressed PNGs are untenable.
brucethemoose2 3y ago
I mean, they published their own metal stable diffusion implementation on GitHub very quickly last year. I dont think this was in doubt
- sroussey 3y ago
  And then… nothing after?
  - haswell 3y ago
    I suspect they prefer to use their standard marketing avenues to better control the narrative. I'm sure we'll be hearing a lot more next week, and in Apple fashion it'll all be grounded in customer facing use cases and associated developer frameworks.
- refulgentis 3y ago
  That's not AI research
  - brucethemoose2 3y ago
    Its desperately needed though. If AMD or Intel started going around github and writing in acceleration for random popular ML repos (and not just one off abandoned demos), that would change so much.
    - refulgentis 3y ago
      These models run on CUDA. Making a version that ran on Metal filled a feature gap on Apple's proprietary GPUs. CPUs aren't involved, the last thing, for example, Stable Diffusion needs is help from AMD and Intel.
      - zeusk 3y ago
        CPUs are actually involved, especially with SIMD like AVX and VNNI instructions. It all depends on the scale of your compute. https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/overview.html https://www.intel.com/content/www/us/en/developer/tools/open...
        refulgentis 3y ago
        Linking to an Intel SDK with “deep learning” in it does prove it’s possible to do matmuls on CPUs. But this a nitpicked to death thread: it’s good to see Apple releasing research, the Metal stuff was to make it run on their GPU, and there simply aren’t any relevant uses of CPUs. Academically, yes, CPUs have SIMD so they can implement the same algorithm that drives GPU perf, but without some massive breakthrough the one that can do more matmuls wins, and nVidia wins massively, in theory and practice.
        zeusk 3y ago
        I'm not just linking it as a proof, I now work at Intel and collaborate with Microsoft on bringing new platform technologies to life. I'm currently involved in VNNI and hybrid core scheduling KPIs. OpenVINO allows developers to target execution engines on client based on their needs and CPU is a well supported engine for lightweight inferencing that is latency sensitive. Matmuls (FMAC especially) did evolve as a SIMD workload on CPUs but it's still forcing vector execution on cores designed around scalar workloads - you just have to be mindful of tradeoffs between executing a bunch of AVX/VNNI instructions for inferencing vs setting up a GPU/VPU/IPU context and performing that inference asynchronously. AFAIK, ARM is also interested in use of NEON/SIMD co-processors as low latency inference engines. Microsoft and Intel have a lot of interesting stuff in the works for AI compute (as I'm sure so do Apple, Nvidia and to a lesser extend AMD).
samwillis 3y ago
Drip by drip gearing up for "one more thing" on Monday...
schappim 3y ago
It has been well documented that Apple has permitted its AI teams to publish papers in a manner that appears to contradict Apple's culture of secrecy. This is explicitly allowed, as previously this was a hindrance to recruiting talent.
nighthawk454 3y ago
They publish research all the time: https://machinelearning.apple.com/research https://machinelearning.apple.com/research Including research about fitting models locally onto Apple Silicon, for example: https://machinelearning.apple.com/research/neural-engine-transformers https://machinelearning.apple.com/research/neural-engine-tra...
- solarkraft 3y ago
  I had no idea. I've never seen one mentioned in the wild.

brucethemoose2 3y ago

Oh, and also, lossless does not necessarily mean readable. Hence they note zlib compressed PNGs are untenable.

brucethemoose2 3y ago

I mean, they published their own metal stable diffusion implementation on GitHub very quickly last year. I dont think this was in doubt

sroussey 3y ago
And then… nothing after?
- haswell 3y ago
  I suspect they prefer to use their standard marketing avenues to better control the narrative. I'm sure we'll be hearing a lot more next week, and in Apple fashion it'll all be grounded in customer facing use cases and associated developer frameworks.
refulgentis 3y ago
That's not AI research
- brucethemoose2 3y ago
  Its desperately needed though. If AMD or Intel started going around github and writing in acceleration for random popular ML repos (and not just one off abandoned demos), that would change so much.
  - refulgentis 3y ago
    These models run on CUDA. Making a version that ran on Metal filled a feature gap on Apple's proprietary GPUs. CPUs aren't involved, the last thing, for example, Stable Diffusion needs is help from AMD and Intel.
    - zeusk 3y ago
      CPUs are actually involved, especially with SIMD like AVX and VNNI instructions. It all depends on the scale of your compute. https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/overview.html https://www.intel.com/content/www/us/en/developer/tools/open...
      - refulgentis 3y ago
        Linking to an Intel SDK with “deep learning” in it does prove it’s possible to do matmuls on CPUs. But this a nitpicked to death thread: it’s good to see Apple releasing research, the Metal stuff was to make it run on their GPU, and there simply aren’t any relevant uses of CPUs. Academically, yes, CPUs have SIMD so they can implement the same algorithm that drives GPU perf, but without some massive breakthrough the one that can do more matmuls wins, and nVidia wins massively, in theory and practice.
        zeusk 3y ago
        I'm not just linking it as a proof, I now work at Intel and collaborate with Microsoft on bringing new platform technologies to life. I'm currently involved in VNNI and hybrid core scheduling KPIs. OpenVINO allows developers to target execution engines on client based on their needs and CPU is a well supported engine for lightweight inferencing that is latency sensitive. Matmuls (FMAC especially) did evolve as a SIMD workload on CPUs but it's still forcing vector execution on cores designed around scalar workloads - you just have to be mindful of tradeoffs between executing a bunch of AVX/VNNI instructions for inferencing vs setting up a GPU/VPU/IPU context and performing that inference asynchronously. AFAIK, ARM is also interested in use of NEON/SIMD co-processors as low latency inference engines. Microsoft and Intel have a lot of interesting stuff in the works for AI compute (as I'm sure so do Apple, Nvidia and to a lesser extend AMD).

samwillis 3y ago

Drip by drip gearing up for "one more thing" on Monday...

schappim 3y ago

It has been well documented that Apple has permitted its AI teams to publish papers in a manner that appears to contradict Apple's culture of secrecy. This is explicitly allowed, as previously this was a hindrance to recruiting talent.

nighthawk454 3y ago

They publish research all the time: https://machinelearning.apple.com/research https://machinelearning.apple.com/research Including research about fitting models locally onto Apple Silicon, for example: https://machinelearning.apple.com/research/neural-engine-transformers https://machinelearning.apple.com/research/neural-engine-tra...

solarkraft 3y ago
I had no idea. I've never seen one mentioned in the wild.