5 ms·

Apple has done a wonderful job making CoreML smoothly integrated with iOS, iWatchOS, iPadOS, and macOS development.

brookst 4y ago
I get the value of the common APIs, but as a developer how do you deal with the wide range of performance in different form factors and product generations? Is there some way to gracefully adapt the same models to a specific device’s capabilities?
- londons_explore 4y ago
  There are a bunch of easy ways to scale neural nets. Quantization and distillation being the main approaches (or some combination of the two). Both typically require more training time, but not much more human-effort. You can normally expect to get way more than half the 'outcome' from a neural net with half the ram/compute/time/power budget. So neural nets scale 'down' pretty well.
ur-whale 4y ago
> Apple has done a wonderful job making CoreML Apple has done a wonderful job of further locking their user into the golden cage they call a platform.
- gjsman-1000 4y ago
  Every platform is a golden cage in some respect. Ask any business who is stuck on ancient Win32 and even DOS applications, source code long gone. (Looking at you my local McDonalds, Menards, Tractor Supply)…
- gcr 4y ago
  I think you're right and you're wrong, it's a bit more complicated. ML is one of the few applications that benefit from platform-specific optimizations, so if you need every ounce of performance, you have your choice of which walled garden you want to tether your application to. The "lock-in" comes from the specific capabilities of your special-purpose hardware, and for serious applications, you're already thinking hard about whether to design your entire implementation around Apple, NVidia, Google/TPU, or even Android devices. For big models, platform-specific needs influence every aspect of model design, including data/model sharding, quantization, training loops... For non-scientific applications, it's usual practice to train your model in platform-agnostic ways using PyTorch or Tensorflow or whatever and then deploy it to devices in platform-specific ways, whether that's XLA, CoreML, Edge TPU, Android NNAPI, TensorflowJS, or hell, custom-written GLSL shaders or whatever. We're just starting to see cross-platform frameworks that abstract model inference: TFLite, PyTorch Mobile, ONNX. To their credit, CoreML can act as a backend for any of these, so you don't even need to worry about your platform.
- bee_rider 4y ago
  The worst thing is, their users don’t even seem to be totally happy with the state of affairs! It’s like they don’t even realize their preferences are wrong. :(
  - bee_rider 4y ago
    This was intended to be obvious sarcasm, but I somehow accidentally added “don’t” which… really just makes it confusing. Oops, haha.

I get the value of the common APIs, but as a developer how do you deal with the wide range of performance in different form factors and product generations? Is there some way to gracefully adapt the same models to a specific device’s capabilities?

londons_explore 4y ago
There are a bunch of easy ways to scale neural nets. Quantization and distillation being the main approaches (or some combination of the two). Both typically require more training time, but not much more human-effort. You can normally expect to get way more than half the 'outcome' from a neural net with half the ram/compute/time/power budget. So neural nets scale 'down' pretty well.

ur-whale 4y ago

> Apple has done a wonderful job making CoreML Apple has done a wonderful job of further locking their user into the golden cage they call a platform.

gjsman-1000 4y ago
Every platform is a golden cage in some respect. Ask any business who is stuck on ancient Win32 and even DOS applications, source code long gone. (Looking at you my local McDonalds, Menards, Tractor Supply)…
gcr 4y ago
I think you're right and you're wrong, it's a bit more complicated. ML is one of the few applications that benefit from platform-specific optimizations, so if you need every ounce of performance, you have your choice of which walled garden you want to tether your application to. The "lock-in" comes from the specific capabilities of your special-purpose hardware, and for serious applications, you're already thinking hard about whether to design your entire implementation around Apple, NVidia, Google/TPU, or even Android devices. For big models, platform-specific needs influence every aspect of model design, including data/model sharding, quantization, training loops... For non-scientific applications, it's usual practice to train your model in platform-agnostic ways using PyTorch or Tensorflow or whatever and then deploy it to devices in platform-specific ways, whether that's XLA, CoreML, Edge TPU, Android NNAPI, TensorflowJS, or hell, custom-written GLSL shaders or whatever. We're just starting to see cross-platform frameworks that abstract model inference: TFLite, PyTorch Mobile, ONNX. To their credit, CoreML can act as a backend for any of these, so you don't even need to worry about your platform.
bee_rider 4y ago
The worst thing is, their users don’t even seem to be totally happy with the state of affairs! It’s like they don’t even realize their preferences are wrong. :(
- bee_rider 4y ago
  This was intended to be obvious sarcasm, but I somehow accidentally added “don’t” which… really just makes it confusing. Oops, haha.