8 ms·
Intel Extension for TensorFlow
- minimaxir 4y agoFor context, in other Python ML packages like scikit-learn running the Intel extension on a compatable CPU can result in a massive performance increase. Although since TensorFlow models should be trained on a GPU unlike sklearn, that's less useful, and there are better tools for CPU inference. (e.g. SavedModels or ONNX)
- make3 4y agoyes, but Intel now has mainstream GPUs that are reasonably (~ RTX 3060) performant, which is what I assume this is for
- Roark66 4y agoAre you talking about Intel Arc? I'm yet to see any ML relevant benchmarks on Intel Arc. If you are aware of any please let me know.
- throwaway1851 4y agoSame here. At 16GB of VRAM and only $349, it could fill a really nice slot for DL.
- tttttt5ts 4y agoYou won't for some time as ARC is only supported on Linux kernel 6+. And honestly, their hardware is pretty bad for ML given you can pick up a used 3090 for $600. I just don't think there will be an ML push for ARC with current gen.
- learndeeply 4y ago> And honestly, their hardware is pretty bad for ML given you can pick up a used 3090 for $600. How do you know it's bad? Do you have benchmarks? Just checked, used 3090s are going for $700-900 on eBay.
- doublepg23 4y agohardwareswap.reddit.com is the best way to buy used computer hardware.
- ShamelessC 4y agoIt won’t be able to make use of CUDA, which is more than enough reason.
- fulafel 4y agoI think a recent Linux version requirement shouldn't hamper benchmarking if you're a somewhat technical user. You can get the latest Linux pacakges for eg Ubuntu here: https://wiki.ubuntu.com/Kernel/MainlineBuilds https://wiki.ubuntu.com/Kernel/MainlineBuilds - probably other distributions have similar easy ways since it's needed so often in bug reporting for users to tell if a hw support bug is in a distro specific kernel change or not.
- gnufx 4y agoIt presumably has to support Aurora urgently: https://www.alcf.anl.gov/aurora https://www.alcf.anl.gov/aurora
- benreesman 4y agoI don’t know much about Intel’s toolchain: people say good stuff about TBB and things like that. Historically doing inference on Intel gear was mostly about whether or not to target AVX2 or AVX512 when building Eigen or whatever. A few years ago the net win was AVX2 because the de-clock and re-clock just killed you. What’s the game these days? Long term I doubt inference will be done on x86, but I think a lot of people still do it.
- dweekly 4y agoFor the curious, Apple has an analogous Tensorflow Metal plugin to allow for Apple Silicon (and AMD GPU) acceleration using the same plugin architecture. https://developer.apple.com/metal/tensorflow-plugin/ https://developer.apple.com/metal/tensorflow-plugin/
- muxamilian 4y agoWhich is basically unusably buggy. For example, tf.sort only sorts up to 16 values and overwrites the rest with -0. Apparently not fixed for over one year: https://developer.apple.com/forums/thread/689299 https://developer.apple.com/forums/thread/689299 Also, tf.random always returns the same random numbers: https://developer.apple.com/forums/thread/69705 https://developer.apple.com/forums/thread/69705 Although I guess these bugs are not the fault of Tensorflow's plugin architecture but rather Apple's implementation.
- wongarsu 4y agoI believe the second link should be https://developer.apple.com/forums/thread/697057 https://developer.apple.com/forums/thread/697057
- londons_explore 4y agoIt is sorta the fault of tensorflow... Tensorflow should have had a test suite for plugins that verifies that all operations at least lead to correct results no matter which backend is in use.
- Anunayj 4y ago> Also, tf.random always returns the same random numbers: reminds of this particular xkcd [1], How someone fucks up that bad is beyond me. 1. https://xkcd.com/221/ https://xkcd.com/221/
- _aavaa_ 4y agoDisappointed by the lack of benchmark comparison in the README
- Roark66 4y agoIntel did a great thing for people interested in ML and numeric research by making their MKL library and compiler free and cross platform compatible. Even today on my AMD zen3 Ryzen machine intel's MKL linked numpy and pytorch are in some operations 10* (yes that is really ten times) faster in comparison with the next best alternative (openBlas etc). I was shocked to discover how much of a difference MKL makes for cpu workloads. This is mostly because it makes use of AVX2 cpu extensions which make certain matrix operations a lot faster.
- ysleepy 4y agoThey crippled the performance on non-Intel CPUs on purpose until recently. Intel's anti competitive behaviour follows them throughout their history.
- Roark66 4y agoYes, this is true, but even back then (during the time they crippled it) it was possible to pretend to run on Intel hardware. I haven't done this, but I read it was possible.
- ysleepy 4y agoI just felt your first sentence, without a qualifier, gave too much credit looking at the context.
- pantalaimon 4y agoThey crippled their own consumer CPUs by retroactively disabling AVX512
- chrchang523 4y agoThat was my previous experience, but have you tried linking to AMD AOCL recently? I would not expect the performance gap between Intel MKL and AMD AOCL to still be as large as you describe on a Zen 3.
- 4y ago
- echelon 4y agoI love the hardware Nvidia makes, but for the good of our field and for progress, we need our ML stacks to work with all other vendors. CUDA is the wrong platform to build our future on.
- thrdbndndn 4y agoSorry for my ignorance, why does TensorFlow in this repo readme have an asterisk? (It writes as TensorFlow*).
- varenc 4y agoI too am wondering that. Doesn’t seem to be any corresponding asterisk explanation.
- londons_explore 4y agoI assume they meant that say "* Tensorflow is a trademark owned by a third party and has no affiliation with Intel"
- pizza 4y agoDoes this work with jax jit?
- learndeeply 4y agohttps://github.com/intel/intel-extension-for-pytorch https://github.com/intel/intel-extension-for-pytorch PyTorch version. Looks like it's been supported a lot longer than TensorFlow, the repo dates back over a year.
- DeathArrow 4y agoDo Nvidia and AMD have something like this?