ajtulloch

12 ms·

61.
by ajtulloch 8y ago
Imagine actually believing this.
62.
by ajtulloch 8y ago
But since everyone (implicitly or explicitly) specifies the precision of the coefficients (eg fp32, fp64, int8, etc), it’s a complete straw man you’re arguing against.
63.
by ajtulloch 8y ago
https://en.m.wikipedia.org/wiki/Capital_strike
64.
by ajtulloch 8y ago
One big problem with the conclusion is that intuitions from low dimensional spaces often don’t carry over to high dimensional spaces. e.g. common example is how the volume of the unit hypersphere intersected with hypercube ratio goes to zer
65.
by ajtulloch 9y ago
Related, and quite an interesting perspective: https://www.rockpapershotgun.com/2018/03/15/all-rise-and-no-...
66.
by ajtulloch 9y ago
They exist and are (nearly all) open-sourced for CPU & Metal (I wrote some of them).
67.
by ajtulloch 9y ago
Look at the first table in https://arxiv.org/pdf/1708.02002.pdf (the RetinaNet paper, code + models included in this release). There’s a substantial accuracy cost for YOLO vs other, similarly fast (within a factor of 2
68.
by ajtulloch 9y ago
FWIW we have some similar R-CNN models using this codebase running in the FB apps on iOS/Android with Caffe2 - with some neat bells and whistles (eg full execution of the compute graph on the device GPU where available via Metal/O
69.
by ajtulloch 9y ago
It's not very much. 100 GFLOPs is about 1-2 Haswell/Broadwell cores (32 FLOP/s * ~2 GHz AVX frequency). Even 2015 era mobile GPUs like the Apple A9 GPU (iPhone 6S) is higher throughput (~150-200GFLOP/s achievable). If yo
70.
by ajtulloch 9y ago
They don't for fp32 or int8 workloads until SVE IMO. High end Skylake as an example has 2 AVX-512-F ports (and not considering AVX-512-4FMAPS), which give you do 64 FLOP/cycle (512 / 32 * 2 * 2). Even 2 NEON units, assuming F
71.
by ajtulloch 9y ago
Not with randomized methods (which you should almost always use IMO).
72.
by ajtulloch 9y ago
A few of the Eigen core devs (Benoit Jacob, Benoit Steiner, ...) work at Google.
73.
by ajtulloch 9y ago
- https://github.com/xmartlabs/Bender/blob/master/Sources/Meta... - https://github.com/caffe2/caffe2/blob/64e52e474/caffe2/contr... :\
74.
by ajtulloch 10y ago
If anyone is interested, a much faster technique is using the randomized methods from https://arxiv.org/abs/0909.4061 - see e.g. http://scikit-learn.org/stable/modules/generated/sklearn.d
75.
by ajtulloch 10y ago
-ffast-math (well, really -funsafe-math-optimizations) is also essential for FP vectorization on ARM CPUs. Compare the generated code of a real-world ARM function with/without funsafe-math-optimizations - https://godbolt.org
76.
by ajtulloch 10y ago
https://github.com/Maratyszcza/NNPACK/blob/master/src/x86_64... (from https://github.com/Maratyszcza/NNPACK ) is also a pretty reasonable AVX2 GEMM implementation written in a P
77.
by ajtulloch 10y ago
"It wraps cuDNN, which is faster than Tensorflow." TF uses CuDNN, so this is a nonsensical statement.
78.
by ajtulloch 11y ago
This is a really great piece of software, awesome stuff Marat! For those folks running convolutional neural networks on CPUs in production, it is a substantial enabler.
79.
by ajtulloch 11y ago
PeachPy is great, thank you @Marat for this project.
80.
by ajtulloch 11y ago
Congrats! It's a brilliant team, looking forward to great things.
81.
by ajtulloch 11y ago
I would recommend instead Facebook's folly::gen library, which provides these functional combinators in a high-performance fashion, by expressing them as streaming combinators. You get all the map/filter/take/drop/r
82.
by ajtulloch 11y ago
That's actually not the case - that's one effect, sure, but when you compared the compiled trees to the flattened trees (the strategy just described), the compiled trees are substantially faster across a range of parameters. See
83.
by ajtulloch 11y ago
The article mentions that "The main drawback of Random Forests is the model size. You could easily end up with a forest that takes hundreds of megabytes of memory and is slow to evaluate." A cool trick for speeding up trained rand
84.
by ajtulloch 11y ago
https://github.com/ajtulloch/dnngraph is an application of these ideas - it's a Haskell DSL for building DNNs for later execution by Caffe/Torch.
85.
by ajtulloch 11y ago
The Haskell monad typeclass _is_ (modulo technical details) a category-theoretic monad on the category `Hask`. As Edward Kmett says in more detail in [1] - the point of calling a monad a monad is not to confuse the reader, but to unlock 70
86.
by ajtulloch 11y ago
This is a false choice - you can serialize/deserialize Thrift objects to/from JSON. I think the choice here is between having a clearly-defined /schema/ or not.
87.
by ajtulloch 11y ago
Nit: gradient boosting isn't an 'augmentation' of random forests - if anything, it's the other way round. AdaBoost is from 1995, the GBM paper was 1999, and Breiman's random forest paper in 2001 explicitly couches i
88.
by ajtulloch 11y ago
The algorithms haven't (fundamentally) changed.
89.
Neil Lawrence: Baidu and the Imagenet Challenge(inverseprobability.com)
1 points by ajtulloch 11y ago|0 comments
90.
by ajtulloch 11y ago
Awesome - and very cool to see ITorch getting some love. Thanks very much @willwhitney.

More ›