fnl

21 ms·

151.
by fnl 9y ago
Nice write-up of the pain points in Java. Though, aggregating and mapping over (in Java or not) (possibly infinite) streams is not really the same as doing that over containers. In other words, Java8 doesn’t have the ability to flatMap et a
152.
by fnl 9y ago
To clarify on the cubic vs square runtime complexity confusion I caused (sorry!): low-rank (to k ranks) SVD of a n x m word embedding indeed scales with O(nmk), while the full SVD would be O(min(n^2m, nm^2)), i.e., squared and cubed runtime
153.
by fnl 9y ago
Finally, to tie this discussion off, two truly official references that explicitly address the issue of runtime complexity. In the best case, as determined by Halko et al., you low-rank k approximation of a n times m term-document matrix is
154.
by fnl 9y ago
And here is a reference to what I mean by non-standard methods. http://sysrun.haifa.il.ibm.com/hrl/bigml/files/Holmes.pdf
155.
by fnl 9y ago
Nope, it's not linear [1]. ADDENDUM: To which I should add, to avoid more discussions, that parallel methods on dense matrices exist that essentially use a prefix sum approach and double the work, but thereby decrease the absolute
156.
by fnl 9y ago
As to [1]: Yes, I was not honest in the sense that non-standard SVD implementations for generating your PMIs will scale with the square of |V|, not the cube. But as I will go on to show, that is not good enough to make count-based approache
157.
by fnl 9y ago
In fact, if the author had actually read Mikolov's w2v NIPS paper (nb, not the arXiv blog posts!) he would have found interesting, insightfull, and, particularly, sound arguments when neutral embeddings work better than count-based PMI
158.
by fnl 9y ago
Some minor space savings, maybe. But SVD runtime still scales with the cube of your vocabulary size. Good luck with SVD on a vocabulary from Wikipedia or Common Crawl. If anything, using traditional count-based approaches is good when yo
159.
by fnl 9y ago
SVD scales with the number of items cubed, w2v scales linearly. Typical real world vocabularies are 1-10M, not 10-100k. This article is FUD and best, and IMO, just plain BS.
160.
by fnl 9y ago
I'd be interested in understanding which design is closest to yours, however. Flink? Akka Streams? Another?
161.
by fnl 9y ago
This phone is as expensive as the latest iOS phones, at least ball-park wise. Yet, Google only supports its hardware for two years, while Apple does so for four, at least (still too little, but at least twice as good). I am using a "br
162.
by fnl 9y ago
That isn't taking into account how frequent relative to all such target objects an attack occurs. If a (much?) smaller percentage of houses get burglarized vs. number of phones broken into, then security requirements are not really com
163.
by fnl 9y ago
I would honestly say we really have no clue, and maybe add that as far as we can tell, activations play as much of a role in intelligence as a myriad of other factors. But more generally, I am just so tied of this "brain metaphor"
164.
by fnl 9y ago
Only in parts, not all instructions, and some functionality it did have was buggy. 4 and 5 are much more advanced/competitive on SIMD issues, it seems. Edit: Oh, sorry you meant that other guy's link to LLVM's vectorization t
165.
by fnl 9y ago
First of all, neurons don't have just one activation function. Each dendrite has. So, anything from dozens to thousands. Second, that definition doesn't cover the entire issue of multiple feedback loops. Third, this doesn't c
166.
by fnl 9y ago
Which is a long-winded way to show that deep learning isn't much more than a concatenation of glorified regression functions... :-) <ducks because="had to get that one out there"/> Edit: There we go with the downvo
167.
by fnl 9y ago
I'd mostly attribute that to the MKL and their ability to just have to deal with their own instructions. But that's just an "educated guess".
168.
by fnl 9y ago
The source of my confusion is that the last time I looked into LLVM's SIMD support was in the context of looking at Rust, a bit more than a year ago or so, and back then my conclusion was that neither (Rust or LLVM, then in version 3)
169.
by fnl 9y ago
Yes, indeed even LLVM 4 already improved its AVX-512 support. http://releases.llvm.org/4.0.0/docs/ReleaseNotes.html Really impressive how many new things came to LLVM this year!
170.
by fnl 9y ago
Impressive; I didn't take a look since the 3.x series, so I am totally stunned by the amount of "love" that LLVM has received lately (4, which was released just a few months ago, and up to the coming version 6, that is under
171.
by fnl 9y ago
Oh, cool, I see LLVM now even sports (basically all of) SSE4.2 and AVX-512. As always, that project amazes... :-)
172.
by fnl 9y ago
But LLVM doesn't support vectorizing, like AVX or SSE4, right? So I don't think that would be nearly as fast as fully (Intel-) CPU optimized code... EDIT: Let me hedge that a bit, to advanced AVX instructions, as LLVM can do sim
173.
by fnl 9y ago
The No Bullshit Guide to Linear Algebra https://gumroad.com/l/noBSLA#
174.
by fnl 9y ago
Well, if properly restricted to criminal fraud, it does make sense to me?
175.
by fnl 9y ago
Sadly, I'd fully agree to this. Things are possible now that were not 10 years ago. But mostly, only performance increased on things we could do 10 years ago, while hardly any new abilities came along. Machine translation, linguistic p
176.
by fnl 9y ago
That implies we have hit or at least know the exact capacity ceiling for, in particular, recurrent/dynamic networks (I do agree that with static networks it seems more clear cut). While it will not be general AI, I'd assume, thing
177.
by fnl 9y ago
Hmm; I guess that means a solution would be to personalize HN rankings on the similarity of voters' earlier votes on other articles with your own voting behavior (as a proxy for article preference...) :-)
178.
by fnl 9y ago
Now this solution looks really cool, as it doesn't even need more than changing the DNS on the router. Thanks!
179.
by fnl 9y ago
Thanks, that looks like an interesting blacklist!
180.
by fnl 9y ago
Thanks! Yeah, I guess HTTPS everywhere is the "enemy" in this case... But at least it gives us some basic protection.

More ›