fizx

124 ms·

1261.
by fizx 18y ago
I routinely run composite jobs of thousands of map-reduces. Imagine an iterative machine learning algorithm where each epoch is a map-reduce job. Imagine a meta-job that runs dozens of these. It's not to bad with Hadoop's job control sys
1262.
by fizx 18y ago
One of ... n/m, its not reddit.
1263.
by fizx 18y ago
Try local search and extremely recent content... Google dusts Yahoo!!. Other than that, they're pretty comparable.
1264.
by fizx 18y ago
XML is like Java. The language isn't friendly, but the platform has so many man-years invested into it that it may be the best tool for the job. Maybe my "taste" is to have a solid platform at the expense of some syntactic niceties. Any a
1265.
by fizx 18y ago
I think your best bet is an RDF tool like Jena or Sesame. 10^9 is pushing these engines though. There's an HBase-SPARQL lib, but that's gotta be two years away from stable. For real world problems, I tend to write custom code that's Java N
1266.
by fizx 18y ago
Sounds nice, although I do like controller inheritance, the ability to mix modules into your controller's scope, etc.
1267.
by fizx 18y ago
Also look at octave. Its OSS, and mostly m-file compatible.
1268.
by fizx 18y ago
According to wikipedia, electrolysis is currently around 65% efficient, with a theoretical cap somewhere around 90%. So this could help, but it's not going to be an order of magnitude improvement. Also, storing large quantities of (I assum
1269.
by fizx 18y ago
+1 for a ban on stories from tdwtf.
1270.
by fizx 18y ago
The next big search engine will answer queries you wouldn't bother giving google. Things like "what mid-priced chinese restaurant near downtown SF would my mother like?" Its not about design, or about patching holes in google's relevance.
1271.
by fizx 18y ago
not reading the social web? (I kid)
1272.
by fizx 18y ago
Generally, we use GP to make optimizations in exponentially sized search spaces. Lets say you're searching in a 60-dimensional space. If every dimension you add doubles your space, 10 years of Moore's law gets you a ~65-dimensional space.
1273.
by fizx 18y ago
gem install facets require "rubygems" require "facets/thread" %w[a b c].threaded_map{|str| puts str } It's multicore if you're using jruby.
1274.
by fizx 18y ago
They occasionally have good content... Maybe a way to aggressively bury TC content would be nice?
1275.
by fizx 18y ago
Hyperbole aside, v2.x has been the best RDF database on the market (or OSS, for that matter). Its a bit rough around the edges (you'll be talking to their very helpful support frequently), but the performance with large datasets is signifi
1276.
by fizx 19y ago
a. You wrote a crawler, not a search engine. b. Ferret will bite you in the ass. c. For a really good off the shelf crawler, look at Heretrix.