4 ms·
Booting the Analytics Application: Ruby, Avro, Pig, Voldemort, Sinatra
- rjurney 15y agoI, the writer, am trying to get more ruby people using Pig and hadoop. Feedback appreciated.
- diego 15y agoThis is very useful to me. I've been at LinkedIn for a few weeks and just yesterday I had some time to play with pig and our Hadoop cluster. Ruby is my preferred language for quick hacks, so I'll follow this series.
- eshvk 15y agoGood stuff here. As a ruby person, you probably also know about Wukong (https://github.com/mrflip/wukong https://github.com/mrflip/wukong ) which lets you do Hadoop stuff without looking at Pig. (I don't know about Pig version 0.9.1 but I have had pretty weird bugs on version 0.8)
- rjurney 15y agoIt looks neat, but it does look like you have to think in mapreduce, which is a big barrier of entry for most people.
- patio11 15y agoRuby person here. I keep wanting an excuse to try Pig. Is there any reason to do so if my entire data set fits comfortably in memory?
- rjurney 15y agoThere is if you want to try dataflow programming, and hope your dataset grows to exceed RAM :)
- ViktorasJucikas 15y agoDoes anyone have experience doing this in Python stack?
- rjurney 15y agoI do, and it is basically the same. Substitute the python avro library for the ruby one, the python voldemort library, and bottle.py for sinatra and you're there.
- rjurney 15y agoI am working on implementing this example in Python. There is a problem, however - python-snappy doesn't currently build on OS X. So I can't build the Python Avro bindings :(