3 ms·
That's an interesting way to go about it. Any reason for using shell scripts to coordinate the flow instead of things like Cython and RPy? (I don't shell scrip
by vikp 13y ago
That's an interesting way to go about it. Any reason for using shell scripts to coordinate the flow instead of things like Cython and RPy? (I don't shell script a lot, so this may be a silly question)
These days, I mostly seem to be able to get away with using single languages for applied machine learning, like a Python webserver that runs background machine learning tasks, or an Android app in Java that connects to a webserver written in Python. But back when I was doing less applied side stuff, I was similar to you in that things were spread across R and Python.
- LukeShu 13y agoThe Unix Shell is basically an IPC language. It's sole purpose is to be able to combine programs, and allow them to communicate. Instead of trying to make a package in one language work with another, it is a lot simpler to use Shell to connect together programs in any language.
- chubot 13y agoIn my experience, RPy is annoying to get working, because you're dealing with Python versions and R versions together. It's brittle and usually unnecessary, as it's you can just serialize to CSV or JSON. If you can get away with using a single language that's good, and Python is probably the only one where that is possible (i.e. can write both prototype and production code, and data ingestion and machine learning). But I often have to use C++ because of the data size, and I think R's plotting is more convenient than anything in Python now. The shell scripts have their messiness and sharp edges, but they definitely save me many many lines of code. There is always some weird thing that needs to be integrated/automated and shell is almost always the right tool for that.