3 ms·
Pretty bad, but in a lot of cases you can use this to set up some kind of dataflow graph you use for long enough to amortize the overhead.
by kragen 5y ago
Pretty bad, but in a lot of cases you can use this to set up some kind of dataflow graph you use for long enough to amortize the overhead.
- zomglings 5y agoApache Beam works this way. Code sample: https://github.com/apache/beam/blob/master/examples/multi-language/python/addprefix.py#L56 https://github.com/apache/beam/blob/master/examples/multi-la...
- kragen 5y agoYeah, it's a very widely used technique to use slow scripting languages (with or without operator overloading) to set up some kind of graph describing a computation that takes a lot longer to run than the scripting-language statement takes to interpret. In addition to Beam, we could mention the Unix shell, Pandas, Numpy, TensorFlow, SQL, SQLAlchemy, APL, Lush, PIL, CSS, some Linux system calls like sendfile(), maybe Apache SPARK, etc.