8 ms·
When someone says they want to work in Perl6/Raku, they likely have vastly different problems than someone using Erlang/Elixer. There is overlap of course (both
by 7thaccount 7y ago
When someone says they want to work in Perl6/Raku, they likely have vastly different problems than someone using Erlang/Elixer. There is overlap of course (both general purpose), but I can't imagine using Erlang for scripting, while Raku is first class here.
- weff_ 7y agoI'm trying to understand, what kind of scripting requires first-class concurrency that isn't fulfilled by say Python?
- Grinnz 7y agoI am not that familiar with Python, but the GIL has long prevented any real in-process concurrency. Perl has concurrency but it's complex, heavy, and poorly supported. Raku's approach to this is built to avoid all these problems (like Elixir).
- weff_ 7y agoI agree the GIL is a problem but it's only an issue for CPU-bound problems. Is there really an important amount of CPU-bound work that is written in a scripting language? If it's CPU-bound, wouldn't you want to use something lower level?
- karmakaze 7y agoIt's not just CPU bound problems, handling multiple overlapping i/o operations is more trouble than it ought to be.
- weff_ 7y agoCan you expand a bit on that? I'm not familiar with the issue you're describing.
- Grinnz 7y agoIndeed and as a Perl developer I make use of XS/external libraries, cooperative multitasking (event loops/promises), and forking to cover these use cases. It doesn't preclude wanting the additional option to take advantage of threads in a useful way, since they do exist.
- nuclear_eclipse 7y agoIf it's entirely CPU bound, you can use multiprocessing to negate most of the GIL issues, and transparently send inputs/outputs between the parent and child processes. If it's I/O bound, then AsyncIO is a great way to express asynchronous workflows. If it's a combination of both I/O and CPU bound workloads, there are ways to mix multiprocessing and AsyncIO to better saturate your cores without losing the simplicity or readability of Python: https://github.com/jreese/aiomultiprocess https://github.com/jreese/aiomultiprocess
- weff_ 7y agoVery true, I had not even thought about the multiprocessing package; it's sometimes not as convenient as multithreading but it'll get those other cores working.
- deleted 7y ago[deleted]
- ajoseps 7y agoI'm guessing because Python's concurrency relies on the Global Interpreter Lock. Although I think concurrent.futures might address that. Haven't worked with python concurrency libraries in a bit.
- weff_ 7y agoas I posted above: I agree the GIL is a problem but it's only an issue for CPU-bound problems. Is there really an important amount of CPU-bound work that is written in a scripting language? If it's CPU-bound, wouldn't you want to use something lower level?
- zaphirplane 7y agoThere is machine learning, which usually calls into numpy or other c extensions With a lot of the data preparation done in python
- rubyn00bie 7y agotldr; Using a scripting language that allows for native threads or has a strong concurrency model builtin to the core would be beneficial for any CPU bound scripting task... ----- Python's concurrency model is good for waiting on network or disk I/O because of its GIL (Global Interpreter Lock): https://realpython.com/python-gil/#the-impact-on-multi-threaded-python-programs https://realpython.com/python-gil/#the-impact-on-multi-threa... If your program is CPU bound the GIL will slow you down. I'm sure since the python ecosystem is quite vast there are ways around the GIL... but then you have to worry about every library you use not supporting "real" multi-threading, or (b)locking for no reason and shitting all over your efforts.
- bjornjaja 7y agos/tldr;/summary:/g
- weff_ 7y agoAs I've posted above, I'm a bit confused by CPU-bound work being processed in a scripting language. If you're planning on doing intense CPU-bound work, maybe use a lower-level language? I'm not saying abandon Python: you can extend Python with C or just use IPC to transfer data between a Python front-end and a computation back-end.
- rubyn00bie 7y agoI totally you feel you, I guess I thought your question was substantially more surface level than it was. My apologies. I'm personally with you. I also don't tend to think object boxing is really the performance bottleneck for most applications, and if/when it is, likely the other requirements should've already ruled out using one (a scripting language). It's like writing Nifs for Elixir, yeah sure you _can_, they have their purpose, but you could also just write another application to do that one thing and like you said, use IPC. So in summary, we agree with each other, here's to: the right tool for the job!
- weff_ 7y ago> the right tool for the job! Hear, hear!
- vaer-k 7y agoWhy should first-class concurrency needs be required to script in Elixir? This question seems to imply that Python is somehow a default language and special requirements must be needed to justify writing in something else. Elixir is general-use and pleasant to write scripts in so seems reasonable to me for someone to do so if that's their thing.
- Grinnz 7y agoWhich is a similar response to the comments asking "why Perl over Python?" I ask, why not (both)?
- b2gills 7y agoEspecially since you can use the Inline::Python module.
- weff_ 7y agoOvid2 said Perl6 has a good concurrency model lliamander replied that Elixir does too and it's worth a look To that, 7thaccount replied that Perl6 and Elixir fill different niches. So far, it seems Perl6 fills a niche that requires scripting and first-class concurrency. My question then is: what is this niche that requires very solid concurrency but also scripting. In other words, what does Perl6 have in terms of concurrency that Python does not (given they are both scripting languages)?
- dnautics 7y agoWell just from experience untangling async calls in python is a nightmare and sometimes hard to reason about. The red/blue function problem is real. Meanwhile dispatching concurrent long-running scripting tasks is basically trivial in elixir (Enum.map over Task.async, then Enum.map over Task.await)
- dragonwriter 7y agoFor concurrency I think Raku has slightly more developed async support than Python, but the bigger advantage, I think, is in parallelism where, aside from the CPython GIL limiting practical parallelism in the main implementation (which is a big deal), Python as a language lacks the parallel iterables produced by the hyper (parallel but constrained to return in order) and race (parallel and not constrained in order) methods on the Iterable role in Raku, or anything like Raku’s hyperoperators that are semantically concurrent and parallelizable at the discretion of the optimizer. (Come to think of it, while also parallel, all those are also high-level concurrency constructs that Python lacks equivalents to.) Python as a language can support parallelism via threads, and CPython as an implementation can via multiprocessing, but those are both very low level abstractions; Raku has higher level abstractions which allow much more succinct expression of parallel operations.
- Too 7y agoSounds like multiprocessing.Pool.imap and imap_unordered? When dealing with io you also have async/await equivalent iterators like asyncio.as_completed().
- b2gills 7y agoRaku has `await` but it doesn't need `async`. sub foo ( Int \n where 1..100 ) { # ___ start is a prefix operator that creates a Promise # V start { sleep n.rand; (1..n).pick } } my $result = foo(10); say $result.status; # Planned say await $result; # 6 (after a delay) # this is nearly identical to foo above sub bar ( Int \n where 1..100 ) { Promise.in( n.rand ).then({ (1..n).pick }) } (Note that `start` does not need a block unless you are grouping more than one statement as I have above.) There are combinators like `Promise.allof` and `Promise.anyof`. You usually don't need to use `Promise.allof`, because you can use `await` on a list. my @promises = Promise.in(1.rand) xx 10; my @results = await @promises; (Note that the left side of `xx` is _thunky_ so there are ten different Promises with ten different random wait times.) --- You should see the `react`/`supply` and `whenever` feature.
- minton 7y agoI guess it depends on what you're comfortable in? I myself have tons of Elixir scripts that I use to automate much of my life. It's replaced ruby as my go-to for one-off scripts.
- xyproto 7y agoI didn't know that Elixir had such a low-threshold for one-off programs. Have you tried Crystal? Both are on my list of programming languages to look into next, together with Elm, Reason and Zig.
- Willamin 7y agoCrystal is wonderful for one-off programs in my experience. Elixir had a long startup time for one-off scripts in my experience, but Elixir’s syntax is still one of my favorites for scripting.
- dnautics 7y agoIs a 100ms startup time really a big deal for something that is either going to be manually triggered or cronjobbed and expected to take on the order of minutes?
- Myrmornis 7y agoThose sorts of startup times discourage writing command-line tools (e.g. composable text processing things one might join with pipes).
- cutler 7y agoRuby scripting has occupied the sysadmin space for over a decade yet on my 2013 Macbook Pro the latest Ruby 2.6.5 takes around 250ms to start. That's a mere 1/4 of a second. Sure, Python, Perl and PHP have shorter startup times but surely this is a non-problem?
- lliamander 7y agoI can't imagine scripting in Erlang. Or rather, I've seen it done and it wouldn't be my first choice. Elixir, on the other hand, has many nice features for scripting, such as better string handling, process pipe-lining, and an excellent set of first-class generic data structures (beyond just map and list). It also happens to have Erlang's fantastic concurrency model.
- dnautics 7y agoAs a testament to this statement I wrote a Google maps API scraper to get a list of driving distances from a pair of csv tables - shortname + address and shortnames to and from, in about 1 hour, and the whole thing clocked in at 90 lines of code including about 40 lines of comments.
- 7thaccount 7y agoThat seems about right for a competent Python coder. It or Powershell or Linux command line tools would have very little code and be very easy to whip something up. Perl would be great here as well or honestly any dynamic language with great library support.
- cutler 7y agoElixir still has to borrow arrays from Erlang and it's a pretty ugly implementation at that so no - I would not agree that Elixir has an excellent set of first-class generic data structures.
- bitwalker 7y agoNeither Elixir or Erlang have real arrays, unless you want to count tuples. That said, I don’t see how that correlates to scripting, the data structures that are first class in Elixir are very ergonomic and certainly flexible enough to tackle pretty much any task you’d use a scripting language for.
- cutler 7y agoElixir/Erlang isn't a general purpose language. You pointed-out one reason why - scripting. Computation is another area where the language is deficient. I'd go so far as to say Elixir's failure to challenge mainstream languages for mindshare is down to to it occupying a niche - lightweight concurrency and distributed systems.
- sanderjd 7y agoTheir point was that elixir is a good language for scripting. I'm not sure if this is true but I don't see any reason it couldn't be. I don't think "distributed systems" are niche... I think every web application I've worked on fits that description, which is basically my whole career at all different kinds of companies.