11 ms·
Show HN: Chili. Rust port of Spice, a low-overhead parallelization library
- n8henrie 2y agoNot related to the content, but as a port of "spice" -- being from New Mexico, we would usually refer to a spicy pepper as (eg green) "chile," or a chunky dish with beans and meat as (eg Texas) "chili." Looks like the AP Stylebook agrees, though Webster's makes it seem less clear, so you can probably get away with it either way.
- skipkey 2y agoIf it has beans it isn’t Texas chili…
- andyferris 2y agoFurther abroad (Australia), Chile is a country and chili is the hot pepper, so I suspect the (English) language here is highly regional. Eg I’ve even lived in North American and never come across that spelling for the pepper.
- anamexis 2y agoThe "chile" spelling is from Spanish, particularly Mexican and Central American Spanish. The word is derived from Nahuatl (Aztec). That said, I'm in northern US and I would spell the pepper "chili"
- ithkuil 2y agoAnd the name of the country Chile has no relation to the nahuatl word for pepper but comes from a word in a south American language (likely Aymara)
- darby_nine 2y agoOddly I associate `chile` with the pepper and `chili` with the stew. Curiously chili pepper seems to be the common spelling around where I live so I have no clue where I picked this up.
- saghm 2y agoYeah, I've lived in the northeast US my whole life and have never seen "Chile" used for anything but the country before; the pepper and the food are both spelled "chili" in my experience. That being said, pronouncing the country's name as a homophone of "chili" is a bit of an anglicism too, since from what I remember of Spanish in high school, the more accurate pronunciation would be something like "chee-lay" rather than "chill-ee". I wouldn't be shocked if both the spelling "chili" and the pronunciation of it had roots in the name of the country and in English usage drifted over time (sort of like the pronunciation of the name of the city "Amarillo" in Texas).
- deleted 2y ago[deleted]
- aidenn0 2y agoWikipedia says "Chili peppers, also spelled chile or chilli"
- hombre_fatal 2y agohttps://en.wikipedia.org/wiki/Chili_pepper https://en.wikipedia.org/wiki/Chili_pepper
- beepbooptheory 2y agoWas just arguing with a friend the other day about distinction between a "chili" and what New Mexican's have in "green chile stew." I have had what by at least strong family-resemblance is a chili but had essentially an adovada meat base, but would never call something chili that had potatoes or any visible vegetables. But in NM, even frito pies get the (superior) "green chile stew" topping.
- thayne 2y ago> or a chunky dish with beans and meat as (eg Texas) "chili." My understanding was that the name of the dish was short for "chili con carne" i.e. chili (the pepper) with meat
- adityaathalyo 2y ago[flagged]
- dangoodmanUT 2y agoI love this, the overhead of rayon always made it hard to gauge whether to use it for small operations (or could be small operations). This seems to solve that, at least for relatively short workloads
- bee_rider 2y agoOut of curiosity, how do Spice/Chili/Rayon compare against the boring answer, OpenMP? I guess they must have some really neat capabilities.
- nestorD 2y agoRayon is known to have a higher overhead for the simple, and arguably most common case in numerical codes, static block size scheduling. Things get better, and you start to benefit from rayon, as work gets less balanced.
- thayne 2y agoWell, for one thing OpenMP is a c/c++ library, and chili and rayon are rust libraries. It's probably possible to use openmp from rust, but the interface probably isn't as nice, you would need some kind of translation layer from rust closures and iterators to the OpenMP API
- bee_rider 2y agoPlus Fortran! There’d definitely be some work translating Rust concepts over. It might be impossible/not worth it. But maybe it would be nice to have Rust, C, and Fortran all talking to the same runtime?
- marmaduke 2y agoThat’s what I was wondering. Iiuc openmp uses work stealing and the spice readme talks about the inefficiency of that: https://github.com/judofyr/spice?tab=readme-ov-file#work-stealing-and-its-inefficiencies https://github.com/judofyr/spice?tab=readme-ov-file#work-ste... so I’d be interested in seeing a benchmark.
- zbowling 2y ago> a "low-overhead" parallelization library Who is out here building high-overhead parallelization libraries?
- internetter 2y agoRelatively, Rayon is higher overhead
- zamalek 2y agoI have almost certainly built high overhead parallelization primitives during my junior years.
- andrepd 2y agoYeah, why are those bastards writing slow code? They should just write fast code instead
- therein 2y agoYeah and people should make sure not to commit bugs too. I look at all these commits with bugs and I'm like why are you committing bugs.
- bravetraveler 2y agoStop publishing software, you don't refine your meal constantly! Enough is enough!
- bionsystem 2y agoIn the early 2000s we were still in high school and a friend of mine told me that it's pointless to chose a carrier in SWE because eventually all the software will be written and there would be nothing to do. Well we both ended up in CS but it's really a funny take nowadays (by then I wasn't so sure and it made me worry a bit).
- 2y ago
- gfs 2y agoI would love to see a blog post or more detail about how this implementation works and why it can beat rayon in some cases.
- judofyr 2y agoThe README of Spice has detailed explanation of the approach: https://github.com/judofyr/spice https://github.com/judofyr/spice
- memco 2y agoThis previous discussion of Spice on HN may help a bit: https://news.ycombinator.com/item?id=41230344 https://news.ycombinator.com/item?id=41230344
- jnordwick 2y agoThe definitive source on how to spell "chili": https://redhotchilipeppers.com/ https://redhotchilipeppers.com/
- jmakov 2y agoIs there anything like ray.io in rust? Nobody running computations on clusters in rust?
- aidanhs 2y agoThe company I work at (Hadean) used to have this as a product - think erlang-like multi machine IPC, with automatic acquisition of cloud resources and language integration for Rust, C, C++, Python. Pretty easy to point it at some machines and get them running a distributed application (as in simulation or big data). But infrastructure for developers is hard to make money with - developers like to build it themselves and people holding the purse strings point at kubernetes and say "that's free". So we just use it as an internal platform for a distributed simulation engine and it works pretty well. I did an analysis of removing it (it's a lot of bespoke code that we have to maintain for something that isn't our actual product) and I think you could probably implement something on top of Nomad that's close enough...but then Nomad went BSL and Kubernetes is a big complexity shift. So...if anyone knows of something out there let me know, I'd love to be able to use it outside of work :)
- jmakov 2y agoray.io seems to be doing pretty well financially...
- aidanhs 2y agoRight, because Anyscale found a niche that distributed compute matters in (AI) and built great libraries/hosted platforms/services around that. I would venture that the money they make from people who pare back things to just ray core is ~0, which is why it's open source. Put another way - building such a platform doesn't preclude commercial success, but (at least for us) it isn't sufficient. Fly.io might be able to pull it off if they want to explore that direction imo. Fwiw if you dig around in the ray core codebase (as I did when I was doing competitor analysis years ago) you can use the core C code from other languages to build such a platform for Rust if you like - they had Java and C++ interfaces at the time, but I haven't looked in the last 5 years.
- aspensmonster 2y agoThe old EE in me got excited for a second there, thinking someone had made a Rust port of [SPICE](https://en.wikipedia.org/wiki/SPICE https://en.wikipedia.org/wiki/SPICE).
- clarkmcc 2y agoInteresting, I wonder if I could use this on embedded. I was trying to get Rayon to work on embedded but was getting stack overflows when I tried. Maybe this will do the trick…