11 ms·

Rust const generics MVP hits beta

• steveklabnik 6y ago
  I am extremely excited for this feature, especially now that I live in "absolutely no allocations" embedded land. But also, beyond that, this is pretty much the last major feature that I've wanted in Rust. I've got some long-form writing in my head about this, but previously, I would have cast it as two or 2.5 features: * const generics * GATs * specialization (this is the half) However, when I've started to think about explaining these sorts of things, GATs (and to some degree specialization) feel much more to me like a removal of a restriction on combinations of features, more than "new features" strictly speaking. I think the line between these two perspectives is fascinating. On some level, you can even cast const generics in this way too: what it's really doing is making arrays a first-class feature of the language. I think that's a bigger stretch than GATs though, so I am not fully sure I'd make that argument. (The minimum feature we're stabilizing now basically does only this, but we do plan on going farther, so it feels true on a technicality now but not later.) Regardless, it's been nice to see how much we've slowed down in adding major things. November of 2019 was the last time we had a feature this large hit stable. 18 months between huge things feels much more like the cadence of more mature languages that have been around a lot longer than Rust. There is still a lot of work to do removing restrictions on existing features, especially const fn and const generics. But Rust is really starting to feel "done" to me, personally.
  • glittershark 6y ago
    I think it probably falls more into the category of removing a restriction rather than a new feature, but I'm still waiting on HKTs - not for traits (as a former die-hard haskeller, I have been gradually converted to the side of "rust doesn't need a Monad trait") but for data types. There's a design pattern in haskell called "Higher-Kinded-Data"[0] where you parametrize a datatype on a type-constructor, and use that to generically define either partial or total versions of that data type- something like (in rust syntax): struct Person<F> { pub name: F<String>, pub age: F<u32>, } where you can then have `Person<Option>` be a person that may or may not have all fields filled, and `Person<Box>` be a person with all fields definitely filled. This is something I find myself reaching for surprisingly frequently when writing rust, and I feel like it's a missed benefit of implementing higher-kinded types. [0]: https://reasonablypolymorphic.com/blog/higher-kinded-data/ https://reasonablypolymorphic.com/blog/higher-kinded-data/ All that said, I'm really excited to have const generics land! Props to all the amazing work by withoutboats and the entire rust team.
    • steveklabnik 6y ago
      I like and enjoy HKTs in other languages, but don't feel a super strong need for them in Rust. I know some other people differ. It is very unclear when, if ever, Rust will get HKTs proper. GATs cover a lot of the same ground. We'll see :)
    • adamch 6y ago
      Wow, higher-kinded data. I've never heard of that, but it would absolutely be useful. E.g. for converting JSON inputs, where any field might be missing, into a version of the data which definitely has values for all the fields. Or a version of the struct where the values are actually read from a cache/database.
      • glittershark 6y ago
        yes! once you use it you start wanting it everywhere, which is a big part of why I wish I could do it in Rust
      • twic 6y ago
        I proposed using it to model lifecycles of entities: https://github.com/tim-group/higher-kinded-lifecycle/blob/master/src/main/scala/Idea.scala https://github.com/tim-group/higher-kinded-lifecycle/blob/ma... (in this code, "idea" is a domain concept from the firm i worked for at the time - basically a recommendation to buy a stock, which is 'opened' on a certain date, and 'closed' when it no longer seems like a good recommendation) My collegues didn't like it, and stuck to using separate types for objects in different stages of the lifecycle!
      • IggleSniggle 6y ago
        TypeScript (as you might expect from a lang built for dealing with JS Objects), has really great tooling for this kind of type narrowing these days (including on string template matching now) and I miss it so much now that I’m writing Go.
    • meetups323 6y ago
      In TS this would be flipped: type Person = { name: string; age: number; } type SomeFieldsMissing = Partial<Person>; type AllFieldsRequired = Required<Person>; // No real change in this case Where Partial and Required are defined like: type Partial<T> = { [P in keyof T]?: T[P] | undefined; } type Required<T> = { [P in keyof T]-?: T[P]; } What would you think of something like this ("mapped types")? They can get fairly powerful: https://www.typescriptlang.org/docs/handbook/2/mapped-types.html https://www.typescriptlang.org/docs/handbook/2/mapped-types....
  • thenewwazoo 6y ago
    After writing embedded Rust, loving it, and then losing that job, I've gotta ask what you're up to these days (mostly so I can live vicariously through you). I am dying to write Rust in the embedded space again. :)
    • cpeterso 6y ago
      Steve works at Oxide Computer and they are hiring. :) https://oxide.computer/careers/software-engineering/ https://oxide.computer/careers/software-engineering/
      • doggodaddo78 6y ago
        TY, probable Moz Rustacean. :hands together emoji: I've been living on savings in self-sabbatical mode while catching-up on life, projects, and reading while keeping current with Rust. I'm soo itching to jump back into workaholism mode.
    • steveklabnik 6y ago
      Oh no! Hope you can get back to it soon. https://oxide.computer/ https://oxide.computer/ https://news.ycombinator.com/item?id=23975445 https://news.ycombinator.com/item?id=23975445 is probably the most accessible thing we've talked about publicly with what we're doing. (This talk was what made me decide to apply.) Still early enough the focus is on building the product rather than talking about it to broad audiences like HN.
      • doggodaddo78 6y ago
        Hire me! Jumps-up-and-down over-eagerly like a nerdy new college grad. I'll wave around cheerleading pompoms or aircraft marshaling wands until you do. ; D Excessive, horn-tooting quasi- resume/cover letter in an HN comment (maybe I should've compressed and uuencoded?). Home: Austin downtown, via much of NorCal { Chico, Sacramento, and Bay Area }, from San Jose. Experience with embedded systems on many architectures and different types of RT deadlines, SMT electronics (PCB design and fabrication, stuffing, and repair), Rust (I sure hope so!), C/C++ (autodidacted at 15), LLVM, several assembly languages (ARM, X86, MIPS), microcontrollers, large-scale industrial firmware development (automotive, mining, & agriculture). I periodically throw 3D printing and Arduino or ESP32 at personal use-cases. And trinocular microscope, Siglent scope, and knock-off compatible JBC solder station. The Fluke 289 DMM is real though. Can do everything from sales engineering, on-prem/remote custom client integration, customer training, feature development, tools support, customer bug forensics, subgroup lead SWE, systems architecture/engineering, customer solutions development. Make the office more funner now and then (especially internal funny 404 pages and easter eggs, tape your chair up, orbeez explosion, get your wife/SO and kids in on a gaslighting long-con). 5¢ charge per instance. Wooden nickels not accepted. No refunds. :) Did one or two minor humblebragging things: - Ported a ridiculous nuclear reactor simulator in Fortran from UNIX to Windows. - IBM Almaden offered me a dark matter research assistant job when I was 15, but my parents weren't supportive and it would've been illegal. :Y U No moon meme here: - Wrote a Java-- to native MIPS (non-JIT, compile-time) compiler from scratch with symmetrical implementations in Java and C++. - Refactored the heck out of a UDP 900 MHz packet radio firmware C++ codebase and added telemetry logging with error tolerance of Flash EEPROM wear beyond mfgr specs. - Restoring & modding a VW camper and getting into paramotoring (parachute wing and motor on your back).
  • the__alchemist 6y ago
    Could you please post an example of how it helps in embedded / no allocator code? I do a good deal of embedded on Rust, but am having a tough time figuring out when or why to use this from the abstract examples in the article. Tangent: What's your take on the Rust HALs? (eg `stm32l4xx-hal`) etc? I'm curious what the take is of someone outside the Github and Matrix communities. They strike me as... remarkably underdeveloped, but have big potential.
    • steveklabnik 6y ago
      > Could you please post an example of how it helps in embedded / no allocator code? In today's Rust, data structures often need to be a fixed size, or dynamically allocate. This will let you write a data structure that has a variable size, but set at compile time. I knocked out a quick ringbuffer recently; I picked a size that seemed fine, but that size is fixed at the moment. If I had used const generics, it could have been more flexible here, I could have written it more easily for any size, where the size is written at compile time. This is not my work code, but back when I was working on a hobby x86 kernel, I did this for a VGA buffer: https://github.com/intermezzOS/kernel/blob/master/vga/src/lib.rs#L16-L22 https://github.com/intermezzOS/kernel/blob/master/vga/src/li... it's generic over "something that can be converted to a slice," so that it uses a slice in production, but a Vec in tests. This also leads to runtime checks https://github.com/intermezzOS/kernel/blob/master/vga/src/lib.rs#L26-L27 https://github.com/intermezzOS/kernel/blob/master/vga/src/li... to make sure that the slice is the right size. This would be much better written with const generics today. The slice trick is neat, but awkward. (This code is also awkward because it grew an internal "frame buffer," so it kinda has both going on. This example needs const generics less since VGA has one single size at compile time ever, but sometimes, you need something where this isn't the case, and this was the first example of code I personally wrote that springs to mind.) > What's your take on the Rust HALs? I haven't used them enough. We use the layer below them, for example, the stm32f4 crate rather than the stm32f4xx-hal crate. That doesn't mean they're bad, I just literally have not used them enough to form an opinion. The namesake of this space is diversity, and so different people make different tradeoffs; we don't really need what those crates are offering right now.
      • 6y ago
  • mamcx 6y ago
    I also wish for extensible enums and enum subset: pub enum Expr { Int, Str } pub enum Expr2 : Expr{ Bool, } fn check(??)-> Expr.Int
    • mamcx 6y ago
      And struct extend!: struct Person { id:i32 } struct Customer: Person { } P.D: This is not subclassing, is not redefine all the same attributes again and again. Is partially supported to copy the values but not defining them.
  • akavel 6y ago
    For me, a feature I'd really hope for is the elusive "if not let" (or however this gets named eventually), to let me escape the currently unavoidable clutches of Rust's rightwards drift and accruing mental context.
    • yazaddaruvala 6y ago
      Lol it would be kinda funny to have this: let input: Option<usize> = Some(1); if let Some(value) != input { return 0; } // use value; such that value: usize = 1
      • nybble41 6y ago
        We already have something close: let input: Option<usize> = Some(1); let value = match input { Some(x) => x, _ => return 0, }; // use value; such that value: usize = 1
        • NobodyNada 6y ago
          That works, but it's clunky enough that it defeats the purpose of writing guard statements IMO. It's not hard to understand, but it's also not possible to figure out what it means at-a-glance. I have to stare at it for a few seconds to try to figure out where all control flow is going to go -- it has two levels of nested subexpressions, along with branches that appear similar but do very different things (returning a value from the expression vs. from the function). The point of a 'guard' clause is to visually distinguish between the "expected" and "unexpected" case. When I see a guard statement: let input: Option<usize> = Some(1); guard let Some(value) = input else { return 0; } // use value; such that value: usize = 1 I immediately know that the rest of the function expects input to contain a value, and that whatever is after the 'else' is some kind of error handler. Your match statement might have the same effect functionally, but visually it's confusing because it's not immediately obvious which branch is the happy-path. I'd rather read a function containing ten guard clauses than a function containing ten of those matches.
      • freeopinion 6y ago
  • RcouF1uZ4gsC 6y ago
    So excited about const generics. Is there any work on variadic templates. That is one of the big things I miss from c++. Macros can fulfill some of the needs, but not always.
    • steveklabnik 6y ago
      Some people have made some proposals, but it's at an extremely early, if ever, state.
  • ComputerGuru 6y ago
    I’m an embedded rust developer myself, and have written my own stack with const_generics and am extremely excited to see const generics land in stable. However (and I’d love your thoughts about this) the one feature I feel has not gotten the love it deserves is DST types and safely creating them. I was writing a zero-alloc no_std h264 and mpegts parser and the hoops I had to jump through to represent extant mpeg structures as valid rust data types was insane and I ran into a lot of edge cases with transmute that made me scratch my head. In the end, I wrote my own macro for creating (in-place new-ing) DST types that transmuted fat pointers (whose representation is not guaranteed but unlikely to change anytime soon) to get things kind of working but the ergonomics and safety were non-existent. I feel like if rust doesn’t pick up the baton for innovating on DSTs it would be a huge shame since from a security and memory safety perspective they cause the lion’s share of overflow and out of bounds accesses in parsers and other high-risk libraries that silently digest all data thrown their way for media previews, thumbnail generation, etc etc. What irked me is that a lot of people consider the checkbox for DST support to be ticked when there’s really only the bare minimum for interesting with types containing single DST elements that are already - somehow - created, typically by the standard library itself. (Again from an embedded perspective: I’m really happy with async/await for embedded development; it really makes reasoning about control flow in an embedded context much more sane!)
  • iamatologist 6y ago
    Upcoming website from Klabnik: www.arewedoneyet.rs
• simias 6y ago
  This is one of my big frustration with Rust and probably the only feature from C++ I rally missed (C++ templates having "non-type parameters"). I'm very happy to soon see it lifted. There were workarounds for some common scenarios already, but it's just so much more simple and convenient to be able to write `fn foo<const N: u32>()`. I'll finally be able to simplify a significant portion of my code.
  • tspiteri 6y ago
    Lucky you :) In my case the only thing I could to do was to change the README of my crate from saying that I plan to migrate from typenum to const generics "when they are supported by the Rust compiler" to now saying "when the Rust compiler support for them is powerful enough". (I need constraints on which ranges are allowed, and that is a little hairier than the MVP; though typenum is perfectly adequate.)
• CodesInChaos 6y ago
  I'm really happy that this finally landed. Unfortunately the limitations of the MVP are severe. For example you can't use associated constant as generic arguments, preventing constructions like this: trait HashFunction { const OUTPUT_SIZE: usize; fn hash(input: &[u8]) -> [u8; Self::OUTPUT_SIZE]; }
  • rmdashrfstar 6y ago
    What’s the progress on supporting something like this in future versions?
    • steveklabnik 6y ago
      https://github.com/rust-lang/rust/issues/76560 https://github.com/rust-lang/rust/issues/76560 is the tracking issue for this specific feature.
  • tuankiet65 6y ago
    Are you literally me 2 months ago? I was trying to implement the same HashFunction trait as yours and I finally settled with: pub trait HashFunction<const OUTPUT_SIZE: usize> { fn hash(data: &[u8]) -> [u8; OUTPUT_SIZE]; } I believe this approach has some shortcomings in my case, but it has been a while and I didn't remember anything. Also for the Rust crowd, is it possible to do something like this? pub trait HashFunction<const OUTPUT_SIZE: usize> { // I want Output to always be [u8; OUTPUT_SIZE], but using associated type means // whatever implements HashFunction can redefine Output. type Output; fn hash(data: &[u8]) -> Output; }
    • CodesInChaos 6y ago
      You can add a constraint of Output. With a bit of abuse you can create a type that's impossible to implement outside your crate and implement it only for arrays in your crate (but if you make the output generic, there is no reason to keep the generic OUTPUT_SIZE parameter).
• est31 6y ago
  Users of the serde-big-array crate can already opt into using min_const_generics by enabling the const-generics feature. The advantage: you no longer have to list a bunch of needed array sizes (or rely on the builtin defaults), but can use whatever size you want. Serde proper needs something like const_evaluatable_checked before it can offer large array support.
• SuperFluffy 6y ago
  Having const generics will also permit much more elegant implementations of low level linear algebra. For example, when writing a `GEMM` (general matrix multiplication) routine, the core building block is a so called kernel, which gets moved over the matrices like a stencil. The size of the kernel however depends on a) the numerical precision (float, double), b) the available SIMD intrinsics, and c) the architecture the code is executed on (haswell, skylake or zen have different latency and throughput for different intrinsics). Right now it's surprisingly painful to write an optimal kernel and buffer, which is different for every architecture and which you need to kind of hardcode. With const generics this should become much easier.
  • porphyra 6y ago
    Yes!!! Linear algebra! I am a huge fan of nalgebra and having const generics would make the API and implementation much nicer. Hopefully the robotics industry can move towards implementing stuff in Rust. Having low level robotics algorithms like state estimation and SLAM in Rust makes perfect sense.
    • gfaure 6y ago
      Have you tried working with rust-ndarray (the closest thing to NumPy in Rust)? While I love Rust for general work, unfortunately for numeric programming, compared to NumPy itself, it's like pulling teeth. There's a lot which is completely missing: the main thing for me is (boolean) masks, which means there's no element-wise comparison of ndarrays.
      • yodelshady 6y ago
        It's a cliche for open-source libraries, but ndarray could use more documentation. A lot more. A really whole lot more. "Pulling teeth" is how I'd describe ndarray currently.
  • mhh__ 6y ago
    I do this in D all the time, and the system is actually so flexible you can even use things like simulated annealing to optimize (if you are completely mad) to optimize the parameters. struct DiscardPastN(uint len) { int[len] buffer; uint state = 0; void put(int x) { buffer[++state % len] = x; //mwuhahahaha static if(len == 4) asm { ud2; } } int[len] get() const { return buffer; } } So extremely useful all over the place let alone linear algebra, I'm glad Rust now has it too
• doggodaddo78 6y ago
  Super frickn cool! I was dying for this feature.
• jlrubin 6y ago
  This is amazing. Kudos rust team for nearing this milestone. I use rust for a (to be released -- comment with your github if you want early access) Bitcoin Smart Contract embedded domain specific language (edsl) that operates sort of like a circuit meta-programming language. Having const generics will drastically simplify my code and enable greater "structural type safety" (checked at the rust level rather than as a part of my edsl). They're not wrong when they say that this is the most highly anticipated feature :)
• sam0x17 6y ago
  This is so huge that it might actually relieve enough of my frustration with Rust that I would try using it again <3 Very needed and cool
• proverbialbunny 6y ago
  This is a pretty big deal. One benefit it gives is it's much easier to write code evaluated at compile time. Most Rust libraries use generics, so if you use a library, compile time support isn't usually available. By adding support for const generics compile time support can become widespread.
• iamatologist 6y ago
  It must be hard to design a zero-cost abstractions language. It seems that there are more and more terms and concepts that come up in order to support more directly-pragmatic PL features.[1] Reminds me a bit of how Haskell comes across to me from the outside with all its GHC extensions. Haskell is a research language but other more specialized functional languages have the luxury of being able to theorize and implement more orthogonal and perhaps more “elegant” concepts and approaches. (Again, merely an impression from the outside.) Consider the design effort behind parametric memory allocators. That seems like a pretty cutting-edge problem. And yet I bet the Rust folks knew that they would want/need to do this way before they started doing that work in earnest, because people from C++ seem to want the same thing (if they don’t have it already?). I idly wonder if one could, if one was in a similar position as Rust was some years ago, just go ahead and design a full-on unapologetic type-level programming language from the start. Because you know that your type-level terms will be worthy of the moniker “language” eventually (and it might not be a compliment as such). Just a nice, high-level language that doesn’t bother with the “bare metal” concepts that Rust the value-level language has to deal with. (Can it even be done? Don’t ask the peanut gallery about that.) Either that or you accidentally build an emergent language that Gankro can write an article about one day titled, I don’t know, Shitting Your Pants With Higher-Order Unsafe Unwind Type-Level Allocator Escape-Suppressing Storm Cellars. [1] In this case: you have more use for compile-time integers than something more general like being able to describe that two nat-indexed lists are of the same length, like you can in Idris.
  • steveklabnik 6y ago
    You might like https://without.boats/blog/revisiting-a-smaller-rust/ https://without.boats/blog/revisiting-a-smaller-rust/, written by someone on the language team, on this topic.
• mhh__ 6y ago
  D Templates can take just about anything in the language as an argument (user defined, builtin, doesn't matter) and I can confirm that being able to do this is extremely useful i.e. if you write a hybrid allocator you can specify the metaparameters as a template parameter. e.g. struct SmallString(size_t N) { /* etc. */ }
• Waterluvian 6y ago
  Newbie question if someone doesn't mind. This seems to make it easier to do stuff without allocating heap memory. How large is the stack? Can I do complex things with only the stack?
  • zucker42 6y ago
    This is an OS question, not a Rust question. Try typing ulimit -s if you are on a Unix-style system to get the maximum stack size (on my system it's in KB). And yes, you can do many things using the stack; it's just not as flexible. But const generics are most useful for increasing the efficiency of certain code while keeping it reusable.
  • grdvnl 6y ago
    I am curious. Why do you think it helps it doing things on the stack. For me, I understand this to provide more generic type level checks on the size/shape of values. Perhaps, I am oversimplifying this?
    • lasagnaphil 6y ago
      Basically, the const-generics feature gives the programmer the ability to create fixed-size custom types that doesn't allocate on the heap. For example, with current Rust you can't make your own generic fixed-size hashmap (as in FixedSizeHashMap<K, V, N: uint>, where N is the maximum number of keys you can use) that doesn't use the heap, since you can't use size-parameterized arrays in structs. Although this isn't a big deal for most general programming purposes, it still matters a lot in domains like embedded/HFT/gamedev/simulation where minimal latency is incredibly important. It's also important when you want to create any sort of linear algebra library (such as Eigen). For me const-generics is an important dealbreaker when choosing between Rust and C++ (where C++ already had the ability to do this for decades).
• senderista 6y ago
  Will I get to specify the node width for BTreeMap now?