7 ms·
Under the hood: Vec<T>
- rossant 1y agoTangent but nice web design.
- herywort 1y agoFirst thing I did was toggle reader view, it's noticeably more responsive scrolling there.
- veber-alex 1y agoThis article feels too long and rambly considering how little actual technical information it provides.
- LoganDark 1y agoI don't think the point of the article is the technical information, I think it's more of an emotional expression. Still valuable, just differently, I suppose.
- germandiago 1y agoLike much of what surrounds Rust. Looks quite emotional to me. If you do not know what I mean, go to the Rust reddit and discuss and compare on solid grounds without using an extremely flattering tone. You will see armies of fanatics voting negative.
- echelon 1y agoIt's what you call an "educational post". The Rust community is doing a fantastic job of training both the next generation as well as those looking to dip their toes in.
- spoiler 1y agoTo be fair, that is just the typical Reddit (or any social media platform) experience
- LoganDark 1y agoI won't deny that there are lots of emotions surrounding Rust, both for myself and for many others. But there are different ways to write about it, and this article looks more of an emotional style ("here's how my journey went") than a technical one ("here's how this works"). I still find it fun to read, but not everyone will, and that's okay.
- steveklabnik 1y agoIf you actually look at the Rust subreddit: https://www.reddit.com/r/rust/top/?sort=top&t=all https://www.reddit.com/r/rust/top/?sort=top&t=all The fifth and eighth articles are explicitly negative about Rust. The seventh is about serious bugs in the Rust ecosystem. Let's look at the top comment on the highest post: https://www.reddit.com/r/rust/comments/1cdqdsi/lessons_learned_after_3_years_of_fulltime_rust/ https://www.reddit.com/r/rust/comments/1cdqdsi/lessons_learn... It's Josh Triplett, long time team member. It starts like this: > First of all, thank you very much for taking the time to write this post. People who leave Rust usually don't write about the issues they have, and that's a huge problem for us, because it means we mostly hear from the people who had problems that weren't serious enough to drive them away. Thank you, seriously, for caring enough to explain the issues you had in detail. This is a very different vibe than the one you're describing. It's true that the Rust subreddit can brush criticism off, but that's also because there are a lot of low-quality criticisms of Rust, and seeing the same thing over and over again can be frustrating. But I've never seen a well thought out post that's critical get downvoted, they're often upvoted, and discussed in a very normal way. That said, it's reddit, so there's always gonna be some garbage posts.
- fuckyah 1y ago[dead]
- commandersaki 1y agoThis comment feels at odds with HN guidelines, if you're going to criticise at least offer some advice on how to improve.
- 5- 1y agorelated: https://doc.rust-lang.org/nomicon/vec/vec.html https://doc.rust-lang.org/nomicon/vec/vec.html
- James_K 1y agoOh, I never knew that Rust had variance. I always just assumed everything was invariant. Strange that they've got no way to write it down in the type system.
- tialaramex 1y agohttps://doc.rust-lang.org/reference/subtyping.html#variance https://doc.rust-lang.org/reference/subtyping.html#variance Lifetimes imply the need for the idea of variance.
- kibwen 1y agoRather, any language with both generics and subtyping needs to consider variance, and Rust doesn't have subtyping in general, but lifetimes do technically have subtyping relationships, so lifetimes are the only place where this matters, and fortunately Rust can just infer the right behavior based on some simple type-level heuristics.
- James_K 1y agoUnfortunately, more like. I'd much rather have some explicit syntax for it that potentially never realise it's a feature. I was baffled beyond belief the first time the compiler asked me to put a ghost in my struct.
- pornel 1y agoIn prehistoric Rust, variance used to be named more explicitly. However, the terminology of covariant and contravariant subtyping of lifetimes is a language theory jargon. This is the right perspective for language design, but programmers using the language don't necessarily use these terms. It's been replaced with a "by example" approach. It's much easier to teach it: just add a fake field that acts if you had this type in your struct. Rust then figures out all of the details it needs.
- Vipsy 1y agoI love how it shows that you can have both performance and safety without jumping through hoops. The little details of memory management really shape how we end up writing code, often without even noticing. Rust makes it possible to get close to the hardware while still keeping you out of the usual trouble spots. It’s one of those things you appreciate more the longer you use it
- alickz 1y agoI come from a Swift/Kotlin background and I've been learning Rust for fun in my spare time From what I heard online I was expecting it to be a lot harder to understand! The moving/borrowing/stack/heap stuff isn't simple by any means, and I'm sure as I go it will get even harder, but it's just not as daunting as I'd expected It helps that I love compiler errors and Rust is full of them :D Every error the compiler catches is an error my QA/users don't
- LoganDark 1y ago> It helps that I love compiler errors and Rust is full of them :D Every error the compiler catches is an error my QA/users don't amen! I despise Python even though I used to love it, and that's because it's full of runtime errors and unchecked exceptions. The very instant I learned more strongly typed languages, I decided never to go back.
- zelphirkalt 1y agoAnd then comes the point, where you get a scenario, which is actually, you'd think, at least, something simple, but then gets really tough to express in strongly, statically typed languages, and you might take a step back and consider for a moment, how simple it would be to express this in a language like Python, while still being memory safe. Statically typing things is great, and I enjoy it too, when it is practical, but I don't enjoy it, when it becomes more complicated than the actual thing I want to express, due to how the type system of that particular language or third party tool (in Python) works.
- James_K 1y agoWhile safe Rust may be relatively simple to write, and certainly easier to write than safe C, this article has someone added to my belief that unsafe Rust is far too difficult to write. Perhaps some of this is deliberate, as a kind of defence mechanism against people using it willy-nilly, it still seems over-designed.
- SonOfLilit 1y agoWhen I was young I peeked into the C++ stdlib, which is probably the best thing to compare this to. It was orders of magnitude worse.
- tonyedgecombe 1y agostdlib code was always impenetrable to me, this was one of the reasons I moved away from C++.
- James_K 1y agoWould you consider this to be indicative of the quality of the C++ language? I think the standard library should usually be an example of optimal code with respect to performance, and when it is very complicated this would indicate, to me, that the language makes good code complicated. If you look at an advanced C library, you'll see perhaps some odd tricks or nifty algorithms, but you probably won't see something that leaves you scratching your head about what the code is even asking the computer to do.
- tialaramex 1y agoUnsafe Rust does have tooling to help you not make some kinds of mistakes, but by its nature you have to be able to press on after the tools can't help you. For example MIRI can understand pointer twiddling under strict provenance. It'll synthesize provenance for the not-really-pointers it is working with and check what you wrote works when executed. But if your algorithm can't work with strict provenance (and thus wouldn't work for CHERI hardware for example) but your target is maybe an x86-64 Windows PC so you don't care, you can write exposed or even just arbitrary "You Gotta Believe Me" pointer twiddling, MIRI just can't check it any more when you do that, so, hope you never make mistakes.
- gblargg 1y agohttps://archive.is/Cqr4E https://archive.is/Cqr4E (if their site blocks your browser)
- constantcrying 1y agoI gave up half way through. The constant description of deep emotional feelings when it comes to something as sober as data types in the rust standard library make this rather hard to read. It is difficult to untangle the point the author is trying to make from the constant bombardment with deep emotional writing.
- swiftcoder 1y agoIf a little lighthearted banter counts as "deeply emotional", I guess I understand why your username is "constant crying"
- constantcrying 1y agoEven if you do not expect a professional tone it is totally bizarre to write a technical article (which I would have liked to understand, since the question is interesting to me) as if you are talking to a toddler. The allusions to Grand conspiracy theories and grave emotional turns which are everywhere make the article sound as if it was written for children. I do not like lighthearted banter, but this is not the problem. The problem is that the author has a hard time writing a single paragraph without it. They should seriously consider doing some training in technical or scientific writing. >, I guess I understand why your username is "constant crying" What a novel comment. But yes, I do hate the constant positivity, especially in the face of a subpar product.
- swiftcoder 1y agoSome people like to have fun. Is it so hard to look away from the things you don't personally like? As to the underlying topic, this is a very introductory article - I warrant 20 minutes with the source code, and you'd reach the same level of understanding.
- assbuttbuttass 1y agoI'm sure all these layers are good engineering and provide meaningful safety guarantees, but it sure makes the code harder to understand if you want to see how things are implemented. Another example, I was trying to see how i64::isqrt is implemented, but first you have to wade through layers of macros
- ozgrakkurt 1y agoThis is just how rust and c++ code is in my experience.
- germandiago 1y agoC++ is bc of compatibility with god knows how many standards, but at the end it is simpler than it looks at first sight. The uglification of names in the std lib make it disgusting to read. If only that was removed it would improve a lot.
- Narishma 1y agoEven C is like that in its most popular stdlib implementation (glibc).
- ameliaquining 1y ago"The standard library internally overuses macros and you probably shouldn't use them that much in your own code" is at least a respectable opinion in the Rust community, I think.
- steveklabnik 1y agoYep. Heck, I've barely even written any macros at all.
- tialaramex 1y agoBecause this is focused on how the data structure works it doesn't mention lots of nice API design choices in Rust. The one I particularly want to call out because it came up this morning is providing both Vec::reserve and Vec::reserve_exact Vec::reserve lets us hint about our upcoming capacity expectations without damaging the O(1) amortized growth which is the whole point of this collection type, but it can waste some memory to achieve this. This is probably what you want in most code. Vec::reserve_exact is a more narrow idea - we can hint about the ultimate capacity needed, if we're wrong and later need more capacity this has a significant performance cost because we thew away the amortized growth promise to get this, but we don't waste memory. C++ only provides the equivalent of Vec::reserve_exact which makes this a footgun but if you use this call when you really needed the other one you trash your perf, as a result of which people teaching C++ tend to just say don't use reservation to uh, reserve capacity, but now you're leaving perf on the table so that's not great.
- smallstepforman 1y agoFrom your description, I cannot determine the difference. Engineers like exact things, whats this fuzzy concept called “hint”?
- throwaway173738 1y agoSuppose I think I may need 128 entries at some point, but the vector is allocated with room for 16 entries by default. I may not want to allocate and then immediately allocate again. But if I get to a 17th entry then I’m already causing allocation. So I might as well allocate 128 at that time so there are no more allocations at all.
- demurgos 1y agoI believe that you are describing `Vec::with_capacity` which allows to change the initial reserved memory on construction. `reserve` and `reserve_exact` are used when mutating an existing vec. What you provide is not the total wanted capacity but the additional wanted capacity. `reserve` allows to avoid intermediate allocation. Let's say that you have a vec with 50 items already and plan to run a loop to add 100 more (so 150 in total). The initial internal capacity is most likely 64, if you just do regular `push` calls without anything else, there will be two reallocations: one from 64 to 128 and one from 128 to 256. If you call `reserve(100)`, you'll be able to skip the intermediate 64 to 128 reallocation: it will do a single reallocation from 64 to 256 and it will be able to handle the 100 pushes without any reallocation. If you call `reserve_exact(100)`, you'll get a single reallocation for from 64 to 150 capacity, and also guarantee no reallocation during the processing loop. The difference is that `reserve_exact` is better if these 100 items were the last ones you intended to push as you get a full vec of capacity 150 and containing 150 items. However, if you intend to push more items later, maybe 100 more, then you'd need to reallocate and break the amortized cost guarantees. With `reserve`, you don't break the amortized cost if there are follow-up inserts; at the price of not being at 100% usage all the time. In the `reserve` case, the capacity of 256 would be enough and let you go from 150 to 250 items without any reallocation. In short, a rule of thumb could be: - If creating a vec and you know the total count, prefer `Vec::with_capacity` - If appending a final chunk of items and then no longer adding items, prefer `Vec::reserve_exact` - If appending a chunk of items which may not be final, prefer `Vec::reserve`
- blacklion 1y agoIt is not clear, does compiler explicitly knows about properties of `NonNull` (for example, that it has neutral value to collapse `Option<NonNull<T>>`), so it is part of compiler, or it is expressed in type system, so it is part of standard library? Same question about other «signal» types.
- conradludgate 1y agoIt's a mix of both. https://doc.rust-lang.org/src/core/ptr/non_null.rs.html#72-73 https://doc.rust-lang.org/src/core/ptr/non_null.rs.html#72-7... #[rustc_layout_scalar_valid_range_start(1)] #[rustc_nonnull_optimization_guaranteed] This declares that NonNull cannot be outside of the range (1..). These magic attributes are only allowed to be used by the stdlib, and they're direct compiler instructions. Nonnull is special because the optimisation is guaranteed so the compiler does have special code to handle it, but other types can be optimised without compiler changes.
- zdimension 1y agoFor anyone wondering, NonNull (along with other "constrained" types like NonZero integers) uses internal compiler attributes (https://doc.rust-lang.org/src/core/ptr/non_null.rs.html#72 https://doc.rust-lang.org/src/core/ptr/non_null.rs.html#72): #[rustc_layout_scalar_valid_range_start(1)] This gives rustc enough information to perform niche optimizations such as collapsing Option<NonNull<T>>. You can technically use those for your own types, but it requires nightly, obviously.
- tialaramex 1y agoHowever, Rust does automatically provide a niche if you make a simple enumeration which doesn't occupy all bit patterns and we don't need compiler-only features, or even unstable Rust, this Just Works™. If you have a USHoliday enum, with values like Juneteenth the fact is there's not that many US national holidays, so both USHoliday and Option<USHoliday> will be the same size, one byte with no extra work.
- dathinab 1y ago> part of compiler, or it is expressed in type system, so it is part of standard library compiler + standard library (core library to be more specific) `NonNull` is mostly "just" a library type in the core libary, but it uses two unstable/nightly attributes to tell the compiler about it's special properties (see other answers, also to be clear its not a `lang_item` attribute but something other types can use, too). The special thing about core/std in rust is that due to it being part of the specific compiler version it can use (some, careful considered) subset of unstable features as long as they work for that specific use case and don't "leak"/accidentally stabilize through it. But this attributes are in progress of getting replaces with a new mechanism of pattern annotations. I.e. NonNull will be annotated with attribute which contains a `1..` pattern (i.e. value is 1 or larger) but other patterns are getting support, too. (The second attribute to enforce/guarantee niche optimizations might still be needed for stability guarantees.) And I think??? there are plans to stabilize this new mechanism, but I'm absolutely not sure what the state of this is (RFC accepted?, stabilization open/likely/decided? implementation done/partial?). I'm currently not in the loop. So in the long future you probably can write your own types with other niches e.g. a u8 with `..127` (MSB unused).
- germandiago 1y agoCan anyone compare it to C++ in terms of memory layout for typical implementations?
- username223 1y agoThe vector implementation on my machine uses three pointers: start, end, and end-of-storage. It's basically the same, but uses addresses rather than counts. The valarray implementation uses a pointer and a single count.
- sesm 1y agoThat's one of major problems I have with Rust docs and community. The best way to explain most things in Rust to experienced dev, is by comparison to C++. But all the docs and blog posts explicitly target newbies and avoid comparison with C++ at all cost.
- zozbot234 1y agoI disagree, Rust and C++ are very different languages with significant impedance mismatches once you go beyond the common C-like subset. Referencing C++ as a matter of course in docs and blog posts would just cause confusion. If you want a modern language that really is a lot closer to C++ you may want to check out Carbon.
- sesm 1y agoRust was created as a C++ replacement, borrows the 'zero cost abstractions' motto from C++, relies on RAII for resource management like C++ (not many languages do it), has the same approach to concurrency (in-place mutation guarded by locks), uses the codegen backend that was created for C++. It's mostly a C++ subset with more guardrails. Edit: RAII
- zozbot234 1y agoIt's nowhere near a C++ subset. The way it works under the hood is quite different, and even trying to add some of the underlying mechanisms to C++ (such as what they call "trivially relocatable types", or "destructing moves") has been quite difficult.
- sedatk 1y ago> Since a NonNull<T> can never be null, the compiler knows it can use the 0 address to represent the None variant when it sees an Option<NonNull<T>> I didn't know this about Rust. That's awesome.
- danhau 1y agoSame optimizations apply to the various NonZero types, IIRC.
- jojomodding 1y agoThe explanation of `Unique` is a bit botched. `Unique` was originally intended to convey uniqueness in the sense of `noalias`, so that more compiler optimizations are possible similar to what happens around mutable references. This is what the article seems to suggest is going on . But this is wrong, it is no longer the case that `Unique` implies any extra optimizations, since it was since recognized that having these optimizations on a low-level unsafe building block is actually undesirable. Further, the `as_ref` functions mentioned in the article are not used by `Vec`'s internals. They are anyways only marked unsafe because they create references, not because of anything special with `Unique`. Nowadays, all that `Unique` does is interact with `dropck` in a quirky way, the "exception to the exception." It is quite complicated and I always have to read up on it. It is explained in the Rustonomocon, in the "Click here to see why" section of this chapter: https://doc.rust-lang.org/nomicon/phantom-data.html#an-exception-the-special-case-of-the-standard-library-and-its-unstable-may_dangle https://doc.rust-lang.org/nomicon/phantom-data.html#an-excep...
- marekzan 1y agoHey, author here :). Thank you for your critique and you are right, I struggled with this chapter a lot. I will read the link to get a better understanding of this type and update the section. If I may ask, do you have more sources where I can read up on the history and current uses of unique besides the rustonomicon? Thank you again for your engagement! It is very helpful to get these insights from people with more knowledge.