4 ms·
> terminology taken from Go, whose slices are similar though with an extra word Interesting; I've always thought of Rust slices as being rather different from
by saghm 9y ago
> terminology taken from Go, whose slices are similar though with an extra word
Interesting; I've always thought of Rust slices as being rather different from Go slices. A Rust slice is always used through a reference and doesn't own its data, whereas a Go slice is not generally used through a pointer and sometimes points to a heap allocated section of memory, so it's basically the union of Rust's vector and slices.
Tangentially, the inability to tell whether some value is heap allocated or not from the type is one of my main gripes when working with Go as opposed to Rust; in Rust, I can be sure that `Vec`, `String`, `Box`, `Rc`, and `Arc` are all heap allocated and that slices, arrays, `str`, `&T`, and `&mut T` are not. In Go, slices and pointers might be heap allocated--or they might not.
- sriram_malhar 9y ago> the inability to tell whether some value is heap allocated or not from the type is one of my main gripes when working with Go Why is this important? The whole point of GC is to not spend time debating this point, as long as performance is good enough
- jmgao 9y agoThere's problems when you're taking a slice and storing it for a potentially long time. If the slice is actually a subslice of a super large array, the backing slice will be kept around until the subslice goes away (assuming the implementation doesn't try to be clever about things, but it probably doesn't).
- zaarn 9y agoIf the Go compiler can't tell anything about the length of storage of a slice, it'll end up in heap. The stack is usually only used when escape analysis determines that the value does not survive the function call, atleast IIRC. Slices should be capable of being partially deallocate so long as the backing arrays are not referenced anymore.
- saghm 9y agoNormally, it isn't! Unfortunately, "good enough" is relative, and for some applications this is vitally important. I'm not trying to knock Go's performance here; from a naive standpoint, GC'ing only some pointers is better than GC'ing all of them like you have in more traditional garbage-collected languages, but it's still easier to know what exactly is being heap allocated and what isn't in a language like Java precisely because you know that all objects are on the heap. From what I've seen of low-level optimizations in Go code, it relies heavily on techniques like generating flame graphs to analyze where allocations are occurring, which IMO isn't a very good workflow, whereas in Rust you could do this much more easily by just looking at the types that are used. I don't think this approach is necessarily incompatible with garbage collection; theoretically a language like Go could have separate vector and slice types like Rust does, and I think that would make these types of optimizations much easier! (I'm also not sure why you were downvoted for asking this; it's a perfectly reasonable question)
- lmm 9y agoFor the record, in modern JVMs with escape analysis you may find that some objects aren't actually on the heap.
- hyperpape 9y agoGo also uses escape analysis, though the effects could be different because of the compilation model.
- cdoxsey 9y ago> I don't think this approach is necessarily incompatible with garbage collection; theoretically a language like Go could have separate vector and slice types like Rust does, and I think that would make these types of optimizations much easier! Absolutely. It's not a GC issue, it's a design issue. Adding this kind of control would make the language harder to use. Making the hard case easier to handle for experts makes the easy case harder to handle for everyone.
- saghm 9y ago
- littlestymaar 9y agoGo doesn't have a generational & compacting GC (where allocation can be made really efficient: just bumping a pointer), hence heap allocation are expensive in Go (but idk the details, and maybe they're not as expensive as they are in C or Rust). Then to avoid performance penalties, you need to reduce allocations to the minimum, but since Go use escape analysis to decide whether to allocate on the heap or not, you don't have full control on what is heap-allocated or not, and avoiding allocations can be quite tricky.
- linkregister 9y agoIsn't it the case that the compiler allocates objects on the heap if the reference escapes the function, and allocates on the stack if the reference doesn't?
- masklinn 9y ago> if the reference escapes the function The difficulty is knowing when that happens, and you're probably guessing wrong (the -m gcflag will tell you). Furthermore fitting Go's theme the escape analysis is pretty simplistic, so there are many cases where it will somewhat unexpectedly assume escape (note: link is from 1.15, some have been fixed since like the …arg one or the slice assignment): https://docs.google.com/document/d/1CxgUBPlx9iJzkz9JWkb6tIpTe5q32QDmz8l0BouG0Cw/preview# https://docs.google.com/document/d/1CxgUBPlx9iJzkz9JWkb6tIpT...
- kibwen 9y agoRather than being a comment on where each was stored, my comment was intended to highlight how both Rust and Go slices are fixed-size pointer+metadata "windows" into some underlying array.
- saghm 9y agoFair enough! Not having done any PL work in a while, I tend to think of types in terms of the properties of how I use them rather than their implementation, so I was just surprised to see them compared this way.
- dan00 9y ago> in Rust, I can be sure that `Vec`, `String`, `Box`, `Rc`, and `Arc` are all heap allocated and that slices, arrays, `str`, `&T`, and `&mut T` are not. That's not really true, because you can easily create e.g. a `&str` out of a `String` or a `&[T]` out of a `Vec<T>`.
- merb 9y agois there a way to force a copy? like String copy to &str (on the stack?)
- kaoD 9y ago`String` copies are always `String`s and will live on the heap. AFAIK there are no stack strings (plain `str` which I guess is what you mean?) IIRC small fixed-size byte arrays (`[N; u8]`) are sometimes allocated on the stack depending on their size, but they are plain bytes and not full-fledged UTF8 strings. You can convert them into `String` but that would heap-allocate them. To expand on this: in Rust copy strinctly means there is a new owner that is tasked with deallocating that data once it goes out of scope. Move is the same but the ownership is transferred (thus the old owner is no longer responsible for deallocating anything) insted of having a new copy and an additional owner. Copy always results in a new object of the same type. & types are always borrowing. You can't copy into a reference since references are just borrowing of data, and owners of references won't deallocate anything (since they assume that, as long as they can hold an &, the data they reference is still alive, which is true because owners can't deallocate anything if there is a borrow in place). EDIT: I was wrong. As sibling comment says, you can convert a stack-allocated fixed-size array slice into `&str` with `str::from_utf8`. Check the last example here: https://doc.rust-lang.org/std/str/fn.from_utf8.html#examples https://doc.rust-lang.org/std/str/fn.from_utf8.html#examples
- deleted 9y ago[deleted]
- kaoD 9y ago> you can easily create e.g. a `&str` out of a `String` or a `&[T]` out of a `Vec<T>`. I think what op means is both `&str` and `&[T]` live on the stack, even if they reference slices of heap-allocated (or static) data.
- littlestymaar 9y ago> `Vec`, `String`, `Box`, `Rc`, and `Arc` are all heap allocated and that slices, arrays, `str`, `&T`, and `&mut T` are not. if T is Box<u32>, &T is heap allocated. Imho, it's more accurate to say that `&` and `&mut` doesn't cause heap allocation. </nitpick>
- steveklabnik 9y agoWith pointers, you have to differentiate where the pointer is from where the pointer is pointing to is. let b = Box::new(5); let r = &*b; Here, r is on the stack (or in a register), but is pointing to something on the heap, so "&T is heap allocated" feels wrong.
- littlestymaar 9y ago> Here, r is on the stack (or in a register), but is pointing to something on the heap, so "&T is heap allocated" feels wrong. What's the difference between `r` and `b` here ? (no pun intended)
- steveklabnik 9y agoThey both point to the same place. However, r doesn't have ownership, so when r goes out of scope, the memory won't be freed. However, b does, and so when b goes out of scope, the memory will be freed.
- littlestymaar 9y agook, but how does it link to where the value is allocated ? My question was, is it wrong to say that `b` isn't heap allocated either since : «Here, b is on the stack (or in a register), but is pointing to something on the heap.»
- steveklabnik 9y agoIt's wrong to say that b isn't heap allocated because it's not stored on the heap. &Ts can refer to something anywhere, heap or stack, and can also be anywhere, heap or stack. A Box<&i32> is going to have a &T on the heap.
- oconnor663 9y ago> whereas a Go slice is not generally used through a pointer I think this is referring to a syntactic difference more than an implementation difference. In Rust, a &[u8] (usually called a "slice" but maybe more technically a "slice reference") is a pointer + a length. This is basically the same as Go's []byte, which is a pointer + a length + a capacity. Rust also sometimes uses the [u8] type (without the &). This is an "exotic" type, in that it has no fixed size. It refers to the bytes inside the slice, but it's not really a pointer to them -- it is the bytes themselves, however many of them that might be. This mostly comes up when you're dealing with generic traits like AsRef or Deref, which will put the & back in all of their method signatures.
- saghm 9y agoI'm not sure that I'd agree that `[]byte` in Go is "basically the same as `&[u8]` in Rust; making a `&[u8]` will never cause a heap allocation, which is what I was trying to get at with my original comment. If I want to track down all of my heap allocations in my program in Rust, I can safely ignore all of my slices, whereas in Go, I have to carefully reason about each usage of one.
- oconnor663 9y agoGot it. In that sense a Go slice and a Go array pointer (a &[n]byte) could have the same effect, is that right? What I wanted to emphasize was that whether you're reading bytes through a Go []byte or a Rust &[u8], the same "number of hops" is happening at runtime.