3 ms·

strings.Split does return subslices of the original string. It's not making a copy of the string. Allocations still have to be made for slice that holds the res

by SPBS 3y ago

strings.Split does return subslices of the original string. It's not making a copy of the string. Allocations still have to be made for slice that holds the result of strings.Split. That's where the allocations are coming from.

saghm 3y ago
From looking into Rust's implementation of the iterator that's returned when splitting a string (which takes a bit of mental parsing, given the heavy use of macros: https://doc.rust-lang.org/src/core/str/iter.rs.html#731-747 https://doc.rust-lang.org/src/core/str/iter.rs.html#731-747), it seems like it has a reference to the original string and an index that it increments until it finds the next pattern to "split" on. This seems like something that would be pretty easy to do in Go as well, except Go doesn't really have any standard iterator API from what I can tell. This seems like a pretty clear argument for why having one is preferable to just returning slices everywhere though; even if you end up wanting to iterator through everything immediately, you avoid allocating unless you actually have to. In Rust, you can do this by just calling `collect` on the iterator, and although I suspect whoever implemented this API for Go wouldn't like the "magic" of it generically supporting collecting into whatever container you want, it feels like even just having an iterator with a method to collect into a slice and nothing else would be preferable.
- masklinn 3y ago
  > it feels like even just having an iterator with a method to collect into a slice and nothing else would be preferable. Most langages just have constructors or factory functions from iterator (or even iterable) to container. It’s a less generic (especially if constructors and functions are segregated) and / or efficient but otherwise works fine. No need for the iterator whatever (protocol, interface, trait) to have specific knowledge of individual collections.
- pstuart 3y ago
  There's going to be experimental support for built-in iteration in 1.22: https://github.com/golang/go/issues/61897 https://github.com/golang/go/issues/61897
  - ikiris 3y ago
    oh nice. i hope this lands.
    - pstuart 3y ago
      Being that its rsc's pet project I'm confident that it will. It will also bring in coroutines so this should make processing large streams of data more performant in an idiomatic way.

by SPBS 3y ago

saghm 3y ago

From looking into Rust's implementation of the iterator that's returned when splitting a string (which takes a bit of mental parsing, given the heavy use of macros: https://doc.rust-lang.org/src/core/str/iter.rs.html#731-747 https://doc.rust-lang.org/src/core/str/iter.rs.html#731-747), it seems like it has a reference to the original string and an index that it increments until it finds the next pattern to "split" on. This seems like something that would be pretty easy to do in Go as well, except Go doesn't really have any standard iterator API from what I can tell. This seems like a pretty clear argument for why having one is preferable to just returning slices everywhere though; even if you end up wanting to iterator through everything immediately, you avoid allocating unless you actually have to. In Rust, you can do this by just calling `collect` on the iterator, and although I suspect whoever implemented this API for Go wouldn't like the "magic" of it generically supporting collecting into whatever container you want, it feels like even just having an iterator with a method to collect into a slice and nothing else would be preferable.

masklinn 3y ago
> it feels like even just having an iterator with a method to collect into a slice and nothing else would be preferable. Most langages just have constructors or factory functions from iterator (or even iterable) to container. It’s a less generic (especially if constructors and functions are segregated) and / or efficient but otherwise works fine. No need for the iterator whatever (protocol, interface, trait) to have specific knowledge of individual collections.
pstuart 3y ago
There's going to be experimental support for built-in iteration in 1.22: https://github.com/golang/go/issues/61897 https://github.com/golang/go/issues/61897
- ikiris 3y ago
  oh nice. i hope this lands.
  - pstuart 3y ago
    Being that its rsc's pet project I'm confident that it will. It will also bring in coroutines so this should make processing large streams of data more performant in an idiomatic way.