4 ms·
Adding manual arena memory management seems like a weird choice for a supposedly memory-safe and GC'd language where this is supposed to be an implementation is
by astrange 4y ago
Adding manual arena memory management seems like a weird choice for a supposedly memory-safe and GC'd language where this is supposed to be an implementation issue.
I mean, you should hear the screams from Rust/Go people about how bad C is, and now you're just going to do it?
- slimsag 4y ago> now you're just going to do it? "you can’t import it by default", "you probably shouldn’t be using it at all", "experimental arena package", and 'you must opt-in to even be able to use it via an GOEXPERIMENT environment variable' As for screams from Rust/Go people about how bad C is, it said: > This is highly efficient, but also highly dangerous. What if the programmer makes a mistake [...] > > To mitigate the risk of these kinds of bugs, the arena package will deliberately cause a panic if can detect someone reusing memory after it has been freed. and goes on to explain that each arena has its own unique/distinct address space, so that if a pointer to an object still exists and is dereference it will get a memory access fault, causing the Go program to terminate, with an error message specifying that arena as the cause of the problem. Sounds like a pretty reasonable/cautious experiment to me.
- lnxg33k1 4y agoYeah I think a language is safe or unsafe according to the ecosystem they live in. A project a developer works on is hardly 100% self-developed and depends on library and third party code, so I understand you have experiments you need to enable, but once it starts spreading in libraries, I think the whole safety guarantee just goes out of the window? As a rust student myself, even the unsafe{} block is stupid
- kaba0 4y agoRust’s unsafe is still much safer than your average C code. The borrow checker checks inside unsafe blocks as well, you can just circumvent it through pointers. But the great thing with unsafe blocks is that you get to create a safe API abstraction on top. You manually verify closely the unsafe parts, and if you got that short part right you can use the safe wrapper wherever you want safely.
- cormacrelf 4y ago> and goes on to explain that each arena has its own unique/distinct address space That does not seem to be how it is implemented now. Try the example use-after-free on https://uptrace.dev/blog/posts/go-memory-arena.html https://uptrace.dev/blog/posts/go-memory-arena.html under 'Address Sanitizer'. There is no error at all when you dereference, it silently keeps working as freed pointers often do (until they don't). Maybe they will add the virtual address space thing later. Edit: the issue (https://github.com/golang/go/issues/51317 https://github.com/golang/go/issues/51317) in which that comment was made also has some people referring to the arena being freed "when the runtime gets around to it" rather than immediately. I can't imagine why the virtual address space wouldn't be made poisonous inside the arena.Free method, but it's possible they do do it later. So your pointers maybe do stop working at an indeterminate time. Sounds pretty much like C. I even tried adding a few `runtime.GC()` calls before dereferencing but to no avail, couldn't get it to crash. The Java 20 implementation has no such issues (https://gist.github.com/cormacrelf/8ddd3cc1b086e4ade93c029a98327d86 https://gist.github.com/cormacrelf/8ddd3cc1b086e4ade93c029a9...), partly because there is no API for allocating a generic T in the arena, so all dereferences are through the managed MemorySegment API. It only seems to offer raw memory, which you can use to implement arrays of unboxed C-style structs, which is great for FFI and network buffers etc. This would be a good tradeoff for Go as well, surely.
- klodolph 4y agoIf you want network buffers in Go, you can just use []byte. The Go GC is designed to make this work well, so nobody in Go is asking for a separate buffer type like you have in Java. In Java you have Buffer in the first place because there are good reasons why you might not want to use byte[]. Unboxed C types in Go can be done with cgo (it's not necessarily pleasant, but it works).
- kaba0 4y ago> why you might not want to use byte[] Such as? Java buffers use byte arrays by default as underlying storage.
- 4y ago
- latch 4y agoSeems to fit the goroutine model well: initialize the arena at goroutine start, destroy it on goroutine end (I assume this happens automatically when it falls out of scope). Like Erlang's process-specific GC. It would be nice for some debug/testing mechanism (akin to -race, or zig's detectLeaks) to proactively verify nothing has outlived the arena.
- zozbot234 4y ago> I mean, you should hear the screams from Rust/Go people about how bad C is, and now you're just going to do it? Wait until you learn how Rust now has a crate that implements seamless concurrent tracing GC, just like Go... https://redvice.org/2023/samsara-garbage-collector/ https://redvice.org/2023/samsara-garbage-collector/
- astrange 4y agoSounds fine to me. That's safe. But if you have a compacting GC, you shouldn't need an arena. You should be able to get away with fixing the GC.
- klodolph 4y ago> You should be able to get away with fixing the GC. Practically speaking, "fixing the GC" is just a really, super hard problem. You're making a tradeoff between memory usage, CPU usage, pause times, and the performance of various features like pinned objects. It makes sense that you may want different tradeoffs in different parts of your program. "Fixing the GC" is right up there with "just make a smart enough compiler" type sentiment that gets you in trouble--it's easy to say that you desire a better GC or better compiler, but when you actually try to make a better GC or better compiler, you find out that it doesn't solve the problems you were hoping it would solve. Go's GC is tuned aggressively for short pause times. A lot of people actually want short pause times in their GCs, enough so that it's a selling point for Go. In Go, those short pause times were achieved partly by sacrificing CPU efficiency. You can't just go in and "fix" the CPU efficiency problems, but you can make new APIs that give you an escape hatch.
- jchw 4y agoI mean, it's opt-in. If you really want to crap your pants, in today's Go, you can import C and call malloc and free, then import unsafe and make pointers into the void. The point of memory safe programming languages is NOT that they entirely disallow all unsafe operations, it's that the language has a proper model for memory safety. In Rust, unsafe code needs to be in unsafe blocks. In Go, unsafe code is only possible by importing unsafe or C. Adding more unsafe tools to Go or Rust do not diminish their safety guarantees. In fact, when Go and Rust add new unsafe features, they generally do so in a fashion that is much more constrained and has better defined edges than traditional methods, so that it's easier to use them correctly and easier to detect misuses. For example, the borrow checker still runs in Rust unsafe blocks; you still have to circumvent the borrow checker even once you're in unsafe. As for this arena package, it seems to have a few safety mechanisms, and hell, I don't even think it's off the table that it could be made entirely memory safe. This is especially true since memory safe does not mean "free of runtime errors", so all that is needed is to be able to detect the error conditions without allowing potentially undefined behavior to occur first. Programming language experiments like Go's new arena package are good; they allow exploring what you can do to improve old concepts.
- heywhatupboys 4y ago> In Go, unsafe code is only possible by importing unsafe or C. of course, this is completely wrong, for anyone reading this. that is simply not how systems work
- the-smug-one 4y agoOK, do you intend to say why?
- thiht 4y agoHow do "systems" (whatever you're talking about) work then?
- mikedelago 4y agoI think they are speaking without the context that "unsafe = lack of memory safety" and with the context that "unsafe = you can still drop the database"
- beltsazar 4y ago> Adding manual arena memory management seems like a weird choice for a supposedly memory-safe and GC'd language where this is supposed to be an implementation issue. Yes, it's because Go's GC is primarily tuned for latency at the cost of (memory allocation) throughput. Furthermore, there are only a few knobs for Go's GC. If Go's GC is implemented a bit differently and has a few more knobs so that it can be optimized for memory allocation throughput, maybe Go doesn't need an arena library. I don't even know other GC-ed languages that have/need an arena library because their GC is customizable enough.
- kubb 4y agoGarbage collector is too slow, so they're reintroducing malloc and free. Life writes the best scenarios.