7 ms·
Stupid alloc – What if memory allocation was annoying
- B1FF_PSUVM 3y agoSounds like a good feature for valgrind(1)
- eek2121 3y agoSo, a couple decades ago I was a C/C++ developer (oh and ASM!). I loved life, enjoyed the language. Then, I learned the higher level stuff. SQL, Visual Basic, ASP, C#, PHP, Python, Ruby, etc. I REALLY enjoyed that stuff, and it has fueled my career. This morning I found out about an interesting language called Hare, which kinda looks like a certain other language...then I saw this post. I hate you. I hate you for making me hate low level programming. I mean...I don't hate it. I don't even hate memory management...but YOU...YOU sir...
- VWWHFSfQ 3y agoI love Rust. But it seems like there's some kind of religion about allocations. Don't use String you fool, you want &str! Don't use Vec you fool, use &[]! It really puts people off. Because now they have to learn all about lifetimes and that's a fucking nightmare. My advice: Use String. Use Vec. Use Hashmaps. It's all still going to be a million time faster than whatever you were doing in Python, or Java, or Go, or Javascript, or whatever.
- tadfisher 3y agoThis isn't really a Rust problem. Even Java forces you to reckon with heap vs. stack allocation with its boxed types, and you'll get annoying warnings when you use Integer when int would do. This is just programming culture; given two ways of doing something, with one being harder but more performant, we gravitate to the harder one at all costs.
- deleted 3y ago[deleted]
- smabie 3y agoConsidering how mind boggling slow most software is, doesn't seem we are gravitating to the more performant options..
- jonhohle 3y agoThere are only two times to use boxed types in Java: when they’re in collection types that take objects (e.g. generics), or when the value needs to be nullable. The first is a language design flaw (imho) that can’t be easily fixed due to backwards compatibility desires. The second is almost always a boundary (DB to app, app to API, etc.) issue that should be resolved quickly, imho. In existing codebases I’ve worked with, often 90% or more boxed primitives are unnecessary, introduce unnecessary errors (assumed non-null refs vs. guaranteed non-null values) and performance penalties. They have a purpose, but there are libraries that address the first concern and Optional${Type} that are more expressive for the second.
- xxs 3y ago>or when the value needs to be nullable. I'd discount this part entirely. Using a boxes/non-primitive type for nullability is likely a design mistake. I can go in a full rant mode, in practical terms I tend to reject pull requests that do that.
- wredue 3y agoI mean. They gave example of where it might be commonplace, which is at API boundaries. Especially at DB boundaries, there’s decisions to make. If you put yourself in a position of your db having entirely non-nullable fields, you end up in one of two situations: 1) your whole db is key/value 2) you have magic values Making all key/value doesn’t solve the nullability issue at the code boundary if you’re representing as an object, so you’re left with magic values. I think we all know how nice magic values are. Ideology is the enemy of practicality.
- lmm 3y ago> you'll get annoying warnings when you use Integer when int would do. You can turn them off though, and most of the culture will support you in ignoring them if that's appropriate to your use case.
- winrid 3y agoI agree, but I don't know about a million :) https://programming-language-benchmarks.vercel.app/rust-vs-java https://programming-language-benchmarks.vercel.app/rust-vs-j... For giggles, also checkout the amount of generated machine code in godbolt with a simple Hello world that uses &str vs String. Its like 10x the instructions :P
- rolobio 3y agoThat link does not work for me?
- satvikpendem 3y agoNot sure why they posted an AMP link, here's the correct one: https://programming-language-benchmarks.vercel.app/rust-vs-java https://programming-language-benchmarks.vercel.app/rust-vs-j...
- winrid 3y agoI fixed it. Didn't trim enough of the URL on mobile..
- xxs 3y agoWhile I agree about million times, [micro]benchmarks that run seconds (or milliseconds) have extremely limited use, esp. when combined with non-compiled languages, that's effectively comparing compilation, guidance, and run time. I'd consider the entire site/process flawed.
- winrid 3y agoIt's useful in demonstrating there isn't a factor of a million difference, which was the discussion.
- return_to_monke 3y agoDo the java benchmarks include JVM startup?
- mostlylurks 3y agoIf you don't understand how lifetimes and associated aspects of the language work, you might very realistically run into situations where your code will be a million times slower than the python, java, etc. equivalent, not faster. For instance, copying a massive dynamically allocated container in every iteration of a hot loop instead of passing a reference (much less likely to do on accident than in C++, but still possible), if you don't understand value vs reference semantics, copying, moving, etc, since garbage collected languages don't force you to learn these things, and passing around references to large collections is cheap in those languages.
- astrange 3y agoThat sounds like the problem is the container itself contains value types rather than references, but then doesn't implement copy-on-write, which is a simple way to solve this.
- gpderetta 3y ago'Some people when confronted with container copy problems think "I know, I'll use COW". Now they have two problems.' With apologies to jwz.
- xxs 3y ago>For instance, copying a massive dynamically allocated container in every iteration Java has references only by design, so you can't do that.
- RGBCube 3y agoExactly, that's the point.
- raggi 3y agoso then you do that, making stuff on the heap, doing extra zero'ing and so on, and your debug build gets faster, and you call it good. then someone profiles a release build, removes the allocation and watches the optimizer wipe out the entire copy because all you were really doing is touching a single field. there's no general case here.
- mikebenfield 3y agoWell you're not going to be much of a Rust user if you don't learn the basics of lifetimes anyway. I don't mean to get all gatekeepy, but if someone's not comfortable with &str and &[T], they're missing half the point of the language and won't be able to use much library code anyway.
- smabie 3y agoIf you don't care about allocations why are you using Rust in the first place? Also if you are super sloppy with allocations I would expect something like Java or Go would be faster since GC is more efficient.
- ripley12 3y ago> If you don't care about allocations why are you using Rust in the first place? Because Rust excels on a lot of dimensions other than "systems programming"; Cargo is fantastic, the ecosystem of packages is great, it's one of the only languages that takes cross-platform support (including Windows) seriously, and it's got sum types. Or, as withoutboats put it: https://twitter.com/withoutboats/status/1658536526043570179 https://twitter.com/withoutboats/status/1658536526043570179
- pjmlp 3y agoAny ML influenced language has sum types. Including some that run on the JVM and 30 years of Java, makes Maven Central certainly beat cargo.io in package count. So there are options.
- Ygg2 3y ago> Any ML influenced language has sum types. > Including some that run on the JVM True. However, most that want to keep interoperability with Java, in order to get benefit of huge deposit of packages. That often means having to repeat Java's mistakes to maintain compatibility.
- kstrauser 3y agoI like to pre-allocate a &str buffer at launch, then create &str variables pointing into that buffer. I separate them with 0x00 bytes so I don’t have to store an extra size counter or invent a new data type.
- int0x2e 3y agoThat works for some types of code and fails for other types. If you have very long lived but memory heavy apps, allocations really matter...
- astrange 3y agoThere's other ways to avoid the problem. One is to just restart the whole process every so often. Also, copies are not bad and they're often a better tradeoff than microoptimizing all your lifetimes.
- constantcrying 3y ago>now they have to learn all about lifetimes If you do not want to learn about lifetimes do not use Rust. The advice is also terrible since inevitably you are going to use someone elses code, which does use slices.
- pkkm 3y ago> It's all still going to be a million time faster than whatever you were doing in Python, or Java, or Go, or Javascript, or whatever. Not at all. Systems languages don't magically make all code faster, they just help you write fast code. It's not that hard to tank performance if you're careless with allocations. I've seen some really slow C++, written by people who liked to use new everywhere like it's Java. If you want not to care about this stuff, I think that Java would fit your programming style better (or Kotlin, if you dislike Java's verbosity). The JVM is quite tolerant of doing lots of unnecessary allocations.
- dbetteridge 3y agoIf?
- andrewmcwatters 3y agoI love it. I’d also like to see a bellalloc which plays the system warning sound on every allocation.[1] [1]: https://en.m.wikipedia.org/wiki/Bell_character https://en.m.wikipedia.org/wiki/Bell_character
- jswrenn 3y agoIt already exists! https://github.com/cuviper/alloc_geiger https://github.com/cuviper/alloc_geiger
- andrewmcwatters 3y agoHahaha, perfect! Edit: Wow, I love that this does specifically that, too.
- LiamPowell 3y agoNot Rust, but it's very easy to wrap malloc under gcc for this kind of thing (or something more useful). Compile with --wrap=malloc and: void *__real_malloc(size_t); void *__wrap_malloc(size_t size) { ... return __real_malloc(size); }
- andrewmcwatters 3y agoThank you for sharing this!
- PaulDavisThe1st 3y agoGiven the implementation of this little thing, I'm mildly curious about how easy it is/will be for a C/C++ project/programmer to use these Rust crates. The other way around is relatively well defined and has been the subject of a lot of attention for several decades. Is the Rust-lib-called-from-C even feasible?
- lionkor 3y agoI feel like Rust was designed to fully replace old languages, and is now facing the same issue as every language before it - C and C++ interop (both ways). Zig nailed it, Rust fucked it up. Time will tell if thats big enough of an issue to warrant some bigger changes to fix it.
- zokier 3y agoThat is mischaracterization. Rust has been intended to be able to incrementally integrate to C/C++ codebases, of course most significantly integrating Rust/Servo components to Firefox/Gecko. See for example this rsvg presentation[1] and gstreamer blog post[2] for examples of the interop [1] https://viruta.org/docs/fmq-porting-c-to-rust.pdf https://viruta.org/docs/fmq-porting-c-to-rust.pdf [2] https://coaxion.net/blog/2016/05/writing-gstreamer-plugins-and-elements-in-rust/ https://coaxion.net/blog/2016/05/writing-gstreamer-plugins-a...
- exDM69 3y agoRust can be called from C the same way as C++ can. You need to export unmangled extern "C" symbols. Not quite frictionless but there are helper utilities for this. If the Rust code uses the "StupidAlloc" allocator, it should work as expected. There may be some constructor/destructor code involved with its usual complications. But it would not replace malloc/free in C code nor will it be compatible with C++ allocators for std containers.
- conradludgate 3y agoIt has been practical for ages. You define a wrapper function as no_mangle and extern "C", then compile the lib as a staticlib. When compiled, it produces a .a file you can link into your C application
- _dain_ 3y agoCool, I've always wanted to allocate memory using a network drive.
- amelius 3y agoNice, now I can grep through my memory as a non-root user. Fits with the Unix philosophy where everything can be done to an object through a file handle.
- Someone 3y agoI guess they will soon deprecate the non-interactive option because of a grave security issue: it uses predictable file names (it creates files in *std::env::temp_dir(), and uses static ALLOC_FILE_COUNTER: AtomicU64 = AtomicU64::new(0); for the variable part of file names) Also, fun is ensured if you run multiple programs in parallel using this allocator. AFAICT, if they share temp_dir, they’ll share memory. Performance tip: run it from a RAM disk.