13 ms·
Is there any practical programming language that is memory safe in its "entirety"? Python, for example, certainly is not. It has unsafe escape hatches (via ffi,
by burntsushi 4y ago
Is there any practical programming language that is memory safe in its "entirety"? Python, for example, certainly is not. It has unsafe escape hatches (via ffi, at the very least). Yet, everyone I know of says and thinks of Python as a memory safe language. I do as well.
> which makes it easier for developers to compartmentalize code to achieve memory-safety
The problem here is that this is incomplete. Many many many languages have achieved this before Rust. Where Rust is (somewhat although not entirely) unique is bringing this compartmentalization into a context that (mostly) lacks a runtime and garbage collection.
I have no problems calling Rust a "memory safe language" precisely because I have no problems calling Java or Python "memory safe languages." What matters isn't whether the language is "entirely" memory safe. What matters is what its default is. C and C++ are by default unsafe everywhere. Rust, Java, Python and many others are all safe by default everywhere. This notion is, IMO, synonymous with the more pithy "memory safe language."
- masklinn 4y ago> It has unsafe escape hatches (via ffi, at the very least). Yep, ctypes is part of the stdlib and lets you corrupt the VM on the fly. Fun stuff like changing the value of cached integers and everything. But ctypes being a terrifying pain in the ass, people tread very carefully around it. Cffi’s a lot better though it requires an external package. At the end of the day I think I’d be more enclined to bind through pyo3 or cython than write C in python (which is what ctypes has you do without even what little type system C has, to say nothing of -Wall -Weverything).
- rraval 4y ago> But ctypes being a terrifying pain in the ass, people tread very carefully around it. I'm not sure how much people treading carefully actually translates into safety in practice. CPython in particular has ad-hoc refcounting semantics where references can either be borrowed or stolen and you have to carefully verify both the documentation and implementation of functions you call because it's the wild west and nothing can be trusted: https://docs.python.org/3.9/c-api/intro.html#reference-count-details https://docs.python.org/3.9/c-api/intro.html#reference-count... This ad-hoc borrowed vs stolen references convention bleeds into cffi as well. If you annotate an FFI function as returning `py_object`, cffi assumes that the reference is stolen and thus won't increment the ref count. However, if that same function instead returns a `struct` containing a `py_object`, cffi assumes the reference is borrowed and will increment the ref count instead. So a harmless looking refactoring that changes a directly returned `py_object` into a composite `struct` containing a `py_object` is now a memory leak. Memory leaks aren't so bad (even Rust treats them as safe after the leakpocalypse [1] [2]). It's when you go the other way and treat what should have been a borrowed reference as stolen that real bad things happen. Here's a quick demo that deallocates the `None` singleton: Python 3.9.13 (main, May 17 2022, 14:19:07) [GCC 11.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.getrefcount(None) 4584 >>> import ctypes >>> ctypes.pythonapi.Py_DecRef.argtypes = [ctypes.py_object] >>> for i in range(5000): ... ctypes.pythonapi.Py_DecRef(None) ... 0 0 0 0 0 [snip] Fatal Python error: none_dealloc: deallocating None Python runtime state: initialized Current thread 0x00007f28b22b7740 (most recent call first): File "<stdin>", line 2 in <module> fish: Job 1, 'python3' terminated by signal SIGABRT (Abort) [1]: https://rust-lang.github.io/rfcs/1066-safe-mem-forget.html https://rust-lang.github.io/rfcs/1066-safe-mem-forget.html [2] https://cglab.ca/~abeinges/blah/everyone-poops/ https://cglab.ca/~abeinges/blah/everyone-poops/
- masklinn 4y ago> Here's a quick demo that deallocates the `None` singleton: As I said, you can trivially corrupt the VM through ctypes. However I don't think I've ever seen anyone wilfully interact with the VM for reasons other than shit and giggles. The few uses of ctypes I've seen were actual FFI (interacting with native libraries), and IME it's rare enough and alien enough that people tread quite carefully around that. I've actually seen a lot less care with the native library on the other side of the FFI call than on the FFI call itself (I've had to point issues with that just this morning during a code review, if anything the ctypes call was over-protected, otoh the update to the so's source had multiple major issues).
- dahfizz 4y agoI think a distinction can be made in that you never really need to use unsafe operations in python or Java. In rust, you need unsafe. Just about every data structure in the stdlib uses unsafe. I think it's fair to call Rust a memory safe language. But I don't think it's on the same tier as a fully managed language like python.
- burntsushi 4y agoI suppose reasonable people can disagree, but I don't think it's anywhere near as clear cut as you seem to be implying. You talk about data structures in std using unsafe, but you don't mention the heaps and piles of C code used to implement CPython's standard library. It's not like you need `unsafe` in Rust to build every data structure. I build oodles of data structures on top of the fundamental primitives provided by std without using any `unsafe` explicitly whatsoever. And it is not at all uncommon to write application code in Rust that doesn't utter `unsafe` at all. Even ripgrep has almost none of it. At the "application" level it has exactly two uses: one related to PCRE2 shenanigans and one related to the use of file backed memory maps. Both of those things are optional. Then there's another whole perspective here, which is that if you're using Rust in the first place, there's a non-trivial chance you're working on something "low level" that might require `unsafe`. Where as with Python you probably aren't doing "low level" work and just don't care much about perf within certain contexts. That has less (albeit not "nothing") to do with the design of the languages and more to do with the problems you're trying to solve. To be clear, I am not saying you're definitely wrong. But as someone who has written many tens of thousands of lines of both Rust and Python, I would put them on the same or very very close level in terms of memory safety personally. Certainly within the same tier.
- bluGill 4y ago> Is there any practical programming language that is memory safe in its "entirety"? This isn't possible. Eventually you are sitting at a block of memory and need to write the allocator. Maybe (like python) your allocator is written in C and you hide it, but there is always something that isn't memory safe sitting under your language. You could write a language for an actual Turing machine which since it has infinite memory is by definition memory safe. However as soon as you need to run on real hardware you have to work with something unsafe. You can of course prove a memory allocator is correct, but it would still have to use unsafe in rust. I supposed you could them implement this alloator in hardware, and make rust use that - but since this doesn't seem like it will happen I'm going with all languages have unsafe somewhere at the bottom.
- AnimalMuppet 4y agoThat depends on your definition of "practical" and "entirety". The article was about languages being used to implement Android. Clearly, no, you can't have an entirely memory safe language that can be used to implement Android, for the reason you said. But there's a wide gap between "practical for doing useful work of any kind" and "practical for implementing Android". Then, "entirely". What's "entirely"? Entirely until you get to library calls? Entirely until you get to OS calls? Entirely including the OS? If you include the OS then again, you are right for the reason you said. But if you exclude the OS, I'm not so certain.
- burntsushi 4y agoYes, exactly. That's why I asked the question: to drive out the point that the ontology the GP was using was probably not terribly useful. Although I did use the weasel word "practical" to narrow the field. If you don't limit yourself to general purpose languages, then I'm sure you can find one that is "entirely" safe.
- saagarjha 4y agoSure, but the language doesn’t have to expose it to you. There’s a bunch of other processes running on your system too aside from your program, but the OS prevents you from scribbling all over their address space.
- cesarb 4y ago> It has unsafe escape hatches (via ffi, at the very least). Playing devil's advocate, I can think of at least one language which has no escape hatches: Javascript running within a web page.
- burntsushi 4y agoYes, but that's not just a language. It's a language within a certain context. But it is a worthy mention.
- kaba0 4y agoIt is very easy to create a safe language. Hell, most brainfuck interpreter is likely completely safe, you allocate a large enough array and just iterate over its basic instructions that only ever modify that array and print a character. A Turing machine in itself can do no harm. The hard part comes at allowing it to do something useful, but only the parts I believe should be able to. E.g. plugging in file system access to our brainfuck interpreter will make it quite unsafe. Node for example does have C FFI.
- fiedzia 4y ago> Is there any practical programming language that is memory safe in its "entirety"? Whatever can be compiled to BPF meets this requirement. The price though is that it wouldn't be very useful.
- burntsushi 4y agoRight, that's why I used the word "practical."
- afdbcreid 4y agoSurely this is true, but I still have the feeling that libraries in Rust tend to have more unsafe code than Java, Python, C# or others, maybe even more unsafe code than needed. Perhaps this is related to the problem domain.
- burntsushi 4y agoMaybe. I don't know. You'd have to collect some data. > Perhaps this is related to the problem domain. Yes, I included that possibility in my comment here: https://news.ycombinator.com/item?id=33821787 https://news.ycombinator.com/item?id=33821787
- saagarjha 4y agoRust code frequently is used in a systems programming context, where it interoperates with unsafe code or needs to occasionally overrule the compiler to satisfy performance requirements.
- tialaramex 4y agoYou would definitely need to control for domain. A Rust library for some sort of mathematical modelling might well need no unsafe at all, while a Java library for controlling some hardware might soon turn into JNI talking to some C++ code and oops you're unsafe. In C# you need to reach for unsafe to do some of the stuff Rust can just do safely anyway. Did you know a C# struct with an array of 8 ints in it, doesn't actually have the eight ints baked inside the struct? It was easier in the CLR not to do that, so they didn't. Which means C# structs which look like a compact single object that surely lives in a single cache line don't actually do that in safe C#. You need unsafe.
- kaba0 4y ago> You need unsafe. You beed it for that feature. It is questionable whether you really want to mandate a special memory layout (because you can’t really do that even in Rust, you don’t have explicit control of struct alignments, paddings, order(!) )
- 4y ago
- saagarjha 4y agoJavaScript? It’s not typical to provide it with any access to unsafe APIs.
- burntsushi 4y agoSomeone already mentioned that. That only works if you restrict yourself to JavaScript in the browser. There's a huge ecosystem for using JavaScript outside of the browser.
- saagarjha 4y agoIt’s not very useful to talk about the memory safety of languages as a whole without looking at specific implementations. JavaScript in a browser is memory safe. JavaScript with access to /proc/mem is no longer memory safe. C on most hardware is not memory safe. C running on the abstract machine itself can be.
- burntsushi 4y agoThis looks like a comment in response to https://news.ycombinator.com/item?id=33820918 https://news.ycombinator.com/item?id=33820918 and not to me. The high level idea of my original rebuke was this idea that Rust was somehow lesser because it isn't "entirely" memory safe, and that its purpose was to divide safe from unsafe. But that really misses some very big points, because the programming language implementations used to build programs virtually everywhere are similarly not "entirely" memory safe, and many many many languages before Rust divided safe from unsafe. Notice how I modified my rebuke to include your caveat. Does my point change? Does the strength of my rebuttal change? Does anything materially change at all, other than using yet more word vomit to account for caveat? No, I don't think there's anything materially different other than more words. I tried to sidestep all of this by using the weasel word "practical." So next time I'll just say, "any practical non-sandboxed programming language." You might still chide me for confusing "programming language" with "implementation of programming language," but I've never much cared for that semantic because the ambiguity is almost always obviously resolvable from the context. > It’s not very useful to talk about the memory safety of languages as a whole without looking at specific implementations. Not sure I would agree with this, but it probably depends on what you mean. We can meaningfully discuss the memory safety properties of the programming languages (not just the implementations) of Rust, C and C++. I think you have to still acknowledge the practical realities of any particular implementation that others will use to build real programs, but I contend you need not do so more than what the language design does on its own already. Because languages aren't designed in a vacuum. Even if you can build an abstract machine, for example, C was not designed to be an abstract machine. It was designed to get stuff done in the real world, and the real world influenced that design. Same for Rust. Things like CHERI will potentially change this conversation quite a bit. I was even thinking about it when I wrote my original comment in this thread. But I think it is, at present, covered by the weasel word "practical." It isn't practical to use CHERI yet, as far as I know.
- lmm 4y agoStandard ML is entirely memory safe (some but not all implementations offer nonstandard escape hatches). I've heard someone here claim that the strictly standard version is a practical programming language, although I'm not sure I believe them.
- kaba0 4y agoWell, there is still an important difference between Java and Rust — are you driving with a guardrail on a field vs are you driving next to a cliffhanger. The JVM has well-defined bad execution as well, e.g. data racing is well-defined. Safe Rust does prevent data races statically, but if they do happen due to a bad unsafe block, you are entirely on your own. While memory safety can abruptly stop both processes, FFI is very rare in Java, it is an almost completely pure platform being Java all the ways down, so in my experience the former is safer from this aspect.
- burntsushi 4y agoI don't have much Java experience, so I'll have to take your word for it. But it's not completely obvious to me that you're correct. We've moved from an absolutist idea of memory safety to trying to build an implicit ontology of tiers of memory safety based on usage. Now you're talking about going and doing surveys of code and trying to measure the relative frequency of certain things and then using that to drive a tiered hierarchy of memory safety in programming languages. Sounds hard to do and you also haven't accounted for what problems are being solved in each language. I can pretty much decide to never ever use `unsafe` again, but I'll be leaving perf on the table. If I were writing Java, I would probably be fine with that. But I'm working on interesting problems that want the most perf possible, and so U do very occasionally justify `unsafe` when writing Rust.
- kaba0 4y agoAs others mentioned, there is no absolute safety, nor memory, no anything. The hardware can have bugs, the verification toolkit can have, or the properties to be verified could have been incorrectly specified to begin with. I’m just saying that corrupting the heap is much easier with Rust than with Java, and there is no coming back from heap corruption on a process basis, while most exceptional cases are recoverable by the JVM (hence the cliff analogy). And Java can have surprisingly good performance, especially in multi-threaded code that has a non-predictable allocation pattern (where ARC is just not too good) — if you want significant performance improvements you really have to go down the inline asm road, which you can do from anywhere.