4 ms·

I like handles too. Are there any programming languages that natively support them? Like, something that's physically a tuple of (data structure pointer, index

I like handles too. Are there any programming languages that natively support them? Like, something that's physically a tuple of (data structure pointer, index), but with the syntax allowing you to directly access it like any other reference. Bonus points for "get all handles to me" as an operation.

Macha 6y ago
If I'm understanding your ask correctly, this is basically what implementing Deref in Rust or operator-> in C++ gives you.
- jtolmar 6y ago
  That can give you the syntax! But what I'm curious about is more the implications of actually having this as a part of the language. Does it open up interesting space in optimization or memory safety?
  - eslaught 6y ago
    This is a bit like what Regent does. "Pointers" are really offsets into regions, which means that we can transparently move them around a distributed machine on the user's behalf. http://regent-lang.org/ http://regent-lang.org/
- derefr 6y ago
  Few things that expect a pointer do it by expecting an opaque dereferenceable thing. Most things that expect pointers actually expect a specific concrete pointer type (even if it's a pointer to an opaque thing.) IMHO, a language with "real suppport" for handles would use them everywhere, instead of pointers—i.e. make all places where pointers pass through "wide enough" for handles to be passed instead—with reduction to a pointer being a kind of WPO for cases where it's proven to be possible.
  - Blikkentrekker 6y ago
    Implementing `Deref` in Rust implements deref coercions, meaning that it can be used in any function that can accept an appropriate reference. This is purely syntactic sugar, and no different the compiler inserting `x.deref()` instead of `x` for function arguments. — it's simple q.o.l..
WalterBright 6y ago
The D language uses that to represent slices. It's a native type that has two fields - a pointer and a length. They're used like pointers, but with bounds checking. Goodbye buffer overflows. https://dlang.org/spec/arrays.html#dynamic-arrays https://dlang.org/spec/arrays.html#dynamic-arrays
- koolba 6y ago
  Not just good by overflows, it’s a better paradigm because you’re passing the context of the totality of the data to the target. It’s impossible to get the “wrong” length because it’s an intrinsic part of the reference.
  - WalterBright 6y ago
    I've proposed this as a backwards-compatible improvement to C: https://www.digitalmars.com/articles/C-biggest-mistake.html https://www.digitalmars.com/articles/C-biggest-mistake.html but there's been no uptake on it. I don't get this reluctance to exterminate the biggest source of security bugs in C code, but that sort of reluctance is why I started the D project.
    - MaulingMonkey 6y ago
      > I don't get this reluctance to exterminate the biggest source of security bugs in C code The forward thinking people who care about security bugs in C code - those ones who aren't reluctant - did exactly what I'd argue you've done: given up on C. It's selection bias in action. You started the D project. I'm RIIRing. Others prefer Python. Or perhaps one might use a language that compiles down into C, allowing you the best of both worlds - a decent programming language, and the ability to target whatever wretched excuse for a C89 compiler that has been forced upon you by whatever niche platform you're targeting. Fixing C's biggest mistake won't fix C. Fixing C's 10 biggest mistakes won't fix C. Fixing C's 100 biggest mistakes won't fix C, and by that point, it will no longer be C. There's an argument to be made for incremental improvements, so I won't discourage your efforts, but it's also the kind of pragmatic, boring, middle ground, half answer that won't make anyone outright excited either.
dataflow 6y ago
> (data structure pointer, index) One problem you encounter if you have something like this is that you end up with unnecessary states in your program. Like you start storing these in arrays and then frequently find the data structure pointers for all your objects are the same, because the owner of that array already assumes the same underneath data structure. Doubly (or multiply) so if the object itself has another pointer to its owning(?) data structure inside it. This inserts degrees of freedom into your program that you don't have any use for, and probably wouldn't have otherwise. What's the downside here? It's hard to predict sometimes, but in general, there's increased potential for redundant fields to go out of sync. (And if your arrays are large, it can end up being a giant waste of memory too.) One thing I've gradually learned over time is that one of the best mechanisms for preventing your program from entering an erroneous state is to make it so that erroneous states cannot exist in the first place. If something can't go wrong, it won't go wrong. That means avoiding degrees of freedom in your program that you don't actually need. Which is precisely the opposite of what we have here, where we've redundantly replicated the same information copied across lots of objects. And there are also other side-effects possible here, e.g., now you'll run into trouble if you ever decide to use the handles atomically (since now it's 2 pointers' worth of data), and you might also run into trouble with your IDE given that you can no longer use some built-in facilities for OOP, debugging, and/or visualization. There are likely more that I'm failing to remember off the top of my head. Of course there are some advantages too, but my point is: it's hardly an obvious design choice, and it can work out to your disadvantage in ways you don't expect.
- loup-vaillant 6y ago
  OP specifically uses that redundancy to prevent use-after-free errors. That redundancy is limited to the handle, and checked at every dereference (in the get_pointer() so you don't forget to check). Desyncs between a handle and the object it points to ought not to be possible at this point. If you remove the redundancy, you have to give up on use-after-free checks. If you want to prevent use-after-free anyway, you need to use a simpler allocation pattern (stack or arena), or fall back to heap allocated pointers (smart pointers in C++, various kind of references in Rust…). But heap allocated pointers were precisely what we were trying to avoid in the first place, most notably because they're not cache friendly, cause memory fragmentation, make leaks harder to detect, and in many cases simply take up much more space than handles (remember, pointers are 8 bytes nowadays). Using handles atomically is probably a consequence of a very bad idea: threads with shared mutable state and locks. As much as humanely possible, you don't want to share any mutable memory to begin with. Send messages instead, it's less error prone, and with current computers, it's often faster as well. See http://ithare.com/ http://ithare.com/ on (re)actors for the gory details. About debugging: some people read code by running it in the default step by step debugger provided by their IDE. I don't understand that way of thinking, so I don't bother catering to it. My code then is optimised for readability without special tools. Which I hardly ever need in practice. (Of course, sometimes tools like GDB or Valgrind do save my life, but that's a rare enough occurrence that losing a bit of time when that happens is not a big deal.) Last quip: the existence of OOP tools is hardly a criterion when choosing how OOP your program should be. If pointer fest hurts performance too much, or if your domain (there are many) is not suited to OOP, tools won't compensate for the impedance mismatch.
Blikkentrekker 6y ago
Rust allows one to implement custom referencing on any datatype as one chposes, so the effect is the same. It's not that important, however, it simply makes `x` syntactic sugar for `x.deref()`.
ajuc 6y ago
PL/SQL :)
fulafel 6y ago
In a safer language you can use normal language provided references, no? The HotSpot JVM implements compressed object references that work conceptually a bit like this under the hood, but it's invisible semantically.

Macha 6y ago

If I'm understanding your ask correctly, this is basically what implementing Deref in Rust or operator-> in C++ gives you.

jtolmar 6y ago
That can give you the syntax! But what I'm curious about is more the implications of actually having this as a part of the language. Does it open up interesting space in optimization or memory safety?
- eslaught 6y ago
  This is a bit like what Regent does. "Pointers" are really offsets into regions, which means that we can transparently move them around a distributed machine on the user's behalf. http://regent-lang.org/ http://regent-lang.org/
derefr 6y ago
Few things that expect a pointer do it by expecting an opaque dereferenceable thing. Most things that expect pointers actually expect a specific concrete pointer type (even if it's a pointer to an opaque thing.) IMHO, a language with "real suppport" for handles would use them everywhere, instead of pointers—i.e. make all places where pointers pass through "wide enough" for handles to be passed instead—with reduction to a pointer being a kind of WPO for cases where it's proven to be possible.
- Blikkentrekker 6y ago
  Implementing `Deref` in Rust implements deref coercions, meaning that it can be used in any function that can accept an appropriate reference. This is purely syntactic sugar, and no different the compiler inserting `x.deref()` instead of `x` for function arguments. — it's simple q.o.l..

WalterBright 6y ago

The D language uses that to represent slices. It's a native type that has two fields - a pointer and a length. They're used like pointers, but with bounds checking. Goodbye buffer overflows. https://dlang.org/spec/arrays.html#dynamic-arrays https://dlang.org/spec/arrays.html#dynamic-arrays

koolba 6y ago
Not just good by overflows, it’s a better paradigm because you’re passing the context of the totality of the data to the target. It’s impossible to get the “wrong” length because it’s an intrinsic part of the reference.
- WalterBright 6y ago
  I've proposed this as a backwards-compatible improvement to C: https://www.digitalmars.com/articles/C-biggest-mistake.html https://www.digitalmars.com/articles/C-biggest-mistake.html but there's been no uptake on it. I don't get this reluctance to exterminate the biggest source of security bugs in C code, but that sort of reluctance is why I started the D project.
  - MaulingMonkey 6y ago
    > I don't get this reluctance to exterminate the biggest source of security bugs in C code The forward thinking people who care about security bugs in C code - those ones who aren't reluctant - did exactly what I'd argue you've done: given up on C. It's selection bias in action. You started the D project. I'm RIIRing. Others prefer Python. Or perhaps one might use a language that compiles down into C, allowing you the best of both worlds - a decent programming language, and the ability to target whatever wretched excuse for a C89 compiler that has been forced upon you by whatever niche platform you're targeting. Fixing C's biggest mistake won't fix C. Fixing C's 10 biggest mistakes won't fix C. Fixing C's 100 biggest mistakes won't fix C, and by that point, it will no longer be C. There's an argument to be made for incremental improvements, so I won't discourage your efforts, but it's also the kind of pragmatic, boring, middle ground, half answer that won't make anyone outright excited either.

dataflow 6y ago

> (data structure pointer, index) One problem you encounter if you have something like this is that you end up with unnecessary states in your program. Like you start storing these in arrays and then frequently find the data structure pointers for all your objects are the same, because the owner of that array already assumes the same underneath data structure. Doubly (or multiply) so if the object itself has another pointer to its owning(?) data structure inside it. This inserts degrees of freedom into your program that you don't have any use for, and probably wouldn't have otherwise. What's the downside here? It's hard to predict sometimes, but in general, there's increased potential for redundant fields to go out of sync. (And if your arrays are large, it can end up being a giant waste of memory too.) One thing I've gradually learned over time is that one of the best mechanisms for preventing your program from entering an erroneous state is to make it so that erroneous states cannot exist in the first place. If something can't go wrong, it won't go wrong. That means avoiding degrees of freedom in your program that you don't actually need. Which is precisely the opposite of what we have here, where we've redundantly replicated the same information copied across lots of objects. And there are also other side-effects possible here, e.g., now you'll run into trouble if you ever decide to use the handles atomically (since now it's 2 pointers' worth of data), and you might also run into trouble with your IDE given that you can no longer use some built-in facilities for OOP, debugging, and/or visualization. There are likely more that I'm failing to remember off the top of my head. Of course there are some advantages too, but my point is: it's hardly an obvious design choice, and it can work out to your disadvantage in ways you don't expect.

loup-vaillant 6y ago
OP specifically uses that redundancy to prevent use-after-free errors. That redundancy is limited to the handle, and checked at every dereference (in the get_pointer() so you don't forget to check). Desyncs between a handle and the object it points to ought not to be possible at this point. If you remove the redundancy, you have to give up on use-after-free checks. If you want to prevent use-after-free anyway, you need to use a simpler allocation pattern (stack or arena), or fall back to heap allocated pointers (smart pointers in C++, various kind of references in Rust…). But heap allocated pointers were precisely what we were trying to avoid in the first place, most notably because they're not cache friendly, cause memory fragmentation, make leaks harder to detect, and in many cases simply take up much more space than handles (remember, pointers are 8 bytes nowadays). Using handles atomically is probably a consequence of a very bad idea: threads with shared mutable state and locks. As much as humanely possible, you don't want to share any mutable memory to begin with. Send messages instead, it's less error prone, and with current computers, it's often faster as well. See http://ithare.com/ http://ithare.com/ on (re)actors for the gory details. About debugging: some people read code by running it in the default step by step debugger provided by their IDE. I don't understand that way of thinking, so I don't bother catering to it. My code then is optimised for readability without special tools. Which I hardly ever need in practice. (Of course, sometimes tools like GDB or Valgrind do save my life, but that's a rare enough occurrence that losing a bit of time when that happens is not a big deal.) Last quip: the existence of OOP tools is hardly a criterion when choosing how OOP your program should be. If pointer fest hurts performance too much, or if your domain (there are many) is not suited to OOP, tools won't compensate for the impedance mismatch.

Blikkentrekker 6y ago

Rust allows one to implement custom referencing on any datatype as one chposes, so the effect is the same. It's not that important, however, it simply makes `x` syntactic sugar for `x.deref()`.

ajuc 6y ago

PL/SQL :)

fulafel 6y ago

In a safer language you can use normal language provided references, no? The HotSpot JVM implements compressed object references that work conceptually a bit like this under the hood, but it's invisible semantically.