5 ms·

> You should be able to forge a number into a pointer, since that’s how hardware works. It's a C-centric view of the world and I wonder what old school FORTRAN

> You should be able to forge a number into a pointer, since that’s how hardware works. It's a C-centric view of the world and I wonder what old school FORTRAN77 people or lispers would think of this. Anyway, I think the right to do this should be a privilege of the compiler. As soon as you claim this right, you abandon all possible support she can give you in battling all kinds of silly mistakes.

alxlaz 8y ago
I'm not an "old-school lisper", but I have done some Lisp development related to embedded systems. The phrasing may be somewhat unfortunate but it ultimately does describe how hardware works: you need to write something at a numerical address that you find in the reference guide or the datasheet. At the end of the line, you do have to forge a number into a pointer. In James Micken's words, "You can’t just place a LISP book on top of an x86 chip and hope that the hardware learns about lambda calculus by osmosis." "Higher-level" languages (like Lisp) can help you in other regards. They can include proper primitives (or give you the proper tools to write the proper primitives yourself) to ensure that direct-to-hardware access is safe, can give you better tools to manipulate DMA buffers (e.g. proper support for coroutines when manipulating double-buffered data is cool), can help you write more generic state-applying and state-reading code (i.e. code for "turn these bytes from this circular buffer into this structure" or "take this structure that represents a peripheral's current config and use it to write the proper magic numbers in the right registers"). They can offer you better semantics for translating human-readable(-ish) configuration into the right stream of bytes, or manipulate peripherals in real-time. They can help you write a better VM (or their runtime can outright be the better VM you need). But at the end of the day, writing to the config space of a memory-mapped device is still going to consist of taking a number (or building it from several numbers) and writing bytes at that address, no matter what language you're using. Better runtime support for safe operation under this scenario, better semantics -- they're all important, but what the author expresses is not C-centric in any way.
Athas 8y ago
Pointers are not even necessary for writing operating systems or memory allocators. The Oberon system writes those low-level components using intrinsic peek/poke functions (like some ancient BASIC!) that are recognised specially by the compiler and turned into direct memory modifications. This is clearly just as unsafe as pointers (probably more so), but it means you don't generally pollute the language to support some very specialised code. The rest of Oberon is memory-safe and garbage-collected (with the garbage collector also written in Oberon).
- josteink 8y ago
  > The Oberon system writes those low-level components using intrinsic peek/poke functions (like some ancient BASIC!) that are recognised specially by the compiler and turned into direct memory modifications. This is clearly just as unsafe as pointers (probably more so), but it means you don't generally pollute the language to support some very specialised code. How is "val = PEEK adr" and "POKE adr, val" any less polluting than allowing "val = &adr" and "&adr = val" in select places?
  - pjmlp 8y ago
    It shouts to you, "Take care something dangerous going on". Hieroglyphs are easy to pass by.
    - josteink 8y ago
      Lots of languages (like C#) allow pointers, but only inside blocks explicitly declared to be unsafe. void DoSomeUnsafeStuffHere() { // regular code here. pointers verboten. unsafe { // pointery stuff here } } I think that is equally clear, if not even more. And again: I don't think this is any more polluting than weird out of place PEEK/POKE statements.
      - eterps 8y ago
        What makes it unclear is exactly this 'pointery stuff'. There's nothing inherently unsafe about strongly typed pointers, but that's not the same as direct memory modification (which is obviously unsafe).
        josteink 8y ago
        Strongly typed pointers is just another name for direct memory access. The following is perfectly typesafe: SuperClass[] items = GetArrayOfSubClass(); Now if we try to do unsafe things to items, we may end up accidentally accessing memory we shouldn't. unsafe { for (int i=0; i < items.Length; i++) { var currentItem = &items[i*sizeof(SuperClass)]; } } This may or may not work, based on the runtime memory-layout of the subclass. Basically the presence of unsafe {} means that beyond this point correctness cannot be guaranteed by the compiler. That's in fact what the keyword is for. And that's the marker you are looking for. No need to go digging deeper into the code looking for the actual pointers themselves.
pandaman 8y ago
I am not sure what you mean, it's not like we cast integers to pointers by hand is it? The compiler does the casting. And sure, there is usually no support from the compiler when you are programming any device other than the CPU but these devices need to be programmed. For you to read this message somebody had to program the display device and for you to even be able to access this web site somebody had to program a whole bunch of things starting from your NIC, going through various routers and to the different NIC on the HN's server. No compiler had been able to do this so far.
m_mueller 8y ago
As someone who likes Fortran90+ for numerics in HPC, I'd put it this way: * Pointers are required mainly so you can do pointer swaps and be sure there is no unnecessary allocation (in simulations this often means being sure there's no 10-100GB allocation each timestep, which is where it really matters). * Pointer math is unnecessary and only there because C has no/poor support of multidimensional arrays. Use a reasonable language like Julia or Fortran instead. In fact, the potential for pointer math has a big negative impact on performance since compilers have to assume aliasing. The __restrict solution is clunky in C and completely non-standard in C++, Fortran does not require this at all (no aliasing allowed except where explicitly stated). * For almost all usecases (except swaps) a semi-managed approach like Fortran's allocatables or Objective-C's ARC or even retain/release seems to me the most straight-forward.
- saagarjha 8y ago
  > Pointers are required mainly so you can do pointer swaps and be sure there is no unnecessary allocation Many languages are getting around this issue by hiding pointers in their type system, performing copy-on-write, and taking the decision of whether allocations are performed on the stack or heap out of the hands of the programmer.
  - m_mueller 8y ago
    > out of the hands of the programmer see, and that's where these languages become really clunky for HPC purposes. A compiler/runtime with HPC support (and I think also systems programming) should provide (a) performant defaults with safety as a second (but still high) priority and (b) the ability for programmers to go and set things a certain way when it is clear to them how things should be implemented on the machine. Otherwise, in large applications, there's just too many moving parts that the compiler (optimization) can mess up. As long as we have no AGI baked into compilers it's an illusion to think that compilers will do the job of HPC engineers, so if you take away these tools we just have to look elsewhere.
    - pjmlp 8y ago
TheCondor 8y ago
Every operating system I’ve worked on has had a tiny few lines of relatively isolated code for this sort of thing. Even some crappy RTOSes, it seems it is natural to want to abstract it. You need to forge a number into a physical address pointer, not just a pointer. Look at some Linux drivers, they will have some offsets defined but they don’t just create a void* and assign it to a memory location, they use macros and helper functions to fix things up. They can even check if the address is in reserved io spaces and do things like that. There are also helpers to write the bytes to those addresses. It’s a tiny amount of code that is usually already abstracted. I don’t know. Seems like a lot of kernels do extra things so driver writers can write fairly normal looking C and produce fairly safe drivers without just arbitrarily assigning pointers addresses from numbers. And even if you use C as your comparison, those helpers will often be written with some assembly language (read C isn’t completely good enough for the task all by itself) I don’t know of it existing but a kernel in go with ref counting GC and a handful of cgo primitives seems very possible and maybe even delightful to work on.

alxlaz 8y ago

I'm not an "old-school lisper", but I have done some Lisp development related to embedded systems. The phrasing may be somewhat unfortunate but it ultimately does describe how hardware works: you need to write something at a numerical address that you find in the reference guide or the datasheet. At the end of the line, you do have to forge a number into a pointer. In James Micken's words, "You can’t just place a LISP book on top of an x86 chip and hope that the hardware learns about lambda calculus by osmosis." "Higher-level" languages (like Lisp) can help you in other regards. They can include proper primitives (or give you the proper tools to write the proper primitives yourself) to ensure that direct-to-hardware access is safe, can give you better tools to manipulate DMA buffers (e.g. proper support for coroutines when manipulating double-buffered data is cool), can help you write more generic state-applying and state-reading code (i.e. code for "turn these bytes from this circular buffer into this structure" or "take this structure that represents a peripheral's current config and use it to write the proper magic numbers in the right registers"). They can offer you better semantics for translating human-readable(-ish) configuration into the right stream of bytes, or manipulate peripherals in real-time. They can help you write a better VM (or their runtime can outright be the better VM you need). But at the end of the day, writing to the config space of a memory-mapped device is still going to consist of taking a number (or building it from several numbers) and writing bytes at that address, no matter what language you're using. Better runtime support for safe operation under this scenario, better semantics -- they're all important, but what the author expresses is not C-centric in any way.

Athas 8y ago

Pointers are not even necessary for writing operating systems or memory allocators. The Oberon system writes those low-level components using intrinsic peek/poke functions (like some ancient BASIC!) that are recognised specially by the compiler and turned into direct memory modifications. This is clearly just as unsafe as pointers (probably more so), but it means you don't generally pollute the language to support some very specialised code. The rest of Oberon is memory-safe and garbage-collected (with the garbage collector also written in Oberon).

josteink 8y ago
> The Oberon system writes those low-level components using intrinsic peek/poke functions (like some ancient BASIC!) that are recognised specially by the compiler and turned into direct memory modifications. This is clearly just as unsafe as pointers (probably more so), but it means you don't generally pollute the language to support some very specialised code. How is "val = PEEK adr" and "POKE adr, val" any less polluting than allowing "val = &adr" and "&adr = val" in select places?
- pjmlp 8y ago
  It shouts to you, "Take care something dangerous going on". Hieroglyphs are easy to pass by.
  - josteink 8y ago
    Lots of languages (like C#) allow pointers, but only inside blocks explicitly declared to be unsafe. void DoSomeUnsafeStuffHere() { // regular code here. pointers verboten. unsafe { // pointery stuff here } } I think that is equally clear, if not even more. And again: I don't think this is any more polluting than weird out of place PEEK/POKE statements.
    - eterps 8y ago
      What makes it unclear is exactly this 'pointery stuff'. There's nothing inherently unsafe about strongly typed pointers, but that's not the same as direct memory modification (which is obviously unsafe).
      - josteink 8y ago
        Strongly typed pointers is just another name for direct memory access. The following is perfectly typesafe: SuperClass[] items = GetArrayOfSubClass(); Now if we try to do unsafe things to items, we may end up accidentally accessing memory we shouldn't. unsafe { for (int i=0; i < items.Length; i++) { var currentItem = &items[i*sizeof(SuperClass)]; } } This may or may not work, based on the runtime memory-layout of the subclass. Basically the presence of unsafe {} means that beyond this point correctness cannot be guaranteed by the compiler. That's in fact what the keyword is for. And that's the marker you are looking for. No need to go digging deeper into the code looking for the actual pointers themselves.

pandaman 8y ago

I am not sure what you mean, it's not like we cast integers to pointers by hand is it? The compiler does the casting. And sure, there is usually no support from the compiler when you are programming any device other than the CPU but these devices need to be programmed. For you to read this message somebody had to program the display device and for you to even be able to access this web site somebody had to program a whole bunch of things starting from your NIC, going through various routers and to the different NIC on the HN's server. No compiler had been able to do this so far.

m_mueller 8y ago

As someone who likes Fortran90+ for numerics in HPC, I'd put it this way: * Pointers are required mainly so you can do pointer swaps and be sure there is no unnecessary allocation (in simulations this often means being sure there's no 10-100GB allocation each timestep, which is where it really matters). * Pointer math is unnecessary and only there because C has no/poor support of multidimensional arrays. Use a reasonable language like Julia or Fortran instead. In fact, the potential for pointer math has a big negative impact on performance since compilers have to assume aliasing. The __restrict solution is clunky in C and completely non-standard in C++, Fortran does not require this at all (no aliasing allowed except where explicitly stated). * For almost all usecases (except swaps) a semi-managed approach like Fortran's allocatables or Objective-C's ARC or even retain/release seems to me the most straight-forward.

saagarjha 8y ago
> Pointers are required mainly so you can do pointer swaps and be sure there is no unnecessary allocation Many languages are getting around this issue by hiding pointers in their type system, performing copy-on-write, and taking the decision of whether allocations are performed on the stack or heap out of the hands of the programmer.
- m_mueller 8y ago
  > out of the hands of the programmer see, and that's where these languages become really clunky for HPC purposes. A compiler/runtime with HPC support (and I think also systems programming) should provide (a) performant defaults with safety as a second (but still high) priority and (b) the ability for programmers to go and set things a certain way when it is clear to them how things should be implemented on the machine. Otherwise, in large applications, there's just too many moving parts that the compiler (optimization) can mess up. As long as we have no AGI baked into compilers it's an illusion to think that compilers will do the job of HPC engineers, so if you take away these tools we just have to look elsewhere.
  - pjmlp 8y ago

TheCondor 8y ago

Every operating system I’ve worked on has had a tiny few lines of relatively isolated code for this sort of thing. Even some crappy RTOSes, it seems it is natural to want to abstract it. You need to forge a number into a physical address pointer, not just a pointer. Look at some Linux drivers, they will have some offsets defined but they don’t just create a void* and assign it to a memory location, they use macros and helper functions to fix things up. They can even check if the address is in reserved io spaces and do things like that. There are also helpers to write the bytes to those addresses. It’s a tiny amount of code that is usually already abstracted. I don’t know. Seems like a lot of kernels do extra things so driver writers can write fairly normal looking C and produce fairly safe drivers without just arbitrarily assigning pointers addresses from numbers. And even if you use C as your comparison, those helpers will often be written with some assembly language (read C isn’t completely good enough for the task all by itself) I don’t know of it existing but a kernel in go with ref counting GC and a handful of cgo primitives seems very possible and maybe even delightful to work on.