16 ms·

Pointer Free Programming and the Future of Nim

205 points by mindB 9y ago

pavlov 9y ago
I’m not a huge fan of the modern C++ style that obsessively avoids null pointers and instead uses object references that are created in an invalid/empty state. With a pointer there is a single way to represent “nothing”, regardless of what the object represents. With a reference I have to go read the API to understand the “nothingness” states. Also, when you’re thinking in pointers, it’s easy to add levels of indirection with the same mental tools (pointers to pointers, etc.) Personally I find it’s easier to solve problems with a limited orthogonal toolset than a sprawling array of marginally differently balanced optimizations, which is what most of C++ has become.
- NiceGuy_Ty 9y ago
  While null may be the simplest way to represent nothing between a large amount of data types, I think the biggest issue (beyond how far null can propagate from the source in error scenarios) is that it's overloaded with too many possible meanings. Does it mean nothing was found? Was an error thrown? Is the value itself null? At least with a well defined API, you can determine these scenarios from a glance. I think that handling types like, None, Err("reason"), or Some(None) (using Rust types) is far clearer than looking at a null pointer and an int error code (which one may ignore).
  - pavlov 9y ago
    I do like the Rust way because it’s standard (so far anyway). The situation in C++ is more like “a thousand blooms of nullishness”, where each library and API is frustratingly different in terms of something so fundamental. Objective-C with its messageable nil is so much better in this respect.
    - oddity 9y ago
      Messageable nil has a special place in my heart, but as a default mode of operation, it's certainly a fun way to hide bugs.
      - delinka 9y ago
        Messages to nil ... sure, OK, fine. Allow that, but CRASH because someone sent an unhandled message? That’s just mean. Either crash when messaging nil, or don’t crash when improperly messaging an object.
        catnaroek 9y ago
        It doesn't matter how a language “handles nil or ‘not understood’ [sic] messages” - burning the computer would be okay. As with any other undesired behavior, it's your responsibility to prove it doesn't happen.
        tomsmeding 9y ago
        I think that everyone here knows that programming without bugs is the ideal, but having the language work against doesn't really help in that regard, does it?
        catnaroek 9y ago
        I wouldn't count “burning the computer under impossible conditions” as “working against”. It's just the principle of explosion at work, and pretty much any optimizing compiler that takes advantage of undefined behavior uses it.
        yoz-y 9y ago
        No. In this case there is no undefined behaviour, the way this is handled is defined, but to the parent it seems inconsistent. Undesired behaviour can not be really leveraged for optimisation, and in any case, high(er) level languages should not have UB in their design.
- delinka 9y ago
  Creating objects in “empty” states is lazy programming. The point of references is to guarantee an object exists. But don’t pretend an object exists just to satisfy the reference. Rethink your code to take advantage of the guarantee.
  - quotemstr 9y ago
    You've illustrated why I'm against the kind of simplistic programming maxims you'll find in "Effective X" books. It's easy to brand certain syntactic constructs as "bad", but unless you teach people good taste (which is fiendishly difficult to learn), all you do by telling people to avoid certain easily-recognizable syntactic patterns is to get them to come up with new horrors.
    - notduncansmith 9y ago
      As GP said, rethink the code, from as far out as you need to. Granted, much code has been written under the explicit constraint of not doing this. In that case, do the nasty thing and then if it matters that much to you, find somewhere else to work that cares a similar amount about producing maintainable software.
    - leni536 9y ago
      > You've illustrated why I'm against the kind of simplistic programming maxims you'll find in "Effective X" books. Another thing you won't find in this book is the C++ Gospel, the One True Path to perfect C++ software. Each of the Items in this book provides guidance on how to develop better designs, how to avoid common problems, or how to achieve greater efficiency, but none of the Items is universally applicable. ... If you follow all the guidelines all the time, you are unlikely to fall into the most common traps surrounding C++, but guidelines, by their nature, have exceptions. That's why each Item has an explanation. The explanations are the most important part of the book. Scott Meyers - Effective C++ The problem is not "Effective X" books. The problems is people treating these rules in these books as dogma.
      - jstimpfle 9y ago
- echelon 9y ago
  This is where `Option` or `Optional` types shine. They force you to unwrap the nilable value at compile time, and are zero-cost abstractions. It's not difficult to use option types, either--especially if the language features syntactic sugar to promote their ergonomics. Rust, Swift, Guava, etc. all get this right. Option types need to become a language feature for all statically-typed languages. C++ should adopt it too.
  - Arcsech 9y ago
    While I love option types, retrofitting them into languages which already have nulls has problems. Ask me about using a Java library that could return an option type, which could be null, so you have to check for both the None return value AS WELL AS null.
    - catnaroek 9y ago
      Retrofitting anything into anything always results in warts. Moral of the story: design things right, right from the beginning.
      - tomsmeding 9y ago
        ... which is practically impossible with larger projects that you can't oversee in a glance.
        catnaroek 9y ago
        So don't bite more than you can chew?
    - pjmlp 9y ago
      The situation is known and will be fixed when the JVM gets value types, with minimal value types already on the horizon. https://wiki.openjdk.java.net/display/valhalla/Minimal+Value+Types https://wiki.openjdk.java.net/display/valhalla/Minimal+Value... Until then it is an issue that we have to live with.
- albinofrenchy 9y ago
  Is this really considered modern c++ style? Best practice as far as I've ever known it is to disallow wherever possible the allowance of having an invalid state; typically with constructers locking those details down. Obviously references are better in most circumstances but only because you don't have to check for an empty or invalid state. They offer no performance or optimization on their own.
  - tomjakubowski 9y ago
    I think GP meant "smart pointers" where they said "object references". At least, that was the only reading that made sense to me, since it would be a little bit insane (and UB) to create a C++ reference (T&) from a null pointer.
    - adrianratnapala 9y ago
      But then what is the difference between what he is complaining about and a traditional pointer. I always assumed that a default-constructed unique_ptr<T> was simply a NULL pointer under the hood. So you have exactly the same representations of "nothing" as you do with a T
      - albinofrenchy 9y ago
        Yes, both unique and shared ptrs default to null and in many ways are drop in replacements for naked pointers. If the empty state thing was in reference to smart pointers, it'd be a remarkable misunderstanding of how they work and how they should be used.
        tomjakubowski 9y ago
        > If the empty state thing was in reference to smart pointers, it'd be a remarkable misunderstanding of how they work and how they should be used. No, I don't think so. Many C++ libraries, especially those originally developed before C++11, implement their own smart pointer templates, which may end up having diverging semantics. That's exactly what OP was describing. Examples off the top of my head include: CEF, Chromium, v8, Qt, glibmm, and the internal codebase where I work.
- d__k 9y ago
  What is really undesirable is to have both pointers and references in one programming language as two separate constructs.
- jstimpfle 9y ago
  I don't get the buzz about null pointers. They are not a problem. They become a problem when you start checking for null where null is not acceptable (most of the places null can't be a meaningful input), which is where is where the original intent starts to become unclear. Let it go. Let it crash. Just assume inputs are non-null (except where null makes sense). Even C crashes safely on null pointer dereference.
  - IshKebab 9y ago
    The problem is that the type system doesn't encode whether a point can be null or not, so unless your documentation is amazing (I doubt it) you'll eventually end up getting passed a pointer from someone else, or giving a pointer to someone else who has a different assumption about whether null is allowed. Boom segfault. There are several solutions in C++. 1. Always check for null. Kind of annoying and lots of people don't for whatever reason. 2. Use references. As you say, annoying because then you can't have null (sensibly) even when it would be really useful. 3. Use std::optional<int&> or something like that. I only just thought of this and don't know if it would work, but I bet it's a pain. So no great solution. Personally I would use a smart pointer type, document it as well as I can, and always check for null.
    - gpderetta 9y ago
      > Always check for null. Kind of annoying and lots of people don't for whatever reason. use assert or some custom always-assert-even-on-release macro. > Use references. As you say, annoying because then you can't have null (sensibly) even when it would be really useful. well, only use references for non nullable pointers. Otherwise use plain (or smart) pointers. > Use std::optional<int&> or something like that. I only just thought of this and don't know if it would work, but I bet it's a pain. std::optional<T&> is unfortunately not part of C++17, mostly because people couldn't agree on operator= semantics: int x = 1; int y = 2; std::optional<int&> oint = x; oint = y; // which one is true? assert(x == y); // deep assign assert(&*oint == &y); // rebind boost::optional supports references and IIRC rebinds on plain assignment.
    - smelterdemon 9y ago
- dan00 9y ago
  > I’m not a huge fan of the modern C++ style that obsessively avoids null pointers and instead uses object references that are created in an invalid/empty state. If you've objects with an invalid state then you've nothing won compared to a null pointer. But the point is to make invalid state not representable as much as possible, then the reasoning of code gets a lot easier, because the invalid state can't distribute throughtout the application.
- gcc_programmer 9y ago
  Have you heard of operator bool() ? If the library is implemented by people who know their stuff, each object will define operator bool() and you can then have code like: Foo foo; foo.init_from_file("/tml/lol"); if (foo) { // do some stuff } we could be getting Foo objects by value from a factory, which constructs and moves the Foo objects. References are meant for situations where the object CANNOT be null/invalid by contract, so you are missing the point amigo.
  - pavlov 9y ago
    The accumulated corpus of C++ libraries has been implemented by people who may or may not have known their stuff at some point in the past 25 years. You say operator bool() is the one true way. Many other people define "empty" states for their objects that sometimes make sense only to the library author. Others use class wrappers. Someone else came up with a cool trick for a Maybe-style functor, I'm sure. Yet another bright person has built his very own mind-bending approach using move semantics and some odd corner of the C++14 spec. With C++, you never know what to expect until you dig into the API. It's an impediment to doing anything with the language unless you work on a huge project where coding standards are rigid and external dependencies come in every once a century (which many C++ projects are, certainly).
    - weeklyrust 9y ago
      I fully agree. C++ never had a clear goal. The features of C++ were added almost at random. Stroustrup's original idea was essentially "C is cool and OOP is cool so let's bolt OOP onto C". Retrospectively, OOP was massively overhyped and is the wrong tool for the job for most of the people most of the time. Half of the GoF design patterns just emulate idioms from functional programming. Real OOP languages like Smalltalk and IO express many useful things that C++ cannot. The feature that C needed most was perhaps parametric polymorphism (aka generics in Java and C#, first seen in ML in the late 1970s) but instead of that C++ got templates that weren't designed to solve any particular problem but rather to kind of solve several completely unrelated problems (e.g. generics and metaprogramming). Someone actually discovered by accident that C++ templates are Turing complete and they wrote and published a program that computed prime numbers at compile time. Wow. A remarkable observation that led to decades of template abuse where people used templates to solve problems much better solved by other pre-existing solutions such Lisp macros and ML polymorphism. Worse, this abuse led to even more language features being piled on top, like template partial specialization. The massive incidental complexity in C++ made it almost impossible to write a working compiler. For example, it remains extremely difficult to write a parser for the C++ language. The syntax also has horrible aspects like List<Set<int>> being interpreted as logical shift right. None of the original C++ compilers were reliable. Only after two decades did we start to see solid C++ compilers (by which time C++ was in decline in industry due to Java and C#). C++ is said to be fast but the reality is that C++ is essentially only fast when you write C-like code and even then it is only fast for certain kinds of programs. Due to the "you don't pay for what you don't use" attitude, C++ is generally inefficient. RAII injects lots of unnecessary function calls at the end of scope, sometimes even expensive virtual calls. These calls often require data that would otherwise be dead so the data are kept alive, increasing register pressure and spilling and decreasing performance. The C++ exception mechanism is very inefficient (~6x slower than OCaml) because it unwinds the stack frame by frame calling destructors rather than long jumping. Allocation with new and delete is slow compared to a modern garbage collector so people are encouraged to use STL collections but these pre-allocate huge blocks of memory in comparison so you've lost the memory-efficiency of C and then you are advised to write your own STL allocator which is no better than using C in the first place. One of the main long-standing advantages of C over modern languages is the unpredictable latency incurred by garbage collectors. C++ offers the worst of both worlds by not having a garbage collector (making it impossible to leverage useful concepts like purely functional data structures properly) but it encourages all destructors to avalanche so you get unbounded pause times (worse than any production GC). Although templates are abused for metaprogramming they are very poor at it and C++ has no real support for metaprogramming. For example, you cannot write an efficient portable regular expression library in C++ because there is no way to do run-time code generation and compilation as you can in Java, C# and languages dating back to Lisp (1960). So while Java and C# have had regular expressions in their standard libraries for well over 10 years, C++ only just got them and they are slow. C++ is so complicated that even world experts make rookie mistakes with it. Herb Sutter works for Microsoft and sits on the C++ standards committee where he influences the future of C++. In a lecture he gave his favorite 10-line C++ program, a thread-safe object cache. My personal feeling is that the new Rust programming language is what C++ should have been. It has useful known features like generics, discriminated unions and pattern matching and useful new features like memory safety without garbage collection.
- deleted 9y ago
  [deleted]
- smelterdemon 9y ago
  Creating objects in invalid states is definitely not idiomatic C++. I do agree that people are a little too fearful of using bare pointers for optional references, if you're using a linter that will catch possible null dereferences it's often the most sensible technique.
- CyberDildonics 9y ago
  C++17 has std::optional If you've ever used unstable software that crashes in a dozen different ways you can thank null pointers. Modern C++ isn't about substituting references for pointers, it is about dealing with values and removing a level of indirection that pointers gives. The old style of of using pointers everywhere frequently usually implies a lot of small heap allocations which is also a big performance problem. Both these things end up being huge detriments to speed and stability.
duneroadrunner 9y ago
First let me say that I'm not familiar with Nim. But I understand that it compiles/transpiles to C/C++. And it sounds like they're now trying to move away from dependency on the GC. In that case, might I suggest they consider switching the transpile target to SaferCPlusPlus[1]. It might make things easier as it already addresses the "efficient memory-safety via scope lifetimes without a borrow-checker" issue. And also the (data race) safe sharing of objects between threads. (Note that the documentation is currently rather out-of-date, and a major update is coming.) [1] shameles plug: https://github.com/duneroadrunner/SaferCPlusPlus https://github.com/duneroadrunner/SaferCPlusPlus
- mratsim 9y ago
  Nim compiles to C/C++/Objective-C and Javascript and all memory safety is handled on the Nim side. The C/C++ code generated by the Nim compiler is then free to use unsafe constructs including the dreaded goto.
seertaak 9y ago
I'm a C++ dev and this sounds extremely interesting. I have nim a spin a year ago and really liked what I saw, but GC was a real issue because I'm doing audio development where you end up having to circumvent the GC anyway. I would love to allow my users to create MIDI plugins with nim and this is a key step towards that.
- mratsim 9y ago
  I'd like to add that "circumventing the GC" is supported by default. There are 3 object types in Nim - `type Foo = object` --> Value type, on the stack - `type Foo = ref object` --> Reference type, managed by GC - `type Foo = ptr object` --> Pointer type, manual memory management (with Nim's equivalent for malloc, free and memcopy) Also Nim does not have 1 GC but several: - Default Deferred Reference Counting - Not stop the world - Boehm GC - Mark and Sweep - Memory Regions - Real-time GC (with tunable max-pause) - GC disabled GC can also be deactivated per thread (setupForeignThreadGc) or for a section of the code (GC_ref, GC_unref) even on Ref types to improve interoperability with other languages.
- baldfat 9y ago
  You make Zenaudio ALK! That is a really interesting sequencer and looper. Congrats on a great concept.
dom96 9y ago
Hello guys. I'm one of the core Nim developers, together with Araq (the creator of Nim) who should also be hanging around this thread. Feel free to ask us anything. This submission is likely the result of the recent livestreams that myself and Araq have been making. You can check out my past livestreams[1] and Araq's[2] at the links below. If you're into watching us live the next time we create a livestream then follow d0m96 (https://go.twitch.tv/d0m96 https://go.twitch.tv/d0m96) and araq4k (https://go.twitch.tv/araq4k https://go.twitch.tv/araq4k) on Twitch. 1 - https://www.youtube.com/watch?v=UQ4RvUlXIDI&index=3&list=PLm-fq5xBdPkrMuVkPWuho7XzszB6kJ2My https://www.youtube.com/watch?v=UQ4RvUlXIDI&index=3&list=PLm... 2 - https://www.youtube.com/watch?v=E2qlDKm_WzE https://www.youtube.com/watch?v=E2qlDKm_WzE, https://www.youtube.com/watch?v=UV38gQfcb9c https://www.youtube.com/watch?v=UV38gQfcb9c
- qaq 9y ago
  You guys are doing amazing work! Found Nim through a mention on HN recently and having a ton of fun learning and playing with it!
  - athenot 9y ago
    Same here. Nim has this elegance that is hard to qualify and impossibe to quantify, but it makes it a real pleasure to write code in it. Can't wait to get deeper into it.
  - jonathanstrange 9y ago
    I like it, too. Just wish mimx was in a better shape. I need rich text with images and links.
- squarefoot 9y ago
  This one just to thank you for everything, Dom! BTW, your book is on its way to my desk. I briefly skimmed the pdf, but can't wait to get the dead tree version!:) Don't forget about embedded systems: they could be the killer platform which makes Nim mainstream overnight.
  - 9y ago
tomsmeding 9y ago
That sink concept sounds really cool. I would really want that semantic in my C and C++ code.
- Boxxed 9y ago
  Pretty sure that's the semantics of an `auto_ptr`: http://www.cplusplus.com/reference/memory/auto_ptr/ http://www.cplusplus.com/reference/memory/auto_ptr/
audunw 9y ago
Does this mean that Nim 1.0 will be postponed even further? That's OK, it's better to get it right, and it's still useful for small projects in the meantime. But Nim seems like a language that is never truly finished. I like this direction. I tried to avoid "ref" in my last Nim project, but it's too hard to do that for every type the way the language is currently designed.
- Tiberium 9y ago
  It won't be postponed AFAIK
  - dom96 9y ago
    Indeed, Araq's plan is to release v1 ASAP (he said by the end of this year on IRC) and implement the ideas described in this article afterwards for a Nim v2.
- nimmer 9y ago
  > Nim seems like a language that is never truly finished True for every language. "1.0" will freeze the spec and it's not like the current versions are unstable or unusable.
tempodox 9y ago
Completely off-topic, bit I feel compelled to mention this: The timestamping of this article is exemplary. Even if it only showed a date, and no time, the time zone is relevant. The Internet is supposed to be for the whole world, not just one time zone.
ndh2 9y ago
Suggestion: Add a date, the author, and possibly some version number that indicate what iteration of Nim this refers to.
userbinator 9y ago
The syntax and title reminds me of standard Pascal, which has very constrained pointers that, among other things, you can't do arithmetic with --- eventually leading to code that basically reinvents memory itself by using a large array and lots of indexing operations. (Look at Donald Knuth's TeX for an example of this style.)
- pjmlp 9y ago
  With the added benefit of bounds checking. However all Pascal dialects had extensions for low level pointer manipulation.
- mratsim 9y ago
  Nim was written in Pascal at the very beginning. But you can't do pointer arithmetics in Nim. Either - `type Foo = object` -> Value type, on the stack - `type Foo = ref object` -> Ref type, managed by GC - `type Foo = ptr object` -> Pointer type, manual memory management (malloc, free, memcopy ...)
  - dom96 9y ago
    > But you can't do pointer arithmetics in Nim. Actually you can, by using casts.
    - mratsim 9y ago
      Oops, I obviously meant "you can" but I probably typed too fast :P
jamesu 9y ago
The other year I tried to implement a GC on top of an existing scripting language to handle circular dependency cases. A big problem however was the existing system for referencing objects wasn't particularly well designed for this particular case (e.g. no roots, no thread safety), and I couldn't come up with a solution I found satisfactory and fool-proof. The fact the author had so many problems with GC bugs is somewhat reassuring. Would be interesting to see a GC-less Nim in a production use-case.

205 points by mindB 9y ago

I’m not a huge fan of the modern C++ style that obsessively avoids null pointers and instead uses object references that are created in an invalid/empty state. With a pointer there is a single way to represent “nothing”, regardless of what the object represents. With a reference I have to go read the API to understand the “nothingness” states. Also, when you’re thinking in pointers, it’s easy to add levels of indirection with the same mental tools (pointers to pointers, etc.) Personally I find it’s easier to solve problems with a limited orthogonal toolset than a sprawling array of marginally differently balanced optimizations, which is what most of C++ has become.

NiceGuy_Ty 9y ago
While null may be the simplest way to represent nothing between a large amount of data types, I think the biggest issue (beyond how far null can propagate from the source in error scenarios) is that it's overloaded with too many possible meanings. Does it mean nothing was found? Was an error thrown? Is the value itself null? At least with a well defined API, you can determine these scenarios from a glance. I think that handling types like, None, Err("reason"), or Some(None) (using Rust types) is far clearer than looking at a null pointer and an int error code (which one may ignore).
- pavlov 9y ago
  I do like the Rust way because it’s standard (so far anyway). The situation in C++ is more like “a thousand blooms of nullishness”, where each library and API is frustratingly different in terms of something so fundamental. Objective-C with its messageable nil is so much better in this respect.
  - oddity 9y ago
    Messageable nil has a special place in my heart, but as a default mode of operation, it's certainly a fun way to hide bugs.
    - delinka 9y ago
      Messages to nil ... sure, OK, fine. Allow that, but CRASH because someone sent an unhandled message? That’s just mean. Either crash when messaging nil, or don’t crash when improperly messaging an object.
      - catnaroek 9y ago
        It doesn't matter how a language “handles nil or ‘not understood’ [sic] messages” - burning the computer would be okay. As with any other undesired behavior, it's your responsibility to prove it doesn't happen.
        tomsmeding 9y ago
        I think that everyone here knows that programming without bugs is the ideal, but having the language work against doesn't really help in that regard, does it?
        catnaroek 9y ago
        I wouldn't count “burning the computer under impossible conditions” as “working against”. It's just the principle of explosion at work, and pretty much any optimizing compiler that takes advantage of undefined behavior uses it.
        yoz-y 9y ago
        No. In this case there is no undefined behaviour, the way this is handled is defined, but to the parent it seems inconsistent. Undesired behaviour can not be really leveraged for optimisation, and in any case, high(er) level languages should not have UB in their design.
delinka 9y ago
Creating objects in “empty” states is lazy programming. The point of references is to guarantee an object exists. But don’t pretend an object exists just to satisfy the reference. Rethink your code to take advantage of the guarantee.
- quotemstr 9y ago
  You've illustrated why I'm against the kind of simplistic programming maxims you'll find in "Effective X" books. It's easy to brand certain syntactic constructs as "bad", but unless you teach people good taste (which is fiendishly difficult to learn), all you do by telling people to avoid certain easily-recognizable syntactic patterns is to get them to come up with new horrors.
  - notduncansmith 9y ago
    As GP said, rethink the code, from as far out as you need to. Granted, much code has been written under the explicit constraint of not doing this. In that case, do the nasty thing and then if it matters that much to you, find somewhere else to work that cares a similar amount about producing maintainable software.
  - leni536 9y ago
    > You've illustrated why I'm against the kind of simplistic programming maxims you'll find in "Effective X" books. Another thing you won't find in this book is the C++ Gospel, the One True Path to perfect C++ software. Each of the Items in this book provides guidance on how to develop better designs, how to avoid common problems, or how to achieve greater efficiency, but none of the Items is universally applicable. ... If you follow all the guidelines all the time, you are unlikely to fall into the most common traps surrounding C++, but guidelines, by their nature, have exceptions. That's why each Item has an explanation. The explanations are the most important part of the book. Scott Meyers - Effective C++ The problem is not "Effective X" books. The problems is people treating these rules in these books as dogma.
    - jstimpfle 9y ago
echelon 9y ago
This is where `Option` or `Optional` types shine. They force you to unwrap the nilable value at compile time, and are zero-cost abstractions. It's not difficult to use option types, either--especially if the language features syntactic sugar to promote their ergonomics. Rust, Swift, Guava, etc. all get this right. Option types need to become a language feature for all statically-typed languages. C++ should adopt it too.
- Arcsech 9y ago
  While I love option types, retrofitting them into languages which already have nulls has problems. Ask me about using a Java library that could return an option type, which could be null, so you have to check for both the None return value AS WELL AS null.
  - catnaroek 9y ago
    Retrofitting anything into anything always results in warts. Moral of the story: design things right, right from the beginning.
    - tomsmeding 9y ago
      ... which is practically impossible with larger projects that you can't oversee in a glance.
      - catnaroek 9y ago
        So don't bite more than you can chew?
  - pjmlp 9y ago
    The situation is known and will be fixed when the JVM gets value types, with minimal value types already on the horizon. https://wiki.openjdk.java.net/display/valhalla/Minimal+Value+Types https://wiki.openjdk.java.net/display/valhalla/Minimal+Value... Until then it is an issue that we have to live with.
albinofrenchy 9y ago
Is this really considered modern c++ style? Best practice as far as I've ever known it is to disallow wherever possible the allowance of having an invalid state; typically with constructers locking those details down. Obviously references are better in most circumstances but only because you don't have to check for an empty or invalid state. They offer no performance or optimization on their own.
- tomjakubowski 9y ago
  I think GP meant "smart pointers" where they said "object references". At least, that was the only reading that made sense to me, since it would be a little bit insane (and UB) to create a C++ reference (T&) from a null pointer.
  - adrianratnapala 9y ago
    But then what is the difference between what he is complaining about and a traditional pointer. I always assumed that a default-constructed unique_ptr<T> was simply a NULL pointer under the hood. So you have exactly the same representations of "nothing" as you do with a T
    - albinofrenchy 9y ago
      Yes, both unique and shared ptrs default to null and in many ways are drop in replacements for naked pointers. If the empty state thing was in reference to smart pointers, it'd be a remarkable misunderstanding of how they work and how they should be used.
      - tomjakubowski 9y ago
        > If the empty state thing was in reference to smart pointers, it'd be a remarkable misunderstanding of how they work and how they should be used. No, I don't think so. Many C++ libraries, especially those originally developed before C++11, implement their own smart pointer templates, which may end up having diverging semantics. That's exactly what OP was describing. Examples off the top of my head include: CEF, Chromium, v8, Qt, glibmm, and the internal codebase where I work.
d__k 9y ago
What is really undesirable is to have both pointers and references in one programming language as two separate constructs.
jstimpfle 9y ago
I don't get the buzz about null pointers. They are not a problem. They become a problem when you start checking for null where null is not acceptable (most of the places null can't be a meaningful input), which is where is where the original intent starts to become unclear. Let it go. Let it crash. Just assume inputs are non-null (except where null makes sense). Even C crashes safely on null pointer dereference.
- IshKebab 9y ago
  The problem is that the type system doesn't encode whether a point can be null or not, so unless your documentation is amazing (I doubt it) you'll eventually end up getting passed a pointer from someone else, or giving a pointer to someone else who has a different assumption about whether null is allowed. Boom segfault. There are several solutions in C++. 1. Always check for null. Kind of annoying and lots of people don't for whatever reason. 2. Use references. As you say, annoying because then you can't have null (sensibly) even when it would be really useful. 3. Use std::optional<int&> or something like that. I only just thought of this and don't know if it would work, but I bet it's a pain. So no great solution. Personally I would use a smart pointer type, document it as well as I can, and always check for null.
  - gpderetta 9y ago
    > Always check for null. Kind of annoying and lots of people don't for whatever reason. use assert or some custom always-assert-even-on-release macro. > Use references. As you say, annoying because then you can't have null (sensibly) even when it would be really useful. well, only use references for non nullable pointers. Otherwise use plain (or smart) pointers. > Use std::optional<int&> or something like that. I only just thought of this and don't know if it would work, but I bet it's a pain. std::optional<T&> is unfortunately not part of C++17, mostly because people couldn't agree on operator= semantics: int x = 1; int y = 2; std::optional<int&> oint = x; oint = y; // which one is true? assert(x == y); // deep assign assert(&*oint == &y); // rebind boost::optional supports references and IIRC rebinds on plain assignment.
  - smelterdemon 9y ago
dan00 9y ago
> I’m not a huge fan of the modern C++ style that obsessively avoids null pointers and instead uses object references that are created in an invalid/empty state. If you've objects with an invalid state then you've nothing won compared to a null pointer. But the point is to make invalid state not representable as much as possible, then the reasoning of code gets a lot easier, because the invalid state can't distribute throughtout the application.
gcc_programmer 9y ago
Have you heard of operator bool() ? If the library is implemented by people who know their stuff, each object will define operator bool() and you can then have code like: Foo foo; foo.init_from_file("/tml/lol"); if (foo) { // do some stuff } we could be getting Foo objects by value from a factory, which constructs and moves the Foo objects. References are meant for situations where the object CANNOT be null/invalid by contract, so you are missing the point amigo.
- pavlov 9y ago
  The accumulated corpus of C++ libraries has been implemented by people who may or may not have known their stuff at some point in the past 25 years. You say operator bool() is the one true way. Many other people define "empty" states for their objects that sometimes make sense only to the library author. Others use class wrappers. Someone else came up with a cool trick for a Maybe-style functor, I'm sure. Yet another bright person has built his very own mind-bending approach using move semantics and some odd corner of the C++14 spec. With C++, you never know what to expect until you dig into the API. It's an impediment to doing anything with the language unless you work on a huge project where coding standards are rigid and external dependencies come in every once a century (which many C++ projects are, certainly).
  - weeklyrust 9y ago
    I fully agree. C++ never had a clear goal. The features of C++ were added almost at random. Stroustrup's original idea was essentially "C is cool and OOP is cool so let's bolt OOP onto C". Retrospectively, OOP was massively overhyped and is the wrong tool for the job for most of the people most of the time. Half of the GoF design patterns just emulate idioms from functional programming. Real OOP languages like Smalltalk and IO express many useful things that C++ cannot. The feature that C needed most was perhaps parametric polymorphism (aka generics in Java and C#, first seen in ML in the late 1970s) but instead of that C++ got templates that weren't designed to solve any particular problem but rather to kind of solve several completely unrelated problems (e.g. generics and metaprogramming). Someone actually discovered by accident that C++ templates are Turing complete and they wrote and published a program that computed prime numbers at compile time. Wow. A remarkable observation that led to decades of template abuse where people used templates to solve problems much better solved by other pre-existing solutions such Lisp macros and ML polymorphism. Worse, this abuse led to even more language features being piled on top, like template partial specialization. The massive incidental complexity in C++ made it almost impossible to write a working compiler. For example, it remains extremely difficult to write a parser for the C++ language. The syntax also has horrible aspects like List<Set<int>> being interpreted as logical shift right. None of the original C++ compilers were reliable. Only after two decades did we start to see solid C++ compilers (by which time C++ was in decline in industry due to Java and C#). C++ is said to be fast but the reality is that C++ is essentially only fast when you write C-like code and even then it is only fast for certain kinds of programs. Due to the "you don't pay for what you don't use" attitude, C++ is generally inefficient. RAII injects lots of unnecessary function calls at the end of scope, sometimes even expensive virtual calls. These calls often require data that would otherwise be dead so the data are kept alive, increasing register pressure and spilling and decreasing performance. The C++ exception mechanism is very inefficient (~6x slower than OCaml) because it unwinds the stack frame by frame calling destructors rather than long jumping. Allocation with new and delete is slow compared to a modern garbage collector so people are encouraged to use STL collections but these pre-allocate huge blocks of memory in comparison so you've lost the memory-efficiency of C and then you are advised to write your own STL allocator which is no better than using C in the first place. One of the main long-standing advantages of C over modern languages is the unpredictable latency incurred by garbage collectors. C++ offers the worst of both worlds by not having a garbage collector (making it impossible to leverage useful concepts like purely functional data structures properly) but it encourages all destructors to avalanche so you get unbounded pause times (worse than any production GC). Although templates are abused for metaprogramming they are very poor at it and C++ has no real support for metaprogramming. For example, you cannot write an efficient portable regular expression library in C++ because there is no way to do run-time code generation and compilation as you can in Java, C# and languages dating back to Lisp (1960). So while Java and C# have had regular expressions in their standard libraries for well over 10 years, C++ only just got them and they are slow. C++ is so complicated that even world experts make rookie mistakes with it. Herb Sutter works for Microsoft and sits on the C++ standards committee where he influences the future of C++. In a lecture he gave his favorite 10-line C++ program, a thread-safe object cache. My personal feeling is that the new Rust programming language is what C++ should have been. It has useful known features like generics, discriminated unions and pattern matching and useful new features like memory safety without garbage collection.
deleted 9y ago
[deleted]
smelterdemon 9y ago
Creating objects in invalid states is definitely not idiomatic C++. I do agree that people are a little too fearful of using bare pointers for optional references, if you're using a linter that will catch possible null dereferences it's often the most sensible technique.
CyberDildonics 9y ago
C++17 has std::optional If you've ever used unstable software that crashes in a dozen different ways you can thank null pointers. Modern C++ isn't about substituting references for pointers, it is about dealing with values and removing a level of indirection that pointers gives. The old style of of using pointers everywhere frequently usually implies a lot of small heap allocations which is also a big performance problem. Both these things end up being huge detriments to speed and stability.

duneroadrunner 9y ago

First let me say that I'm not familiar with Nim. But I understand that it compiles/transpiles to C/C++. And it sounds like they're now trying to move away from dependency on the GC. In that case, might I suggest they consider switching the transpile target to SaferCPlusPlus[1]. It might make things easier as it already addresses the "efficient memory-safety via scope lifetimes without a borrow-checker" issue. And also the (data race) safe sharing of objects between threads. (Note that the documentation is currently rather out-of-date, and a major update is coming.) [1] shameles plug: https://github.com/duneroadrunner/SaferCPlusPlus https://github.com/duneroadrunner/SaferCPlusPlus

mratsim 9y ago
Nim compiles to C/C++/Objective-C and Javascript and all memory safety is handled on the Nim side. The C/C++ code generated by the Nim compiler is then free to use unsafe constructs including the dreaded goto.

seertaak 9y ago

I'm a C++ dev and this sounds extremely interesting. I have nim a spin a year ago and really liked what I saw, but GC was a real issue because I'm doing audio development where you end up having to circumvent the GC anyway. I would love to allow my users to create MIDI plugins with nim and this is a key step towards that.

mratsim 9y ago
I'd like to add that "circumventing the GC" is supported by default. There are 3 object types in Nim - `type Foo = object` --> Value type, on the stack - `type Foo = ref object` --> Reference type, managed by GC - `type Foo = ptr object` --> Pointer type, manual memory management (with Nim's equivalent for malloc, free and memcopy) Also Nim does not have 1 GC but several: - Default Deferred Reference Counting - Not stop the world - Boehm GC - Mark and Sweep - Memory Regions - Real-time GC (with tunable max-pause) - GC disabled GC can also be deactivated per thread (setupForeignThreadGc) or for a section of the code (GC_ref, GC_unref) even on Ref types to improve interoperability with other languages.
baldfat 9y ago
You make Zenaudio ALK! That is a really interesting sequencer and looper. Congrats on a great concept.

dom96 9y ago

Hello guys. I'm one of the core Nim developers, together with Araq (the creator of Nim) who should also be hanging around this thread. Feel free to ask us anything. This submission is likely the result of the recent livestreams that myself and Araq have been making. You can check out my past livestreams[1] and Araq's[2] at the links below. If you're into watching us live the next time we create a livestream then follow d0m96 (https://go.twitch.tv/d0m96 https://go.twitch.tv/d0m96) and araq4k (https://go.twitch.tv/araq4k https://go.twitch.tv/araq4k) on Twitch. 1 - https://www.youtube.com/watch?v=UQ4RvUlXIDI&index=3&list=PLm-fq5xBdPkrMuVkPWuho7XzszB6kJ2My https://www.youtube.com/watch?v=UQ4RvUlXIDI&index=3&list=PLm... 2 - https://www.youtube.com/watch?v=E2qlDKm_WzE https://www.youtube.com/watch?v=E2qlDKm_WzE, https://www.youtube.com/watch?v=UV38gQfcb9c https://www.youtube.com/watch?v=UV38gQfcb9c

qaq 9y ago
You guys are doing amazing work! Found Nim through a mention on HN recently and having a ton of fun learning and playing with it!
- athenot 9y ago
  Same here. Nim has this elegance that is hard to qualify and impossibe to quantify, but it makes it a real pleasure to write code in it. Can't wait to get deeper into it.
- jonathanstrange 9y ago
  I like it, too. Just wish mimx was in a better shape. I need rich text with images and links.
squarefoot 9y ago
This one just to thank you for everything, Dom! BTW, your book is on its way to my desk. I briefly skimmed the pdf, but can't wait to get the dead tree version!:) Don't forget about embedded systems: they could be the killer platform which makes Nim mainstream overnight.
- 9y ago

tomsmeding 9y ago

That sink concept sounds really cool. I would really want that semantic in my C and C++ code.

Boxxed 9y ago
Pretty sure that's the semantics of an `auto_ptr`: http://www.cplusplus.com/reference/memory/auto_ptr/ http://www.cplusplus.com/reference/memory/auto_ptr/

audunw 9y ago

Does this mean that Nim 1.0 will be postponed even further? That's OK, it's better to get it right, and it's still useful for small projects in the meantime. But Nim seems like a language that is never truly finished. I like this direction. I tried to avoid "ref" in my last Nim project, but it's too hard to do that for every type the way the language is currently designed.

Tiberium 9y ago
It won't be postponed AFAIK
- dom96 9y ago
  Indeed, Araq's plan is to release v1 ASAP (he said by the end of this year on IRC) and implement the ideas described in this article afterwards for a Nim v2.
nimmer 9y ago
> Nim seems like a language that is never truly finished True for every language. "1.0" will freeze the spec and it's not like the current versions are unstable or unusable.

tempodox 9y ago

Completely off-topic, bit I feel compelled to mention this: The timestamping of this article is exemplary. Even if it only showed a date, and no time, the time zone is relevant. The Internet is supposed to be for the whole world, not just one time zone.

ndh2 9y ago

Suggestion: Add a date, the author, and possibly some version number that indicate what iteration of Nim this refers to.

userbinator 9y ago

The syntax and title reminds me of standard Pascal, which has very constrained pointers that, among other things, you can't do arithmetic with --- eventually leading to code that basically reinvents memory itself by using a large array and lots of indexing operations. (Look at Donald Knuth's TeX for an example of this style.)

pjmlp 9y ago
With the added benefit of bounds checking. However all Pascal dialects had extensions for low level pointer manipulation.
mratsim 9y ago
Nim was written in Pascal at the very beginning. But you can't do pointer arithmetics in Nim. Either - `type Foo = object` -> Value type, on the stack - `type Foo = ref object` -> Ref type, managed by GC - `type Foo = ptr object` -> Pointer type, manual memory management (malloc, free, memcopy ...)
- dom96 9y ago
  > But you can't do pointer arithmetics in Nim. Actually you can, by using casts.
  - mratsim 9y ago
    Oops, I obviously meant "you can" but I probably typed too fast :P

jamesu 9y ago

The other year I tried to implement a GC on top of an existing scripting language to handle circular dependency cases. A big problem however was the existing system for referencing objects wasn't particularly well designed for this particular case (e.g. no roots, no thread safety), and I couldn't come up with a solution I found satisfactory and fool-proof. The fact the author had so many problems with GC bugs is somewhat reassuring. Would be interesting to see a GC-less Nim in a production use-case.