7 ms·

As an ex security consultant almost all the bugs I found in Go applications were crashes that happened because of nil. It’s crazy how Rust just gets rid of this

• brundolf 5y ago
  > It’s crazy how Rust just gets rid of this whole class of bugs As do TypeScript, and Kotlin, and Dart, and Python (with MyPy), and C#, and Swift... It's honestly inexcusable that Go, as a relatively new statically-typed language, doesn't have strict null-checks
  • tticvs 5y ago
    Unfortunately it's easy enough to confuse MyPy that you can't really fit in with the rest of this list.
  • rowanG077 5y ago
    Unfortunately in any real world typescript or python the type checking is almost always partial. Which means it doesn't actually save you from these kind of errors.
    • cageface 5y ago
      I have not found this to be the case in Typescript code I've worked on. The main issue I've found with TS is stuff coming in from outside the system as JSON that doesn't conform to expected formats.
      • CryZe 5y ago
        One issue I've noticed is that indexing into a T[] array where T can't be undefined, results in the expression to be of type T, not T | undefined, despite an out of bounds index evaluating to undefined. I believe I've seen that they recently added a setting that changes this though.
        • cageface 5y ago
          A lot of statically typed languages leave array indexing as a loophole for convenience but yeah this can also be a source of errors.
      • andy_ppp 5y ago
        Consider using https://github.com/gcanti/io-ts https://github.com/gcanti/io-ts to help with that...
      • jakelazaroff 5y ago
        You can solve the JSON issue by validating that it deserializes to a known type at the point of ingress. I wrote a small library that I use in basically every project to help with that: https://www.npmjs.com/package/narrows https://www.npmjs.com/package/narrows
  • mbStavola 5y ago
    I don't know about C# and Swift, but Kotlin, Python, and Typescript are only null-safe-ish; there is always a relatively easy path to smuggling a null value into your program regardless of the guarantees of the types you use. This isn't to say it's useless, just that it's not as useful as it _could_ be. I've seen codebases in all three languages where there have been real production issues due to null values, even though the type decls say otherwise. Kotlin probably does the best job here but using any JVM library introduces potential (K)NPEs.
    • unscaled 5y ago
      This situation is unfortunately quite hard to avoid in the languages above since they either added null safety (or indeed any kind of type-checking) at a later stage (C# and Python) or they need to maintain compatibility with a language or platform that is not null-safe (Kotlin with Java, Swift with Objective C, TypeScript with JavaScript). Languages like Rust, Haskell or ML do not have this issue since they do not need to support such a legacy layer.
      • pjmlp 5y ago
        Kind of true, there is still the possibility to smuggle null values via unsafe, or FFI, but then the fault is on whoever certified the code as safe.
        • unscaled 5y ago
          I was hesitating whether to mention unsafe code blocks or FFI, but the no language that gets thing done can avoid abuse of its escape hatches. Even Haskell has to deal with nulls when doing FFI.
    • wildbook 5y ago
      As much as I like C#, your points apply to it too.
    • skohan 5y ago
      How can you get an unchecked nil into a swift program without the unsafe API?
  • andy_ppp 5y ago
    As someone working on a Typescript codebase with "string? | undefined" everywhere I wish this were true.
  • virgilp 5y ago
    C# only sort-of-fixed this problem and it did so rather inelegantly IMO. E.g. nullable types are not options - you can't "map" them (you can't invoke Select, Where etc). It's easy to "get value or default" (via operator `??`), but you can't do the other, equally frequent thing, of "apply transformation on value if not-null". Or... well, you can, but here's the syntax: var newVal = val == null ? (resultType?) null : compute(val.Value); instead of var newVal = val.Select(compute);
    • CodesInChaos 5y ago
      For that I use my own extension function like `?.Map(x => ...)`, defined as `Map(this T input, Func<T, TResult> map) => map(input)`. Still a bit ugly, but much nicer than the ternary operator. Of course that doesn't fix all the other issues with null handling in C# (inconsistency between null references and null value types, inability to nest options, ...)
    • mathw 5y ago
      Various bits of C# and its libraries also variously ignore non-nullability when it suits them. I know it's done the way it's done for compatibility reasons, but it's really quite confusing around the edges as a result. And despite something being non-nullable in the code it could still be null at runtime thanks in my experience particularly to the joys of deserialisation libraries.
• cageface 5y ago
  I did iOS dev for many years. When I switched from Obj-C to Swift almost all my runtime errors vanished because the compiler caught them for me. Now I consider nil safety as an essential feature of any modern language.
• rightbyte 5y ago
  Segfaults are a nice runtime assert. Better that than having erroneous error handling code that hides the bug.
• pjmlp 5y ago
  As ML did already in 1976, and Ada in 1983, with Eiffel in the mid-90's being the first OOP language to have non-nullable references. While I like Rust, it would be good if people actually did their research while crediting Rust for things it didn't implement.
  • ebingdom 5y ago
    > while crediting Rust for things it didn't implement. * invent (Other than this minor typo, I'm not sure why you're being downvoted. It's crazy that people in 2021 think non-nullable types are novel/"crazy". Why is nullability the default in most people's brains?)
    • onei 5y ago
      I would guess null is the default because many of us start out with languages like C or Java. I've seen some interns have FP experience with Haskell, but it tends to be a single module and they often don't appreciate the nuance and FP is still not particularly mainstream in industry. More generally, I don't think many programmers get to see the better way because they're not exposed to it. I can rhapsodise about Rust, but I doubt my company is going to buy into it because they already picked Go.
      • cies 5y ago
        > I would guess null is the default because many of us start out with languages like C or Java. Go is not a language that "just happened" (what could be said of JS and PHP). Go is designed. Designed by heavyweight language designers (from Wikipedia): Robert Griesemer, Rob Pike and Ken Thompson. These people knew of Haskell, type systems, and the merits of type safety. I would expect them to know of "billion dollar mistake[1]" that is null. But sadly Go still carries on with the mistake. I too care more about null-safety and proper sum types (in combination with nice switch/match statements and/or pattern matching) than generics. In the Elm language I found an experience that not having generics is perfectly okay (just a little annoying sometimes). I find "not null safe languages" not okay nowadays, and I hold this opinion since before Go's first appearance (2009). I really wonder how the designers came to this decision. I'm afraid this mistake can never be fixed, as null checks are already idiomatic Go. Java also could not fix it (which may be one of the main reasons behind Kotlin). Maybe the best thing we can hope for is Kotlin kind of language for Go (question marks after types to indicate nullability). It's just sad. [1]: https://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare/ https://www.infoq.com/presentations/Null-References-The-Bill...
• kitd 5y ago
  With generics, Optionals in Go become much more ergonomic.
  • tgv 5y ago
    But they add an unnecessary overhead, and still don't prevent you actually writing x := nil; print(x.a);. And everyone can have their own Optional, meaning you'll going to install dozens of different generics libraries for just this feature. There are simple, go-like solutions. There's a character for pointer, so why not a character for a non-nil pointer?
    • kitd 5y ago
      And everyone can have their own Optional, meaning you'll going to install dozens of different generics libraries for just this feature. Tbf, this is simply solved by putting a generic optional in the stdlib.
      • tgv 5y ago
        How would it work, BTW? In Rust, you're forced to match against the enum before you can look at the value, but in Go you even can't match. So unless it's implemented as a type cast, testing optionality would be just like testing against nil. The current internal "optional" package just panics, which is just as useful as a panic on dereferencing nil.
        • kitd 5y ago
          Similar to a Java Optional<T> [1] Give it an interface that allows you to attach behaviours for not null and null values. [1] https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Optional.html https://docs.oracle.com/en/java/javase/11/docs/api/java.base...
          • tgv 5y ago
            Function passing/callbacks, no thanks. You still won't get a compiler error when passing a potentially nil value, and I'd rather have readable code. I hope Optional is not going to be in the way of progress in this area.
• lima 5y ago
  > It’s crazy how Rust just gets rid of this whole class of bugs with option and result types (and sum types in general). As a Rust security consultant, many bugs we find in Rust applications are crashes due to .unwrap()... it's easier to spot, but still essentially the same class of bug.
  • rowanG077 5y ago
    I disagree. A nil error in Go is most likely a dev forgetting a check. An unwrap() is a dev explicitly saying this should always contain a value. In fact fixing unwrap is as easy as search for unwrap in your codebase and factoring them all out.
    • vaylian 5y ago
      > An unwrap() is a dev explicitly saying this should always contain a value Isn't expect() supposed to cover that case?
      • rowanG077 5y ago
        No it's the opposite. Unwrap() means the dev said this will never not be set. Expect() is saying if this value is not available exit the program with this user facing error message.
      • ModernMech 5y ago
        expect() is like unwrap() but you can give a message as an argument.
  • mathw 5y ago
    It is a shame there are so many Rust examples out there with unwrap() all over them, because it's really not something I would want to see in production code. expect() with a message if you know that the only reason this fails is because something else messed up that should have prevented the call in the first place, sure. unwrap()? No thanks. That's why we have the ? operator now.
• BoxOfRain 5y ago
  Option types in general are a wonderful thing, it's a big part of why I prefer Kotlin to Java. Kotlin's syntax for option types is really nice too, it's just `Int` vs `Int?` as opposed to Rust's `Option<i32>`. I definitely prefer Rust-style result types when it comes to error handling though.
• jtaft 5y ago
  I don’t think rust got rid of the null dereference problem. Just traded it for something slightly different. ie, calling unwrap when no value exists causes the program to panic.
  • arnsholt 5y ago
    Sure, but the crucial difference is that all the unwraps in your code are then hooks that a linter can find, or simply grepping for unwrap in your codebase and do a manual audit of those pieces of code. In my experiments with Rust that's been a very nice way of working: first build a very barebones version getting the basic happy path right, and then getting all the tedious stuff right afterwards by eliminating all the unwraps. By making it explicit, you now have a concrete thing you can audit specifically, rather than every pointer access in your entire program and every pointer you pass off to a library.
  • pornel 5y ago
    It did. It really makes a huge difference. In Go some common types are forced to be nillable, and you can't express "this is never nil". In Rust, "never nil" is the default, even for slices and by-reference types, so right of the bat for the majority of types nullability disappears entirely. You can't make a mistake of `unwrap()`ing something that doesn't support unwrapping. `unwrap()` is the laziest/worst way of handling optionals, but it's still better than Go's behavior. `unwrap()` is local and explicit, so you can find and review every potential failure point (unlike e.g. finding every use of a nil map in Go). And of course Rust has plenty of better, graceful ways of handling optionals, so you can also ban this method entirely with a lint.
    • moldavi 5y ago
      Doesnt Rust have implicit panics on indexing out of bounds? I wonder if any codebases lint those away.
      • masklinn 5y ago
        > Doesnt Rust have implicit panics on indexing out of bounds? It does yes. A fair number of other constructs can panic as well. > I wonder if any codebases lint those away. Clippy has a lint for indexing so probably. For the general case, it's almost impossible unless you're working on very low-level software (embedded, probably kernel-rust eventually) e.g. `std` assumes allocations can't fail, so any allocation will show up as a panic path. https://github.com/Technolution/rustig https://github.com/Technolution/rustig can actually uncover panic paths, but because of the above the results are quite noisy, and while it's possible to uncover bugs thanks to rustig it requires pretty ridiculous amounts of filtering.