6 ms·
Parse, Don't Validate and Type-Driven Design in Rust
- dang 8mo agoRecent and related: Parse, Don't Validate (2019) - https://news.ycombinator.com/item?id=46960392 https://news.ycombinator.com/item?id=46960392 - Feb 2026 (172 comments) also: Parse, Don’t Validate – Some C Safety Tips - https://news.ycombinator.com/item?id=44507405 https://news.ycombinator.com/item?id=44507405 - July 2025 (73 comments) Parse, Don't Validate (2019) - https://news.ycombinator.com/item?id=41031585 https://news.ycombinator.com/item?id=41031585 - July 2024 (102 comments) Parse, don't validate (2019) - https://news.ycombinator.com/item?id=35053118 https://news.ycombinator.com/item?id=35053118 - March 2023 (219 comments) Parse, Don't Validate (2019) - https://news.ycombinator.com/item?id=27639890 https://news.ycombinator.com/item?id=27639890 - June 2021 (270 comments) Parsix: Parse Don't Validate - https://news.ycombinator.com/item?id=27166162 https://news.ycombinator.com/item?id=27166162 - May 2021 (107 comments) Parse, Don’t Validate - https://news.ycombinator.com/item?id=21476261 https://news.ycombinator.com/item?id=21476261 - Nov 2019 (230 comments) Parse, Don't Validate - https://news.ycombinator.com/item?id=21471753 https://news.ycombinator.com/item?id=21471753 - Nov 2019 (4 comments) (p.s. these links are just to satisfy extra-curious readers - no criticism is intended! I add this because people sometimes assume otherwise)
- hacker_homie 8mo ago[flagged]
- jaggederest 8mo agoYou can go even further with this in other languages, with things like dependent typing - which can assert (among other interesting properties) that, for example, something like get_elem_at_index(array, index) cannot ever have index outside the bounds of the array, but checked statically at compilation time - and this is the key, without knowing a priori what the length of array is. "In Idris, a length-indexed vector is Vect n a (length n is in the type), and a valid index into length n is Fin n ('a natural number strictly less than n')." Similar tricks work with division that might result in inf/-inf, to prevent them from typechecking, and more subtle implications in e.g. higher order types and functions
- VorpalWay 8mo agoHow does that work? If the length of the array is read from stdin for example, it would be impossible to know it at compile time. Presumably this is limited somehow?
- jaggederest 8mo agoIf the length is read from outside the program it's an IO operation, not a static variable, but there are generally runtime checks in addition to the type system. Usually you solve this as in the article, with a constructor that checks it - so you'd have something like "Invalid option: length = 5 must be within 0-4" when you tried to create the Fin n from the passed in value
- ratorx 8mo agoIt doesn’t have to be a compile time constant. An alternative is to prove that when you are calling the function the index is always less than the size of the vector (a dynamic constraint). You may be able to assert this by having a separate function on the vector that returns a constrained value (eg. n < v.len()).
- mdm12 8mo agoOne option is dependent pairs, where one value of the pair (in this example) would be the length of the array and the other value is a type which depends on that same value (such as Vector n T instead of List T). Type-Driven Development with Idris[1] is a great introduction for dependently typed languages and covers methods such as these if you're interested (and Edwin Brady is a great teacher). [1] https://www.manning.com/books/type-driven-development-with-idris https://www.manning.com/books/type-driven-development-with-i...
- marcosdumay 8mo agoIf you check that the value is inside the range, and execute some different code if it's not, then congratulations, you now know at compile time that the number you will read from stdin is in the right range.
- dernett 8mo ago
- cmovq 8mo agoDividing a float by zero is usually perfectly valid. It has predictable outputs, and for some algorithms like collision detection this property is used to remove branches.
- woodruffw 8mo agoI think “has predictable outputs” is less valuable than “has expected outputs” for most workloads. Dividing by zero almost always reflects an unintended state, so proceeding with the operation means compounding the error state. (This isn’t to say it’s always wrong, but that having it be an error state by default seems very reasonable to me.)
- deleted 8mo ago[deleted]
- kibwen 8mo agoThe problem is that "division" in the context of computation refers to multiple different operations, so that we all end up talking past each other. Some people want a `division_where_zero_is_an_illegal_denominator`, some people want `division_but_return_zero_if_denominator_is_zero`, and I'm sure other people want weirder things. Honestly, I'm coming around to the idea that for any precise and reliable system, the fact that we only have a single "operator" (+ - * / etc) that we arbitrarily assign to one of the many possible operations that we want to do does more harm than good. Consider how many different ways there are just to add a number (panic on overflow, error value on overflow, saturate on overflow, wrap on overflow, return tuple with optional carry flag on overflow...).
- noitpmeder 8mo agoThis reminds me a bit of a recent publication by Stroustrup about using concepts... in C++ to validate integer conversions automatically where necessary. https://www.stroustrup.com/Concept-based-GP.pdf https://www.stroustrup.com/Concept-based-GP.pdf { Number<unsigned int> ii = 0; Number<char> cc = '0'; ii = 2; // OK ii = -2; // throws cc = i; // OK if i is within cc’s range cc = -17; // OK if char is signed; otherwise throws cc = 1234; // throws if a char is 8 bits }
- strawhatguy 8mo agoThe alternative is one type, with many functions that can operate on that type. Like how clojure basically uses maps everywhere and the whole standard library allows you to manipulate them in various ways. The main problem with the many type approach is several same it worse similar types, all incompatible.
- packetlost 8mo agoI don't really get why this is getting flagged, I've found this to be true but more of a trade off than a pure benefit. It also is sort of besides the point: you always need to parse inputs from external, usually untrusted, sources.
- doublesocket 8mo agoAgree with this. Mismatching types are generally an indicator of an underlying issue with the code, not the language itself. These are areas AI can be helpful flagging potential problems.
- fiddlerwoaroof 8mo agoYeah, there's something of a tension between the Perlis quote "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures" and Parse, don't validate. The way I've thought about it, though, is that it's possible to design a program well either by encoding your important invariants in your types or in your functions (especially simple functions). In dynamically typed languages like Clojure, my experience is that there's a set of design practices that have a lot of the same effects as "Parse, Don't Validate" without statically enforced types. And, ultimately, it's a question of mindset which style you prefer.
- strawhatguy 8mo agoThere's probably a case for both. Core logic might benefit from hard types deep in the bowels of unchanging engine. The real world often changes though, and more often than not the code has to adapt, regardless of how elegant are systems are designed.
- sam0x17 8mo agobtw the “quoth” crate makes it really really easy to implement scannerless parsing in rust for arbitrary syntax, use it on many of my projects
- hutao 8mo agoNote that the division-by-zero example used in this article is not the best example to demonstrate "Parse, Don't Validate," because it relies on encapsulation. The principle of "Parse, Don't Validate" is best embodied by functions that transform untrusted data into some data type which is correct by construction. Alexis King, the author of the original "Parse, Don't Validate" article, also published a follow-up, "Names are not type safety" [0] clarifying that the "newtype" pattern (such as hiding a nonzero integer in a wrapper type) provide weaker guarantees than correctness by construction. Her original "Parse, Don't Validate" article also includes the following caveat: > Use abstract datatypes to make validators “look like” parsers. Sometimes, making an illegal state truly unrepresentable is just plain impractical given the tools Haskell provides, such as ensuring an integer is in a particular range. In that case, use an abstract newtype with a smart constructor to “fake” a parser from a validator. So, an abstract data type that protects its inner data is really a "validator" that tries to resemble a "parser" in cases where the type system itself cannot encode the invariant. The article's second example, the non-empty vec, is a better example, because it encodes within the type system the invariant that one element must exist. The crux of Alexis King's article is that programs should be structured so that functions return data types designed to be correct by construction, akin to a parser transforming less-structured data into more-structured data. [0] https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-type-safety/ https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-...
- Sharlin 8mo agoEven the newtype-based "parse, don't validate" is tremendously useful in practice, though. The big thing is that if you have a bare string, you don't know "where it's been". It doesn't carry with it information whether it's already been validated. Even if a newtype can't provide you full correctness by construction, it's vastly easier to be convinced of the validity of an encapsulated value compared to a naked one. For full-on parse-don't-validate, you essentially need a dependent type system. As a more light-weight partial solution, Rust has been prototyping pattern types, which are types constrained by patterns. For instance a range-restricted integer type could be simply spelled `i8 is 0..100`, or a nonempty slice as `[T] is [_, ..]`. Such a feature would certainly make correctness-by-construction easier in many cases. The non-empty list implemented as a (T, Vec<T>) is, btw, a nice example of the clash between practicality and theoretical purity. It can't offer you a slice (consecutive view) of its elements without storing the first element twice (which requires immutability and that T: Clone, unlike normal Vec<T>), which makes it fairly useless as a vector. It's okay if you consider it just an abstract list with a more restricted interface.
- fph 8mo agoThe article quickly mentions implementing addition: ``` impl Add for NonZeroF32 { ... } impl Add<f32> for NonZeroF32 { ... } impl Add<NonZeroF32> for f32 { ... } ``` What type would it return though?
- alfons_foobar 8mo agoWould have to be F32, no? I cannot think of any way to enforce "non-zero-ness" of the result without making it return an optional Result<NonZeroF32>, and at that point we are basically back to square one...
- MaulingMonkey 8mo ago> Would have to be F32, no? Generally yes. `NonZeroU32::saturating_add(self, other: u32)` is able to return `NonZeroU32` though! ( https://doc.rust-lang.org/std/num/type.NonZeroU32.html#method.saturating_add https://doc.rust-lang.org/std/num/type.NonZeroU32.html#metho... ) > I cannot think of any way to enforce "non-zero-ness" of the result without making it return an optional Result<NonZeroF32>, and at that point we are basically back to square one... `NonZeroU32::checked_add(self, other: u32)` basically does this, although I'll note it returns an `Option` instead of a `Result` ( https://doc.rust-lang.org/std/num/type.NonZeroU32.html#method.checked_add https://doc.rust-lang.org/std/num/type.NonZeroU32.html#metho... ), leaving you to `.map_err(...)` or otherwise handle the edge case to your heart's content. Niche, but occasionally what you want.
- alfons_foobar 8mo ago> `NonZeroU32::saturating_add(self, other: u32)` is able to return `NonZeroU32` though! I was confused at first how that could work, but then I realized that of course, with _unsigned_ integers this works fine because you cannot add a negative number...
- fph 8mo agoYou'd still have to check for overflow, I imagine. And there are other gotchas, for instance it seems natural to assume that NonZeroF32 * NonZeroF32 can return a NonZeroF32, but 1e-25 * 1e-25 = 0 because of underflow.
- deleted 8mo ago[deleted]
- the__alchemist 8mo agoThe examples in question propagate complexity throughout related code. I think this is a case I see frequently in Rust of using too many abstractions, and its associated complexities. I would just (as a default; the situation varies)... validate prior to the division and handle as appropriate. The analogous situation I encounter frequently is indexing, e.g. checking if the index is out of bounds. Similar idea; check; print or display an error, then fail that computation without crashing the program. Usually an indication of some bug, which can be tracked down. Or, if it's an array frequently indexed, use a (Canonical for Rust's core) `get` method on the whatever struct owns the array. It returns an Option. I do think either the article's approach, or validating is better than runtime crashes! There are many patterns in programming. Using Types in this way is something I see a lot of in OSS rust, but it is not my cup of tea. Not heinous in this case, but I think not worth it. This is the key to this article's philosophy, near the bottom: > I love creating more types. Five million types for everyone please.
- unixpickle 8mo agoThe `try_roots` example here is actually a _counterexample_ to the author's main argument. They explicitly ignore the "negative discriminant" case. What happens if we consider it? If we take their "parse" approach, then the types of the arguments a, b, and c have to somehow encode the constraint `b^2 - 4ac >= 0`. This would be a total mess--I can't think of any clean way to do this in Rust. It makes _much_ more sense to simply return an Option and do the validation within the function. In general, I think validation is often the best way to solve the problem. The only counterexample, which the author fixates on in the post, is when one particular value is constrained in a clean, statically verifiable way. Most of the time, validation is used to check (possibly complex) interactions between multiple values, and "parsing" isn't at all convenient.
- sbszllr 8mo agoI was thinking a similar thing when reading the article. Often, the validity of the input depends on the interaction between some of them. Sure, we can follow the advice of creating types that represent only valid states but then we end up with `fn(a: A, b: B, c: C) transformed into `fn(abc: ValidABC)`
- deleted 8mo ago[deleted]
- AxiomLab 8mo ago[dead]
- Garlef 8mo agocurious: what kind of tooling would you use here?
- qsera 8mo ago[flagged]
- throw310822 8mo agoI only half understand this stuff, but all this encapsulation of values so that they are guaranteed to remain valid across manipulations... isn't it called Object Oriented Programming?
- mrkeen 8mo agoWas the original blog post wrong?
- qsera 8mo agowere you validating during parsing before?
- instig007 8mo agoValidating during parsing is still parsing, there's a reason why `Alternative f` exists after all: you have to choose between branches of possibilities and falsehoods. Now consider that there's another kind of validation that happens outside of program boundaries (where broader-than-needed data is being constrained in a callee rather than the calling site) that should've been expresed as `Alternative f` during parsing instead. That's the main point of the article, but you seem to only focus on the literal occurence of the word "validation" here and there.
- qsera 8mo agoSo you are saying that if at a certain point in parsing the only expected terms are 'a', 'b' and 'c', one should not put the corresponding parsed entry in a `char` (after checking it is either of these aka validating), and instead it should be put in some kind of enum type (parsed via `Alternative f`). Right?
- 8mo ago
- MarcLore 8mo ago[dead]
- bbkane 8mo agoI try to do this in Go as well, but the amount of pointers used to represent optional fields (more generally the lack of algebraic types) is making it less effective/ more verbose than I'd like
- barnacs 8mo agoEvery time you introduce a type for a "value invariant" you lose compatibility and force others to make cumbersome type conversions. To me, invalid values are best expressed with optional error returns along with the value that are part of the function signature. Types are best used to only encode information about the hierarchy of structures composed of primitive types. They help define and navigate the representation of composite things as opposed to just having dynamic nested maps of arbitrary strings.
- mrkeen 8mo ago> They help define and navigate the representation of composite things as opposed to just having dynamic nested maps of arbitrary strings. What would you say to someone who thinks that nested maps of arbitrary strings have maximum compatibility, and using types forces others to make cumbersome type conversions?
- barnacs 8mo agoIf the fields of a structure or the string keys of an untyped map don't match then you don't have compatibility either way. The same is not true for restricting the set of valid values. edit: To put it differently: To possibly be compatible with the nested "Circle" map, you need to know it is supposed to have a "Radius" key that is supposed to be a float. Type definitions just make this explicit. But just because your "Radius" can't be 0, you shouldn't make it incompatible with everything else operating on floats in general.
- ubixar 8mo agoC# gets close to this with records + pattern matching, F# discriminated unions are even better for this with algebraic data types built right in. A Result<'T,'Error> makes invalid states unrepresentable without any ceremony. C# records/matching works for now, but native DUs will make it even nicer.
- cadamsdotcom 8mo agoParsing over validation, and newtypes for everything, fall over when you don’t know the full range of possibilities that can occur in the wild. It is a handy way to prevent divide by zero as in the article, or to have fun with lambda calculus by asking the type system if 3 + 4 == 8. You can reason about the full range of inputs. Same for file format parsing - making as many failure modes as possible fail as early as possible! But be VERY wary of using them to represent business logic or state machines that allow only the transitions you believe can exist at this point in time. You just don’t know what wild things people will want to do in business logic, and if your software can’t handle those scenarios, people will just work around it and your stuff no longer matches reality.