4 ms·

Type declarations in Common Lisp are unsafe. Violating a declaration at runtime is UB. What SBCL is doing to Common Lisp is similar to what C compiler vendors

Type declarations in Common Lisp are unsafe. Violating a declaration at runtime is UB. What SBCL is doing to Common Lisp is similar to what C compiler vendors did to C. In order to win at benchmarks, C compiler vendors started to use UB as a license to miscompile optimized code, which got them better benchmarks. Common Lisp has very little UB, so the C strategy is harder to execute. SBCL basically first needs to sucker their users into overusing one corner of the language that does have UB, type declarations. They get their benchmarks, at the cost of saturating the ecosystem with code that overuses UB, basically poisoning the ecosystem.

kbp 8y ago
> What SBCL is doing to Common Lisp is similar to what C compiler vendors did to C. In order to win at benchmarks, C compiler vendors started to use UB as a license to miscompile optimized code, which got them better benchmarks. I sort of see what you're getting at, but I think what SBCL does isn't really comparable to the C situation. The complaint people have with UB in C is compiler-writers treating it as a licence to do very unintuitive optimisations; it uses it as a justification for being extremely hostile to the programmer and then the defence is "well the standard said the behaviour was undefined, why would you think there's "obvious" code for the compiler to emit here?". In the SBCL situation, the standard doesn't guarantee any behaviour around that feature, and SBCL is using it as a licence to be more helpful to the programmer, so I think most of the issues people might have with the C situation don't really transfer. For the great majority of programs, the difference doesn't matter either: breaking type declarations being an error (as in SBCL) vs being undefined (as in portable CL) aren't incompatible, as long as your program doesn't rely on a condition being signalled when you violate a type declaration. I can see how relying on SBCL's interpretation of the standard would cause problems, but when the difference is that SBCL just changed "the behaviour is undefined" to "it is an error", I don't see that introducing a wave of unportable code. > They get their benchmarks Type declarations that can't be statically verified transparently degrade to runtime assertions; I don't think that choice was made to do better on benchmarks. > at the cost of saturating the ecosystem with code that overuses UB, basically poisoning the ecosystem. I don't think there's that much code out there that relies on errors being signalled when it violates type declarations as a part of normal execution (quick, link a few projects that make use of this), certainly not enough to say that trend poisoned the ecosystem.
stassats 8y ago
That's not at all what SBCL is doing. Each type declaration is treated as an assertion, and only then it performs optimization. And there is no undefined behavior at all. And even without type declarations SBCL is actively deriving types and optimizing things that are proved to have a restricted type. Spreading misinformation like this is what's poisoning the ecosystem.
- kbp 8y ago
  > And there is no undefined behavior at all. Well, there is. Violating type declarations is UB in Common Lisp, the only thing the standard says about it is "the consequences are undefined". In SBCL, an error will be signalled, but there's no guarantees about what other compilers might do. This could show up if a Lisp programmer who primarily used SBCL was relying on the type system to do some of their input validation for them when portable code would have to actually check it (ie, in their head they might think their program will raise an error if you pass it garbage, when really it just invokes UB if you pass it garbage). I still don't think it's that big of an issue, and mentioning it alongside benchmarks certainly indicates some misunderstanding, but it's not completely baseless.
  - lispm 8y ago
    It's only undefined in the standard. Most implementations document their behavior somehow. Generally I run my code with safety 3 and only selected portions might have unsafe code - but with args check before in surrounding code.
  - stassats 8y ago
    That is theoretically true, but what implementation really does that with SAFETY being 3? Claiming that SBCL following the standard is poisoning anything is completely baseless, though.
    - kbp 8y ago
      > what implementation really does that with SAFETY being 3? LispWorks: CL-USER 1 > (declaim (optimize safety)) NIL CL-USER 2 > (defun f (x) (declare (fixnum x)) x) F CL-USER 3 > (f "foo") "foo"
lispm 8y ago
SBCL checks type declarations at compile time and at runtime. The default fully safe code in SBCL is already fast enough for many cases. You need to check the manual of SBCL sometime. http://sbcl.org/manual/index.html#Handling-of-Types http://sbcl.org/manual/index.html#Handling-of-Types
- YouAreGreat 8y ago
  Thanks. Seems my assessment and SBCL's self-description match perfectly, as they explicitly aim to "reward the use of type declarations throughout development" and exhort users to "always declare the types of function arguments and structure slots as precisely as possible". They do want their users to use one of Common Lisp's least safe features pervasively. If the point was to mine assertions for type information, they could have encouraged the use of assertions. But they don't. They encourage the use of declarations, which is entirely not the same thing, and it's a bad thing for the Common Lisp source code ecosystem, even if not for the SBCL ecosystem. Hurray for SBCL.
  - lispm 8y ago
    > They do want their users to use one of Common Lisp's least safe features pervasively It's not an unsafe Common Lisp feature. The standard does say nothing about its unsafeness. Only implementations may be unsafe. The Common Lisp standard says nothing how the implementation deal with type declarations. There is a wide range of behavior in Common Lisp implementations dealing with type declarations. Some will do nothing with type declarations - thus its not more or less safe if one define types. Some will add more runtime checks when types are declare. Thus the code is MORE safe at runtime. Some compilers may add less runtime checks and will create specialized code -> less safe. SBCL provides several ways to deal with that depending on compiler settings. That SBCL advanced features are a disadvantage is nonsense.
    - YouAreGreat 8y ago
      You skipped the important part: > If the point was to mine assertions for type information, they could have encouraged the use of assertions. With that in mind: > Some will add more runtime checks when types are declare. Thus the code is MORE safe at runtime. Users can portably and safely use assertions if they want that behavior. Encouraging the use of declarations instead is neither portable nor safe since, as you said: > Some compilers may add less runtime checks and will create specialized code -> less safe.
dang 8y ago
> poisoning the ecosystem Please don't take HN threads into programming language flamewar; we've all seen what this leads to elsewhere. It's both possible and much preferable to simply make your points flame-free. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- YouAreGreat 8y ago
  You are right, I misjudged the inflammatory power of my choice of phrasing.
flavio81 8y ago
>Type declarations in Common Lisp are unsafe. Violating a declaration at runtime is UB. What SBCL is doing to Common Lisp is similar to what C compiler vendors did to C. In order to win at benchmarks, C compiler vendors started to use UB as a license to miscompile optimized code, which got them better benchmarks. Fear, uncertainty and doubt. Type declarations are part of the ANSI Common Lisp standard. Lisp implementations can decide what to do with type mismatches (thus the UB), but how can this be a problem? Code will still be standards-compliant.
kazinator 8y ago
Type declarations in Common Lisp support (safety <n>) and (speed <n>) syntax for configuring, you guessed it, safety and speed. The undefined behavior kicks in when you explicitly defeat safety. (safety 0) (speed 3) means "throw out the life jackets and assume that these declarations are true, making the code as fast as possible". If you don't have the confidence for this over some piece of code, then just ... don't do that. Safety is on by default. If some implementation changes that, that is bad. The C language lacks any means to specify these preferences over a specific scope of code. Some compilers have certain #pragma directives, and beyond that, there are only module-level compiler flags. Those don't specifically give you safety. Usually there is just an optimization level. If optimization is omitted or perhaps specified as -O0 or whatever, that makes code sort of behave more safely in some handwaving manner in that certain "confusing, optimizy things" don't happen. Also note that Lisp has a defined-order of evaluation. A function call like (f (inc i) (inc i)) is not comparable to the C f(++i, ++i) because it performs the left increment, obtains its value, then the right increment, and obtains its value. Also note that C requires declarations everywhere. Lisp declarations are optional. Before Lisp declarations can be used in a wrong way that causes a malfunction, they have to be used, period. We can use Lisp declarations to cause an integer overflow, like in C. If we don't use them, the integer type automatically prevents them by switching to bignums. In C, even the function which is called once, main, is stuffed with declarations and compiled for speed, often with optimization on in actual real-world projects. The SBCL people would have a long way to go before they screw up anything to the level of C.
- YouAreGreat 8y ago
  > Type declarations in Common Lisp support (safety <n>) and (speed <n>) syntax To be pedantic, those aren't type declarations. More importantly, the (un)safety of a type declaration isn't predicated on any specific safety/speed settings. The mere use of a type declaration implies that the programmer certifies their accuracy inside its scope; an additional optimize declaration is not required. It bears repeating: While Common Lisp has "type declarations", they don't behave as they do in statically typed languages. The are certificates, assurances by the programmer to be exploited at will by the implementation.
PuercoPop 8y ago
You are missing the point of the Lisp's type system. > There has been some confusion about the difference between type checking for the purposes of compiling traditional languages, and type checking for the purposes of ensuring a program's correctness. [0] Lisp type system is mainly designed for the former. Abstract data types are useful for the later use. [0]: http://home.pipeline.com/~hbaker1/TInference.html http://home.pipeline.com/~hbaker1/TInference.html

kbp 8y ago

> What SBCL is doing to Common Lisp is similar to what C compiler vendors did to C. In order to win at benchmarks, C compiler vendors started to use UB as a license to miscompile optimized code, which got them better benchmarks. I sort of see what you're getting at, but I think what SBCL does isn't really comparable to the C situation. The complaint people have with UB in C is compiler-writers treating it as a licence to do very unintuitive optimisations; it uses it as a justification for being extremely hostile to the programmer and then the defence is "well the standard said the behaviour was undefined, why would you think there's "obvious" code for the compiler to emit here?". In the SBCL situation, the standard doesn't guarantee any behaviour around that feature, and SBCL is using it as a licence to be more helpful to the programmer, so I think most of the issues people might have with the C situation don't really transfer. For the great majority of programs, the difference doesn't matter either: breaking type declarations being an error (as in SBCL) vs being undefined (as in portable CL) aren't incompatible, as long as your program doesn't rely on a condition being signalled when you violate a type declaration. I can see how relying on SBCL's interpretation of the standard would cause problems, but when the difference is that SBCL just changed "the behaviour is undefined" to "it is an error", I don't see that introducing a wave of unportable code. > They get their benchmarks Type declarations that can't be statically verified transparently degrade to runtime assertions; I don't think that choice was made to do better on benchmarks. > at the cost of saturating the ecosystem with code that overuses UB, basically poisoning the ecosystem. I don't think there's that much code out there that relies on errors being signalled when it violates type declarations as a part of normal execution (quick, link a few projects that make use of this), certainly not enough to say that trend poisoned the ecosystem.

stassats 8y ago

That's not at all what SBCL is doing. Each type declaration is treated as an assertion, and only then it performs optimization. And there is no undefined behavior at all. And even without type declarations SBCL is actively deriving types and optimizing things that are proved to have a restricted type. Spreading misinformation like this is what's poisoning the ecosystem.

kbp 8y ago
> And there is no undefined behavior at all. Well, there is. Violating type declarations is UB in Common Lisp, the only thing the standard says about it is "the consequences are undefined". In SBCL, an error will be signalled, but there's no guarantees about what other compilers might do. This could show up if a Lisp programmer who primarily used SBCL was relying on the type system to do some of their input validation for them when portable code would have to actually check it (ie, in their head they might think their program will raise an error if you pass it garbage, when really it just invokes UB if you pass it garbage). I still don't think it's that big of an issue, and mentioning it alongside benchmarks certainly indicates some misunderstanding, but it's not completely baseless.
- lispm 8y ago
  It's only undefined in the standard. Most implementations document their behavior somehow. Generally I run my code with safety 3 and only selected portions might have unsafe code - but with args check before in surrounding code.
- stassats 8y ago
  That is theoretically true, but what implementation really does that with SAFETY being 3? Claiming that SBCL following the standard is poisoning anything is completely baseless, though.
  - kbp 8y ago
    > what implementation really does that with SAFETY being 3? LispWorks: CL-USER 1 > (declaim (optimize safety)) NIL CL-USER 2 > (defun f (x) (declare (fixnum x)) x) F CL-USER 3 > (f "foo") "foo"

lispm 8y ago

SBCL checks type declarations at compile time and at runtime. The default fully safe code in SBCL is already fast enough for many cases. You need to check the manual of SBCL sometime. http://sbcl.org/manual/index.html#Handling-of-Types http://sbcl.org/manual/index.html#Handling-of-Types

YouAreGreat 8y ago
Thanks. Seems my assessment and SBCL's self-description match perfectly, as they explicitly aim to "reward the use of type declarations throughout development" and exhort users to "always declare the types of function arguments and structure slots as precisely as possible". They do want their users to use one of Common Lisp's least safe features pervasively. If the point was to mine assertions for type information, they could have encouraged the use of assertions. But they don't. They encourage the use of declarations, which is entirely not the same thing, and it's a bad thing for the Common Lisp source code ecosystem, even if not for the SBCL ecosystem. Hurray for SBCL.
- lispm 8y ago
  > They do want their users to use one of Common Lisp's least safe features pervasively It's not an unsafe Common Lisp feature. The standard does say nothing about its unsafeness. Only implementations may be unsafe. The Common Lisp standard says nothing how the implementation deal with type declarations. There is a wide range of behavior in Common Lisp implementations dealing with type declarations. Some will do nothing with type declarations - thus its not more or less safe if one define types. Some will add more runtime checks when types are declare. Thus the code is MORE safe at runtime. Some compilers may add less runtime checks and will create specialized code -> less safe. SBCL provides several ways to deal with that depending on compiler settings. That SBCL advanced features are a disadvantage is nonsense.
  - YouAreGreat 8y ago
    You skipped the important part: > If the point was to mine assertions for type information, they could have encouraged the use of assertions. With that in mind: > Some will add more runtime checks when types are declare. Thus the code is MORE safe at runtime. Users can portably and safely use assertions if they want that behavior. Encouraging the use of declarations instead is neither portable nor safe since, as you said: > Some compilers may add less runtime checks and will create specialized code -> less safe.

dang 8y ago

> poisoning the ecosystem Please don't take HN threads into programming language flamewar; we've all seen what this leads to elsewhere. It's both possible and much preferable to simply make your points flame-free. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

YouAreGreat 8y ago
You are right, I misjudged the inflammatory power of my choice of phrasing.

flavio81 8y ago

>Type declarations in Common Lisp are unsafe. Violating a declaration at runtime is UB. What SBCL is doing to Common Lisp is similar to what C compiler vendors did to C. In order to win at benchmarks, C compiler vendors started to use UB as a license to miscompile optimized code, which got them better benchmarks. Fear, uncertainty and doubt. Type declarations are part of the ANSI Common Lisp standard. Lisp implementations can decide what to do with type mismatches (thus the UB), but how can this be a problem? Code will still be standards-compliant.

kazinator 8y ago

Type declarations in Common Lisp support (safety <n>) and (speed <n>) syntax for configuring, you guessed it, safety and speed. The undefined behavior kicks in when you explicitly defeat safety. (safety 0) (speed 3) means "throw out the life jackets and assume that these declarations are true, making the code as fast as possible". If you don't have the confidence for this over some piece of code, then just ... don't do that. Safety is on by default. If some implementation changes that, that is bad. The C language lacks any means to specify these preferences over a specific scope of code. Some compilers have certain #pragma directives, and beyond that, there are only module-level compiler flags. Those don't specifically give you safety. Usually there is just an optimization level. If optimization is omitted or perhaps specified as -O0 or whatever, that makes code sort of behave more safely in some handwaving manner in that certain "confusing, optimizy things" don't happen. Also note that Lisp has a defined-order of evaluation. A function call like (f (inc i) (inc i)) is not comparable to the C f(++i, ++i) because it performs the left increment, obtains its value, then the right increment, and obtains its value. Also note that C requires declarations everywhere. Lisp declarations are optional. Before Lisp declarations can be used in a wrong way that causes a malfunction, they have to be used, period. We can use Lisp declarations to cause an integer overflow, like in C. If we don't use them, the integer type automatically prevents them by switching to bignums. In C, even the function which is called once, main, is stuffed with declarations and compiled for speed, often with optimization on in actual real-world projects. The SBCL people would have a long way to go before they screw up anything to the level of C.

YouAreGreat 8y ago
> Type declarations in Common Lisp support (safety <n>) and (speed <n>) syntax To be pedantic, those aren't type declarations. More importantly, the (un)safety of a type declaration isn't predicated on any specific safety/speed settings. The mere use of a type declaration implies that the programmer certifies their accuracy inside its scope; an additional optimize declaration is not required. It bears repeating: While Common Lisp has "type declarations", they don't behave as they do in statically typed languages. The are certificates, assurances by the programmer to be exploited at will by the implementation.

PuercoPop 8y ago

You are missing the point of the Lisp's type system. > There has been some confusion about the difference between type checking for the purposes of compiling traditional languages, and type checking for the purposes of ensuring a program's correctness. [0] Lisp type system is mainly designed for the former. Abstract data types are useful for the later use. [0]: http://home.pipeline.com/~hbaker1/TInference.html http://home.pipeline.com/~hbaker1/TInference.html