6 ms·

What aspects of security are you seeking to understand vis-a-vis macros? Macros are "just" functions whose inputs are S-expressions and outputs are S-expressio

What aspects of security are you seeking to understand vis-a-vis macros? Macros are "just" functions whose inputs are S-expressions and outputs are S-expressions, and they are run at compile-time and not run-time. The semantics of the execution of a macro corresponds more or less to the semantics of Lisp itself, which, in a simplified view, can be completely understood in a first course in computer science (as in SICP). Any statements about correctness of a macro will derive from statements about correctness of code more broadly, for which there's a lot of literature and is an ongoing active and popular field of research. Any statements about the security implications of a macro will derive from (1) statements about the security of a compiler more generally and (2) statements about code more broadly, both of which have lots of literature, and both of which have active research communities. I don't know if there are many useful things to frame around security and correctness about macros specifically (without resorting to the above derivations), which is probably why there's no literature that I know of about it.

astrange 4y ago
I said security of Lisp, not of macros (or maybe I just meant to). Nb “my implementation is memory safe” as a response just means you don’t even know what the problems are - the world’s most popular Scheme implementation, JavaScript, has security issues all the time.
- reikonomusha 4y ago
  Do you mean of a programming language specification (e.g., ANSI Common Lisp, RnRS Scheme), or of a specific implementation? There is no the implementation of JavaScript. There are several, all with their own issues, security and otherwise.
  - astrange 4y ago
    Any arbitrary implementation. Do specifications have security issues? (The implementation is usually the one with the security issues, not the programs you run on it. Of course, either one’s possible if it crosses a trust boundary.)
    - reikonomusha 4y ago
      Open-source projects like Steel Bank Common Lisp (SBCL) and commercial projects like Allegro Common Lisp alike respond to bugs reports of all manners, including security issues. I can't speak for Google, but looking at their commit history on SBCL, you can see various security-improving enhancements through the years, especially around the runtime written in C. I don't think anybody has done a formal study or written some kind of report on the security of said implementations, however. A gentleman named Paul Dietz implemented something called the "ANSI compliance test suite" [1], which is an implementation-agnostic battery of tests that examines how well an implementation adheres to the ANSI specification of Common Lisp. Dietz is also known for his prolific fuzz and randomized testing of Common Lisp implementations. He's been a source of obscure bug reports for years, and those bugs are always promptly fixed. [1] https://ansi-test.common-lisp.dev/ https://ansi-test.common-lisp.dev/
      - astrange 4y ago
        Thanks, that is informative, although now I’m confused why Google cares about SBCL. That seems like a Yegge-only thing. I did think of a specification-level security issue - bignums reduce math errors but mean operations are O(N), which puts you at great risk of DoS issues and sidechannel leaks in security-sensitive math. And of course Lisp’s features like macros tend to be powerful, and security has a “rule of least power” meaning those are harder to verify if you can’t statically analyze them.
        reikonomusha 4y ago
        Google cares about SBCL because their flight search software that powers Google Flights, Orbitz, Kayak, etc. is written in Common Lisp.
    - lispm 4y ago
      > Do specifications have security issues? Sure, the Common Lisp standard specifies a default syntax mode, where the s-expression reader (-> the function READ) can execute arbitrary code at read time.
- pronlover723 4y ago
  I see this claim often. I'm not seeing how JavaScript is anything like Scheme. Scheme still ingests s-expressions and you can build compile time macros and manipulate those s-expressions at compile time before you get to runtime. What is the equivalent in JavaScript?
  - astrange 4y ago
    JavaScript is literally a Scheme “implementation”, as in it started as Scheme in a browser but they removed the s-exprs (the interface) and kept the implementation. It does still have the other similarities; it has closures and a prototype based object system. In the 90s we only had boring Blub languages, so most of the influence that allowed such powerful concepts came from Lisp. (Though I think the prototypes are from Self.)
    - agumonkey 4y ago
      You say it yourself, js is part scheme part self, and the algol { clothing } shift it a large step away from scheme since the functional aspects became culturally secondary for more than a decade. The imperative aspects are the one causing bugs IMO.
      - astrange 4y ago
        Well remember “functional vs imperative” isn’t a real distinction - Lisp is imperative. That showed up elsewhere where the “pure functional” languages like Haskell are the not-imperative ones. JS has other differences on the “correctness” spectrum, like all numbers are doubles instead of bignums. But even if that’s “incorrect”, just imagine how much more memory your Chrome tabs would use if you let people use bignums everywhere.
        agumonkey 4y ago
        Scheme pushed for functional much more than early lisp (and Id love for early lispers to pitch in about the imperative traits of lisp, it's a bit fuzzy if it was a desired trait or a commercially necessary addition facing fortan). Bye, why would a fully numeric tower js only use the least machine friendly format everywhere ? Did I miss something ?

astrange 4y ago

I said security of Lisp, not of macros (or maybe I just meant to). Nb “my implementation is memory safe” as a response just means you don’t even know what the problems are - the world’s most popular Scheme implementation, JavaScript, has security issues all the time.

reikonomusha 4y ago
Do you mean of a programming language specification (e.g., ANSI Common Lisp, RnRS Scheme), or of a specific implementation? There is no the implementation of JavaScript. There are several, all with their own issues, security and otherwise.
- astrange 4y ago
  Any arbitrary implementation. Do specifications have security issues? (The implementation is usually the one with the security issues, not the programs you run on it. Of course, either one’s possible if it crosses a trust boundary.)
  - reikonomusha 4y ago
    Open-source projects like Steel Bank Common Lisp (SBCL) and commercial projects like Allegro Common Lisp alike respond to bugs reports of all manners, including security issues. I can't speak for Google, but looking at their commit history on SBCL, you can see various security-improving enhancements through the years, especially around the runtime written in C. I don't think anybody has done a formal study or written some kind of report on the security of said implementations, however. A gentleman named Paul Dietz implemented something called the "ANSI compliance test suite" [1], which is an implementation-agnostic battery of tests that examines how well an implementation adheres to the ANSI specification of Common Lisp. Dietz is also known for his prolific fuzz and randomized testing of Common Lisp implementations. He's been a source of obscure bug reports for years, and those bugs are always promptly fixed. [1] https://ansi-test.common-lisp.dev/ https://ansi-test.common-lisp.dev/
    - astrange 4y ago
      Thanks, that is informative, although now I’m confused why Google cares about SBCL. That seems like a Yegge-only thing. I did think of a specification-level security issue - bignums reduce math errors but mean operations are O(N), which puts you at great risk of DoS issues and sidechannel leaks in security-sensitive math. And of course Lisp’s features like macros tend to be powerful, and security has a “rule of least power” meaning those are harder to verify if you can’t statically analyze them.
      - reikonomusha 4y ago
        Google cares about SBCL because their flight search software that powers Google Flights, Orbitz, Kayak, etc. is written in Common Lisp.
  - lispm 4y ago
    > Do specifications have security issues? Sure, the Common Lisp standard specifies a default syntax mode, where the s-expression reader (-> the function READ) can execute arbitrary code at read time.
pronlover723 4y ago
I see this claim often. I'm not seeing how JavaScript is anything like Scheme. Scheme still ingests s-expressions and you can build compile time macros and manipulate those s-expressions at compile time before you get to runtime. What is the equivalent in JavaScript?
- astrange 4y ago
  JavaScript is literally a Scheme “implementation”, as in it started as Scheme in a browser but they removed the s-exprs (the interface) and kept the implementation. It does still have the other similarities; it has closures and a prototype based object system. In the 90s we only had boring Blub languages, so most of the influence that allowed such powerful concepts came from Lisp. (Though I think the prototypes are from Self.)
  - agumonkey 4y ago
    You say it yourself, js is part scheme part self, and the algol { clothing } shift it a large step away from scheme since the functional aspects became culturally secondary for more than a decade. The imperative aspects are the one causing bugs IMO.
    - astrange 4y ago
      Well remember “functional vs imperative” isn’t a real distinction - Lisp is imperative. That showed up elsewhere where the “pure functional” languages like Haskell are the not-imperative ones. JS has other differences on the “correctness” spectrum, like all numbers are doubles instead of bignums. But even if that’s “incorrect”, just imagine how much more memory your Chrome tabs would use if you let people use bignums everywhere.
      - agumonkey 4y ago
        Scheme pushed for functional much more than early lisp (and Id love for early lispers to pitch in about the imperative traits of lisp, it's a bit fuzzy if it was a desired trait or a commercially necessary addition facing fortan). Bye, why would a fully numeric tower js only use the least machine friendly format everywhere ? Did I miss something ?