3 ms·
Lisp advocacy (for the last few decades) is always about how you can write anything in 10 lines of macros, but I never see anyone discuss security or whether th
by astrange 4y ago
Lisp advocacy (for the last few decades) is always about how you can write anything in 10 lines of macros, but I never see anyone discuss security or whether the result actually does what you meant it to do.
- reikonomusha 4y agoWhat aspects of security are you seeking to understand vis-a-vis macros? Macros are "just" functions whose inputs are S-expressions and outputs are S-expressions, and they are run at compile-time and not run-time. The semantics of the execution of a macro corresponds more or less to the semantics of Lisp itself, which, in a simplified view, can be completely understood in a first course in computer science (as in SICP). Any statements about correctness of a macro will derive from statements about correctness of code more broadly, for which there's a lot of literature and is an ongoing active and popular field of research. Any statements about the security implications of a macro will derive from (1) statements about the security of a compiler more generally and (2) statements about code more broadly, both of which have lots of literature, and both of which have active research communities. I don't know if there are many useful things to frame around security and correctness about macros specifically (without resorting to the above derivations), which is probably why there's no literature that I know of about it.
- astrange 4y agoI said security of Lisp, not of macros (or maybe I just meant to). Nb “my implementation is memory safe” as a response just means you don’t even know what the problems are - the world’s most popular Scheme implementation, JavaScript, has security issues all the time.
- reikonomusha 4y agoDo you mean of a programming language specification (e.g., ANSI Common Lisp, RnRS Scheme), or of a specific implementation? There is no the implementation of JavaScript. There are several, all with their own issues, security and otherwise.
- astrange 4y agoAny arbitrary implementation. Do specifications have security issues? (The implementation is usually the one with the security issues, not the programs you run on it. Of course, either one’s possible if it crosses a trust boundary.)
- reikonomusha 4y agoOpen-source projects like Steel Bank Common Lisp (SBCL) and commercial projects like Allegro Common Lisp alike respond to bugs reports of all manners, including security issues. I can't speak for Google, but looking at their commit history on SBCL, you can see various security-improving enhancements through the years, especially around the runtime written in C. I don't think anybody has done a formal study or written some kind of report on the security of said implementations, however. A gentleman named Paul Dietz implemented something called the "ANSI compliance test suite" [1], which is an implementation-agnostic battery of tests that examines how well an implementation adheres to the ANSI specification of Common Lisp. Dietz is also known for his prolific fuzz and randomized testing of Common Lisp implementations. He's been a source of obscure bug reports for years, and those bugs are always promptly fixed. [1] https://ansi-test.common-lisp.dev/ https://ansi-test.common-lisp.dev/
- astrange 4y agoThanks, that is informative, although now I’m confused why Google cares about SBCL. That seems like a Yegge-only thing. I did think of a specification-level security issue - bignums reduce math errors but mean operations are O(N), which puts you at great risk of DoS issues and sidechannel leaks in security-sensitive math. And of course Lisp’s features like macros tend to be powerful, and security has a “rule of least power” meaning those are harder to verify if you can’t statically analyze them.
- reikonomusha 4y agoGoogle cares about SBCL because their flight search software that powers Google Flights, Orbitz, Kayak, etc. is written in Common Lisp.
- pronlover723 4y agoI see this claim often. I'm not seeing how JavaScript is anything like Scheme. Scheme still ingests s-expressions and you can build compile time macros and manipulate those s-expressions at compile time before you get to runtime. What is the equivalent in JavaScript?
- astrange 4y agoJavaScript is literally a Scheme “implementation”, as in it started as Scheme in a browser but they removed the s-exprs (the interface) and kept the implementation. It does still have the other similarities; it has closures and a prototype based object system. In the 90s we only had boring Blub languages, so most of the influence that allowed such powerful concepts came from Lisp. (Though I think the prototypes are from Self.)
- agumonkey 4y agoYou say it yourself, js is part scheme part self, and the algol { clothing } shift it a large step away from scheme since the functional aspects became culturally secondary for more than a decade. The imperative aspects are the one causing bugs IMO.
- astrange 4y agoWell remember “functional vs imperative” isn’t a real distinction - Lisp is imperative. That showed up elsewhere where the “pure functional” languages like Haskell are the not-imperative ones. JS has other differences on the “correctness” spectrum, like all numbers are doubles instead of bignums. But even if that’s “incorrect”, just imagine how much more memory your Chrome tabs would use if you let people use bignums everywhere.
- agumonkey 4y agoScheme pushed for functional much more than early lisp (and Id love for early lispers to pitch in about the imperative traits of lisp, it's a bit fuzzy if it was a desired trait or a commercially necessary addition facing fortan). Bye, why would a fully numeric tower js only use the least machine friendly format everywhere ? Did I miss something ?
- chrisseaton 4y agoWhat do macros have to do with security? And what do you mean about doing what you mean it to? The semantics of macro expansion are well-defined.
- astrange 4y agoYou’re aware that it’s possible to make mistakes while writing a program, yes? What are the techniques to avoid doing this?
- chrisseaton 4y agoI don’t get the point though? Does Lisp do anything more than other similar languages to help you avoid mistakes in meaning? I guess not. But it doesn’t claim to, as you say, so what’s the problem? Lisp advocates don’t talk about something Lisp doesn’t claim to do and most other languages don’t do. Why’s that surprising?
- astrange 4y agoWell, the unique bit is images and homoiconic metaprogramming, not just metaprogramming. You can get that unsafely (code generation scripts) or safely (program extraction from a proof language) a lot of other places. But the main argument is that Lisp is hyperproductive if you’re an individual brain genius who’s memorized every emacs key combo. And my point is that it doesn’t matter if you can produce something fast if you can’t know if that thing actually works. This is the traditional “static vs dynamic typing” argument but you know, a lot of new languages have new kinds of safety features going beyond “static typing”. And of course Lisp does make unusual decisions affecting safety. Making all numbers bignums means math is more likely to be correct since it won’t overflow, but untrusted user input can pretty easily DoS you.
- deleted 4y ago[deleted]
- hayley-patton 4y ago
- agumonkey 4y agoAbove the 10loc macro it's also identifying larger scale structure / needs in your logic but make it as invisible as possible in your code, as opposed to verbose indirections. And I believe there are 40+ years of people ensuring macros do what they're supposed to, and discovering all the subtle failures, in many dimensions. You can have OnLisp semi hygienic raw macros, scheme hygienic macro systems, Matthew Flatt siloed macro systems.. same goes for other metaprogrammable languages. I guess same goes for security.
- drog 4y agoSmaller programs are easier to understand and often you can be sure that they are correct just by looking at them. Macros allows you to decompose and simplify problems in a way that is impossible with functions. Macros are especially good at removing boilerplate and writing syntactic glue code. Since macros are programs making sure that they do what you meant them to do you can use similar techniques as you use for other programs: making them small and obvious, testing.