5 ms·

I have some cons! Last time I checked on it, QuickLisp doesn't support fetching packages over anything except for plain http, with no encryption and no verific

I have some cons! Last time I checked on it, QuickLisp doesn't support fetching packages over anything except for plain http, with no encryption and no verification mechanism in place to detect files that may have been tampered with during transmission. I think not supporting encryption or authentication for something as important as fetching source code makes QL a non-starter for me and hopefully for anyone else who cares about security. Another issue I have ran into, is that SBCL is hosted on sourceforge, which has in the past injected malware into projects downloadable archives! I consider this to also be a security issue, and sourceforge in general is not pleasant to work with. I don't think there are any valid reasons to continue to use sourceforge today, so why such an important project continues to use it confuses me a lot. I don't see these issues mentioned by anyone else which is bizarre to me. I really like lisps and common lisp specifically but things like this has driven me away from using it and it doesn't appear that anyone cares about fixing these things.

schemescape 3y ago
Quicklisp doesn’t use TLS or signatures? How have I not heard this before? That would be unbelievably irresponsible. Has this really not been addressed by the CL community? Edit: here’s the issue: https://github.com/quicklisp/quicklisp-client/issues/167 https://github.com/quicklisp/quicklisp-client/issues/167 Thanks for bringing this up!
- koito17 3y ago
  The reason for this is quite simple: portability. Quicklisp also uses plain TAR files to distribute dists. Why? Because quicklisp has a built-in TAR extractor written in 100% standard/portable CL. This allows Quicklisp to run on just about everything, from your computer to real LispMs and operating systems like Mezzano. TLS comes up every time someone discusses Quicklisp, but nobody bothers to go ahead and actually implement it portably (and even if they did, have fun with performance and side channel attacks, both of which require you to break portability to implement well for every platform you want to target). If you would like a more stereotypical package manager, consider using CLPM. Though one of the big reasons to use CLPM is not encryption IMO, but versioning. ASDF supports locking versions of dependencies, but Quicklisp doesnt ever use this and instead constantly pushes latest of everything from git repositories. This IMO sucks a lot more than using plain HTTP given that this actually breaks code, whereas some MITM from plain HTTP connection to Quicklisp would require so much coordination (and specificity of target) that it's just not in my threat model at all.
  - schemescape 3y ago
    > Quicklisp doesnt ever use this and instead constantly pushes latest of everything from git repositories Yeah, I didn't recall off hand, but this was one of my main complaints with Quicklisp vs. other package managers I've used (for other ecosystems--not CL). > whereas some MITM from plain HTTP connection to Quicklisp would require so much coordination (and specificity of target) that it's just not in my threat model at all I hope you're right, but it still seems like an unnecessary risk. Even if I can't imagine a scenario where someone is able to MITM me (or, more likely, a server I'm deploying code to), there's still the lingering feeling that it's possible. I certainly wouldn't download an executable over HTTP and run it, and downloading library code is fairly similar (although easier to inspect, at least).
  - deleted 3y ago
    [deleted]
  - ristos 3y ago
    This does keep coming up, and it's a few years old now. I think Quicklisp can easily still support https while supporting the older packages that are tar+http, which could easily be mirrored in a git repo. Quicklisp has unfortunately taken over the entire ecosystem, making it hard to use anything else, and you often need to depend on it to use a lot of tools in the ecosystem. It sort of reminds me of Systemd in that way. I agree on the version pinning being a worse situation, and also not having something like "node_modules" for lisp. I haven't tried CLPM since a while back, it was kind of hard to setup back then. I have a little package manager thing, cl-micropm, that just uses Quicklisp to fetch everything via docker (should probably support podman too), and an .envrc file to tell ASDF to look in the project directory (a project-local node_modules-like folder called "lisp-systems") for systems. That way I can pin my deps manually by picking the commits + git submodule in lisp-systems/, and it's isolated to my local project. I looked into using the Docker container to rewrite the requests to use https, bypassing whatever Quicklisp is doing, but I never got around to that. I'm looking to switch it to something even simpler/explicit though, cl-pm, that'll only optionally need/use Quicklisp via podman _only_ to figure out what the dependencies are, and then just have a function that uses wget/curl/git-pull to conveniently explicitly pull them in on request. That way you can decide to add a git mirror for an old http library, or pin a specific version, etc. It's slightly more manual than Quicklisp or CLPM, not a big deal, but very easy for anyone with just a little bit of lisp knowledge to understand the whole thing in under an hour.
- atgreen 3y ago
  Try ocicl instead of quicklisp. System tarballs are hosted in an OCI registry, and are downloaded via TLS connections (obeying proxies). Tarballs are signed and signatures are stored in the sigstore rekor transparency log for later inspection. https://github.com/ocicl/ocicl https://github.com/ocicl/ocicl
  - schemescape 3y ago
    Just wanted to say I did see your other comment and am intrigued by ocicl. Thanks!
sussmannbaka 3y ago
> I have some cons I’m sure you do :°)
ristos 3y ago
These issues get mentioned a lot, you just haven't noticed I guess. Sourceforge is also an issue with some C libraries too, I'm guessing because it was done a long time ago? not sure. I use ECL because it has really good C interop. It actually lets you inline C and access macros directly, making it a great glue language for C libraries. It's what I'm using it for now. I think you might even be able to avoid the GC entirely and use it to script C programs together in a performant way, by using the C FFI to allocate and manage the memory, including the ECL types, instead of the GC. And that's actually doable because of how good the inspector/debugger for lisp is. You can even inline assembly. I'm working on a bunch of CL stuff around this sort of thing, I plan to do a writeup of it and share it once I've developed it more. Lisp has it's downsides, but the C FFI/embeddability, along with the excellent low-level debugger/inspector, interactivity, and conditions and restarts, makes it worth the time for me to invest in it. And the stability of the language. My main gripe is the reader, but it's easy-ish enough to avoid the problems with named-readtables, or a simple lisp parser for `read` or whatever. I like Clojure, but it's missing some key stuff from the old lisp world that I'd love to see. Shadow-cljs is awesome.
momentoftop 3y ago
When I started using CL 20 years ago, libraries were stored on cliki and any malicious user could put malware there. Any source you asdf-installed was generally GPG signed and the installer automatically checked signatures against your personal trust-chain. Learning CL back then was my first introduction to GPG (and Emacs, and Linux)
- aidenn0 3y ago
  > When I started using CL 20 years ago, libraries were stored on cliki and any malicious user could put malware there. Any source you asdf-installed was generally GPG signed and the installer automatically checked signatures against your personal trust-chain. Which, in practice, involved downloading GPG public keys from cliki because I didn't know every single CL developer.
vindarel 3y ago
Solutions for the lack of https: - add in https://github.com/rudolfochrist/ql-https https://github.com/rudolfochrist/ql-https (downloads packages with curl) - use another package manager, CLPM: https://www.clpm.dev https://www.clpm.dev (or the newest ocicl) > CLPM comes as a pre-built binary, supports HTTPS by default, supports installing multiple package versions, supports versioned systems, and more. - use mitmproxy: https://hiphish.github.io/blog/2022/03/19/securing-quicklisp-through-mitmproxy/ https://hiphish.github.io/blog/2022/03/19/securing-quicklisp...
lukego 3y ago
Nix has a really convenient new CL libraries packaging upstream now. That verifies everything with sha256. It's quite complete because it's seeded from Quicklisp and had more packages added on (aswell as their native library dependcies.) Nix isn't to everyone's taste but it demonstrates that you can treat security/reproducibility/etc as orthogonal to Quicklisp and Sourceforge (and to Lisp native tooling in general.)
LispSporks22 3y ago
> Last time I checked on it, QuickLisp doesn't support fetching packages over anything except for plain http, with no encryption and no verification mechanism in place to detect files that may have been tampered with during transmission. I know it's not an excuse, but it was fun as heck booting up "capital M" MacOS (9.2.1) and loading Quicklisp into MCL without any trouble. I'm not even sure that's a supported platform by Quicklisp. https://code.google.com/archive/p/mcl/ https://code.google.com/archive/p/mcl/
- p_l 3y ago
  I think there might have been bits and pieces somewhere to run Quicklisp on lisp machines... or at least ASDF, which is the core dependency

schemescape 3y ago

Quicklisp doesn’t use TLS or signatures? How have I not heard this before? That would be unbelievably irresponsible. Has this really not been addressed by the CL community? Edit: here’s the issue: https://github.com/quicklisp/quicklisp-client/issues/167 https://github.com/quicklisp/quicklisp-client/issues/167 Thanks for bringing this up!

koito17 3y ago
The reason for this is quite simple: portability. Quicklisp also uses plain TAR files to distribute dists. Why? Because quicklisp has a built-in TAR extractor written in 100% standard/portable CL. This allows Quicklisp to run on just about everything, from your computer to real LispMs and operating systems like Mezzano. TLS comes up every time someone discusses Quicklisp, but nobody bothers to go ahead and actually implement it portably (and even if they did, have fun with performance and side channel attacks, both of which require you to break portability to implement well for every platform you want to target). If you would like a more stereotypical package manager, consider using CLPM. Though one of the big reasons to use CLPM is not encryption IMO, but versioning. ASDF supports locking versions of dependencies, but Quicklisp doesnt ever use this and instead constantly pushes latest of everything from git repositories. This IMO sucks a lot more than using plain HTTP given that this actually breaks code, whereas some MITM from plain HTTP connection to Quicklisp would require so much coordination (and specificity of target) that it's just not in my threat model at all.
- schemescape 3y ago
  > Quicklisp doesnt ever use this and instead constantly pushes latest of everything from git repositories Yeah, I didn't recall off hand, but this was one of my main complaints with Quicklisp vs. other package managers I've used (for other ecosystems--not CL). > whereas some MITM from plain HTTP connection to Quicklisp would require so much coordination (and specificity of target) that it's just not in my threat model at all I hope you're right, but it still seems like an unnecessary risk. Even if I can't imagine a scenario where someone is able to MITM me (or, more likely, a server I'm deploying code to), there's still the lingering feeling that it's possible. I certainly wouldn't download an executable over HTTP and run it, and downloading library code is fairly similar (although easier to inspect, at least).
- deleted 3y ago
  [deleted]
- ristos 3y ago
  This does keep coming up, and it's a few years old now. I think Quicklisp can easily still support https while supporting the older packages that are tar+http, which could easily be mirrored in a git repo. Quicklisp has unfortunately taken over the entire ecosystem, making it hard to use anything else, and you often need to depend on it to use a lot of tools in the ecosystem. It sort of reminds me of Systemd in that way. I agree on the version pinning being a worse situation, and also not having something like "node_modules" for lisp. I haven't tried CLPM since a while back, it was kind of hard to setup back then. I have a little package manager thing, cl-micropm, that just uses Quicklisp to fetch everything via docker (should probably support podman too), and an .envrc file to tell ASDF to look in the project directory (a project-local node_modules-like folder called "lisp-systems") for systems. That way I can pin my deps manually by picking the commits + git submodule in lisp-systems/, and it's isolated to my local project. I looked into using the Docker container to rewrite the requests to use https, bypassing whatever Quicklisp is doing, but I never got around to that. I'm looking to switch it to something even simpler/explicit though, cl-pm, that'll only optionally need/use Quicklisp via podman _only_ to figure out what the dependencies are, and then just have a function that uses wget/curl/git-pull to conveniently explicitly pull them in on request. That way you can decide to add a git mirror for an old http library, or pin a specific version, etc. It's slightly more manual than Quicklisp or CLPM, not a big deal, but very easy for anyone with just a little bit of lisp knowledge to understand the whole thing in under an hour.
atgreen 3y ago
Try ocicl instead of quicklisp. System tarballs are hosted in an OCI registry, and are downloaded via TLS connections (obeying proxies). Tarballs are signed and signatures are stored in the sigstore rekor transparency log for later inspection. https://github.com/ocicl/ocicl https://github.com/ocicl/ocicl
- schemescape 3y ago
  Just wanted to say I did see your other comment and am intrigued by ocicl. Thanks!

sussmannbaka 3y ago

> I have some cons I’m sure you do :°)

ristos 3y ago

These issues get mentioned a lot, you just haven't noticed I guess. Sourceforge is also an issue with some C libraries too, I'm guessing because it was done a long time ago? not sure. I use ECL because it has really good C interop. It actually lets you inline C and access macros directly, making it a great glue language for C libraries. It's what I'm using it for now. I think you might even be able to avoid the GC entirely and use it to script C programs together in a performant way, by using the C FFI to allocate and manage the memory, including the ECL types, instead of the GC. And that's actually doable because of how good the inspector/debugger for lisp is. You can even inline assembly. I'm working on a bunch of CL stuff around this sort of thing, I plan to do a writeup of it and share it once I've developed it more. Lisp has it's downsides, but the C FFI/embeddability, along with the excellent low-level debugger/inspector, interactivity, and conditions and restarts, makes it worth the time for me to invest in it. And the stability of the language. My main gripe is the reader, but it's easy-ish enough to avoid the problems with named-readtables, or a simple lisp parser for `read` or whatever. I like Clojure, but it's missing some key stuff from the old lisp world that I'd love to see. Shadow-cljs is awesome.

momentoftop 3y ago

When I started using CL 20 years ago, libraries were stored on cliki and any malicious user could put malware there. Any source you asdf-installed was generally GPG signed and the installer automatically checked signatures against your personal trust-chain. Learning CL back then was my first introduction to GPG (and Emacs, and Linux)

aidenn0 3y ago
> When I started using CL 20 years ago, libraries were stored on cliki and any malicious user could put malware there. Any source you asdf-installed was generally GPG signed and the installer automatically checked signatures against your personal trust-chain. Which, in practice, involved downloading GPG public keys from cliki because I didn't know every single CL developer.

vindarel 3y ago

Solutions for the lack of https: - add in https://github.com/rudolfochrist/ql-https https://github.com/rudolfochrist/ql-https (downloads packages with curl) - use another package manager, CLPM: https://www.clpm.dev https://www.clpm.dev (or the newest ocicl) > CLPM comes as a pre-built binary, supports HTTPS by default, supports installing multiple package versions, supports versioned systems, and more. - use mitmproxy: https://hiphish.github.io/blog/2022/03/19/securing-quicklisp-through-mitmproxy/ https://hiphish.github.io/blog/2022/03/19/securing-quicklisp...

lukego 3y ago

Nix has a really convenient new CL libraries packaging upstream now. That verifies everything with sha256. It's quite complete because it's seeded from Quicklisp and had more packages added on (aswell as their native library dependcies.) Nix isn't to everyone's taste but it demonstrates that you can treat security/reproducibility/etc as orthogonal to Quicklisp and Sourceforge (and to Lisp native tooling in general.)

LispSporks22 3y ago

> Last time I checked on it, QuickLisp doesn't support fetching packages over anything except for plain http, with no encryption and no verification mechanism in place to detect files that may have been tampered with during transmission. I know it's not an excuse, but it was fun as heck booting up "capital M" MacOS (9.2.1) and loading Quicklisp into MCL without any trouble. I'm not even sure that's a supported platform by Quicklisp. https://code.google.com/archive/p/mcl/ https://code.google.com/archive/p/mcl/

p_l 3y ago
I think there might have been bits and pieces somewhere to run Quicklisp on lisp machines... or at least ASDF, which is the core dependency