9 ms·
Using SBCL Common Lisp as a Dynamic Library
- math-dev 5y agoGreat post!
- christophilus 5y agoI love lispy languages— Clojure being the one I’m most familiar with, but it’s hard to give up static types. Anyone here worked on a large lisp codebase? How big was it? How big was the team? What’s the secret sauce?
- andi999 5y agoI havent. But my hunch is it works best for rapid prototyping with small teams.
- bachmeier 5y agoI suppose it depends on what you mean by "static types" since you can optionally specify types in Common Lisp. SBCL even does precise type checking[1]: "If a variable is declared to be (integer 3 17) then its value must always be an integer between 3 and 17. If multiple type declarations apply to a single variable, then all the declarations must be correct; it is as though all the types were intersected producing a single and type specifier." [1] http://www.sbcl.org/manual/#Handling-of-Types http://www.sbcl.org/manual/#Handling-of-Types
- maydup-nem 5y agoI would like to add that optimizations may also be performed in addition to type-checking (type-checking, by the way, depends on safety optimization level). What's also handy is that you can specify the optimization levels (space, safety, speed, debug) per function or per file (or in a `locally' block). Function return types may also be specified, with serapeum's ->, it may be as simple as (-> f (fixnum fixnum) boolean) for instance, right above the function. Classes and structs let you specify types for slots. Expressions may be typed with `the' special operator aka (the fixnum (first list)). PS CL has gradual typing.
- christophilus 5y agoDang. I didn’t know sbcl did this. I’ve never used it, but now I want to try. Thanks!
- bitwize 5y agoType systems at least as rich as Hindley-Milner are pretty much table stakes in 2022. Common Lisp's type system is not as rich as Hindley-Milner. I believe it doesn't even support user-defined parametric types (so it's kind of like pre-generics Go in that regard). It lets you type function args, but it doesn't incorporate the args' types into the function type. So no, Common Lisp's type system, even with static type checking enabled, is not adequate for software engineering in the present day. Coalton, however, solves many of these problems.
- deleted 5y ago[deleted]
- KerrAvon 5y agoWhat languages in common use in 2022 support a Hindley-Milner type system? This sounds a bit like pointless academic gatekeeping…
- bitwize 5y agoBasically, any statically typed language with parametric polymorphism, so Ada, C++, Java, C#, Rust, TypeScript, Python with static-typing support, post-Delphi Pascal, and recently Go all qualify.
- reikonomusha 5y agoParametric polymorphism != Hindley-Milner, not even basically. H-M is a construction for describing and ascertaining types of terms in a lambda calculus. Many of those languages don't even typically idiomatically achieve polymorphism through generic types (though of course it's possible), but instead from subtype and ad hoc polymorphism. I don't think it's accurate to imply their designs orbit H-M. I don't want to belabor on technicalities. Your original point was about "table stakes" for type systems in order to be relevant for modern-day software engineering. While I agree a language that attempts to incorporate fashionable computational constructs (like variable binding and lambda terms) ought to derive its type system from something H-M-esque—if only because it's so well understood—it's certainly not a requirement for the language to be taken seriously, and it's certainly not without significant peril in the presence of mutation constructions. I honestly think C, vanilla Common Lisp, and pre-generic Go are sufficiently disparate examples of that.
- cardanome 5y agoYou might want to look into Coalton as statically typed, functional language on top of Common Lisp: https://github.com/coalton-lang/coalton https://github.com/coalton-lang/coalton That said, Common Lisp while dynamically typed, is also strongly typed. Many type errors are prevented at runtime. Also you can declare types and have some limited form of static type checking in SBCL.
- fiddlerwoaroof 5y agoOne advantage of lisps is that you work at run time. So, you can access runtime information in your editor, which makes up for some of the limitations of dynamic types: static types propagate information ahead of time that a Lisp can discover through introspection.
- agumonkey 5y agoThere was a thread about Franz Lisp founder where he explains that DARPA D.A.R.T project was done in C, about 30M LoC, and failed hard, while the Lisp implementation was running nicely, easy to fix in hours not days. I'm not an experienced lisper, but it's not the first time I read this kind of report.
- rwallace 5y agoThat sounds interesting! Google doesn't find any mention of that, only https://en.wikipedia.org/wiki/Dynamic_Analysis_and_Replanning_Tool https://en.wikipedia.org/wiki/Dynamic_Analysis_and_Replannin... that says DART, far from failing hard, was a huge success. Do you have a link to the thread?
- agumonkey 5y agohere https://news.ycombinator.com/item?id=31044510 https://news.ycombinator.com/item?id=31044510 btw, I, too, couldn't find traces of the lisp side of this project after a 15 minute search. I was curious about the fix times and lisp codebase size
- rwallace 5y agoThanks! Right, so the successful version was the one written in Lisp. Okay, that's actually understandable. At that time, managed languages were not widely used, and C is really not a good choice for that kind of project. If those are the two options, I can certainly see how Lisp would do better.
- reikonomusha 5y agoI have been using Common Lisp professionally for over a decade. Largest codebase was probably ~100k lines for an embedded optical fingerprint analysis and booking system for US law enforcement agencies. The largest Lisp team I've worked on was about 30 people for satcom hardware. Every time I've described to people on the internet what makes Lisp nice, I've been met with a riposte that such aspects aren't unique to Lisp—be them interactive/incremental development, macros, efficient machine code, editor integration, whatever—or that a laundry list of features means nothing and instead Lisp's value should somehow be proven through its (hitherto comparatively dismal) popularity with employers and software engineers. I myself have definitely given up on an academic approach to proselytizing the language, and instead have just focused my energy in building applications and libraries people (especially employers) find useful, at smaller cost and in shorter time than other people are able to provide. It's like classical music. I can't convince you such music is incredibly rich and pleasant, especially not by talking about the harmonic theory of common practice. Instead, you just have to listen and agree or disagree, I think. And Lisp certainly has no secrets; it's old as can be and extraordinarily and thoroughly documented. Countless people (myself included) have spilled ink on the (alleged) virtues of Lisp. It's all out there. When it comes down to it, it's an incredibly flexible language useful for prototyping and rapid development, while at the same time supporting everything you need to transform your application into a rock solid, robust, static binary. All of Lisp's features balance in such a way that developer time and energy are prioritized and consequently optimized. P.S., Like a sibling comment mentioned, you can have Haskell-level static typing in Common Lisp with Coalton: https://github.com/coalton-lang/coalton https://github.com/coalton-lang/coalton
- someguy101010 5y agohow do you go about producing static binaries for sbcl?
- reikonomusha 5y agoLoad your application up in SBCL and call `save-lisp-and-die` with the options you want (like what the `main` function should be).
- 5y ago
- Decabytes 5y agoIf you used Typed Racket you get static typing checked at compile time ^1 1. https://docs.racket-lang.org/ts-guide/quick.html https://docs.racket-lang.org/ts-guide/quick.html
- gus_massa 5y agoHave you tried Typed Racket https://docs.racket-lang.org/ts-guide/quick.html https://docs.racket-lang.org/ts-guide/quick.html ? You can add static type, and the compiler will complain if there is a mismatch. (The types are also used by the compiler to make optimizations of the code, so compilation is slower than (untyped) Racket, but the final code may be faster.)
- vnorilo 5y agoThis is purely my speculation, but I think it is OOP and dynamic types that is the toxic mix. Lisp codebases tend not do it as much (CLOS notwithstanding). Data tends to be flatter and simpler. Data shape is in theory just as hard to refactor properly, but IME Lisp projects just do a lot less of that. Of course there are also alternatives like gradual types or libs like spec in Clojure.
- beders 5y agoWorking with several teams of combined 10+ devs on a few large code-bases in Clojure. It works just fine. Very often, the context of your changes are local to a handful of functions - most of them being pure and simple to test. I'm actually surprised how well it works compared to Java - for example - where long-distance dependencies can bite you in surprising ways - even with refactoring tools. Clojure does have types and you can add compile-time tyoe-checking as an option, but it often is just in the way. I don't miss types for modelling a domain at all. Maps/sets/vectors/lists are sufficient. Granted: Your names better be good and you better use something like spec/malli etc. on the boundaries of your app/module for safety. If bugs happen it typically is due to nil-punting and bad tests. The nice thing: Those things are easily fixed. There's nothing complicated in Clojure. What I can't get back to: compile, run, fail loop (vs. read, eval, print, loop). The REPL allows you to live inside your app and it is just beautiful.
- wglb 5y agoI built an application for low-latency financial application in SBCL using genetic programming. This runs about 75k lines. I have a bunch of small projects totaling about 70k lines. This includes * Simple library to do application logging. Provides time stamp options, file naming options, others. * Pull apart excel spreadsheet to generate exception report. * Download images from Montana roadside cameras (15 minute cycle), Glacier park cameras (5 minute cycle). Includes crude detection of stars for dark images. * Convert rtf format to text to enable reasonable text-bases search. Used in scanning all household documents. * Parse burp logs to prepare for statistical analysis * configurations as s-expressions * Parse, report ham radio logs from trlog, wsjtx others. * Create makefiles for LaTeX projects * Command launcher to simplify working with many emacs projects, starting x-terminals for other systems, monitoring application status, tracking ntp sync of multiple systems. * Very small program to bump version number as part of lisp program builds. * Generate invoice from journal entries produced by https://github.com/wglb/for-hire-tools * Generate full penetration test report in LaTex from findings, glossary entries, vulnerability counts. * Simple api utils package used for web penetration tests * Pinboard api access used to generate pages for https://ciexinc.com/blog/solarwinds-articles/ * Library to read, dissect pcap files. (Unfinished) And countless tiny replacements for shell scripts.
- tlavoie 5y agoOut of curiosity, what sorts of processing were you doing for burp and penetration test data? I'm in security, and reporting on tests is always something of a pain point.
- wglb 5y agoSo the burp log analyzed certain statistics about cookies and url segments. Nowadays there might be ways to do that within burp. Could also look for secrets in bodies. The report one combines all elements of a pen test report including descriptive articles (e.g., what are CSRF, clickjacking, secure headers, and the like), one page for each vulnerability. These pages would include severity, impact, CVE links and CWE links, whether the vuln is closed, open, or risk accepted. Included is a tally of vulns by severity. A glossary of terms is also included. At the end is the glossary and index. A table of contents of course is at the beginning. One improvement is a pie chary of vulnerabilities. The lisp program generates LaTeX and references the article document type. Vastly simplified generating the report as the test went along. Document was one unit and no paring was required.
- puredanger 5y agoThere are a number of companies with 100s of Clojure developers and I know of several Clojure companies with 100ks of LOC. Most larger projects I know of use ancillary systems to validate map attributes via spec, Schema, Malli, etc. Statically typed libs and programs seem to still have issue trackers full of lots of bugs, and issue trackers for dynamically typed programs are not (in my experience) mostly type-related problems. That is, static types don't actually help you as much as you might first assume at writing correct programs.
- abrax3141 5y agoThe opposite —- calling python libraries directly from Lisp, like you can from Julia — would be much more useful. No adult wants to use python, but for some reason it is replete with yummy libraries.
- reikonomusha 5y agopy4cl2: https://digikar99.github.io/py4cl2/ https://digikar99.github.io/py4cl2/ burgled-batteries: https://github.com/pinterface/burgled-batteries https://github.com/pinterface/burgled-batteries
- pritambaral 5y agoFor Python3: https://github.com/snmsts/burgled-batteries3 https://github.com/snmsts/burgled-batteries3
- pabs3 5y agoThat sounds kind of similar to Hy: https://github.com/hylang/hy https://github.com/hylang/hy
- abrax3141 5y agoHy is Lisp embedded in Python. Most embedded lisps are bad lisps (or at least not CL compliant). Also lisp in python is definitely going to be slower than Lisp, which is generally fully compiled: https://programming-language-benchmarks.vercel.app/lisp-vs-python https://programming-language-benchmarks.vercel.app/lisp-vs-p...
- wyager 5y agoPython has trillions of libraries, but in my experience many (most?) of them are so bad that you're better off not trying to use them.
- abrax3141 5y agoReally? I work at a large scientific facility surrounded by serious physicists and computer scientists, and pretty much everyone there uses python and the python mathematical, machine learning, and scientific libraries. (Except me...I use Lisp, hence my desire to call python so that I can use the code I'm surrounded by without having to do horrible hacks like passing files or home-brew stream sharing!)
- smnrchrds 5y agoSteel Bank SBCL Common Lisp?
- smegsicle 5y agoisn't complaining about certain words being provided as context just because they happen to be in the acronym a bit pedantic?
- SnowHill9902 5y agoHe just really wanted to say Common Lisp.
- tempodox 5y agoThe article is about SBCL specifically, but the submitted title is editorialized.
- latenightcoding 5y agoInteresting, one of the reasons I use Scheme over Common Lisp is that doing this in CL is not common or easy. Scheme has few options to generate C code and it's so simple that I can write my own transpiler in no time.
- lispm 5y agoECL is typically used for integration into C. CLASP is a derivative for deep integration with C++. There are a bunch of other to C transpilers, some are in-house. For example PTC's large Creo Design CAD system uses an in-house maintained Common Lisp to C compiler.
- ngcc_hk 5y agoExternal callable is good and is that easy. But is lisp good for library construction and for other system to use. Many languages are not in this niche (is swift or apple os easily callable if not using objective c), but Common Lisp gives an impression it is more like it to be the centre, if not hotel California like. In fact one key feature or bug is it’s dynamic nature - debugging, typing, object and method etc - might be lost. In this regards callable from outside and unlike c which is not built for others (as os is to be called) one wonder what is to be gained. Obviously it could be one of the foundation of a large system. But can the satellite be others is my question. Can I have CL core and say python front end etc.
- vindarel 5y agoawesome to see SBCL so active. newcomers to Lisp, check out: - https://github.com/CodyReichert/awesome-cl https://github.com/CodyReichert/awesome-cl (there might be more libraries than you think) - https://lispcookbook.github.io/cl-cookbook/ https://lispcookbook.github.io/cl-cookbook/ - how to create a project (with my project generator): https://www.youtube.com/watch?v=XFc513MJjos https://www.youtube.com/watch?v=XFc513MJjos