7 ms·

Meh the problem is "Which Lisp?" There are dozens of incompatible Lisps. Even this site is written in a Lisp dialect written by its author (Arc). In fact I c

• armitron 5y ago
  The canonical Lisps still widely used today are Common Lisp, Scheme and Emacs Lisp. They all belong in the same family, and syntax / semantics are close. Porting code from Scheme to Common Lisp can be a lot easier than going from Python 2 to Python 3. Clojure is something else entirely which is why a lot of people don't consider it a Lisp. > Honest question: how do you communicate between two Lisp processes on two different machines? If you want to use built-in object serialization, there is print and read.
  • iak8god 5y ago
    > Common Lisp, Scheme and Emacs Lisp... all belong in the same family Could you say more about what you mean by this? Is there another family of Lisps that excludes these three? I've met people who make a big deal about lisp-1 vs lisp-2 (https://en.wikipedia.org/wiki/Lisp-1_vs._Lisp-2 https://en.wikipedia.org/wiki/Lisp-1_vs._Lisp-2), and which is the right way to be a Lisp, but I think maybe those people just enjoy being pedantic.
    • armitron 5y ago
      I was referring to Clojure which going by syntax/semantics does not belong in the same family as Common Lisp, Scheme, Emacs Lisp.
      • iak8god 5y ago
        Makes sense. I think many people would put Scheme in its own branch as well. E.g. https://gist.github.com/Aethaeryn/3036597 https://gist.github.com/Aethaeryn/3036597
• Jtsummers 5y ago
  > Honest question: how do you communicate between two Lisp processes on two different machines? I know Clojure has EDN (which is sort of like JSON : JavaScript), but I haven't heard of the solutions for other Lisps. Probably TCP or UDP based protocols like essentially every cross-network communication in every language today. EDIT: Also, it should be noted that JSON does not, itself, allow you to communicate across a network. It's just a serialization format. You still have to select some protocol for actually doing the communication. If your answer to the question "How do you communicate between two JavaScript processes on two different machines?" is "JSON", you've failed at actually answering the question.
  • chubot 5y ago
    Right, so that is what I'm getting at. If you have two Lisp programs on different machines, or two programs written in different Lisp dialects, the way you compose them is basically "Unix" -- serialize to some common format and send over a pipe / socket.
    • gmfawcett 5y ago
      What alternatives are you possibly allowing for, if you're taking IP and wire protocols off of the table?
      • chubot 5y ago
        I agree there aren't many practical alternatives to serialize -> wire -> parse, because that's how all storage and networking hardware works. But I keep having this conversation with people who are mad at Unix (and Unix shell) and searching for an alternative. Some of them are just fantasizing, as with this post, but some of them are actively working in that direction. One possible alternative is if the kernel grew a mechanism to pass linked data structures directly, instead of requiring serialization and parsing (a little like Go channels perhaps). I suppose you can kind of do that with shared memory -- but not really because copying is roughly equivalent to serialization. This might sound like an unrealistic idea, but people are seriously arguing for it. And of course they always argue in favor of their favorite language's data structures, not realizing it comes up with a boatload of design choices specific a problem domain, and that other problems require different data structures. This post is arguing for something even more radical: an entire machine in Lisp, with a single address space (really bad idea). ---- The other more practical thing I'm arguing against in the post is a certain design of alternative Unix shell [1] that I've been seeing a lot. The design is that basically that all the tools like 'ls' and 'rm' and even networking tools are expected to be statically linked into the shell binary. And many of them pass structured data directly, specifically to get rid of serialization and parsing. There are at least 3 or 4 shells with this design. My point is that this creates a "two-tiered" shell design and that composition is inhibited. In my mind the point of the shell, and the power of it, is to be situated within or "woven into" the OS. It interacts with everything on the system. It's not a "world" unto its own. You should be able to whip up a script in your favorite language and call it from your shell, and compose it with other tools you didn't write. You shouldn't be forced to "escape" the shell to call outside tools, or write algorithms in shell like you would write algorithms in Python or C. The shell is for coarse-grained composition and coordination. [1] list here: https://github.com/oilshell/oil/wiki/Alternative-Shells https://github.com/oilshell/oil/wiki/Alternative-Shells ---- Another possible alternative is something like https://www.unison-lang.org/ https://www.unison-lang.org/ where source code is not text; it's structured data. This has big consequences that I hinted at in the prior post, and have in my notes for upcoming posts: They are writing their own version control system and editor because they don't use text. (I think experiments like this are interesting; it's definitely worth some portion of our efforts.) It is true that they have to use byte streams at the end of the day because they have to use storage and networking hardware, but the user is supposedly insulated from any of that. That is, as long as you stay in this hypothetical universe. Again I will bring up interop as a big problem. A lot of people seem to imagine "monoglot" worlds where they wrote everything from scratch and no other code exists. Everything done in the past must be thrown out because we know better now. :) Elm is a more widely used language with similar interop problems.
• zozbot234 5y ago
  An unappreciated means of code reuse under *nix is the static and dynamic library. This seems to be the go-to whenever you need something more involved than simply reusing a full binary via pipes.
  • scj 5y ago
    C's greatest feature is that it trivially maps onto .so files. Linking to and creating .so files isn't just cheap, it's effectively free. Most higher level languages I've worked with seem to focus on using .so files rather than producing them. This means the lowest common denominator for the unix ecosystem is what C can provide. Otherwise stated, unix is marching forward at the pace of C.
    • pjmlp 5y ago
      No wonder, given that C was created to turn UNIX from an originally Assembly written OS into a portable one, UNIX is effectively a C Machine.
    • samatman 5y ago
      This is one of the things I like so much about Lua, LuaJIT in particular: it's designed around being just another .so file, from another clib's perspective Lua is a library for manipulating a calling stack, and the LuaJIT ffi makes it short work to adapt a header file so that structures and functions can be used from within Lua.
• bitwize 5y ago
  You are aware that Lisp machines understood several different flavors of Lisp? The Symbolics ones understood Zetalisp and Common Lisp at least. Were they on the market today they could be convinced to run Clojure and Scheme as well. There are a few old-timers developing Common Lisp applications to run on modern hardware, using Symbolics hardware. In fact, Symbolics shipped compilers for other non-Lisp programming languages, including C and Ada. These interoperated smoothly with Lisp code, much more so than they do under Unix. In this demo, Kalman Reti compiles a JPEG decoder written in C, and replaces the Lisp JPEG decoder that came with the system with it, yielding a performance boost: https://www.youtube.com/watch?v=o4-YnLpLgtk https://www.youtube.com/watch?v=o4-YnLpLgtk
  • chubot 5y ago
    OK interesting, I will check out the link. I still think various Lisps don't interoperate enough today, but I'm not very familiar with the Lisp machines of the past. If it can interoperate with C and Ada that's interesting. But I also wonder about interop with JavaScript :) i.e. not just existing languages but FUTURE languages. These are the M x N problems and extensibility problems I'm talking about on the blog.
    • fiddlerwoaroof 5y ago
      If you’re fine with ES3, there’s https://marijnhaverbeke.nl/cl-javascript/ https://marijnhaverbeke.nl/cl-javascript/ (I’ve been intending to use Babel to compile modern JS to ES3 and then load it into CL, but no time) Or parenscript: https://parenscript.common-lisp.dev/ https://parenscript.common-lisp.dev/ Or CLOG: https://github.com/rabbibotton/clog https://github.com/rabbibotton/clog And, there’s always the option of writing an HTTP or websocket server and serving JSON. I personally use the SLIME repl in emacs for a ton of things I used to use bash one-liners for and you can push it pretty far (this is an experiment, I’ve wrapped it up more nicely into my .sbclrc since): https://twitter.com/fwoaroof/status/1502881957012668417?s=21&t=81OfHTvxdkkNYDZiASP1gA https://twitter.com/fwoaroof/status/1502881957012668417?s=21...
    • bitwize 5y ago
      I'm not sure what you mean by "The lowest common denominator between a Common Lisp, Clojure, and Racket program is a Bourne shell script (and eventually an Oil script)." You can get two Lisp programs (or two Python programs, or two mixed-language programs, etc.) to intercommunicate without involving the shell at all. It'd be more accurate to say "The lowest common denominator between a Common Lisp, Clojure, and Racket program is sexpr notation." By using sexprs over a pipe, named pipe, or network socket, you can very easily get any of those Lisps to intercommunicate deeply structured data with any other. This is how SLIME and its SWANK protocol work. I don't even think the shell is involved; Emacs handles spawning the inferior Lisp and setting up the communication channel itself. The thing the Lisp machines had was a very robust ABI. Lisp has a specific notion about what a function is in memory. This is largely because Lisp is always "on" on a Lisp machine, there is no point at which you cannot take advantage of the entire Lisp runtime, including the compiler, in your own programs. Accordingly the Lisp machine C compiler output Lisp functions containing code compiled from C, that could be called by Lisp functions directly (and vice versa). Presumably a JavaScript runtime for Lisp machines would be able to do the same thing. By contrast, C has no notion of what a function is; and the C ABI used by many operating systems presents a function as simply a pointer in memory that gets called after some parameters are pushed to the stack or left in registers. (How many parameters, their type, and their size, is unspecified and simply agreed upon by caller and callee.) Nothing about memory management is provided by the runtime either. All that has to be provided by user programs. All this adds friction to interoperation by function call, and makes IPC the most straightforward way to interoperate. But oh well. Our computers are all slowly turning into JavaScript machines anyway, so maybe those Lisp/Smalltalk happy days can return again soon.
• throw10920 5y ago
  > Meh the problem is "Which Lisp?" Not a problem - you don't need (or want) a single Lisp. A hypothetical Lisp OS would support a standard runtime and typed IPC system that host programs can use (a la Windows and COM, or dbus), and nothing prevents you from using your own custom communication protocol over pipes/sockets instead. I don't agree with your implicit assertion that there is a single reason why Unix is more popular than Lisp (machines) - I think that there are a multitude of reasons. Certainly, the most impactful one isn't a lack of interoperability between Lisps - it would be something like the bandwagon effect, or the fact that Bell Labs gave away Unix to universities, or the inefficient implementations of early Lisps. I'm also fairly confident that the lack of a Lisp ecosystem is not because of a particular lack of interoperability (after all, Python has a massive ecosystem that is built almost exclusively on (a) C FFI and (b) other Python code), but for cultural reasons - Lispers just don't like collaborating together, and enjoy reinventing the wheel. These tendencies have been documented in [1] and many other places, and are highly consistent with my own experience in the Common Lisp community. > Honest question: how do you communicate between two Lisp processes on two different machines? Use PRINT to serialize an object on one machine, ferry it over to another one through whatever means you wish, and then READ it on the other. Or: "in the same way that two processes in most other languages on two different machines communicate". Serialize, transport, deserialize. Yes, the ecosystem is far less mature, and you'll have to do more legwork, but fundamentally the process is the same as in, say, Python. (Erlang might be an exception and have language-lever support for this, I'm not sure) This method works for different implementations of the same Lisp, and even for different Lisps, under some constraints (e.g. while a list with the integers 1, 2, and 3 is represented as the s-expression (1 2 3) in almost every Lisp you can find, CL represents "true" as T, while Scheme represents it as #true, so you'll have to work around that). If you will, you can just use JSON or XML as a serialization format instead - every non-trivial Lisp has libraries for those formats, and some have libraries for ASN.1, too. >> The lowest common denominator between a Common Lisp, Clojure, and Racket program is a Bourne shell script (and eventually an Oil script). All of those languages share basic s-expression syntax described above, which is rather higher-level than a Bourne shell script. Why do you say that the latter is the "lowest common denominator"? For that matter, why do you exclude the idea that JSON or XML aren't "common denominators" between Lisp programs, or even between Lisps, Python, and C++? ---------------- Your article says "Text Is The Only Thing You Can Agree On", but "text" isn't even a single thing. The set of bytes allowed by ASCII vs UTF-8 vs UTF-16 aren't the same. Even if it was, plain text is purely sequential and flat. Program interoperability requires structure. If the structure isn't directly encoded in a given "substrate" (such as text), and you need another layer on top, then that substrate isn't actually the one providing the interoperability. You say "Text is the most structured format they all agree on" but text isn't even structured at all, and nobody completely agrees on it - "bytes" is the only thing that fits into place here (which is equivalent to saying that there's no structured communication method that all programs agree on, which is true). Put another way - programs do not communicate using plain text. They communicate with either an ad-hoc protocol built on top of text (that still has a set of constraints that make it incompatible with other protocols built on top of text), or they use a standardized format like JSON or XML, or even something like ASN.1 that isn't a subset of text. Communication using text does not make programs interoperable. Bytes might be a "narrow waist", but text is factually not - if it was, you wouldn't need to use sed/awk/perl one-liners to connect various Unix utilities to each other, because the very fact that they were all using text input/output would make them interoperable. You say "Tables and documents are essential structures in software, and expressing them in JSON is awkward." but you can express them in JSON. You cannot express those structures in "plain text", because "plain text" is not capable of expressing structure at all, and the best you can do is build a protocol that is a subset of plain text that can express structure (JSON, XML, CSV, etc.) ------------ If anything, Lisps are more interoperable than Unix, because the "lowest common denominator" of Unix utilities is "plain text" (which by definition cannot encode structure), while the lowest common denominator of Lisps is some common subset of their s-expression formats, which can encode structure. ------------ > I would say the design of Unix is "rotting", but the answer is to IMPROVE Unix. You say this, but you haven't given a reason for why that's the answer. Here's a reason why improving Unix is not the answer: because "everything is text" is a fundamentally flawed paradigm that introduces needless complexity and fragility into the whole system design. > Not dream of clean slate designs that will never be deployed. You're mixing the normative and the positive - "Unix should be improved" with "clean-slate designs won't be deployed". Are you making an argument about what should happen, or what will happen? (there's no guarantee that any improvements to Unix will be deployed, either) > Plus this post doesn't actually propose anything. If you actually start trying to build your Lisp machine, I believe you will run into dozens of reasons why it's not a good idea. Oh, yes, and I can start with a few: first, there's good reason to believe that high-level CPUs are a bad idea (as suggested by the fact that nobody has been able to make an effective one[2]); second, the security-less design of older Lisp machines is a terrible idea (in a vacuum); third, that a machine that only runs Lisp is going to be unpopular and there's no single Lisp and Lisps are not the final stage of PL evolution. ...but the arguments made by the article as to why Unix is inadequate are still solid, even if the author is suggesting a suboptimal solution due to nostalgia. [1] https://www.lambdassociates.org/blog/bipolar.htm https://www.lambdassociates.org/blog/bipolar.htm [2] http://yosefk.com/blog/the-high-level-cpu-challenge.html http://yosefk.com/blog/the-high-level-cpu-challenge.html
• fiddlerwoaroof 5y ago
  > Honest question: how do you communicate between two Lisp processes on two different machines? Depends what level of integration you want: custom tcp/udp protocols, HTTP and websockets are all pretty easy. But, you can also use something like this over a trusted network/vpn: https://github.com/brown/swank-client https://github.com/brown/swank-client
• convolvatron 5y ago
  > Honest question: how do you communicate between two Lisp processes on two different machines? using continuations with sexps is insanely powerful send (+ 1 3 (lambda (x) `(handle-response, x)))
• openfuture 5y ago
  It's all going to be datalisp, mark my word :) Although you have no idea what I am talking about yet just wait a bit more :))
• syrrim 5y ago
  pretty sure hn was ported away from arc at some point.
• foobarian 5y ago
  > In fact I conjecture that this is the reason Unix is more popular than Lisp -- because Lisps don't interoperate well. They haven't built up a big ecosystem of reusable code. And why are there so many? IMO the language is too flexible for its own good. It promotes this curious intellectual solo-competition where you try to prove you are worth of the elite who get all the fancy FP stuff and make all this bespoke functionality. It's almost impossible for Lisp to be popular. To be popular means a lot of people use it, but that means it can't be complicated much above the median. But because it lets individual programmers push intellectual boundaries it self-selects itself out of this pool. Any big collaborative project will attract this type of developer and soon the lesser developers (who don't dare object for fear of appearing dumb) are not able to contribute. Just my opinion, if a little dramatic.
  • zozbot234 5y ago
    Haskell has its share of fancy FP stuff, and people manage to develop workable things in it. I still think Lisp really is too dynamic to be useful beyond a small scale of development.
• spacemanmatt 5y ago
  > how do you communicate between two Lisp processes on two different machines? Same as every other language: you pick a protocol and use it on both sides. Many of us already have enough JSON in play that it makes sense to start there.