5 ms·

This is a really great explanation, and I appreciate you taking time to share in such depth. One question I have is on version control. I understand the value

This is a really great explanation, and I appreciate you taking time to share in such depth. One question I have is on version control. I understand the value of reference images and release images and debugging images (with a rare bug front and center) and so on. Would this play well with diffing as I am familiar with it from a carpentry perspective, e.g git diff? Would you serialize and dump a set of source code from an image? Would you build the reference image from a source code project and manually handle the consistency between source files and images? I know it's a vague multi-question, but I hope you understand what I'm getting at, and I would really appreciate learning more about this style of development.

Y_Y 8y ago
Exactly the same question ocurred to me as I was reading. In a lisp the is certainly possible, the environment exists as a giant sexp (possibly with some small compiled components). No idea how smalltalk works though. I've tried something like this with ipython, since you can save the session's input as a .py file. As long as you don't depend on the outside world you can recreate your image by using the bare repl as your "base image" and just load the (gittable) previous sessions input (which itself possibly loads older and older sessions).
- mikelevins 8y ago
  A Smalltalk system has data structures that record a comprehensive inventory of every memory object. If anything, Smalltalk makes it even easier than Lisp to keep track of all the objects in memory.
tonyg 8y ago
Most Squeak/Pharo Smalltalk development takes place these days in the context of either git itself or a git-like DVCS called Monticello. Use of git/monticello within the image directly mirrors use of git at the command-line in Unix. You use git/monticello to clone a remote repo, installing the code it contains into the live image. Because of other aspects of the Smalltalk design, this has an additional effect of being a bit like package installation - the code isn't just present, it's installed and ready to run. Unlike Unix, the lack of an explicit system-compilation step in Smalltalk means it's also ready to edit in-place, which means, if you've started some service instances, ready to edit live. It's very much like Docker or working with a Unix VM: you start from a known image, like a Docker FROM command or a Debian ISO, and install files/code/packages from there, using version control tools or not as you like.
mikelevins 8y ago
Other comments explain that nowadays Smalltalk programmers use git or Monticello for revision control. Lisp programmers usually use git or other conventional revision control tools. Yes, traditional Smalltalk systems can serialize sources to files, and can read them back into a running image. In Smalltalk jargon that's called "filing out" and "filing in". Lisp systems generally use plain old text files as the canonical form of storage for source code. A given project consists of a set of source files, just like a C or Java program. The utility of image files is that they give you (1) the starting Lisp environment; (2) a reference image customized for your local site that starts up fully customized and ready to go; and (3) working images that record a specific state that you were working on, to preserve your context. Git and tools like it are not great for managing image files. Images are large binary files, and successive versions of images tend to have collections of small changes to them. This is exactly the kind of file that git does not handle well at all. Consequently, I don't generally use git or similar tools to manage images, and I don't imagine a lot of other Smalltalk and Lisp programmers do, either (correct me if I'm wrong!). It would be great if we had some sort of git for large binary files. I don't mean something like git-annex--a way for git to duck the problems of managing versioned large binary files by outsourcing them to some other storage. I mean it would be great to have a revision-control system that could manage versioned binaries as well as git manages versioned text. Lisp and Smalltalk programmers are not the only people who would benefit. Any project that needs to manage a lot of media files would benefit. One obvious example is in game development. Even small games tend to need to manage versioned files containing thousands of image and sound resources. It's a giant pain. Git for binaries would be a godsend for projects like that. And for managing Lisp and Smalltalk image files, too, but we can't rely on that demand. The number of people who need to manage a lot of Lisp and Smalltalk images is even smaller than the number of people regularly using Lisp and Smalltalk in their daily work.
mikelevins 8y ago
I feel like I should answer part of your question more directly than I did: "Would you serialize and dump a set of source code from an image? Would you build the reference image from a source code project and manually handle the consistency between source files and images?" Yes, this is a normal way to manage changes in Lisp projects. I should mention that, even when I'm using images as I've described, I use text files, not image files, as the canonical storage for my code. I use images in the ways I've described, but I don't use them as the main way to store the reference version of my code. Consequently, the normal way I manage Lisp projects (and the way I've seen other Lisp hackers do it) is to use git or similar tools to manage versions of the source code. I build reference images, working images, and release images from those source files. Smalltalk systems have always been more oriented toward managing everything from within the image, and the Smalltalkers I've worked with generally treated text files as a serialization format for exchanging code with others. The canonical form of the source code was in the image. (Well, really, it was in the sources file, but that's basically just backing store for the image.) I work with Lisp much more than with Smalltalk, so I'll defer to other commenters on the current best practices for managing Smalltalk projects with revision-control tools. I will mention that some older Lisp systems were a little more like Smalltalk systems, in that they had more tools for version management than currently-available systems. Interlisp, for example, had a pretty impressive change-management facility called Masterscope, and Lisp Machines had a versioned file system--the machine's filesystem was a revision-control system.

Y_Y 8y ago

Exactly the same question ocurred to me as I was reading. In a lisp the is certainly possible, the environment exists as a giant sexp (possibly with some small compiled components). No idea how smalltalk works though. I've tried something like this with ipython, since you can save the session's input as a .py file. As long as you don't depend on the outside world you can recreate your image by using the bare repl as your "base image" and just load the (gittable) previous sessions input (which itself possibly loads older and older sessions).

mikelevins 8y ago
A Smalltalk system has data structures that record a comprehensive inventory of every memory object. If anything, Smalltalk makes it even easier than Lisp to keep track of all the objects in memory.

tonyg 8y ago

Most Squeak/Pharo Smalltalk development takes place these days in the context of either git itself or a git-like DVCS called Monticello. Use of git/monticello within the image directly mirrors use of git at the command-line in Unix. You use git/monticello to clone a remote repo, installing the code it contains into the live image. Because of other aspects of the Smalltalk design, this has an additional effect of being a bit like package installation - the code isn't just present, it's installed and ready to run. Unlike Unix, the lack of an explicit system-compilation step in Smalltalk means it's also ready to edit in-place, which means, if you've started some service instances, ready to edit live. It's very much like Docker or working with a Unix VM: you start from a known image, like a Docker FROM command or a Debian ISO, and install files/code/packages from there, using version control tools or not as you like.

mikelevins 8y ago

Other comments explain that nowadays Smalltalk programmers use git or Monticello for revision control. Lisp programmers usually use git or other conventional revision control tools. Yes, traditional Smalltalk systems can serialize sources to files, and can read them back into a running image. In Smalltalk jargon that's called "filing out" and "filing in". Lisp systems generally use plain old text files as the canonical form of storage for source code. A given project consists of a set of source files, just like a C or Java program. The utility of image files is that they give you (1) the starting Lisp environment; (2) a reference image customized for your local site that starts up fully customized and ready to go; and (3) working images that record a specific state that you were working on, to preserve your context. Git and tools like it are not great for managing image files. Images are large binary files, and successive versions of images tend to have collections of small changes to them. This is exactly the kind of file that git does not handle well at all. Consequently, I don't generally use git or similar tools to manage images, and I don't imagine a lot of other Smalltalk and Lisp programmers do, either (correct me if I'm wrong!). It would be great if we had some sort of git for large binary files. I don't mean something like git-annex--a way for git to duck the problems of managing versioned large binary files by outsourcing them to some other storage. I mean it would be great to have a revision-control system that could manage versioned binaries as well as git manages versioned text. Lisp and Smalltalk programmers are not the only people who would benefit. Any project that needs to manage a lot of media files would benefit. One obvious example is in game development. Even small games tend to need to manage versioned files containing thousands of image and sound resources. It's a giant pain. Git for binaries would be a godsend for projects like that. And for managing Lisp and Smalltalk image files, too, but we can't rely on that demand. The number of people who need to manage a lot of Lisp and Smalltalk images is even smaller than the number of people regularly using Lisp and Smalltalk in their daily work.

mikelevins 8y ago

I feel like I should answer part of your question more directly than I did: "Would you serialize and dump a set of source code from an image? Would you build the reference image from a source code project and manually handle the consistency between source files and images?" Yes, this is a normal way to manage changes in Lisp projects. I should mention that, even when I'm using images as I've described, I use text files, not image files, as the canonical storage for my code. I use images in the ways I've described, but I don't use them as the main way to store the reference version of my code. Consequently, the normal way I manage Lisp projects (and the way I've seen other Lisp hackers do it) is to use git or similar tools to manage versions of the source code. I build reference images, working images, and release images from those source files. Smalltalk systems have always been more oriented toward managing everything from within the image, and the Smalltalkers I've worked with generally treated text files as a serialization format for exchanging code with others. The canonical form of the source code was in the image. (Well, really, it was in the sources file, but that's basically just backing store for the image.) I work with Lisp much more than with Smalltalk, so I'll defer to other commenters on the current best practices for managing Smalltalk projects with revision-control tools. I will mention that some older Lisp systems were a little more like Smalltalk systems, in that they had more tools for version management than currently-available systems. Interlisp, for example, had a pretty impressive change-management facility called Masterscope, and Lisp Machines had a versioned file system--the machine's filesystem was a revision-control system.