6 ms·

Talking with Mikel Evins about the Lisp-based Newton OS from Apple

68 points by curtis 12y ago

_mhr_ 12y ago
Previous discussion here: https://news.ycombinator.com/item?id=320646 https://news.ycombinator.com/item?id=320646
deleted 12y ago
[deleted]
deleted 12y ago
[deleted]
S4M 12y ago
I am getting an "access forbidden" error :(
- BruceM 12y ago
  IIRC, the machine hosting this doesn't always have a lot of resources / bandwidth. I'm pretty sure that he used to discourage linking to stuff there for that sort of reason. It works for me now at the moment though.
  - S4M 12y ago
    Well, it would be good if someone who can access to the website could dump a copy on pastebin.
    - BruceM 12y ago
      http://pastebin.com/0vcwKsjk http://pastebin.com/0vcwKsjk
      - S4M 12y ago
        Thanks!
BruceM 12y ago
Mikel has many stories to tell and it would be great if he would tell more of them (publicly). We talk occasionally due to our shared interest and history with Dylan (http://opendylan.org/ http://opendylan.org/) and he's always got valuable points of view on things like development tools, frame-based knowledge systems, interactive development, and so on. After his time on Dylan and the Lisp-based Newton at Apple, he went on to work on SK8 (http://en.wikipedia.org/wiki/SK8 http://en.wikipedia.org/wiki/SK8). If you dig around online, you can find the SK8 sources which were openly published at some point. They aren't that useful now given that they're written for Macintosh Common Lisp on the original Mac OS, but they're definitely interesting to read through. These days, Mikel is still doing Common Lisp work, but he's also worked on his own language, Bard: https://github.com/mikelevins/bard https://github.com/mikelevins/bard and he's slowly been working with us in the Open Dylan community to add support back for the old s-expression syntax.
S4M 12y ago
Mikel said that they could build the OS with only a team of 5-6 people, in 50-100k lines of code (excluding the code for the Kernel). I don't know much about OS implementation, but I find it quite impressive, and it seems a real shame that those efforts and technology were not reused somewhere else.
- ANTSANTS 12y ago
  If Rob Pike is to be believed, the bulk of the effort in writing an OS goes into providing compatibility with the myriad externally-imposed standards that people depend on to get things done and run the programs they need to do so. From what I know of the Newton, it's almost like they tried as hard as possible to be incompatible with existing standards and established modes of computing. I guess it's not surprising, then, that they were able to do a lot with not much code; they completely skipped the hardest part.
  - S4M 12y ago
    It's true that they had much less restrictions on hardware since they were developing for only one type of model, but still it's impressive.
  - fit2rule 12y ago
    >>externally-imposed standards That is the crux of it. I've also seen teams build a complete stack - OS to App - in short order in similar fashion in the industrial embedded computing world .. it seems to be that a tight, motivated team, really can pull of extreme wonders when it works right for the specific, internally-agreed-/co-created feature set. The scale aspect is actually arbitrary; but if you have a group of people who are willing to complete the stack, full-system wise, the scale is a human factor, observed after application; i.e. its not inherent to the subject of software development that there is some dividing line between the 'difficulties' of the whole system development ethos, but rather a function of the differences, and/or similarities, of the people involved, doing something with 'the' technology .. which is after all little more than a social imposition in the first place.
delackner 12y ago
I swear I have read a very similar article elsewhere that talked more about WHY the project got cancelled, but this line in the linked article stood out to me as a hint: "Size in RAM was about twice the size of the C++ version". This is back when memory was actually expensive, so yeah that is pretty big.
- pjmlp 12y ago
  True, but probably it wasn't fully optimized. In any case, as Alan Kay says: "if you want to see the future, you need to be willing to build the hardware from 10 - 15 years in the future." Xerox PARC systems would never had been built if they would just use PDP like systems.
- BruceM 12y ago
  Well, keep in mind this point as well: At the time, there was much frustration that Apple Cambridge spent so much time on their new development environment, and not as much as we wanted on making improvements to the Dylan runtime (for example, we hired Ken because we concluded that Cambridge would not get around to threads in time to matter to us). That said, I have nothing but respect for what Apple Cambridge accomplished. Oliver Steele who was involved with the Dylan IDE on the Apple Cambridge team has said that they spent too much time on IDE features, if I recall correctly. Given that, it is pretty likely that they lacked a tree-shaker to reduce executable sizes (https://groups.google.com/forum/#!topic/comp.lang.lisp/pspFr1XByZk https://groups.google.com/forum/#!topic/comp.lang.lisp/pspFr...). It is also pretty unlikely that they'd spent enough time on reducing the memory usage and so on. (It is worth noting that even today, Open Dylan hasn't got a tree shaker.) Bringing up a whole new ecosystem from scratch is hard work!
- kabdib 12y ago
  I joined Newton just as the transition from the Dylan stuff was happening. In the space of about a week the group moved from Dylan to "We're going to get this done now, in a small form factor [called Junior], really fast and in C++". I think the switch was due to resource constraints (512K of RAM was a big deal in a handheld product) as well as a sense that they needed to get real and ship something. The impetus may have been a small group of people (e.g., Capps and Walter and some other heavies) having a Serious Talk while at a trade show in June of 1992, but obviously management must have been involved, too. Anyway, I joined the group the week a bunch of people were out of town, and when they came back everything was different. There were a number of sad-faced LISP hackers roaming the halls, clutching copies of the C++ Annotated Reference Manual. The ARM (the book, not the CPU) must have been hell for them. From that point on the Dylan stuff was pretty much sidelined. The "Senior" product (a tablet) was kept alive for quite some time, but the group's focus was almost 100% on Junior. There were occasional releases of Dylan (and I got a copy of the book), but little true interest. Paul Mercer went an extra step further in reducing risk and complexity; he proposed ditching the microkernel in favor of a relatively simple a 68K-based handheld running an event loop and some apps, and he even got a lot of it working in a demo on some prototype hardware he whipped up. This was a little too radical, since there would have been no handwriting recognition, and he wound up leaving (and later founded Pixo, which was contracted to do the first UI for the iPod and was eventually bought by Apple). I'll note that what Paul proposed is essentially what the Palm Pilot did, with a limited character-by-character recognition interface that actually worked pretty well.
calibraxis 12y ago
The danger of listening to superficial programmers and their rejectionism: "I believe now that I was wrong: the infix syntax did not cause Dylan to succeed, and I find it distinctly inferior in practice to the s-expression syntax that I used for several years." It's true, there is an advantage to mainstream appearance. But analysis suggests weighing tradeoffs: what are the disadvantages? Are there ways to minimize disadvantages while keeping advantages?
- BruceM 12y ago
  This is a subject that keeps coming up over the years in the Dylan community (and why, in part, Mikel is slowly working on an s-expression reader for the current compiler). It is impossible to know what might have happened if Dylan had kept the s-expression syntax and how that syntax might have evolved. When the switch was made, the language didn't have all of the features that it soon had in the infix syntax. The macro system is just a single example of this. (And Dylan was one of, if not, the first infix language to have a macro system like it does. See http://people.csail.mit.edu/jrb/Projects/dexprs.pdf http://people.csail.mit.edu/jrb/Projects/dexprs.pdf for a discussion of it and some extensions.) In the s-expression syntax, you defined a new method by: (define-method odd? ...) In the infix syntax, you do: define method odd? ... ... end; However, you can supply adjectives as well, which weren't present in the s-expression syntax: define sealed inline method odd? ... ... end; Personally, I don't really care if I'm using the infix syntax or an updated form of the s-expression syntax. But to deal with the features present in the infix syntax, it would have to be a somewhat modified version of the old s-expression syntax. Given the history of the last 20+ years since Dylan was created, there are a lot of other things that could've been done differently in the syntax as well: * Something more terse and compact. * Not requiring semicolons. * Using braces or whitespace instead of 'end' statements. * Stuck with s-expressions since Lisp is more acceptable today (ala Clojure). * <Your bikeshed here.> One of the Dylan designers, David Moon, went on design (but not implement) a new language with some differing takes on the syntax: http://users.rcn.com/david-moon/PLOT3/ http://users.rcn.com/david-moon/PLOT3/ ... it is interesting, but without an implementation. There's also the question of whether Dylan failed due to the syntax. There were a lot of factors that led to Dylan not seeing the adoption that was desired: * Java was a big factor. * The financial troubles at Apple. * The collapse of Harlequin. Harlequin had an implementation of Dylan that lives on today as Open Dylan, but had a very advanced IDE on Windows and a pretty solid foundation. * The business model of Harlequin. Harlequin sold expensive commercial development tools. Lispworks lives on with that business model, but it can't be said to be a massive success. * The team at CMU switched to Java and away from their Gwydion Dylan implementation. (This was partly due to research dollars.) And the failings of the Apple Dylan product itself didn't help. It was late to market, buggy, slow, and lacked PowerPC support in the initial release. Much of the above wasn't due to or related to the syntax. Yet despite all of that, Dylan still has a special place in my heart and that of many others. We've put out new releases, fixed a lot of bugs, improved platform portability and more in the last few years. Now we're looking towards the future and considering potentially larger changes.

68 points by curtis 12y ago

_mhr_ 12y ago

Previous discussion here: https://news.ycombinator.com/item?id=320646 https://news.ycombinator.com/item?id=320646

deleted 12y ago

[deleted]

deleted 12y ago

[deleted]

S4M 12y ago

I am getting an "access forbidden" error :(

BruceM 12y ago
IIRC, the machine hosting this doesn't always have a lot of resources / bandwidth. I'm pretty sure that he used to discourage linking to stuff there for that sort of reason. It works for me now at the moment though.
- S4M 12y ago
  Well, it would be good if someone who can access to the website could dump a copy on pastebin.
  - BruceM 12y ago
    http://pastebin.com/0vcwKsjk http://pastebin.com/0vcwKsjk
    - S4M 12y ago
      Thanks!

BruceM 12y ago

Mikel has many stories to tell and it would be great if he would tell more of them (publicly). We talk occasionally due to our shared interest and history with Dylan (http://opendylan.org/ http://opendylan.org/) and he's always got valuable points of view on things like development tools, frame-based knowledge systems, interactive development, and so on. After his time on Dylan and the Lisp-based Newton at Apple, he went on to work on SK8 (http://en.wikipedia.org/wiki/SK8 http://en.wikipedia.org/wiki/SK8). If you dig around online, you can find the SK8 sources which were openly published at some point. They aren't that useful now given that they're written for Macintosh Common Lisp on the original Mac OS, but they're definitely interesting to read through. These days, Mikel is still doing Common Lisp work, but he's also worked on his own language, Bard: https://github.com/mikelevins/bard https://github.com/mikelevins/bard and he's slowly been working with us in the Open Dylan community to add support back for the old s-expression syntax.

S4M 12y ago

Mikel said that they could build the OS with only a team of 5-6 people, in 50-100k lines of code (excluding the code for the Kernel). I don't know much about OS implementation, but I find it quite impressive, and it seems a real shame that those efforts and technology were not reused somewhere else.

ANTSANTS 12y ago
If Rob Pike is to be believed, the bulk of the effort in writing an OS goes into providing compatibility with the myriad externally-imposed standards that people depend on to get things done and run the programs they need to do so. From what I know of the Newton, it's almost like they tried as hard as possible to be incompatible with existing standards and established modes of computing. I guess it's not surprising, then, that they were able to do a lot with not much code; they completely skipped the hardest part.
- S4M 12y ago
  It's true that they had much less restrictions on hardware since they were developing for only one type of model, but still it's impressive.
- fit2rule 12y ago
  >>externally-imposed standards That is the crux of it. I've also seen teams build a complete stack - OS to App - in short order in similar fashion in the industrial embedded computing world .. it seems to be that a tight, motivated team, really can pull of extreme wonders when it works right for the specific, internally-agreed-/co-created feature set. The scale aspect is actually arbitrary; but if you have a group of people who are willing to complete the stack, full-system wise, the scale is a human factor, observed after application; i.e. its not inherent to the subject of software development that there is some dividing line between the 'difficulties' of the whole system development ethos, but rather a function of the differences, and/or similarities, of the people involved, doing something with 'the' technology .. which is after all little more than a social imposition in the first place.

delackner 12y ago

I swear I have read a very similar article elsewhere that talked more about WHY the project got cancelled, but this line in the linked article stood out to me as a hint: "Size in RAM was about twice the size of the C++ version". This is back when memory was actually expensive, so yeah that is pretty big.

pjmlp 12y ago
True, but probably it wasn't fully optimized. In any case, as Alan Kay says: "if you want to see the future, you need to be willing to build the hardware from 10 - 15 years in the future." Xerox PARC systems would never had been built if they would just use PDP like systems.
BruceM 12y ago
Well, keep in mind this point as well: At the time, there was much frustration that Apple Cambridge spent so much time on their new development environment, and not as much as we wanted on making improvements to the Dylan runtime (for example, we hired Ken because we concluded that Cambridge would not get around to threads in time to matter to us). That said, I have nothing but respect for what Apple Cambridge accomplished. Oliver Steele who was involved with the Dylan IDE on the Apple Cambridge team has said that they spent too much time on IDE features, if I recall correctly. Given that, it is pretty likely that they lacked a tree-shaker to reduce executable sizes (https://groups.google.com/forum/#!topic/comp.lang.lisp/pspFr1XByZk https://groups.google.com/forum/#!topic/comp.lang.lisp/pspFr...). It is also pretty unlikely that they'd spent enough time on reducing the memory usage and so on. (It is worth noting that even today, Open Dylan hasn't got a tree shaker.) Bringing up a whole new ecosystem from scratch is hard work!
kabdib 12y ago
I joined Newton just as the transition from the Dylan stuff was happening. In the space of about a week the group moved from Dylan to "We're going to get this done now, in a small form factor [called Junior], really fast and in C++". I think the switch was due to resource constraints (512K of RAM was a big deal in a handheld product) as well as a sense that they needed to get real and ship something. The impetus may have been a small group of people (e.g., Capps and Walter and some other heavies) having a Serious Talk while at a trade show in June of 1992, but obviously management must have been involved, too. Anyway, I joined the group the week a bunch of people were out of town, and when they came back everything was different. There were a number of sad-faced LISP hackers roaming the halls, clutching copies of the C++ Annotated Reference Manual. The ARM (the book, not the CPU) must have been hell for them. From that point on the Dylan stuff was pretty much sidelined. The "Senior" product (a tablet) was kept alive for quite some time, but the group's focus was almost 100% on Junior. There were occasional releases of Dylan (and I got a copy of the book), but little true interest. Paul Mercer went an extra step further in reducing risk and complexity; he proposed ditching the microkernel in favor of a relatively simple a 68K-based handheld running an event loop and some apps, and he even got a lot of it working in a demo on some prototype hardware he whipped up. This was a little too radical, since there would have been no handwriting recognition, and he wound up leaving (and later founded Pixo, which was contracted to do the first UI for the iPod and was eventually bought by Apple). I'll note that what Paul proposed is essentially what the Palm Pilot did, with a limited character-by-character recognition interface that actually worked pretty well.

calibraxis 12y ago

The danger of listening to superficial programmers and their rejectionism: "I believe now that I was wrong: the infix syntax did not cause Dylan to succeed, and I find it distinctly inferior in practice to the s-expression syntax that I used for several years." It's true, there is an advantage to mainstream appearance. But analysis suggests weighing tradeoffs: what are the disadvantages? Are there ways to minimize disadvantages while keeping advantages?

BruceM 12y ago
This is a subject that keeps coming up over the years in the Dylan community (and why, in part, Mikel is slowly working on an s-expression reader for the current compiler). It is impossible to know what might have happened if Dylan had kept the s-expression syntax and how that syntax might have evolved. When the switch was made, the language didn't have all of the features that it soon had in the infix syntax. The macro system is just a single example of this. (And Dylan was one of, if not, the first infix language to have a macro system like it does. See http://people.csail.mit.edu/jrb/Projects/dexprs.pdf http://people.csail.mit.edu/jrb/Projects/dexprs.pdf for a discussion of it and some extensions.) In the s-expression syntax, you defined a new method by: (define-method odd? ...) In the infix syntax, you do: define method odd? ... ... end; However, you can supply adjectives as well, which weren't present in the s-expression syntax: define sealed inline method odd? ... ... end; Personally, I don't really care if I'm using the infix syntax or an updated form of the s-expression syntax. But to deal with the features present in the infix syntax, it would have to be a somewhat modified version of the old s-expression syntax. Given the history of the last 20+ years since Dylan was created, there are a lot of other things that could've been done differently in the syntax as well: * Something more terse and compact. * Not requiring semicolons. * Using braces or whitespace instead of 'end' statements. * Stuck with s-expressions since Lisp is more acceptable today (ala Clojure). * <Your bikeshed here.> One of the Dylan designers, David Moon, went on design (but not implement) a new language with some differing takes on the syntax: http://users.rcn.com/david-moon/PLOT3/ http://users.rcn.com/david-moon/PLOT3/ ... it is interesting, but without an implementation. There's also the question of whether Dylan failed due to the syntax. There were a lot of factors that led to Dylan not seeing the adoption that was desired: * Java was a big factor. * The financial troubles at Apple. * The collapse of Harlequin. Harlequin had an implementation of Dylan that lives on today as Open Dylan, but had a very advanced IDE on Windows and a pretty solid foundation. * The business model of Harlequin. Harlequin sold expensive commercial development tools. Lispworks lives on with that business model, but it can't be said to be a massive success. * The team at CMU switched to Java and away from their Gwydion Dylan implementation. (This was partly due to research dollars.) And the failings of the Apple Dylan product itself didn't help. It was late to market, buggy, slow, and lacked PowerPC support in the initial release. Much of the above wasn't due to or related to the syntax. Yet despite all of that, Dylan still has a special place in my heart and that of many others. We've put out new releases, fixed a lot of bugs, improved platform portability and more in the last few years. Now we're looking towards the future and considering potentially larger changes.