32 ms·

X and NeWS history

177 points by colinprince 9y ago

grabcocque 9y ago
I doubt it will come as a surprise to anyone that X was fundamentally ill conceived from the very beginning. It’s one of those unfortunate side effects of unintended consequences that Unix history saddled us with it for twenty years longer than we’d ideally have wanted. It fascinates me that something so widely regarded as Bad and Wrong, proved so hard to replace when it’s baked into a system. I think there’s a cautionary tale about the necessity of getting your core APIs right, because you might never be able to get rid of them.
- pjmlp 9y ago
  Having used a multitude of OSes since the 80's and having digital archeology as one of my hobbies, I think it applies to everything in UNIX's design.
  - MycroftJones 9y ago
    I would like to hear more. The Unix core API is pretty simple and durable; what was better than it? Other than Plan9? SmallTalk doesn't really count.
    - pjmlp 9y ago
      For starters having been written in C, after the Assembly and B rewrites, when we already had safer systems programming languages in the early 60's. IBM for example did most of their RISC research with an OS written in PL/8, where they prototyped many ideas about modular compilers, a few of them visible in how LLVM works. Then by being a plain CLI based OS versus what was being done at Xerox PARC with Xerox Star and Pilot (done in Mesa). NeXTSTep was probably the only variant that got close of the Mac, Amiga and Atari multimedia capabilities, and even it wasn't a pure UNIX, rather used it as a means to bring applications into the NeXT world. Many mainframe ideas about OS security are still not part of any UNIX clone, as another example.
      - MycroftJones 9y ago
        Thank you for explaining. I disagree, but then, I started with the Commodore 64 and BASIC, and have missed them ever since. C and LISP version 1 sort of fit in the same space. I hate any language that hides the machine from me. Also don't really like clicky gooey stuff. Text and typing is where it is at. Matter of taste really. But newLISP is a good mix of hardware access with safety.
- digi_owl 9y ago
  Perfect is the enemy of good, as the saying goes.
  - rayiner 9y ago
    Worse is better, as the saying goes.
agumonkey 9y ago
DonHopkins is on HN from time to time, he's always full of things to say about this. Waiting for the wall of history :)
wwweston 9y ago
> I have a section in the book that I'm writing where I talk about how to design a good API. I pose the question of why none of the original Apple Mac API published in 1985 taking about 1,200 pages is in use today whereas almost all of the UNIX V6 API published in 1975 taking 321 pages is still in use and has been copied by many other systems. I'm sure that everyone on this list knows the answer. I'm not sure I do. But it does seem like a good question. Also: http://www.art.net/~hopkins/Don/unix-haters/x-windows/disaster.html http://www.art.net/~hopkins/Don/unix-haters/x-windows/disast...
- valuearb 9y ago
  My guess is that the original Mac OS provided a great GUI, but not much of an OS. It loaded a single application at a time that had access to all of memory. That's why it had desk accessories, to give you apps you could use without quitting the main application. It had handle based memory management so the OS could move memory blocks more contagiously to create enough open space for new memory allocations. Over time they added cooperative multitasking, which meant the foreground app had to consciously give time to let background apps do stuff. But they could never build a real OS with security and memory protection out of it. So Apple bought NeXT to get industrial strength unix as their core OS, and switched all Apple development to NeXTStep, or at least the modern derivative of it, called Cocoa. My dust covered Inside Mac volumes were made useless nearly 20 years ago.
  - __david__ 9y ago
    Except most of the Inside Mac APIs were brought forward in OS X as Carbon. Carbon only went away relatively recently. So the APIs could support a reasonable OS, it's just Apple couldn't accomplish it for some reason (I waited for Copland for a long time).
    - xenadu02 9y ago
      That is not correct. Over a thousand APIs were dropped or substantially modified to be “carbon clean”. Even Multi-Finder was a lot of work because apps happily poked OS globals. The original Mac OS was more like an embedded app toolkit than an OS. The original implementations were all ROM!
jandrese 9y ago
The comment about how X is really just a distributed database system with occasional visual side effects that could be reduced down to 10 or so API calls gives us a glimpse as to what life could have been. It's a shame that all of the X replacements on the horizon also ditch the network transparency part, even though that's been a killer feature for me. I hate how much of a hack Remote Desktop is on Windows machines.
- theoh 9y ago
  In the old days it was convenient to be able to run a SunOS binary on a remote server and display on a Linux machine -- but this can't be as common today. The way X clients can be forwarded over SSH implies that network transparency doesn't need to be built in to the window system, there just needs to be a "seam" at which it can be interposed. Obviously any kind of direct access to graphics hardware makes this impractical. Plan 9's window system uses the 9p file system protocol to cross the network; another Bell Labs trick (used in Rob Pike's Sam editor) is for the app to be split into two parts with their own internal interface. They can run on different hosts. The internal protocol/interface can be tailored to the app (and can potentially be very simple and/or efficient).
  - mst 9y ago
    Once you've created a seam (possibly a unix socket, many of my designs rely on those) exposing the same socket over TCP becomes relatively trivial and now you effectively have network transparency anyway (see also spiped). But, yeah, so long as the client is a separate process, getting it transited over the network can be done separately.
    - theoh 9y ago
      Agreed, though exposing/using a bare TCP socket comes with security responsibilities that you are forced to address -- it's good to devolve those.
      - mst 9y ago
jimjimjim 9y ago
still better than web apps.
mwcampbell 9y ago
I've read some of Don Hopkins' comments about all the cool things that were done with PostScript running in the NeWS server. I wonder, if NeWS had survived long enough that Sun had to add accessibility for people with disabilities (to sell to governments and schools if nothing else), how would they have done it? Today, the current state of the art is very chatty IPC (meaning tens of microseconds per cross-process call), partially assisted by bulk fetching and caching. To appreciate how bad that can be, think about a screen reader making at least one IPC call per word when reading text, to ask the application about word boundaries and bounding rectangles. That's a problem I hope to solve someday. In a parallel universe, though, maybe the screen reader is running inside the NeWS server, in the same address space as a PostScript-based UI toolkit.
- DonHopkins 9y ago
  I've written up some thoughts and discussions about accessibility and JavaScript, and discussed them a few times here! I discovered the great work of Morgan Dixon and James Fogarty, which proves that you can do some amazing things with screen scraping, pattern matching, visual deconstruction, augmentation and reconstruction! His work needs to be combined with platform specific accessibility APIs via JavaScript. I'm proposing "aQuery", a high level scriptable accessibility tool that is to native user interface components like jQuery is to DOM, for selecting and querying components, matching visual patterns, handing events, abstracting platform dependencies and high level service interfaces, building and scripting higher level widgets and applications, etc. http://donhopkins.com/mediawiki/index.php/AQuery http://donhopkins.com/mediawiki/index.php/AQuery https://news.ycombinator.com/item?id=11520967 https://news.ycombinator.com/item?id=11520967 Morgan Dixon's and James Fogarty's work is truly breathtaking and eye opening, and I would love for that to be a core part of a scriptable hybrid Screen Scraping / Accessibility API approach. Screen scraping techniques are very powerful, but have limitations. Accessibility APIs are very powerful, but have different limitations. But using both approaches together, screencasting and re-composing visual elements, and tightly integrating it with JavaScript, enables a much wider and interesting range of possibilities. Think of it like augmented reality for virtualizing desktop user interfaces. The beauty of Morgan's Prefab is how it works across different platforms and web browsers, over virtual desktops, and how it can control, sample, measure, modify, augment and recompose guis of existing unmodified applications, even dynamic language translation, so they're much more accessible and easier to use! https://news.ycombinator.com/item?id=12425668 https://news.ycombinator.com/item?id=12425668 This link has the most up-to-date links to Morgan's work, and his demo videos! https://news.ycombinator.com/item?id=14182061 https://news.ycombinator.com/item?id=14182061 https://prefab.github.io/ https://prefab.github.io/ Prefab: The Pixel-Based Reverse Engineering Toolkit Prefab is a system for reverse engineering the interface structure of graphical interfaces from their pixels. In other words, Prefab looks at the pixels of an existing interface and returns a tree structure, like a web-page's Document Object Model, that you can then use to modify the original interface in some way. Prefab works from example images of widgets; it decomposes those widgets into small parts, and exactly matches those parts in screenshots of an interface. Prefab does this many times per second to help you modify interfaces in real time. Imagine if you could modify any graphical interface? With Prefab, you can explore this question! https://www.youtube.com/watch?v=w4S5ZtnaUKE https://www.youtube.com/watch?v=w4S5ZtnaUKE Imagine if every interface was open source. Any of us could modify the software we use every day. Unfortunately, we don't have the source. Prefab realizes this vision using only the pixels of everyday interfaces. This video shows how we advanced the capabilities of Prefab to understand interface content and hierarchy. We use Prefab to add new functionality to Microsoft Word, Skype, and Google Chrome. These demonstrations show how Prefab can be used to translate the language of interfaces, add tutorials to interfaces, and add or remove content from interfaces solely from their pixels. Prefab represents a new approach to deploying HCI research in everyday software, and is also the first step toward a future where anybody can modify any interface. More Prefab demos: https://prefab.github.io/videos.html https://prefab.github.io/videos.html
kweinber 9y ago
This is also an important example of how limiting an anti-social attitude can be when trying to collaborate and have a large affect on the world. As brilliant as Jon is, he wasted an inordinate amount of time making up for constantly insulting people, places, and approaches. (Even today he couldn't avoid insulting all of Tulsa OK). Imagine where windowing systems could be if his brilliant work, API thoughts, and insights weren't delivered with insults, arrogance and vitriol. At the end it is sad that he was proud of being right even though he was an _hole (his word). It could have been that he was right and his best ideas were in common use.
- MycroftJones 9y ago
  I didn't realize people were so sensitive in the field. Compared to Linus, Theo de Raadt, Richard Stallman, etc, Jon seemed to be cruising at a comfortable level of Germanic straight-forward truth-telling, no real aholishness I could detect. Guess I should reset my etiquette and politeness filters.
  - adrianratnapala 9y ago
    I note that the people you name all built their communities through work they did well outside the corporate world.
    - digi_owl 9y ago
      And at least two of them have a undeserved reputation.
      - MycroftJones 9y ago
        You mean Theo and Linus? Yes.
        rbanffy 9y ago
        Theo's reputation is very well deserved.
badsectoracula 9y ago
> The only really worthwhile thing about X was the distributed extension registration mechanism. All of the input, graphics and other crap should be moved to extension #1. That way, it won't be mandatory in conforming implementations once that stuff was obsolete. As you probably know, that's where we are today; nobody uses that stuff but it's like the corner of an Intel chip that implements the original instruction set. On the other hand, all of the input, graphics and "other crap" being mandatory means that as a developer you can pretty much guarantee that they are there and if you care about having your stuff work everywhere (well, everywhere X is available anyway) you can simply rely on that without having to support a bunch of different APIs that may or may not be there (see the audio situation a few years ago in Linux). (also FWIW a lot of people use the X primitives, not everyone uses Gtk or Qt)
- adrianratnapala 9y ago
  Yes, the 2d primitives in X are actually fine -- a bunch of dumb obvious primitives of the kind that just about every graphics system has. Where it fell down was all the complication about XVisuals and colours etc. Those things were solving real problems for the era but doing so with an unnecessarily complicated interface that tied the 2d-primitives and much else into the assumptions of that era. Thus as things developed, whole new APIs like XRender had to be invented in order to do the dumb basic primitives again.
  - DonHopkins 9y ago
    I beg to differ. X11 2D primitives are "actually fine" in the same sense that there were "very fine people" marching in Charlottesville. By which I mean "not actually very fine". ;) But I totally agree with you about colors (the X approach to device independence is to treat everything like a MicroVAX framebuffer on acid), and xRender (which was wonderful work that transcended X11 and led to Cairo and html's canvas). And don't get me started about window borders, and the SHAPES extension, and their fractally complex interactions! http://www.art.net/~hopkins/Don/unix-haters/x-windows/disaster.html http://www.art.net/~hopkins/Don/unix-haters/x-windows/disast... A task as simple as filing and stroking shapes is quite complicated because of X's bizarre pixel-oriented imaging rules. When you fill a 10x10 square with XFillRectangle, it fills the 100 pixels you expect. But you get extra "bonus pixels" when you pass the same arguments to XDrawRectangle, because it actually draws an 11x11 square, hanging out one pixel below and to the right!!! If you find this hard to believe, look it up in the X manual yourself: Volume 1, Section 6.1.4. The manual patronizingly explains how easy it is to add 1 to the x and y position of the filled rectangle, while subtracting 1 from the width and height to compensate, so it fits neatly inside the outline. Then it points out that "in the case of arcs, however, this is a much more difficult proposition (probably impossible in a portable fashion)." This means that portably filling and stroking an arbitrarily scaled arc without overlapping or leaving gaps is an intractable problem when using the X Window System. Think about that. You can't even draw a proper rectangle with a thick outline, since the line width is specified in unscaled pixel units, so if your display has rectangular pixels, the vertical and horizontal lines will have different thicknesses enen though you scaled the rectangle corner coordinates to compensate for the aspect ratio.
DonHopkins 9y ago
John Steinhart is outstanding in his field! Sometimes he's even out driving a tractor in his field! ;)
- MycroftJones 9y ago
  In that thread, it mentiond that NeWS had "problems". Can you tell us what those problems were?
  - DonHopkins 9y ago
    I don't know where to begin, or when to stop, so I'll pick something at random and jump around. NeWS 1.1 was layered on top of PixRects (the graphics library that SunView was built on top of), which gave it the weird limitation that sub-windows could not be positioned above or to the left of their parents (sub-window coordinates were unsigned), which was a huge pain in the ass for making all kinds of user interfaces. X11/NeWS solved that problem by having a much nicer PostScript graphics renderer and compositor, but it came with its own set of problems, many having to do with being nailed together with a X11 server with a totally different graphics model and approach to life. But there were some really cool things you could do with X11 and NeWS together, like managing X11 windows with NeWS frames (which could have non-rectangular windows like jutting title tabs, round pie menus, panning virtual desktops, rooms, etc). Another important point is that running the window manager in the same heavy weight Unix process as the window server saves an astronomical number of costly context switches, and results in much smoother user interface with silky local feedback. Plus it's extensible and customizable and drawn in PostScript. Here's a demo of NeWS window frames with tabs and pie menus. We used those to made an ICCCM X11 window manager written in PostScript (that Sun never released) that let you use those tabbed window frames and pie menus and other NeWS components to manage all of your X11 windows (and use them as your default frame for NeWS apps too, of course), seamlessly integrating X11 and NeWS apps together. https://www.youtube.com/watch?v=tMcmQk-q0k4 https://www.youtube.com/watch?v=tMcmQk-q0k4 Here are some flames and notes I wrote years ago about X11 and NeWS window management -- the "OWM" section is about the NeWS X11 window manager. It covers some of the political and technical problems we had integrating X11 and NeWS. Window management is a surprisingly contentious issue! Everybody wants to be on top. http://www.donhopkins.com/home/catalog/unix-haters/x-windows/i39l.html http://www.donhopkins.com/home/catalog/unix-haters/x-windows... Basically, we didn't want to put our round NeWS pie menus and pizza windows inside of their rectangular X11 ICCCM window frames, because it would have been better for everyone the other way around. http://www.donhopkins.com/home/catalog/images/pizzatool.gif http://www.donhopkins.com/home/catalog/images/pizzatool.gif Josh Siegel wrote this beautiful PostScript tour de force swan song for X11/NeWS, an ICCCM window manager that actually worked quite well, with special window subclasses to support all the OPEN LOOK dialog types and window frame features through their standard X11 ICCCM properties! Check out "ClassX11ManagerMixin", which is what it sounds like. It's mixed into FavoriteBaseWindow, which can be redefined to the custom ClassTabBaseWindow supporting tabs and pie menus, shown in the above video, or any other kind of frame you want to wrap around X11 windows: http://www.donhopkins.com/home/archive/NeWS/owm.ps.txt http://www.donhopkins.com/home/archive/NeWS/owm.ps.txt It was a proof of concept, which we wanted to take a lot further by implementing a HyperCard-like user customizable persistent window manager with HyperLook. So users can edit and build their own custom window frames by copying, pasting and configuring scriptable components from object warehouses, edit and extend the user interfaces of running apps, and make high level task-oriented interfaces by composing widgets and apps into "cards", "backgrounds", "stacks" and optional network "clients". But that's not how things turned out. ;/ http://www.donhopkins.com/home/catalog/hyperlook/index.html http://www.donhopkins.com/home/catalog/hyperlook/index.html http://www.donhopkins.com/home/catalog/hyperlook/Warehouse1.gif http://www.donhopkins.com/home/catalog/hyperlook/Warehouse1.... http://www.donhopkins.com/home/catalog/hyperlook/ButtonIdeas4.gif http://www.donhopkins.com/home/catalog/hyperlook/ButtonIdeas... http://www.donhopkins.com/home/catalog/hyperlook/TalkInterfacing.gif http://www.donhopkins.com/home/catalog/hyperlook/TalkInterfa... http://www.donhopkins.com/home/catalog/hyperlook/TalkRunTime.gif http://www.donhopkins.com/home/catalog/hyperlook/TalkRunTime... I'll excerpt the relevant parts of the i39l flames: Who Should Manage the Windows, X11 or NeWS? This is a discussion of ICCCM Window Management for X11/NeWS. One of the horrible problems of X11/NeWS was window management. The X people wanted to wrap NeWS windows up in X frames (that is, OLWM). The NeWS people wanted to do it the other way around, and prototyped an ICCCM window manager in NeWS (mostly object oriented PostScript, and a tiny bit of C), that wrapped X windows up in NeWS window frames. Why wrap X windows in NeWS frames? Because NeWS is much better at window management than X. On the surface, it was easy to implement lots of cool features. But deeper, NeWS is capable of synchronizing input events much more reliably than X11, so it can manage the input focus perfectly, where asynchronous X11 window managers fall flat on their face by definition. Our next step (if you'll pardon the allusion) was to use HyperNeWS (renamed HyperLook, a graphical user interface system like HyperCard with PostScript) to implemented a totally customizable X window manager! Some notes about OWM OWM is the "Open Window Manager" we prototyped in NeWS. We enhanced the NeWS window frames so they sported indexing tabs, pie menus, rooms, and a scrolling virtual desktop. Many of our enhancements were separatly developed, and plugged together orthogonally like legos. All NeWS applications could use these fancy frames, and the Open Window Manager wrapped X clients in the same frames that NeWS windows got! This way, factoring the window frames out as a part of the toolkit, and implementing the X window manager separately, NeWS applications don't have to know a thing about X window management, and X clients can go on doing the same nasty things they've always done, and everybody get the benefits of dynamic extensibility, and a consistent user interface, by using the default window class! X11 window managers must grab the server in order to animate rubber-band feedback over the screen when resizing and moving windows. This grabbing causes many problems with NeWS synchronous interests, that can be demonstrated by pressing the "Help" key while dragging out a rectangle on the root background. NeWS can do a much better job at managing global resources in the server because it is in the same address space and it has facilities like the overlay plane specifically designed to implement such window management functions, without even grabbing the server. This antisocial server grabbing behavior is just one symptom of a general class of problems with external X window management, including other important issues such as keyboard and colomap focus.
DonHopkins 9y ago
"A couple of decades ago I bough a Sun Ultra-60 and asked whether it was worth paying for the dual processor version. The answer that I got was "Yes. You need one processor to run X and another for everything else." People cursed with diskless Sun 3/50's would run xterms and emacs's across the net on each other's workstations instead of locally. It was noticeably less sluggish, because you didn't get hit with all the heavyweight process context switching between X clients and server every keystroke.
DonHopkins 9y ago
"Had he done some real design work and looked at what others were doing he might have realized that at its core, X was a distributed database system in which operations on some of the databases have visual side-effects. I forget the exact number, but X includes around 20 different databases: atoms, properties, contexts, selections, keymaps, etc. each with their own set of API calls. As a result, the X API is wide and shallow like the Mac, and full of interesting race conditions to boot. The whole thing could have been done with less than a dozen API calls." To that end, one of the weirder and cooler re-implementations of NeWS was Cogent's PIX for transputers. It was basically a NeWS-like multiprocessing PostScript interpreter for Transputers, with Linda "tuple spaces" as an interprocess communication primitive: http://ieeexplore.ieee.org/document/301904/ http://ieeexplore.ieee.org/document/301904/ The Cogent Research XTM is a desktop parallel computer based on the INMOS T800 transputer. Designed to expand from two to several hundred processors, the XTM provides a transparent distributed computing environment both within a single workstation and among a collection of workstations. Using Linda tuple spaces as the basis for interprocess communication and synchronization, a Unix-compatible, server-based OS was constructed. A graphic user interface is provided by an interactive PostScript window server called PIX. All processors see the same set of system services, and within protection limits, programs capable of using many processors can spread out over a network of workstations and resource servers, acquiring the services of unused processors. http://wiki.c2.com/?TupleSpace http://wiki.c2.com/?TupleSpace https://en.wikipedia.org/wiki/Tuple_space https://en.wikipedia.org/wiki/Tuple_space
adrianratnapala 9y ago
Through most of that article I was thinking "Yep, that's the kind of train-wreck you get when you design by committee, or worse by political need". But toward the end, he says that the particular kind of technical train-wreck that they got was similar to the Mac API (presumably the this means the pre-OS X OS). "[As a resulting of accidentally being lots of databases] the X API is wide and shallow like the Mac, and full of interesting race conditions to boot." Anyway, I didn't think that design by committee was one of Apples weaknesses.

177 points by colinprince 9y ago

I doubt it will come as a surprise to anyone that X was fundamentally ill conceived from the very beginning. It’s one of those unfortunate side effects of unintended consequences that Unix history saddled us with it for twenty years longer than we’d ideally have wanted. It fascinates me that something so widely regarded as Bad and Wrong, proved so hard to replace when it’s baked into a system. I think there’s a cautionary tale about the necessity of getting your core APIs right, because you might never be able to get rid of them.

pjmlp 9y ago
Having used a multitude of OSes since the 80's and having digital archeology as one of my hobbies, I think it applies to everything in UNIX's design.
- MycroftJones 9y ago
  I would like to hear more. The Unix core API is pretty simple and durable; what was better than it? Other than Plan9? SmallTalk doesn't really count.
  - pjmlp 9y ago
    For starters having been written in C, after the Assembly and B rewrites, when we already had safer systems programming languages in the early 60's. IBM for example did most of their RISC research with an OS written in PL/8, where they prototyped many ideas about modular compilers, a few of them visible in how LLVM works. Then by being a plain CLI based OS versus what was being done at Xerox PARC with Xerox Star and Pilot (done in Mesa). NeXTSTep was probably the only variant that got close of the Mac, Amiga and Atari multimedia capabilities, and even it wasn't a pure UNIX, rather used it as a means to bring applications into the NeXT world. Many mainframe ideas about OS security are still not part of any UNIX clone, as another example.
    - MycroftJones 9y ago
      Thank you for explaining. I disagree, but then, I started with the Commodore 64 and BASIC, and have missed them ever since. C and LISP version 1 sort of fit in the same space. I hate any language that hides the machine from me. Also don't really like clicky gooey stuff. Text and typing is where it is at. Matter of taste really. But newLISP is a good mix of hardware access with safety.
digi_owl 9y ago
Perfect is the enemy of good, as the saying goes.
- rayiner 9y ago
  Worse is better, as the saying goes.

agumonkey 9y ago

DonHopkins is on HN from time to time, he's always full of things to say about this. Waiting for the wall of history :)

wwweston 9y ago

> I have a section in the book that I'm writing where I talk about how to design a good API. I pose the question of why none of the original Apple Mac API published in 1985 taking about 1,200 pages is in use today whereas almost all of the UNIX V6 API published in 1975 taking 321 pages is still in use and has been copied by many other systems. I'm sure that everyone on this list knows the answer. I'm not sure I do. But it does seem like a good question. Also: http://www.art.net/~hopkins/Don/unix-haters/x-windows/disaster.html http://www.art.net/~hopkins/Don/unix-haters/x-windows/disast...

valuearb 9y ago
My guess is that the original Mac OS provided a great GUI, but not much of an OS. It loaded a single application at a time that had access to all of memory. That's why it had desk accessories, to give you apps you could use without quitting the main application. It had handle based memory management so the OS could move memory blocks more contagiously to create enough open space for new memory allocations. Over time they added cooperative multitasking, which meant the foreground app had to consciously give time to let background apps do stuff. But they could never build a real OS with security and memory protection out of it. So Apple bought NeXT to get industrial strength unix as their core OS, and switched all Apple development to NeXTStep, or at least the modern derivative of it, called Cocoa. My dust covered Inside Mac volumes were made useless nearly 20 years ago.
- __david__ 9y ago
  Except most of the Inside Mac APIs were brought forward in OS X as Carbon. Carbon only went away relatively recently. So the APIs could support a reasonable OS, it's just Apple couldn't accomplish it for some reason (I waited for Copland for a long time).
  - xenadu02 9y ago
    That is not correct. Over a thousand APIs were dropped or substantially modified to be “carbon clean”. Even Multi-Finder was a lot of work because apps happily poked OS globals. The original Mac OS was more like an embedded app toolkit than an OS. The original implementations were all ROM!

jandrese 9y ago

The comment about how X is really just a distributed database system with occasional visual side effects that could be reduced down to 10 or so API calls gives us a glimpse as to what life could have been. It's a shame that all of the X replacements on the horizon also ditch the network transparency part, even though that's been a killer feature for me. I hate how much of a hack Remote Desktop is on Windows machines.

theoh 9y ago
In the old days it was convenient to be able to run a SunOS binary on a remote server and display on a Linux machine -- but this can't be as common today. The way X clients can be forwarded over SSH implies that network transparency doesn't need to be built in to the window system, there just needs to be a "seam" at which it can be interposed. Obviously any kind of direct access to graphics hardware makes this impractical. Plan 9's window system uses the 9p file system protocol to cross the network; another Bell Labs trick (used in Rob Pike's Sam editor) is for the app to be split into two parts with their own internal interface. They can run on different hosts. The internal protocol/interface can be tailored to the app (and can potentially be very simple and/or efficient).
- mst 9y ago
  Once you've created a seam (possibly a unix socket, many of my designs rely on those) exposing the same socket over TCP becomes relatively trivial and now you effectively have network transparency anyway (see also spiped). But, yeah, so long as the client is a separate process, getting it transited over the network can be done separately.
  - theoh 9y ago
    Agreed, though exposing/using a bare TCP socket comes with security responsibilities that you are forced to address -- it's good to devolve those.
    - mst 9y ago

jimjimjim 9y ago

still better than web apps.

mwcampbell 9y ago

I've read some of Don Hopkins' comments about all the cool things that were done with PostScript running in the NeWS server. I wonder, if NeWS had survived long enough that Sun had to add accessibility for people with disabilities (to sell to governments and schools if nothing else), how would they have done it? Today, the current state of the art is very chatty IPC (meaning tens of microseconds per cross-process call), partially assisted by bulk fetching and caching. To appreciate how bad that can be, think about a screen reader making at least one IPC call per word when reading text, to ask the application about word boundaries and bounding rectangles. That's a problem I hope to solve someday. In a parallel universe, though, maybe the screen reader is running inside the NeWS server, in the same address space as a PostScript-based UI toolkit.

DonHopkins 9y ago
I've written up some thoughts and discussions about accessibility and JavaScript, and discussed them a few times here! I discovered the great work of Morgan Dixon and James Fogarty, which proves that you can do some amazing things with screen scraping, pattern matching, visual deconstruction, augmentation and reconstruction! His work needs to be combined with platform specific accessibility APIs via JavaScript. I'm proposing "aQuery", a high level scriptable accessibility tool that is to native user interface components like jQuery is to DOM, for selecting and querying components, matching visual patterns, handing events, abstracting platform dependencies and high level service interfaces, building and scripting higher level widgets and applications, etc. http://donhopkins.com/mediawiki/index.php/AQuery http://donhopkins.com/mediawiki/index.php/AQuery https://news.ycombinator.com/item?id=11520967 https://news.ycombinator.com/item?id=11520967 Morgan Dixon's and James Fogarty's work is truly breathtaking and eye opening, and I would love for that to be a core part of a scriptable hybrid Screen Scraping / Accessibility API approach. Screen scraping techniques are very powerful, but have limitations. Accessibility APIs are very powerful, but have different limitations. But using both approaches together, screencasting and re-composing visual elements, and tightly integrating it with JavaScript, enables a much wider and interesting range of possibilities. Think of it like augmented reality for virtualizing desktop user interfaces. The beauty of Morgan's Prefab is how it works across different platforms and web browsers, over virtual desktops, and how it can control, sample, measure, modify, augment and recompose guis of existing unmodified applications, even dynamic language translation, so they're much more accessible and easier to use! https://news.ycombinator.com/item?id=12425668 https://news.ycombinator.com/item?id=12425668 This link has the most up-to-date links to Morgan's work, and his demo videos! https://news.ycombinator.com/item?id=14182061 https://news.ycombinator.com/item?id=14182061 https://prefab.github.io/ https://prefab.github.io/ Prefab: The Pixel-Based Reverse Engineering Toolkit Prefab is a system for reverse engineering the interface structure of graphical interfaces from their pixels. In other words, Prefab looks at the pixels of an existing interface and returns a tree structure, like a web-page's Document Object Model, that you can then use to modify the original interface in some way. Prefab works from example images of widgets; it decomposes those widgets into small parts, and exactly matches those parts in screenshots of an interface. Prefab does this many times per second to help you modify interfaces in real time. Imagine if you could modify any graphical interface? With Prefab, you can explore this question! https://www.youtube.com/watch?v=w4S5ZtnaUKE https://www.youtube.com/watch?v=w4S5ZtnaUKE Imagine if every interface was open source. Any of us could modify the software we use every day. Unfortunately, we don't have the source. Prefab realizes this vision using only the pixels of everyday interfaces. This video shows how we advanced the capabilities of Prefab to understand interface content and hierarchy. We use Prefab to add new functionality to Microsoft Word, Skype, and Google Chrome. These demonstrations show how Prefab can be used to translate the language of interfaces, add tutorials to interfaces, and add or remove content from interfaces solely from their pixels. Prefab represents a new approach to deploying HCI research in everyday software, and is also the first step toward a future where anybody can modify any interface. More Prefab demos: https://prefab.github.io/videos.html https://prefab.github.io/videos.html

kweinber 9y ago

This is also an important example of how limiting an anti-social attitude can be when trying to collaborate and have a large affect on the world. As brilliant as Jon is, he wasted an inordinate amount of time making up for constantly insulting people, places, and approaches. (Even today he couldn't avoid insulting all of Tulsa OK). Imagine where windowing systems could be if his brilliant work, API thoughts, and insights weren't delivered with insults, arrogance and vitriol. At the end it is sad that he was proud of being right even though he was an _hole (his word). It could have been that he was right and his best ideas were in common use.

MycroftJones 9y ago
I didn't realize people were so sensitive in the field. Compared to Linus, Theo de Raadt, Richard Stallman, etc, Jon seemed to be cruising at a comfortable level of Germanic straight-forward truth-telling, no real aholishness I could detect. Guess I should reset my etiquette and politeness filters.
- adrianratnapala 9y ago
  I note that the people you name all built their communities through work they did well outside the corporate world.
  - digi_owl 9y ago
    And at least two of them have a undeserved reputation.
    - MycroftJones 9y ago
      You mean Theo and Linus? Yes.
      - rbanffy 9y ago
        Theo's reputation is very well deserved.

badsectoracula 9y ago

> The only really worthwhile thing about X was the distributed extension registration mechanism. All of the input, graphics and other crap should be moved to extension #1. That way, it won't be mandatory in conforming implementations once that stuff was obsolete. As you probably know, that's where we are today; nobody uses that stuff but it's like the corner of an Intel chip that implements the original instruction set. On the other hand, all of the input, graphics and "other crap" being mandatory means that as a developer you can pretty much guarantee that they are there and if you care about having your stuff work everywhere (well, everywhere X is available anyway) you can simply rely on that without having to support a bunch of different APIs that may or may not be there (see the audio situation a few years ago in Linux). (also FWIW a lot of people use the X primitives, not everyone uses Gtk or Qt)

adrianratnapala 9y ago
Yes, the 2d primitives in X are actually fine -- a bunch of dumb obvious primitives of the kind that just about every graphics system has. Where it fell down was all the complication about XVisuals and colours etc. Those things were solving real problems for the era but doing so with an unnecessarily complicated interface that tied the 2d-primitives and much else into the assumptions of that era. Thus as things developed, whole new APIs like XRender had to be invented in order to do the dumb basic primitives again.
- DonHopkins 9y ago
  I beg to differ. X11 2D primitives are "actually fine" in the same sense that there were "very fine people" marching in Charlottesville. By which I mean "not actually very fine". ;) But I totally agree with you about colors (the X approach to device independence is to treat everything like a MicroVAX framebuffer on acid), and xRender (which was wonderful work that transcended X11 and led to Cairo and html's canvas). And don't get me started about window borders, and the SHAPES extension, and their fractally complex interactions! http://www.art.net/~hopkins/Don/unix-haters/x-windows/disaster.html http://www.art.net/~hopkins/Don/unix-haters/x-windows/disast... A task as simple as filing and stroking shapes is quite complicated because of X's bizarre pixel-oriented imaging rules. When you fill a 10x10 square with XFillRectangle, it fills the 100 pixels you expect. But you get extra "bonus pixels" when you pass the same arguments to XDrawRectangle, because it actually draws an 11x11 square, hanging out one pixel below and to the right!!! If you find this hard to believe, look it up in the X manual yourself: Volume 1, Section 6.1.4. The manual patronizingly explains how easy it is to add 1 to the x and y position of the filled rectangle, while subtracting 1 from the width and height to compensate, so it fits neatly inside the outline. Then it points out that "in the case of arcs, however, this is a much more difficult proposition (probably impossible in a portable fashion)." This means that portably filling and stroking an arbitrarily scaled arc without overlapping or leaving gaps is an intractable problem when using the X Window System. Think about that. You can't even draw a proper rectangle with a thick outline, since the line width is specified in unscaled pixel units, so if your display has rectangular pixels, the vertical and horizontal lines will have different thicknesses enen though you scaled the rectangle corner coordinates to compensate for the aspect ratio.

DonHopkins 9y ago

John Steinhart is outstanding in his field! Sometimes he's even out driving a tractor in his field! ;)

MycroftJones 9y ago
In that thread, it mentiond that NeWS had "problems". Can you tell us what those problems were?
- DonHopkins 9y ago
  I don't know where to begin, or when to stop, so I'll pick something at random and jump around. NeWS 1.1 was layered on top of PixRects (the graphics library that SunView was built on top of), which gave it the weird limitation that sub-windows could not be positioned above or to the left of their parents (sub-window coordinates were unsigned), which was a huge pain in the ass for making all kinds of user interfaces. X11/NeWS solved that problem by having a much nicer PostScript graphics renderer and compositor, but it came with its own set of problems, many having to do with being nailed together with a X11 server with a totally different graphics model and approach to life. But there were some really cool things you could do with X11 and NeWS together, like managing X11 windows with NeWS frames (which could have non-rectangular windows like jutting title tabs, round pie menus, panning virtual desktops, rooms, etc). Another important point is that running the window manager in the same heavy weight Unix process as the window server saves an astronomical number of costly context switches, and results in much smoother user interface with silky local feedback. Plus it's extensible and customizable and drawn in PostScript. Here's a demo of NeWS window frames with tabs and pie menus. We used those to made an ICCCM X11 window manager written in PostScript (that Sun never released) that let you use those tabbed window frames and pie menus and other NeWS components to manage all of your X11 windows (and use them as your default frame for NeWS apps too, of course), seamlessly integrating X11 and NeWS apps together. https://www.youtube.com/watch?v=tMcmQk-q0k4 https://www.youtube.com/watch?v=tMcmQk-q0k4 Here are some flames and notes I wrote years ago about X11 and NeWS window management -- the "OWM" section is about the NeWS X11 window manager. It covers some of the political and technical problems we had integrating X11 and NeWS. Window management is a surprisingly contentious issue! Everybody wants to be on top. http://www.donhopkins.com/home/catalog/unix-haters/x-windows/i39l.html http://www.donhopkins.com/home/catalog/unix-haters/x-windows... Basically, we didn't want to put our round NeWS pie menus and pizza windows inside of their rectangular X11 ICCCM window frames, because it would have been better for everyone the other way around. http://www.donhopkins.com/home/catalog/images/pizzatool.gif http://www.donhopkins.com/home/catalog/images/pizzatool.gif Josh Siegel wrote this beautiful PostScript tour de force swan song for X11/NeWS, an ICCCM window manager that actually worked quite well, with special window subclasses to support all the OPEN LOOK dialog types and window frame features through their standard X11 ICCCM properties! Check out "ClassX11ManagerMixin", which is what it sounds like. It's mixed into FavoriteBaseWindow, which can be redefined to the custom ClassTabBaseWindow supporting tabs and pie menus, shown in the above video, or any other kind of frame you want to wrap around X11 windows: http://www.donhopkins.com/home/archive/NeWS/owm.ps.txt http://www.donhopkins.com/home/archive/NeWS/owm.ps.txt It was a proof of concept, which we wanted to take a lot further by implementing a HyperCard-like user customizable persistent window manager with HyperLook. So users can edit and build their own custom window frames by copying, pasting and configuring scriptable components from object warehouses, edit and extend the user interfaces of running apps, and make high level task-oriented interfaces by composing widgets and apps into "cards", "backgrounds", "stacks" and optional network "clients". But that's not how things turned out. ;/ http://www.donhopkins.com/home/catalog/hyperlook/index.html http://www.donhopkins.com/home/catalog/hyperlook/index.html http://www.donhopkins.com/home/catalog/hyperlook/Warehouse1.gif http://www.donhopkins.com/home/catalog/hyperlook/Warehouse1.... http://www.donhopkins.com/home/catalog/hyperlook/ButtonIdeas4.gif http://www.donhopkins.com/home/catalog/hyperlook/ButtonIdeas... http://www.donhopkins.com/home/catalog/hyperlook/TalkInterfacing.gif http://www.donhopkins.com/home/catalog/hyperlook/TalkInterfa... http://www.donhopkins.com/home/catalog/hyperlook/TalkRunTime.gif http://www.donhopkins.com/home/catalog/hyperlook/TalkRunTime... I'll excerpt the relevant parts of the i39l flames: Who Should Manage the Windows, X11 or NeWS? This is a discussion of ICCCM Window Management for X11/NeWS. One of the horrible problems of X11/NeWS was window management. The X people wanted to wrap NeWS windows up in X frames (that is, OLWM). The NeWS people wanted to do it the other way around, and prototyped an ICCCM window manager in NeWS (mostly object oriented PostScript, and a tiny bit of C), that wrapped X windows up in NeWS window frames. Why wrap X windows in NeWS frames? Because NeWS is much better at window management than X. On the surface, it was easy to implement lots of cool features. But deeper, NeWS is capable of synchronizing input events much more reliably than X11, so it can manage the input focus perfectly, where asynchronous X11 window managers fall flat on their face by definition. Our next step (if you'll pardon the allusion) was to use HyperNeWS (renamed HyperLook, a graphical user interface system like HyperCard with PostScript) to implemented a totally customizable X window manager! Some notes about OWM OWM is the "Open Window Manager" we prototyped in NeWS. We enhanced the NeWS window frames so they sported indexing tabs, pie menus, rooms, and a scrolling virtual desktop. Many of our enhancements were separatly developed, and plugged together orthogonally like legos. All NeWS applications could use these fancy frames, and the Open Window Manager wrapped X clients in the same frames that NeWS windows got! This way, factoring the window frames out as a part of the toolkit, and implementing the X window manager separately, NeWS applications don't have to know a thing about X window management, and X clients can go on doing the same nasty things they've always done, and everybody get the benefits of dynamic extensibility, and a consistent user interface, by using the default window class! X11 window managers must grab the server in order to animate rubber-band feedback over the screen when resizing and moving windows. This grabbing causes many problems with NeWS synchronous interests, that can be demonstrated by pressing the "Help" key while dragging out a rectangle on the root background. NeWS can do a much better job at managing global resources in the server because it is in the same address space and it has facilities like the overlay plane specifically designed to implement such window management functions, without even grabbing the server. This antisocial server grabbing behavior is just one symptom of a general class of problems with external X window management, including other important issues such as keyboard and colomap focus.

DonHopkins 9y ago

"A couple of decades ago I bough a Sun Ultra-60 and asked whether it was worth paying for the dual processor version. The answer that I got was "Yes. You need one processor to run X and another for everything else." People cursed with diskless Sun 3/50's would run xterms and emacs's across the net on each other's workstations instead of locally. It was noticeably less sluggish, because you didn't get hit with all the heavyweight process context switching between X clients and server every keystroke.

DonHopkins 9y ago

"Had he done some real design work and looked at what others were doing he might have realized that at its core, X was a distributed database system in which operations on some of the databases have visual side-effects. I forget the exact number, but X includes around 20 different databases: atoms, properties, contexts, selections, keymaps, etc. each with their own set of API calls. As a result, the X API is wide and shallow like the Mac, and full of interesting race conditions to boot. The whole thing could have been done with less than a dozen API calls." To that end, one of the weirder and cooler re-implementations of NeWS was Cogent's PIX for transputers. It was basically a NeWS-like multiprocessing PostScript interpreter for Transputers, with Linda "tuple spaces" as an interprocess communication primitive: http://ieeexplore.ieee.org/document/301904/ http://ieeexplore.ieee.org/document/301904/ The Cogent Research XTM is a desktop parallel computer based on the INMOS T800 transputer. Designed to expand from two to several hundred processors, the XTM provides a transparent distributed computing environment both within a single workstation and among a collection of workstations. Using Linda tuple spaces as the basis for interprocess communication and synchronization, a Unix-compatible, server-based OS was constructed. A graphic user interface is provided by an interactive PostScript window server called PIX. All processors see the same set of system services, and within protection limits, programs capable of using many processors can spread out over a network of workstations and resource servers, acquiring the services of unused processors. http://wiki.c2.com/?TupleSpace http://wiki.c2.com/?TupleSpace https://en.wikipedia.org/wiki/Tuple_space https://en.wikipedia.org/wiki/Tuple_space

adrianratnapala 9y ago

Through most of that article I was thinking "Yep, that's the kind of train-wreck you get when you design by committee, or worse by political need". But toward the end, he says that the particular kind of technical train-wreck that they got was similar to the Mac API (presumably the this means the pre-OS X OS). "[As a resulting of accidentally being lots of databases] the X API is wide and shallow like the Mac, and full of interesting race conditions to boot." Anyway, I didn't think that design by committee was one of Apples weaknesses.