8 ms·

Xplain – Explaining X11 for the rest of us

456 points by b3h3moth 10y ago

therein 10y ago
Even my manager who worked on the original X/Motif codebase in the 80s when DEC and HP was working on it doesn't enjoy talking about these details. Pretty good write up, though.
shmerl 10y ago
> Several new technologies have appeared You should probably also add Vulkan + WSI to the list. UPDATE: I just realized, this article is from 2013.
- datenwolf 10y ago
  Neither Vulkan nor WSI are essential for operation of X11 or Wayland. Vulkan is "just" a new API to talk to GPUs, completely separated from windowing (just as is OpenGL) and the WSI part of Vulkan aims for a standardized method to allow windowing systems to expose the low level graphics device they're working towards to client applications so that implementations of APIs like Vulkan (but also OpenGL) can talk to it. WSI is kind of a generalized API for what GLX or WGL are doing. This neither revolutionary nor a new technology. Just a unified, agreed upon, window system neutral API for something that's quite old.
daveloyall 10y ago
https://github.com/magcius/xplain/issues/15 https://github.com/magcius/xplain/issues/15
- stonogo 10y ago
  Why isn't this just an email to the author?
  - daveloyall 10y ago
    "praise in public, criticize in private" ?
    - sdegutis 10y ago
      Personally I'd feel a bit uncomfortable if someone posted that kind of comment to me on Github and HN. I'd be grateful of course, but would feel pretty awkward.
      - Jasper_ 10y ago
        Author here, I'm totally OK with it.
      - codemaster3000 10y ago
        The author seems more than ok with receiving these types of comments and almost seems to be fishing for them... Great writeup, but the number of times he's mentioned that it would be 'ok' to thank him or give him praise seems a bit over the top > If you have any questions or comments, _or just want to thank me_, feel free to email me....
  - gkya 10y ago
    Because others can't see it then. Also, it'll appear on his github that he's contributed to the project.
    - daveloyall 10y ago
      That project is a labor of love and the author asked for very little in return, but he did ask for it, publicly: https://github.com/magcius/xplain/blob/gh-pages/article/regions.html#L402-L415 https://github.com/magcius/xplain/blob/gh-pages/article/regi... Had he asked for money, I probably would not have sent any. But he asked for something that I can provide with ease, so I did. And I enjoyed it. Is that really what this is about? Is it wrong to do good if it might benefit the do-gooder in some way? Can we all take a collective moment to think critically about that? :) (anyway, you're wrong(ish?). Posting an issue does not appear to have added my name to https://github.com/magcius/xplain/graphs/contributors https://github.com/magcius/xplain/graphs/contributors . Though I could have sworn that I've seen what you are talking about happen before. \shrug\, that's github's fault.)
      - gkya 10y ago
        An issue tracker is a tracker for issues, not the guestbook to the project. You could've emailed the author, link to him on social media, on your website, submit to reddit etc... but you choose to open an issue on his issue tracker, and put that here on HN in a comment. Even if you intended differently, it's how it look to an outsider. On your last paragraph, yes, github used to show who reports issues as contributed to the project to which they reported. I recall at one point having reported an embarassing silly issue (a very stupid config mistake I made that I don't recall) to Pelican blogware and it went there among the contributed projects as a badge of dishonour. :) But now I checked, that section seems to be gone from the profile page on gihtub.
        daveloyall 10y ago
        ok, you got me on the "issue trackers are for issues" point.
        10y ago
sdegutis 10y ago
The interactivity of this series is just amazingly helpful in visualizing what the heck is going on. Honestly, for me that makes this one of the few links on HN that I might actually read all the way through. And may even use it as a homeschool lesson for our kids, if it seems written simply enough (which at first glance it is).
microtherion 10y ago
Personally, my favorite explanation of X11 is found on page 139 of the Unix Hater's Handbook http://web.mit.edu/~simsong/www/ugh.pdf http://web.mit.edu/~simsong/www/ugh.pdf "A truly portable X application is required to act like the persistent customer in Monty Python’s “Cheese Shop” sketch, or a grail seeker in “Monty Python and the Holy Grail.” Even the simplest applications must answer many difficult questions" Read the original for the hilariously accurate dialog that follows…
- Aloha 10y ago
  this is my favorite quote from UHG "If the designers of X Windows built cars, there would be no fewer than five steering wheels hidden about the cockpit, none of which fol- lowed the same principles—but you’d be able to shift gears with your car stereo. Useful feature, that."
- sampo 10y ago
  > page 139 of the Unix Hater's Handbook http://web.mit.edu/~simsong/www/ugh.pdf http://web.mit.edu/~simsong/www/ugh.pdf For quick access, put page 177 in pdf reader to get to printed page 139.
  - nitrogen 10y ago
    Sometimes these work: http://web.mit.edu/~simsong/www/ugh.pdf#page177 http://web.mit.edu/~simsong/www/ugh.pdf#page177 or http://web.mit.edu/~simsong/www/ugh.pdf#177 http://web.mit.edu/~simsong/www/ugh.pdf#177
    - 0xmohit 10y ago
      The actual form would be http://web.mit.edu/~simsong/www/ugh.pdf#page=177 http://web.mit.edu/~simsong/www/ugh.pdf#page=177 For pdf.js (used by Firefox), refer to https://github.com/mozilla/pdf.js/wiki/Viewer-options https://github.com/mozilla/pdf.js/wiki/Viewer-options If using Adobe Reader, refer https://www.adobe.com/content/dam/Adobe/en/devnet/acrobat/pdfs/pdf_open_parameters.pdf#page=5 https://www.adobe.com/content/dam/Adobe/en/devnet/acrobat/pd... (The second variant mentioned by you that works in Firefox using pdf.js appears to be a undocumented feature!)
Jasper_ 10y ago
Author here, let me know if you have any questions or comments! Given how much attention this is getting, I should probably finally finish one of the three articles I have sitting around in the backlog.
- alfiedotwtf 10y ago
  Thanks for the writeup. I've been wanting to read about X11 for a long time but didn't know where to start.
deleted 10y ago
[deleted]
jd3 10y ago
Excellent description of the (somewhat needlessly) complex X Window System (the Blit and Rio are so beautiful). Still makes me sad that COMPOSITE doesn't build in XQuartz for OS X. I posted on the mailing list, but got a pretty non-helpful response.. [0] [0]: http://comments.gmane.org/gmane.os.apple.xquartz.devel/912 http://comments.gmane.org/gmane.os.apple.xquartz.devel/912
- eschaton 10y ago
  What did you find non-helpful about the response?
  - jd3 10y ago
    It would have been nice to get an update on the status of building the COMPOSITE extension into XQuartz, not a link to a new library.
eschaton 10y ago
The author appears to credit the X Window System or Microsoft Windows with repaint-on-expose. To my knowledge that was first implemented (along with regions) by Bill Atkinson for the Apple Lisa, because he misremembered what he saw on the visit to PARC; overlapped windows in the PARC systems at the time didn't automatically and efficiently repaint only the affected area.
swiley 10y ago
This looks like a great article but on a phone the margins are as wide as the text! And they used that wrapping mode that stretches lines out instead of truncating them which looks great if you don't want to read the text but is super annoying when you are.
- Jasper_ 10y ago
  This article series wasn't designed for phones! It uses heavy interactive figures and diagrams and I actually expected it would bring most phones to a halt with all of the drawing... I don't know much about front-end dev and made something that looked decent on my laptop. PRs are most appreciated if you want to make it presentable on mobile devices.
  - kiwijamo 10y ago
    It seemed to work fine on my iPhone. My only complaint was the font size was a bit too small. Also reader mode didn't seem to work so I wasn't able to use that as an alternative to get a reasonable font size.
jordyjordy 10y ago
We are bank of hackers based in the UK, US, Canada we offer services such as school grades change, DDOS, database hack, credit, criminal records, bank transfer, social network hack, emails, password retrieval and all forms of hack, contact GURULEADS hackers today via email demonteco@outlook.com
SXX 10y ago
For those who might want to understand why Wayland was created. Also tons of fun facts about X11. Really nice talk of Daniel Stone about X server from 2013: https://www.youtube.com/watch?v=RIctzAQOe44 https://www.youtube.com/watch?v=RIctzAQOe44
- digi_owl 10y ago
  Frankly i find that video a case of preaching to the choir.
lil1729 10y ago
https://medium.com/@slavapestov/yesterdays-news-c52f2be95205#.acr1rvmmo https://medium.com/@slavapestov/yesterdays-news-c52f2be95205... "Unlike X11, where the graphics primitives were rather low-level and all input event handling involved round-trips to the client, NeWS was advanced enough that simple widgets, such as scroll bars and sliders, could be implemented entirely server-side, only sending high-level state changes to the client, more along the lines of “slider value is now set to 15” than “mouse button 2 released”. Similarly, the client could ask the server to render a widget in a given state, rather than repeatedly transmitting sequences of graphics primitives."
- pjc50 10y ago
  .. as it was always done in Windows with the native controls.
  - mcculley 10y ago
    Windows native controls could be remoted? The point of the comment above is that NeWS communicated events at a higher semantic level than X between the display server and the application. This communication was network transparent.
    - pjc50 10y ago
      Not remoted, but the semantic communication between the kernel and the application is at the event level for native controls such as the scrollbars example mentioned.
      - mcculley 10y ago
        If you're not remoting, it's not difficult to work at a high level. The API presented to the application is easy to control. The interesting thing about NeWS was that a high level event could be defined by the application. The protocol needn't be designed to accommodate it a priori. The example mentioned sliders, not scrollbars. The display server wouldn't even know what a slider was. It would have been defined by the application.
known 10y ago
Check https://en.wikipedia.org/wiki/X_Window_System#Competitors https://en.wikipedia.org/wiki/X_Window_System#Competitors
foo101 10y ago
How are the interactive windows that are displayed in the page created? Is that a complete simulation of X server done with JavaScript or is it something unrelated to X server?
- Jasper_ 10y ago
  It's a simulation of an X server done with JavaScript. The README.md explains it in a bit more detail: https://github.com/magcius/xplain#demo-framework--xserverjs https://github.com/magcius/xplain#demo-framework--xserverjs Basically, I emulate what an X server would do, semantically, without implementing any of the actual wire protocol. The codebase is, I hope, pretty well commented and designed as something to learn from. https://github.com/magcius/xplain/blob/gh-pages/src/server/server.js https://github.com/magcius/xplain/blob/gh-pages/src/server/s...
hathym 10y ago
the X11 server written in Javascript here: https://magcius.github.io/xplain/src/server/server.js https://magcius.github.io/xplain/src/server/server.js
rikkus 10y ago
Anyone who's interested in X11 (the protocol) and the Ruby language should have a look at Mathieu Bouchard's incredible implementation [1] There's some quite beautiful Ruby code in there, e.g.: module X11 k = Keysymdef = {} c = Cursorfont = {} [k,c].each {|x| x.extend X11::MAssignForHash } k[%w| space exclam quotedbl numbersign dollar percent ampersand |]=32..38 k[%w| quoteright parenleft parenright asterisk plus comma minus |]=39..45 [...] [1] http://artengine.ca/matju/RubyX11/ http://artengine.ca/matju/RubyX11/
digi_owl 10y ago
> While I do personally believe Wayland is going to become The Linux Display Server Of The Future® Best i can understand, wayland is a protocol spec rather than a set of code like X. Each compositor/wm is individually responsible for implementing said protocol and thus take over the job of X (input handling et al). Meaning that there will no longer be an X server to act as a independent arbiter of behavior. Whatever the devs of whatever DE you are logged into will have the final word. The more i learn of Wayland, the more i expect it to turn into a hairball to match X. Only now without a network option ("too insecure"), and with GPU acceleration. BTW, why are we so bent out of shape about this seats concept? Why oh why are we continually trying to turn a single user piece of hardware into a desktop mainframe?!

456 points by b3h3moth 10y ago

Even my manager who worked on the original X/Motif codebase in the 80s when DEC and HP was working on it doesn't enjoy talking about these details. Pretty good write up, though.

shmerl 10y ago

> Several new technologies have appeared You should probably also add Vulkan + WSI to the list. UPDATE: I just realized, this article is from 2013.

datenwolf 10y ago
Neither Vulkan nor WSI are essential for operation of X11 or Wayland. Vulkan is "just" a new API to talk to GPUs, completely separated from windowing (just as is OpenGL) and the WSI part of Vulkan aims for a standardized method to allow windowing systems to expose the low level graphics device they're working towards to client applications so that implementations of APIs like Vulkan (but also OpenGL) can talk to it. WSI is kind of a generalized API for what GLX or WGL are doing. This neither revolutionary nor a new technology. Just a unified, agreed upon, window system neutral API for something that's quite old.

daveloyall 10y ago

https://github.com/magcius/xplain/issues/15 https://github.com/magcius/xplain/issues/15

stonogo 10y ago
Why isn't this just an email to the author?
- daveloyall 10y ago
  "praise in public, criticize in private" ?
  - sdegutis 10y ago
    Personally I'd feel a bit uncomfortable if someone posted that kind of comment to me on Github and HN. I'd be grateful of course, but would feel pretty awkward.
    - Jasper_ 10y ago
      Author here, I'm totally OK with it.
    - codemaster3000 10y ago
      The author seems more than ok with receiving these types of comments and almost seems to be fishing for them... Great writeup, but the number of times he's mentioned that it would be 'ok' to thank him or give him praise seems a bit over the top > If you have any questions or comments, _or just want to thank me_, feel free to email me....
- gkya 10y ago
  Because others can't see it then. Also, it'll appear on his github that he's contributed to the project.
  - daveloyall 10y ago
    That project is a labor of love and the author asked for very little in return, but he did ask for it, publicly: https://github.com/magcius/xplain/blob/gh-pages/article/regions.html#L402-L415 https://github.com/magcius/xplain/blob/gh-pages/article/regi... Had he asked for money, I probably would not have sent any. But he asked for something that I can provide with ease, so I did. And I enjoyed it. Is that really what this is about? Is it wrong to do good if it might benefit the do-gooder in some way? Can we all take a collective moment to think critically about that? :) (anyway, you're wrong(ish?). Posting an issue does not appear to have added my name to https://github.com/magcius/xplain/graphs/contributors https://github.com/magcius/xplain/graphs/contributors . Though I could have sworn that I've seen what you are talking about happen before. \shrug\, that's github's fault.)
    - gkya 10y ago
      An issue tracker is a tracker for issues, not the guestbook to the project. You could've emailed the author, link to him on social media, on your website, submit to reddit etc... but you choose to open an issue on his issue tracker, and put that here on HN in a comment. Even if you intended differently, it's how it look to an outsider. On your last paragraph, yes, github used to show who reports issues as contributed to the project to which they reported. I recall at one point having reported an embarassing silly issue (a very stupid config mistake I made that I don't recall) to Pelican blogware and it went there among the contributed projects as a badge of dishonour. :) But now I checked, that section seems to be gone from the profile page on gihtub.
      - daveloyall 10y ago
        ok, you got me on the "issue trackers are for issues" point.
      - 10y ago

sdegutis 10y ago

The interactivity of this series is just amazingly helpful in visualizing what the heck is going on. Honestly, for me that makes this one of the few links on HN that I might actually read all the way through. And may even use it as a homeschool lesson for our kids, if it seems written simply enough (which at first glance it is).

microtherion 10y ago

Personally, my favorite explanation of X11 is found on page 139 of the Unix Hater's Handbook http://web.mit.edu/~simsong/www/ugh.pdf http://web.mit.edu/~simsong/www/ugh.pdf "A truly portable X application is required to act like the persistent customer in Monty Python’s “Cheese Shop” sketch, or a grail seeker in “Monty Python and the Holy Grail.” Even the simplest applications must answer many difficult questions" Read the original for the hilariously accurate dialog that follows…

Aloha 10y ago
this is my favorite quote from UHG "If the designers of X Windows built cars, there would be no fewer than five steering wheels hidden about the cockpit, none of which fol- lowed the same principles—but you’d be able to shift gears with your car stereo. Useful feature, that."
sampo 10y ago
> page 139 of the Unix Hater's Handbook http://web.mit.edu/~simsong/www/ugh.pdf http://web.mit.edu/~simsong/www/ugh.pdf For quick access, put page 177 in pdf reader to get to printed page 139.
- nitrogen 10y ago
  Sometimes these work: http://web.mit.edu/~simsong/www/ugh.pdf#page177 http://web.mit.edu/~simsong/www/ugh.pdf#page177 or http://web.mit.edu/~simsong/www/ugh.pdf#177 http://web.mit.edu/~simsong/www/ugh.pdf#177
  - 0xmohit 10y ago
    The actual form would be http://web.mit.edu/~simsong/www/ugh.pdf#page=177 http://web.mit.edu/~simsong/www/ugh.pdf#page=177 For pdf.js (used by Firefox), refer to https://github.com/mozilla/pdf.js/wiki/Viewer-options https://github.com/mozilla/pdf.js/wiki/Viewer-options If using Adobe Reader, refer https://www.adobe.com/content/dam/Adobe/en/devnet/acrobat/pdfs/pdf_open_parameters.pdf#page=5 https://www.adobe.com/content/dam/Adobe/en/devnet/acrobat/pd... (The second variant mentioned by you that works in Firefox using pdf.js appears to be a undocumented feature!)

Jasper_ 10y ago

Author here, let me know if you have any questions or comments! Given how much attention this is getting, I should probably finally finish one of the three articles I have sitting around in the backlog.

alfiedotwtf 10y ago
Thanks for the writeup. I've been wanting to read about X11 for a long time but didn't know where to start.

deleted 10y ago

[deleted]

jd3 10y ago

Excellent description of the (somewhat needlessly) complex X Window System (the Blit and Rio are so beautiful). Still makes me sad that COMPOSITE doesn't build in XQuartz for OS X. I posted on the mailing list, but got a pretty non-helpful response.. [0] [0]: http://comments.gmane.org/gmane.os.apple.xquartz.devel/912 http://comments.gmane.org/gmane.os.apple.xquartz.devel/912

eschaton 10y ago
What did you find non-helpful about the response?
- jd3 10y ago
  It would have been nice to get an update on the status of building the COMPOSITE extension into XQuartz, not a link to a new library.

eschaton 10y ago

The author appears to credit the X Window System or Microsoft Windows with repaint-on-expose. To my knowledge that was first implemented (along with regions) by Bill Atkinson for the Apple Lisa, because he misremembered what he saw on the visit to PARC; overlapped windows in the PARC systems at the time didn't automatically and efficiently repaint only the affected area.

swiley 10y ago

This looks like a great article but on a phone the margins are as wide as the text! And they used that wrapping mode that stretches lines out instead of truncating them which looks great if you don't want to read the text but is super annoying when you are.

Jasper_ 10y ago
This article series wasn't designed for phones! It uses heavy interactive figures and diagrams and I actually expected it would bring most phones to a halt with all of the drawing... I don't know much about front-end dev and made something that looked decent on my laptop. PRs are most appreciated if you want to make it presentable on mobile devices.
- kiwijamo 10y ago
  It seemed to work fine on my iPhone. My only complaint was the font size was a bit too small. Also reader mode didn't seem to work so I wasn't able to use that as an alternative to get a reasonable font size.

jordyjordy 10y ago

We are bank of hackers based in the UK, US, Canada we offer services such as school grades change, DDOS, database hack, credit, criminal records, bank transfer, social network hack, emails, password retrieval and all forms of hack, contact GURULEADS hackers today via email demonteco@outlook.com

SXX 10y ago

For those who might want to understand why Wayland was created. Also tons of fun facts about X11. Really nice talk of Daniel Stone about X server from 2013: https://www.youtube.com/watch?v=RIctzAQOe44 https://www.youtube.com/watch?v=RIctzAQOe44

digi_owl 10y ago
Frankly i find that video a case of preaching to the choir.

lil1729 10y ago

https://medium.com/@slavapestov/yesterdays-news-c52f2be95205#.acr1rvmmo https://medium.com/@slavapestov/yesterdays-news-c52f2be95205... "Unlike X11, where the graphics primitives were rather low-level and all input event handling involved round-trips to the client, NeWS was advanced enough that simple widgets, such as scroll bars and sliders, could be implemented entirely server-side, only sending high-level state changes to the client, more along the lines of “slider value is now set to 15” than “mouse button 2 released”. Similarly, the client could ask the server to render a widget in a given state, rather than repeatedly transmitting sequences of graphics primitives."

pjc50 10y ago
.. as it was always done in Windows with the native controls.
- mcculley 10y ago
  Windows native controls could be remoted? The point of the comment above is that NeWS communicated events at a higher semantic level than X between the display server and the application. This communication was network transparent.
  - pjc50 10y ago
    Not remoted, but the semantic communication between the kernel and the application is at the event level for native controls such as the scrollbars example mentioned.
    - mcculley 10y ago
      If you're not remoting, it's not difficult to work at a high level. The API presented to the application is easy to control. The interesting thing about NeWS was that a high level event could be defined by the application. The protocol needn't be designed to accommodate it a priori. The example mentioned sliders, not scrollbars. The display server wouldn't even know what a slider was. It would have been defined by the application.

known 10y ago

Check https://en.wikipedia.org/wiki/X_Window_System#Competitors https://en.wikipedia.org/wiki/X_Window_System#Competitors

foo101 10y ago

How are the interactive windows that are displayed in the page created? Is that a complete simulation of X server done with JavaScript or is it something unrelated to X server?

Jasper_ 10y ago
It's a simulation of an X server done with JavaScript. The README.md explains it in a bit more detail: https://github.com/magcius/xplain#demo-framework--xserverjs https://github.com/magcius/xplain#demo-framework--xserverjs Basically, I emulate what an X server would do, semantically, without implementing any of the actual wire protocol. The codebase is, I hope, pretty well commented and designed as something to learn from. https://github.com/magcius/xplain/blob/gh-pages/src/server/server.js https://github.com/magcius/xplain/blob/gh-pages/src/server/s...

hathym 10y ago

the X11 server written in Javascript here: https://magcius.github.io/xplain/src/server/server.js https://magcius.github.io/xplain/src/server/server.js

rikkus 10y ago

Anyone who's interested in X11 (the protocol) and the Ruby language should have a look at Mathieu Bouchard's incredible implementation [1] There's some quite beautiful Ruby code in there, e.g.: module X11 k = Keysymdef = {} c = Cursorfont = {} [k,c].each {|x| x.extend X11::MAssignForHash } k[%w| space exclam quotedbl numbersign dollar percent ampersand |]=32..38 k[%w| quoteright parenleft parenright asterisk plus comma minus |]=39..45 [...] [1] http://artengine.ca/matju/RubyX11/ http://artengine.ca/matju/RubyX11/

digi_owl 10y ago

> While I do personally believe Wayland is going to become The Linux Display Server Of The Future® Best i can understand, wayland is a protocol spec rather than a set of code like X. Each compositor/wm is individually responsible for implementing said protocol and thus take over the job of X (input handling et al). Meaning that there will no longer be an X server to act as a independent arbiter of behavior. Whatever the devs of whatever DE you are logged into will have the final word. The more i learn of Wayland, the more i expect it to turn into a hairball to match X. Only now without a network option ("too insecure"), and with GPU acceleration. BTW, why are we so bent out of shape about this seats concept? Why oh why are we continually trying to turn a single user piece of hardware into a desktop mainframe?!