6 ms·

Ada implementation of an X Window System Server (1989)

108 points by kickitover 5y ago

DonHopkins 5y ago
I posted this earlier about John Steinhart's XTool (here it is with updated links): https://news.ycombinator.com/item?id=17056516 https://news.ycombinator.com/item?id=17056516 Hasn't somebody reimplemented X11 in JavaScript/canvas/websockets yet? There was an X11 server for Lisp Machines! Not sure who wrote it, but it was probably written inside or at least nearby the X Consortium, and I remember Robert Scheifler used it regularly. https://news.ycombinator.com/item?id=6864364 https://news.ycombinator.com/item?id=6864364 "For example the TI Explorer Lisp Machine came with an X11 server written in Lisp. On my Symbolics Lisp Machine I used the usual MIT X11 server written in C - this was possible because the Symbolics Lisp machine had a C compiler." -lispm John Steinhart wrote XTool, a nice snappy reimplementation of X11 on top of SunView! ;) https://web.archive.org/web/20171105150953/https://minnie.tuhs.org//pipermail/tuhs/2017-September/010471.html https://web.archive.org/web/20171105150953/https://minnie.tu... https://news.ycombinator.com/item?id=15325226 https://news.ycombinator.com/item?id=15325226 >XTool was very small and fast compared to the X sample server because I wrote the server from scratch. I think that I'm the only person to write an X server outside of the X Consortium. One of the things that I learned by doing it was that the X Consortium folks were wrong when they said that the documentation was the standard, not the sample server. There were significant differences between the two. >The only really worthwhile thing about X was the distributed extension registration mechanism. All of the input, graphics and other crap should be moved to extension #1. That way, it won't be mandatory in conforming implementations once that stuff was obsolete. As you probably know, that's where we are today; nobody uses that stuff but it's like the corner of an Intel chip that implements the original instruction set. As an aside, I upset many when working on OpenDoc for Apple and saying the same thing there. >The atom/property mechanism allows clients to allocate memory in the server that can never be freed. Some way to free memory needs to be added. >The bit encodings should be part of a separate language binding, not part of the functional description. >Had he done some real design work and looked at what others were doing he might have realized that at its core, X was a distributed database system in which operations on some of the databases have visual side-effects. I forget the exact number, but X includes around 20 different databases: atoms, properties, contexts, selections, keymaps, etc. each with their own set of API calls. As a result, the X API is wide and shallow like the Mac, and full of interesting race conditions to boot. The whole thing could have been done with less than a dozen API calls.
- rjsw 5y ago
  The TI Explorer X11 server looks to have been written from scratch.
chasil 5y ago
This server appears to have been closed source, and I assume the source is lost. My bet is that OpenBSD's X server has a far better security design, as the server itself no longer runs as root. The aggressive free() also exposed use-after-free bugs never before seen, and OpenBSD has superior mitigations for rop gadget abuse, aslr everywhere, and other exploits beyond the imagination of 1989. The paper documents problems of Ada, particularly in linking to C (where the use of void * pointer casts was particularly difficult for Ada's type system), and that the Ada implementation had nearly double the lines of code as the C reference. In the days of 1989, the fastest CPUs were under 100MHz. It is unlikely that any of this code could be salvaged for a modern server, but the accomplishment is interesting.
- yakubin 5y ago
  > The aggressive free() also exposed use-after-free bugs never before seen What do you mean by that? Is there an overview somewhere about "aggressive free()" that I can read?
  - chasil 5y ago
    I'll look it up for you a little later. OpenBSD altered malloc/free to use mmap instead of sbrk. This allowed free to remove larger allocations from the address space, which caught a lot of programs that were using memory after free.
  - chasil 5y ago
    It looks like the conversion from sbrk to mmap happened in 2009. https://www.openbsd.org/papers/eurobsdcon2009/otto-malloc.pdf https://www.openbsd.org/papers/eurobsdcon2009/otto-malloc.pd... https://undeadly.org/cgi?action=article;sid=20090106070242 https://undeadly.org/cgi?action=article;sid=20090106070242 https://news.ycombinator.com/item?id=3286254 https://news.ycombinator.com/item?id=3286254 https://web.archive.org/web/20111231094521/http://os-blog.com/how-openbsd-leverages-malloc-to-find-bugs/ https://web.archive.org/web/20111231094521/http://os-blog.co... General architecture: https://utcc.utoronto.ca/~cks/space/blog/unix/SbrkVersusMmap https://utcc.utoronto.ca/~cks/space/blog/unix/SbrkVersusMmap
admin786 5y ago
Thanks 7 comment
zetalyrae 5y ago
It's interesting how the X11 server is the only major category of infrastructure where we have one and only one open source implementation. With browsers -- which are massively complex nowadays -- at least we have both WebKit and Gecko. C++ compilers are at a similar level of complexity and there is GCC and Clang. Linux, FreeBSD, OpenBSD etc. are all separate kernels. But for X11 there is only the the X.Org server, the reference implementation. How much work would it be to implement a practical X11 server in a safe language like Ada or Rust, which supports the widely-used extensions?
- naasking 5y ago
  I assume there's only one implementation because for decades people have been saying that X is an antiquated design, and they want a different way. Thus we got Wayland.
- tablespoon 5y ago
  > With browsers -- which are massively complex nowadays -- at least we have both WebKit and Gecko. Isn't that a bit of a historical accident? Almost all alternative browsers are now based on Chromium, and if Firefox dies or drops Gecko, I doubt anyone would go through the trouble to create a new one. I mean Microsoft tried and gave up on it, and they're one of the few organizations with the resources to actually succeed.
- rjsw 5y ago
  The KDrive X11 server was separate to the original one even though it was bundled with the rest of the X.Org software.
  - exikyut 5y ago
    What was the provenance of/motivation behind (writing) KDrive? I always wondered.
    - rjsw 5y ago
      From memory, it was that Keith Packard felt that the bits of the original server that run on the CPU (mfb, cfb) were optimized for CPUs without caches, they unroll the various bit operations and generate C functions for each one which increases code size. One of the first targets for KDrive was the Compaq iPaq with a StrongARM CPU and no GPU. I think the ideas behind KDrive have now been folded into the main X.Org server.
PaulHoule 5y ago
In Ada you write type Score = 0 .. 192 -- enough for a game of American football instead of choosing to use a signed or unsigned byte, short, triplebyte, etc.. The compiler figures out what kind of number to use. That article points out that bit twiddling and other operations you want to do in embedded systems is awkward in Ada because Ada hides the binary nature of math in ordinary computers. It's strange because, by 1980 when Ada was released, 8-bit bytes had won conclusively. The last straggler, Digital's 36-bit PDP-10 was discontinued in 1983.
- Jtsummers 5y ago
  Ada had to target lots of (relatively) bespoke systems, especially DOD systems. Even in 1980, while new systems had 8-bit bytes, that wasn't guaranteed on the existing systems where JOVIAL and other languages Ada was meant to replace were used. For the bit twiddling stuff, it does work with the modular types, at least now. I have no clue when it was added.
- pyuser583 5y ago
  Oh gosh I remember having to code for 36 bit systems. Wow that brings back memeories.

108 points by kickitover 5y ago

I posted this earlier about John Steinhart's XTool (here it is with updated links): https://news.ycombinator.com/item?id=17056516 https://news.ycombinator.com/item?id=17056516 Hasn't somebody reimplemented X11 in JavaScript/canvas/websockets yet? There was an X11 server for Lisp Machines! Not sure who wrote it, but it was probably written inside or at least nearby the X Consortium, and I remember Robert Scheifler used it regularly. https://news.ycombinator.com/item?id=6864364 https://news.ycombinator.com/item?id=6864364 "For example the TI Explorer Lisp Machine came with an X11 server written in Lisp. On my Symbolics Lisp Machine I used the usual MIT X11 server written in C - this was possible because the Symbolics Lisp machine had a C compiler." -lispm John Steinhart wrote XTool, a nice snappy reimplementation of X11 on top of SunView! ;) https://web.archive.org/web/20171105150953/https://minnie.tuhs.org//pipermail/tuhs/2017-September/010471.html https://web.archive.org/web/20171105150953/https://minnie.tu... https://news.ycombinator.com/item?id=15325226 https://news.ycombinator.com/item?id=15325226 >XTool was very small and fast compared to the X sample server because I wrote the server from scratch. I think that I'm the only person to write an X server outside of the X Consortium. One of the things that I learned by doing it was that the X Consortium folks were wrong when they said that the documentation was the standard, not the sample server. There were significant differences between the two. >The only really worthwhile thing about X was the distributed extension registration mechanism. All of the input, graphics and other crap should be moved to extension #1. That way, it won't be mandatory in conforming implementations once that stuff was obsolete. As you probably know, that's where we are today; nobody uses that stuff but it's like the corner of an Intel chip that implements the original instruction set. As an aside, I upset many when working on OpenDoc for Apple and saying the same thing there. >The atom/property mechanism allows clients to allocate memory in the server that can never be freed. Some way to free memory needs to be added. >The bit encodings should be part of a separate language binding, not part of the functional description. >Had he done some real design work and looked at what others were doing he might have realized that at its core, X was a distributed database system in which operations on some of the databases have visual side-effects. I forget the exact number, but X includes around 20 different databases: atoms, properties, contexts, selections, keymaps, etc. each with their own set of API calls. As a result, the X API is wide and shallow like the Mac, and full of interesting race conditions to boot. The whole thing could have been done with less than a dozen API calls.

rjsw 5y ago
The TI Explorer X11 server looks to have been written from scratch.

chasil 5y ago

This server appears to have been closed source, and I assume the source is lost. My bet is that OpenBSD's X server has a far better security design, as the server itself no longer runs as root. The aggressive free() also exposed use-after-free bugs never before seen, and OpenBSD has superior mitigations for rop gadget abuse, aslr everywhere, and other exploits beyond the imagination of 1989. The paper documents problems of Ada, particularly in linking to C (where the use of void * pointer casts was particularly difficult for Ada's type system), and that the Ada implementation had nearly double the lines of code as the C reference. In the days of 1989, the fastest CPUs were under 100MHz. It is unlikely that any of this code could be salvaged for a modern server, but the accomplishment is interesting.

yakubin 5y ago
> The aggressive free() also exposed use-after-free bugs never before seen What do you mean by that? Is there an overview somewhere about "aggressive free()" that I can read?
- chasil 5y ago
  I'll look it up for you a little later. OpenBSD altered malloc/free to use mmap instead of sbrk. This allowed free to remove larger allocations from the address space, which caught a lot of programs that were using memory after free.
- chasil 5y ago
  It looks like the conversion from sbrk to mmap happened in 2009. https://www.openbsd.org/papers/eurobsdcon2009/otto-malloc.pdf https://www.openbsd.org/papers/eurobsdcon2009/otto-malloc.pd... https://undeadly.org/cgi?action=article;sid=20090106070242 https://undeadly.org/cgi?action=article;sid=20090106070242 https://news.ycombinator.com/item?id=3286254 https://news.ycombinator.com/item?id=3286254 https://web.archive.org/web/20111231094521/http://os-blog.com/how-openbsd-leverages-malloc-to-find-bugs/ https://web.archive.org/web/20111231094521/http://os-blog.co... General architecture: https://utcc.utoronto.ca/~cks/space/blog/unix/SbrkVersusMmap https://utcc.utoronto.ca/~cks/space/blog/unix/SbrkVersusMmap

admin786 5y ago

Thanks 7 comment

zetalyrae 5y ago

It's interesting how the X11 server is the only major category of infrastructure where we have one and only one open source implementation. With browsers -- which are massively complex nowadays -- at least we have both WebKit and Gecko. C++ compilers are at a similar level of complexity and there is GCC and Clang. Linux, FreeBSD, OpenBSD etc. are all separate kernels. But for X11 there is only the the X.Org server, the reference implementation. How much work would it be to implement a practical X11 server in a safe language like Ada or Rust, which supports the widely-used extensions?

naasking 5y ago
I assume there's only one implementation because for decades people have been saying that X is an antiquated design, and they want a different way. Thus we got Wayland.
tablespoon 5y ago
> With browsers -- which are massively complex nowadays -- at least we have both WebKit and Gecko. Isn't that a bit of a historical accident? Almost all alternative browsers are now based on Chromium, and if Firefox dies or drops Gecko, I doubt anyone would go through the trouble to create a new one. I mean Microsoft tried and gave up on it, and they're one of the few organizations with the resources to actually succeed.
rjsw 5y ago
The KDrive X11 server was separate to the original one even though it was bundled with the rest of the X.Org software.
- exikyut 5y ago
  What was the provenance of/motivation behind (writing) KDrive? I always wondered.
  - rjsw 5y ago
    From memory, it was that Keith Packard felt that the bits of the original server that run on the CPU (mfb, cfb) were optimized for CPUs without caches, they unroll the various bit operations and generate C functions for each one which increases code size. One of the first targets for KDrive was the Compaq iPaq with a StrongARM CPU and no GPU. I think the ideas behind KDrive have now been folded into the main X.Org server.

PaulHoule 5y ago

In Ada you write type Score = 0 .. 192 -- enough for a game of American football instead of choosing to use a signed or unsigned byte, short, triplebyte, etc.. The compiler figures out what kind of number to use. That article points out that bit twiddling and other operations you want to do in embedded systems is awkward in Ada because Ada hides the binary nature of math in ordinary computers. It's strange because, by 1980 when Ada was released, 8-bit bytes had won conclusively. The last straggler, Digital's 36-bit PDP-10 was discontinued in 1983.

Jtsummers 5y ago
Ada had to target lots of (relatively) bespoke systems, especially DOD systems. Even in 1980, while new systems had 8-bit bytes, that wasn't guaranteed on the existing systems where JOVIAL and other languages Ada was meant to replace were used. For the bit twiddling stuff, it does work with the modular types, at least now. I have no clue when it was added.
pyuser583 5y ago
Oh gosh I remember having to code for 36 bit systems. Wow that brings back memeories.