4 ms·

Wow, this is huge. Is the goal really to have a new language better than Objective-C, and to actually use it in projects, or mainly to demonstrate what an AI ca

by Rochus 10d ago

Wow, this is huge. Is the goal really to have a new language better than Objective-C, and to actually use it in projects, or mainly to demonstrate what an AI can do? As far as I understand, there is a non-trivial optimiser, assumingly also generated by AI, including the associated IR and lowerings. Are there comparisons of the resulting machine code quality and performance compared to e.g. GCC? Did you design and specify the xc language, and did the AI implement the frontend based on your specification, or was it a "full package" AI approach? (sorry if some questions were already answered on the web sites and the code, but it was too much to quickly get the answers)

spacedcowboy 9d ago
Hi, thanks for the kind words and interest :) It is indeed the goal to have a language better than (or at least fixing some of the pain points of) ObjC, and to make it truly cross-platform and just as easy to grok. Coming from C, I think ObjC was pretty much at the sweet-spot for complexity vs flexibility - there’s only a few things you have to learn over and above C and you get so much from that little extra. I’m already using it in one big project - and now this mentioned on HN has utterly failed [grin] I can put a link to https://blewit.net/ https://blewit.net/ - the site isn’t really ready yet, another 6-8 weeks before launch, but if you want to see some of the WASM results, click on the sandbox link :) On the server side of blewit, everything is in xc, no apache, no scripting, and it uses the same classes there as it does in the WASM client. Plenty of C libraries compiled in with #use and then just linked: TLS, redis, Postgres libpq, … The compiler is mainly written with AI, but with a lot of supervision - those 6 months have been pretty much non-stop, with me being retired, and since I used to work for Apple for a couple of decades I have opinions on what the compiler should do (or not do) :) It started off as a project to write a new language for the 6502, if you look at https://atari-xt.com/ https://atari-xt.com/ you’ll see a striking similarity in the website style... I knew I’d want more than one back end (st and xl) so from the start there was this idea of an IR that was flexible enough to handle register-rich and register-poor architectures, then I thought it’d be kind of useful to be able to write and test code on the Mac without loading it into the FPGA, so it was scaling from 8-bit to 64-bit pretty much immediately, with different executable formats; once that ground was laid it wasn’t too hard to extend it to the current multiple platforms. The optimiser is pretty thorough in terms of the classical operations. Where it still lacks is automatic vectorisation, clang vectorises code in lots of places you wouldn’t expect, but it also has a huge head-start over xc. Right now we’re sometimes on a par with a clang-compiled C program, but otherwise between 1.1x and 1.8x slower. There has been the occasional micro-benchmark where xc comes out faster :) I don’t actually have a benchmark suite, this is more informal testing during developmentof the optimiser and that’s something I should fix, not least because it’ll show where the next “bang for the buck” will be. It’s worth mentioning that most of my optimisation tests have been pretty low-level, so the clang version is written in C to make it easier to understand the assembly for this mere human, and xc has a bit more overhead with its automatic reference counting code, so a better future test might be to compare vs ObjC. Originally, I wrote up a 300 (or so) line spec for the language, and another spec for the IR, because it had to include automatic and transparent banked-memory use on the 6502 (so pointers were 3-bytes {bank, hi, lo}). Coming from using modern ObjC, I wanted ARC from the start, so that was built in, but it was only extended to blocks and callbacks (a callback is a ound function, an {object,method} or {nil,function} tuple) when we got onto the serious architecture support. It’s been “interesting” supporting a machine that doesn’t even support ‘mul’ alongside one that has vector simd :) Overall, I’d say the AI has done maybe 80% of the work, possibly 85% if I’m being generous. There have certainly been times I’m in the code changing things because it’s easier to express the difference between what it first came up with, and what I actually want, in actual code rather than English. On the other hand, I’d never even have attempted such a large project on my own. I see the AI as a massive force-multiplier on what can be done. It doesn’t get tired, it prevaricates (“I don’t want to make such a large change without <insert nonsense>”) a lot less than humans, and it has alot of knowledge to draw on. One other thing I guess I should mention is that there is also the beginnings of a cross-platform UI framework, which uses an AppKit-like binding (you get things like TableView, CollectionView, OutlineView etc, with datasource protocols and delegates) but binds to the native widgets to provide a native look/feel/interaction from the same AppKit-like source. The docs go through it in more detail. This is only about 50% done, there’s a lot to finish off here. It helps that String in xc is native UTF8, but the binding part takes time. The fallback default is to draw the widget if there’s no binding, and until you pretty much 100% cover the native UI for a platform, it can look a bit jarring. Alongside that, there’s a tool ‘RoCkS’ which is even younger (maybe 10% there) which is intended to be the equivalent of Interface Builder. RoCkS is spelt that way because of the original atari origins... the GUI designer for GEM was "RCS" - Resource Construction Kit :) You design your UI with drag/drop and springs/struts, then bind UI objects in various layouts {desktop, tablet, phone} in {portrait, landscape} to the same core codebase. All with drag/drop, just like in Interface builder. Then the same application code can use a ‘phone’ UI layout when you compile the iOS/Android version, and use a ‘desktop’ layout when you compile a Windows/Linux/Mac version etc. At that point, I might re-announce on HN, because free and open-source cross-platform (and easy multi-platform) UI isn’t that common :) Again, thanks for the interest :) At least I got 1 comment :)
- Rochus 9d ago
  Interesting, thanks for the detailed explanations. Objective-C has indeed some impressive features, but also some inefficiencies and it got a bit out of favour over the years. Coming from the Wirth school I also understand the virtue and challenge of writing a compiler in his own language, but in practice it is often rather an obstacle than a benefit, and if a language is useful, there should never be a lack of representative projects. I thus write my own compilers always in a conservative C++98 subset or even in C so bootstrapping is never an issue and there is no second compiler necessary. Is your blewit, which sounds yet like another big code base, also AI generated/assisted? If I understand you correctly, you indeed have the engineering expertise to write compilers and to understand its architecture and complexities; so the project is not "vibe coded", but rather "augmented engineering"; I didn't manage to have a detailed look at the large code base yet, but in that case I would expect that it rather looks like a human engineered code base than the often strange looking architecture and code which I already have seen in Claude projects where the AI is just left alone for a few weeks. A performance within 1.1 to 1.8 on average of optimized Clang is a very good start for such a project. As I understand, the AI has implemented the frontend and lowering after your spec, which is amazing and demonstrates that it is able to work not only in and for the languages it was trained on. How was the AI able to demonstrate that your spec has been completely and correctly implemented? Is the language and IR spec somewhere on the web? What formalism did you use for the spec? I well remember GEM and also wrote software for Atari ST many years ago ;-)
  - spacedcowboy 9d ago
    I'd already written a lot of it in ObjC, thinking it would be "just for me" before I moved to wanting it running on the FPGA. Three options - Port the ObjC runtime to my little OS on the FPGA ? - Rewrite everything in C so the native FPGA tools could compile it - Make it compile itself and treat the ObjC version as a bootstrap. #3 seemed the best option :) Blewit is a reasonably-sized project, but it's not on the same order as the compiler. It did shake out a few bugs in the compiler that the unit-tests etc. didn't find. At least, when you own the compiler, you don't submit a bug report to the developers and wait for a patch :) Yep, I'm happy with the optimisation - and I think most of the areas left are actually vectorisation - clang is very eager to vectorise (for good reason, it makes a difference) As for the spec, the original one is https://github.com/ThrudTheBarbarian/xc/blob/main/compiler/docs/xtc.bnf https://github.com/ThrudTheBarbarian/xc/blob/main/compiler/d... but that's now quite out of date (despite claiming to be the reference :) I'll update it, and also add a readable version to the website.. We've really only added things that haven't broken fundamentals recently. The main recent additions were the block type (and since I agree with the thinking that spawned "http://goshdarnblocksyntax.com http://goshdarnblocksyntax.com", we use 'block' not some convoluted ^ notation). Similarly a callback is a 'callback', the types were added and obey the usual grammar rules. The GEM desktop lives on: https://0x0000ff.co.uk/mov/xt/xtos-aug-11.mov https://0x0000ff.co.uk/mov/xt/xtos-aug-11.mov :) - if you look at the bottom you'll see 2 icons (ST, XE) and clicking on that launches the emulation in a window, but since the actual desktop also uses GEM, you can run a GEM app (actually on either the ST or the XL) and open windows, drag them around etc. - There's a lot to do to get this desktop to be as useful as Magic! or Thing, but for various reasons it's on hold atm - mainly that I'm waiting on a physical garage to be built, which (today!) got rafters put on. Once that's in place, I'll have all the electronics stuff set up, and I can debug the motherboard that the FPGA plugs into.
deleted 9d ago
[deleted]

by Rochus 10d ago

spacedcowboy 9d ago

Hi, thanks for the kind words and interest :) It is indeed the goal to have a language better than (or at least fixing some of the pain points of) ObjC, and to make it truly cross-platform and just as easy to grok. Coming from C, I think ObjC was pretty much at the sweet-spot for complexity vs flexibility - there’s only a few things you have to learn over and above C and you get so much from that little extra. I’m already using it in one big project - and now this mentioned on HN has utterly failed [grin] I can put a link to https://blewit.net/ https://blewit.net/ - the site isn’t really ready yet, another 6-8 weeks before launch, but if you want to see some of the WASM results, click on the sandbox link :) On the server side of blewit, everything is in xc, no apache, no scripting, and it uses the same classes there as it does in the WASM client. Plenty of C libraries compiled in with #use and then just linked: TLS, redis, Postgres libpq, … The compiler is mainly written with AI, but with a lot of supervision - those 6 months have been pretty much non-stop, with me being retired, and since I used to work for Apple for a couple of decades I have opinions on what the compiler should do (or not do) :) It started off as a project to write a new language for the 6502, if you look at https://atari-xt.com/ https://atari-xt.com/ you’ll see a striking similarity in the website style... I knew I’d want more than one back end (st and xl) so from the start there was this idea of an IR that was flexible enough to handle register-rich and register-poor architectures, then I thought it’d be kind of useful to be able to write and test code on the Mac without loading it into the FPGA, so it was scaling from 8-bit to 64-bit pretty much immediately, with different executable formats; once that ground was laid it wasn’t too hard to extend it to the current multiple platforms. The optimiser is pretty thorough in terms of the classical operations. Where it still lacks is automatic vectorisation, clang vectorises code in lots of places you wouldn’t expect, but it also has a huge head-start over xc. Right now we’re sometimes on a par with a clang-compiled C program, but otherwise between 1.1x and 1.8x slower. There has been the occasional micro-benchmark where xc comes out faster :) I don’t actually have a benchmark suite, this is more informal testing during developmentof the optimiser and that’s something I should fix, not least because it’ll show where the next “bang for the buck” will be. It’s worth mentioning that most of my optimisation tests have been pretty low-level, so the clang version is written in C to make it easier to understand the assembly for this mere human, and xc has a bit more overhead with its automatic reference counting code, so a better future test might be to compare vs ObjC. Originally, I wrote up a 300 (or so) line spec for the language, and another spec for the IR, because it had to include automatic and transparent banked-memory use on the 6502 (so pointers were 3-bytes {bank, hi, lo}). Coming from using modern ObjC, I wanted ARC from the start, so that was built in, but it was only extended to blocks and callbacks (a callback is a ound function, an {object,method} or {nil,function} tuple) when we got onto the serious architecture support. It’s been “interesting” supporting a machine that doesn’t even support ‘mul’ alongside one that has vector simd :) Overall, I’d say the AI has done maybe 80% of the work, possibly 85% if I’m being generous. There have certainly been times I’m in the code changing things because it’s easier to express the difference between what it first came up with, and what I actually want, in actual code rather than English. On the other hand, I’d never even have attempted such a large project on my own. I see the AI as a massive force-multiplier on what can be done. It doesn’t get tired, it prevaricates (“I don’t want to make such a large change without <insert nonsense>”) a lot less than humans, and it has alot of knowledge to draw on. One other thing I guess I should mention is that there is also the beginnings of a cross-platform UI framework, which uses an AppKit-like binding (you get things like TableView, CollectionView, OutlineView etc, with datasource protocols and delegates) but binds to the native widgets to provide a native look/feel/interaction from the same AppKit-like source. The docs go through it in more detail. This is only about 50% done, there’s a lot to finish off here. It helps that String in xc is native UTF8, but the binding part takes time. The fallback default is to draw the widget if there’s no binding, and until you pretty much 100% cover the native UI for a platform, it can look a bit jarring. Alongside that, there’s a tool ‘RoCkS’ which is even younger (maybe 10% there) which is intended to be the equivalent of Interface Builder. RoCkS is spelt that way because of the original atari origins... the GUI designer for GEM was "RCS" - Resource Construction Kit :) You design your UI with drag/drop and springs/struts, then bind UI objects in various layouts {desktop, tablet, phone} in {portrait, landscape} to the same core codebase. All with drag/drop, just like in Interface builder. Then the same application code can use a ‘phone’ UI layout when you compile the iOS/Android version, and use a ‘desktop’ layout when you compile a Windows/Linux/Mac version etc. At that point, I might re-announce on HN, because free and open-source cross-platform (and easy multi-platform) UI isn’t that common :) Again, thanks for the interest :) At least I got 1 comment :)

Rochus 9d ago
Interesting, thanks for the detailed explanations. Objective-C has indeed some impressive features, but also some inefficiencies and it got a bit out of favour over the years. Coming from the Wirth school I also understand the virtue and challenge of writing a compiler in his own language, but in practice it is often rather an obstacle than a benefit, and if a language is useful, there should never be a lack of representative projects. I thus write my own compilers always in a conservative C++98 subset or even in C so bootstrapping is never an issue and there is no second compiler necessary. Is your blewit, which sounds yet like another big code base, also AI generated/assisted? If I understand you correctly, you indeed have the engineering expertise to write compilers and to understand its architecture and complexities; so the project is not "vibe coded", but rather "augmented engineering"; I didn't manage to have a detailed look at the large code base yet, but in that case I would expect that it rather looks like a human engineered code base than the often strange looking architecture and code which I already have seen in Claude projects where the AI is just left alone for a few weeks. A performance within 1.1 to 1.8 on average of optimized Clang is a very good start for such a project. As I understand, the AI has implemented the frontend and lowering after your spec, which is amazing and demonstrates that it is able to work not only in and for the languages it was trained on. How was the AI able to demonstrate that your spec has been completely and correctly implemented? Is the language and IR spec somewhere on the web? What formalism did you use for the spec? I well remember GEM and also wrote software for Atari ST many years ago ;-)
- spacedcowboy 9d ago
  I'd already written a lot of it in ObjC, thinking it would be "just for me" before I moved to wanting it running on the FPGA. Three options - Port the ObjC runtime to my little OS on the FPGA ? - Rewrite everything in C so the native FPGA tools could compile it - Make it compile itself and treat the ObjC version as a bootstrap. #3 seemed the best option :) Blewit is a reasonably-sized project, but it's not on the same order as the compiler. It did shake out a few bugs in the compiler that the unit-tests etc. didn't find. At least, when you own the compiler, you don't submit a bug report to the developers and wait for a patch :) Yep, I'm happy with the optimisation - and I think most of the areas left are actually vectorisation - clang is very eager to vectorise (for good reason, it makes a difference) As for the spec, the original one is https://github.com/ThrudTheBarbarian/xc/blob/main/compiler/docs/xtc.bnf https://github.com/ThrudTheBarbarian/xc/blob/main/compiler/d... but that's now quite out of date (despite claiming to be the reference :) I'll update it, and also add a readable version to the website.. We've really only added things that haven't broken fundamentals recently. The main recent additions were the block type (and since I agree with the thinking that spawned "http://goshdarnblocksyntax.com http://goshdarnblocksyntax.com", we use 'block' not some convoluted ^ notation). Similarly a callback is a 'callback', the types were added and obey the usual grammar rules. The GEM desktop lives on: https://0x0000ff.co.uk/mov/xt/xtos-aug-11.mov https://0x0000ff.co.uk/mov/xt/xtos-aug-11.mov :) - if you look at the bottom you'll see 2 icons (ST, XE) and clicking on that launches the emulation in a window, but since the actual desktop also uses GEM, you can run a GEM app (actually on either the ST or the XL) and open windows, drag them around etc. - There's a lot to do to get this desktop to be as useful as Magic! or Thing, but for various reasons it's on hold atm - mainly that I'm waiting on a physical garage to be built, which (today!) got rafters put on. Once that's in place, I'll have all the electronics stuff set up, and I can debug the motherboard that the FPGA plugs into.

deleted 9d ago

[deleted]