47 ms·

Porting 58k lines of D and C++ to Jai

212 points by Growtika 4y ago

flykespice 4y ago
I heard about this "promising" language as a youngster and never heard of it again, it doesn't seem to have taken off after all these years, and probably it's too later for it ever take off.
dmitriid 4y ago
Can't wait to see how this progresses. So far the only source of knowledge have been Blow's streams, but he's the author, and he knows his language inside and out. It will be immensely interesting to see other people's experiences.
- throwaway17_17 4y ago
  Tsoding, coding on Twitch and with archives on YouTube has a few streams from after he got into the beta. They give a decent overview of using the language in its current state, and I find Tsoding petty entertaining as well.
  - dmitriid 4y ago
    Thanks! I'll check them out!
chromatin 4y ago
I find the "slow compile time" complaint of D interesting; overall I find ldc2 to be one of the faster compilers out there. I suppose the combination of features and metaprogramming used by the author is responsible for the 3-minute compile times in debug mode. While jai is evidently expected to be substantially faster (author estimates 5 sec for this codebase???), other choices like Rust would be far, far slower than D.
- dmitriid 4y ago
  > author estimates 5 sec for this codebase??? This expectation comes from jai compile times for the game Jonathan Blow is developing: https://www.twitch.tv/j_blow https://www.twitch.tv/j_blow It takes about 2.5 seconds to compile and link over 168k lines of code in debug mode. See e.g. https://www.twitch.tv/videos/1659970118 https://www.twitch.tv/videos/1659970118 at around 30 seconds in
  - Kukumber 4y ago
    My game+engine wich is composed of 37k lines of D + 22k of C code (with D's ImportC), fully recompiles in 1.2 seconds with DMD https://bpa.st/raw/WCVQ https://bpa.st/raw/WCVQ
- p0nce 4y ago
  To get best compile times with D it's important not go over board with CTFE and templates.
gavinray 4y ago
It's worth mentioning that DMD is a reference compiler and not meant as a production tool. DMD is for fast compile time during dev cycles and testing new language features early. You've got the LLVM and GCC compilers for production. The problem with lack of debug info on Windows in D is solved by using LDC with LLDB/GDB to debug You can also ask the compiler to emit CodeView debugging info if you want to stick with Microsoft stuff
- mattarm 4y ago
  I don't think "DMD is a reference compiler and not meant as a production tool" is a great way to characterize it. There are certainly valid reasons for choosing LLVM and GCC for "production", but also valid reasons for choosing DMD.
  - anta40 4y ago
    As far as I know, DMD doesn't offer much optimisations. So you use DMD for development, and once the testing is done, built the production release with LDC or GDC.
    - WalterBright 4y ago
      DMD does do all the basic data flow analysis optimizations, along with register allocation by coloring, etc. The problem is there are just a ton of special cases needed to generate good code, especially with the x86 instruction set which is nothing but special cases <g>. Doing these takes a lot of developer time, one by one.
- Xorlev 4y ago
  The author's chief complaint appears to be that dmd produces incorrect debug info on Windows, something I'd expect a reference compiler to do. A reference compiler doesn't have to be fast, but it does need to be correct.
  - mhh__ 4y ago
    Debug info is one of the backend things that does get priority with dmd. It has gotten some performance increases in recent years but these are mostly just a nicety. Personally I have fixed some debug info generation problems so it is being worked on.
aeturnum 4y ago
I'm really interested to see Jai in the wild, but I have to say I find the Jai v.s. Rust comparison here...unconvincing to say to least. It's particularly funny to be excited about "being able to do anything during compilation" using Jai and then complain to that the #1 problem in programming is "culturally tolerated and even encouraged complexity."
- mjburgess 4y ago
  Complexity often arises from restrictions on "anything", rather than people actually doing anything.
  - owenpalmer 4y ago
    Excellent point!
  - aeturnum 4y ago
    I'm curious - what you think of as a good example? I think of C++ templates & macros, which I would not describe as arising from restrictions?
    - ablob 4y ago
      Template metaprogramming in the absence of constexpr/concepts would be my guess.
      - Ar-Curunir 4y ago
        Traits are more restricted in what they can do than Templates, so the original quote does not apply here
        mort96 4y ago
        Template metaprogramming is the idea of executing arbitrary logic at compile time using the template system as your programming language. The replacement for template metaprogramming is a solid compile-time execution model, not a traits system.
dymk 4y ago
I don’t understand how this language is planning on ever seeing adoption when it’s been restricted from open use since it’s inception. Seems like vaporware.
- diimdeep 4y ago
  See https://github.com/Jai-Community/Jai-Community-Library/wiki/Overview#history https://github.com/Jai-Community/Jai-Community-Library/wiki/... and https://github.com/Ivo-Balbaert/The_Way_to_Jai https://github.com/Ivo-Balbaert/The_Way_to_Jai
  - DannyBee 4y ago
    Your links back up the statement. The first says the compiler is proprietary and unavailable outside of some beta testers, but may be open in the great and nebulous future.
  - p0nce 4y ago
    > Supports x86 architecture only No native macOS then? > No virtual functions > Jai is less dogmatic okay
    - Cyberdog 4y ago
      > No native macOS then? Or Pi, or iOS, or practically all Android hardware… Kind of a non-starter IMO. Maybe this was marginally acceptable when the language started in '14 but it becomes less so every year.
      - imran0 4y ago
        It has an LLVM backend so adding support for other architectures is not difficult. JB already ported the compiler to Nintendo Switch several years ago. https://old.reddit.com/r/NintendoSwitch/comments/cixsoy/jonathan_blows_jai_game_programming_language_now/ https://old.reddit.com/r/NintendoSwitch/comments/cixsoy/jona...
rizky05 4y ago
andai 4y ago
“In the beginning I hinted at expecting the porting process to go somewhat smoothly (which I’m sure I won’t regret later). The reason for this is that I have two systems in my game that will hopefully be immensely helpful in this endeavor: - The game records the inputs (HID, loaded files, network, …) of a play-session into a file and replay them later. When replaying, feeding recorded input into the deterministic game loop leads to the exact same state, down to the bit. - The game hashes game state at various points during execution and saves these hashes to a different file. When replaying, the contents of this file can be used to check that the execution of the replayed session exactly matches the original execution.”
tonetheman 4y ago
Hard to get much out of this since the language is not available to everyone.
Kukumber 4y ago
> ldc2 is awfully slow to compile5 LDC = LLVM backend it's slow for D, for C++, for Zig and also slow for Jai The reference compiler "DMD" is what you use to get fast iteration time > (jai) Reducing compile times from about 60s right now to under 5s, hopefully around 1s Try to compile the DMD compiler and notice how fast it is at compilation So that sounds like it's a "your code" problem, my game engine takes 1.2 seconds to fully recompile with DMD > the documentation is lacking https://dlang.org/phobos/index.html https://dlang.org/phobos/index.html Looks complete with exhaustive code examples that you can run online > Ok, so I don’t like D and C++ and jai looks good. But what about all the other languages? Ok so that explains everything
stephc_int13 4y ago
I'd love to see more about Jai. I don't understand why they are not working in the open, this is a tool, not a game, input for a large community is extremely valuable. But as brilliant as Jon Blow is, he is at least equally as stubborn. Fast compilation and nice, easy to read syntax with good default is exactly what I am looking for. I can understand the hype about metaprogramming, and its potential usefulness, but I also think this is a pandora box. Pretty much like C macros and C++ templates, overuse can lead to a very messy codebase, hard to read, hard to debug, with a lot of unexpected side effects.
- hbn 4y ago
  I don't see why a language can't be one guy's passion project. Even if he releases it and it turns out he's the only person that likes it, so what? It's not like he's had millions in VC or government investment dumped into it or something. He's got some ideas and he's implementing them. I'm interested to see how it turns out. Golang has had success and was primarily made in private by 3 guys and hasn't strayed too far from the original founding principles they came up with.
  - stephc_int13 4y ago
    Yeah, of course he can do whatever he wants. The think is that quite a few people have been convinced by his talks and want to use Jai for real projects.
    - googlryas 4y ago
      If there's enough actual interest, some group of people will just clone the language in an open manner since enough is known about it. But that doesn't seem to be happening at the moment.
      - cyber_kinetist 4y ago
        I thought Odin was one of the clone languages at the start (although later it diverged from Jai in many aspects, and is far ahead Jai in the aspect that it's open-source and is actually being used in production). But really I would like Jai to take some time to mature over the years, instead of rushing out for an immediate public release. It has some very ambitious ideas that would absolutely be killer features, but I think it needs ample time to get polished. (Off to writing C++ code during the time then...)
StreamBright 4y ago
I am surprised by the design of jai. The author has very interesting view on typical programming problems (RAII, exceptions).
poulpy123 4y ago
I'm not sure that if passing from a widely used language to a marginal language wasn't a success why going from a marginal language to a totally obscure one will be
- mariusor 4y ago
  An obscure language custom built for the purpose of writing computer games and by a developer with razor sharp focus on making it better who is very responsive about bugs submitted by the people with access to the beta.
hesdeadjim 4y ago
I’m in the private beta and things are changing constantly. I’m not sure why anyone would pick a language that won’t ever have a real ecosystem of libraries. Coming from C++ and need to remain low-level? Pick Rust, done.
- throwaway17_17 4y ago
  With the almost total incompatibility in mindset and preferences between Jai and Rust, I’m not sure if this is actually practical ‘advice’. I can’t help but think most devs that want to use Jai are avoiding Rust purposefully so advising them to skip Jai as a potential replacement for their low level development needs and use Rust is just not going to happen.
  - Ar-Curunir 4y ago
    Who is avoiding Rust (an open source, publicly accessible, and relatively widespread language) for Jai (a closed vapourware language that is apparently not anywhere near release) for anything but toy projects?
    - throwaway17_17 4y ago
      I was not implying that people were specifically avoiding Rust in favor of Jai. My comment was more to the point that people who would be interested in Jai are most likely the same devs that are not interested in Rust. The reasons devs give for avoiding Rust are mixed but tend to focus on the restrictiveness inherent in Rust’s chosen memory model or the appearance of continual expansion of the language and its complexity or a strong aversion to the manner of dealing with and heavy usage of third party dependency, along with some other topics (syntax, functional-adjacent styling, insistence on idiomatic code at any cost, etc). While some or all of those reasons may be overblown, some devs just do not want Rust but are looking for an alternative to C or C++.
- protoster 4y ago
  >I’m not sure why anyone would pick a language that won’t ever have a real ecosystem of libraries Could you elaborate?
didip 4y ago
This person might as well said that he is not planning to finish his game. Screwing around with tooling and languages seems to be a common pitfall to not finishing a game. Why not just use the best ecosystem like Unreal/Unity and get the game out of the door fast?
- markdog12 4y ago
  > And the most important reason: I just feel like it!
- naikrovek 4y ago
  > Why not just use the best ecosystem like Unreal/Unity laughable. absolutely laughable. I'm not even talking about politics of using an engine versus writing one; unity and unreal are complex tools with their own problems. if you value what you are producing, and if you value quality and you want your game to be just the way you want it, Unity and Unreal will fight you just as hard as D or C++ have been for the blog author. Unity/Unreal are also simply not capable of a lot of things. They are definitely not a pure win for someone making a game.
  - nomel 4y ago
    Could you give an examples of a game that used a custom game engines that couldn’t achieve what they wanted in Unreal or Unity? Edit: The only style I would think would be infinite/strange geometry, like Manifold garden (great game!). But, it’s Unity.
    - jessermeyer 4y ago
      I've worked on several. You haven't heard of them because none of them shipped due to issues with commercial engines (in particular Unreal Engine 4). They are very difficult to use if the game play semantics are complicated and require lots of interaction with world state or world geometry. If you're making a common FPS, they are great.
      - deleted 4y ago
cleandreams 4y ago
I wrote an extremely hard and complex program in which I had to use c++ metaprogramming throughout. Not my decision. I do not understand why you could possibly need more metaprogramming than C++ provides. This is a problematic feature to begin with. Not worthless but not worth much. The debugging was difficult. Many problems show up buried in literally thousands of compiler error messages.
- P5fRxh5kUvp2th 4y ago
  C++ started with templates as a type polymorphism mechanisms. Then it was shown to be turing complete so people started using it as such. The over time it started getting more and more things constexpr such that it's easier to execute during compile time (because it's turing complete). Jai just made the decision that everything is constexpr by default. And the answer to your question is that it's easier to do in Jai (in theory) than C++.
- gallier2 4y ago
  That's the issue with C++ metaprogramming. It's so unwieldy and complex that the benefits are difficult to see and the occasions to be worthwhile limited. When the language has better (i.e. simpler) facilities for meta-programming like D, it will open big venues where it makes sense to use it. It is like programming object oriented in C, it is possible to do but so unwieldy and requiring discipline and verbose boiler plate that there's rarely an occasion where it would be interesting to use. While it is much easier to do in C++ or D or Java (etc.) as they provide the abstractions that makes it useable. C++'s meta-programming facilities are not good enough.
  - bjourne 4y ago
    I disagree. Object-oriented programming is very easy in C even though the syntax itself doesn't guide you. Essentially, all C code I write is object-oriented. Perhaps you are thinking about dynamic dispatch which indeed is very annoying to implement correctly in C?
    - deleted 4y ago
unwind 4y ago
As someone who is aware of jai but not at all following the development, do the terms of the closed beta the OP is in permit code to be shown? I guess "yes", it seems so pointless otherwise. Would be very interesting to see e.g. "here is how this (gnarly) C++ turned into beatiful jai", zoomed-in to actual code level.
- vinyl7 4y ago
  Yea, theres no NDA aside from not distributing the compiler. Theres a bunch of code on github, though it can be hard to find
  - unwind 4y ago
    Okay, great. That totally makes sense, but on the other hand Mr Blow seems to have some rather unusual ideas regarding how to develop this new language (and is a game developer, which sometimes brings its own set of secrecy-first ideas [1]). [1]: For reasons, of course. Been there.
- sbarre 4y ago
  Blow regularly streams himself hacking on jai on Twitch, so I don't think there's any concern about hiding the code.
bachmeier 4y ago
> there’s also the general feeling that the D creators’ vision of what a fixed C++ looks like is just vastly different from mine I have emphasized for years that D is not C++. It can be used as a replacement for C++, in the sense that it does the same things, but it is not accurate to call it a "fixed C++". So many times I have seen C++ programmers disappointed that D is not C++. On the other hand, if you're a C programmer, you'll probably be comfortable writing D code. I think of D and C++ as incompatible forks of C.
- WalterBright 4y ago
  > So many times I have seen C++ programmers disappointed that D is not C++. More generally, every time I learn a new language, it is always frustrating, because I cannot write the same thing in the new language that I was used to writing in the old. It takes time until one starts thinking in terms of the new language, and only then will the frustration fade.
- SilentLambda 4y ago
  Yes, we can agree that D is not like C++ - not at all. The reason C++ programmers are usually disappointed, is because D gets marketed as such (it's the bait you need to get them to look at D afterall). In fact, even on the basic class type, C++ and D are really miles apart. Make no mistake about this - they are miles apart, even on this one construct (the same contruct for there being a C++ in the first place). It is clear when you use D, that it wasn't designed to be like C++. D does have a subset that is C like, and that's primarly because it is, in essence, C. So D is D. D is not at all like C++. A subset of D is like C (cause it is - more or less - C).
ibiza 4y ago
> a big chunk of these vulnerabilities would not exist if C and C++ [...] simply didn’t have zero-terminated string, initialized values by default, had a proper pointer+length type thus replacing 90% of pointer arithmetic with easily bounds-checkable code, and had established a culture that discouraged the prevalent ad-hoc style of memory management. This is Rust's calling-card, so I find this plea for a better lang / eco rather jarring after dismissing Rust for somehow "making the wrong tradeoffs".
- winter_blue 4y ago
  > initialized values by default Doesn't the Linux kernel zero out memory pages that are accessed by a process, before they've been written to? I guess it doesn't matter, since the larger concern is that the process could accidentally read its own freed blocks....
- vvanders 4y ago
  Yeah, I think Bevy[1] is also a great example of something that fits in well with a lot of Rust's strengths while also showing that it's possible to lean in heavily to very gamedev centric approaches with how it approached ECS. I think there's is some truth around try to avoid unsafe, that said the times I've dropped down into it I've found myself chasing heap corruption or use-after-free on more than one occasion :). Some of Jai's AoS/SoA transforms look neat and certainly interested to see what it looks like once it starts opening up more. [1] https://bevyengine.org/ https://bevyengine.org/
- wk_end 4y ago
  If I'm reading OP right, their issue with Rust is the borrow checker - none of those features require one. They're asking for a language that's safer than C, not as safe as Rust, but simpler and easier to ram things through. I don't necessarily agree, but I think that's what they're saying.
- BenFrantzDale 4y ago
  This is also C++20’s calling card. It has `std::span`.
haolez 4y ago
> All information for building a program is contained within the source code of the program. Thus there is no need for a make command or project files to build a Jai program. Ok, now I'm curious. Will have to check it out.
smegsicle 4y ago
> Porting 58k lines of D to C++ in Jail and they call sprinkling a bit of water "cruel and unusual"
pietmichal 4y ago
Very interesting! Apparently a lot of information on the Internet about the Jai language is outdated which makes me curious about the follow up post! I don't know if OP is the blog owner. If yes, I'd love to have an option to subscribe to an email list. I've installed an RSS feed but email is still my preference. Thanks!
zengid 4y ago
> In my eyes, the most important ones are faster compilation and allowing metaprogramming via unrestricted compile time execution. I'm fascinated by languages that are adding more compile-time programming features, in the context of a low-level performance-oriented use case. Would love to know more about the history and state of the art of this area.
- Corendos 4y ago
  That actually makes a lot of sense to improve performance on specific parts. Think about generating lookup tables at compile time for example. Instead of having a separate script generating them, you can keep the generation close to the place you use it, written in the same language and always up to date.
pizza234 4y ago
Putting together the available document "JayPrimer" (which may be obsolete) with the blog post, Jay it seems a language designed by somebody who's was fed up with C++, and deciced to write their own, tailored language; this looks great when seen through the lens of C++, but there isn't anything particularly innovative in the context of modern languages. In particular, I'm personally neutral to Zig, but there seems to be little reason to prefer Jay over it. From the primer: > Arbitrary Compile-Time Code Execution This is the big selling point, but, brought to the extreme, it's not necessarily a good thing. The examples in the primer are intended to look great: - Insert build time data - Download the OpenGL spec and build the most recent gl.h header file - Contact a build server and retrieve/send build data but they're the type of things that turn a build into a monster. I guess comptime execution is big in the gamedev area (I can't say, I have little experience), but I suppose Zig fits the typical gamedev use cases (curious to hear devs with hands-on experience). > Code Refactoring The example presented seems to be "extract to function", which sufficiently advanced IDEs should support. It's also unclear if it's currently implemented. > Integrated Build Process I don't see this as a good thing. It's good from the perspective of old programming languages, whose build tools are a mess. But having a separate tool is actually an advantage, as long as it's standardized and well integrated (I suppose modern languages have this support). > SOA AND AOS This seems to be a very niche feature. > Reflection and Run-Time Type Information This is very convenient, but again, nothing unique. > FUNCTION POLYMORPHISM It seems to be an odd (flexible/inferred) generics implementation. One of the language objectives is to never perform automated type casting, but in this example, it is performed. > THE ANY TYPE I guess this is a polarizing feature. > STRUCT POINTER OWNERSHIP Is this syntactic sugar for a C++ destructor? > Other Cool Stuff > Specific data types for 8, 16, and 32 bit integers No 64/128? :^) Regarding the post, the non-trivial selling points are described as: > Reducing compile times from about 60s right now to under 5s, hopefully around 1s This is certainly very appealing. > having debuggers work Uh? That's based on the bad D experience. > Replacing build-scripts with jai code > Catching more errors by introducing custom compilation checks using metaprogramming > Replacing complex metaprogramming code with simpler, imperative code See Zig. The rest are trivialities.
fefe23 4y ago
Please choose more descriptive titles for your submissions here, people. Some dude I don't know is porting a game I know nothing about to a language I know nothing about and his reason is "because I feel like it". He has grievances about the current state of things and the port hasn't gotten anywhere yet. I have a hard time thinking of a blog post that would be less useful to me. No lessons learned, no "why you should consider this obscure programming language you have never heard anything about". Also you need to go to wikipedia to find out the programming language is targeted at games. Programming games is not one of my interests. You'd think they at least explain why jai is good for programming games. But they don't. The blog post can't, in fact, because he hasn't gotten far enough yet to draw any conclusions. At least make it look like you are making an effort to not waste my time.
- _gabe_ 4y ago
  You seem to just not be the target audience for this blog post. I enjoyed it because it fits in with a lot of the subjects that I've been looking into and it resonated with me. Your comment is the equivalent of saying that some research paper about a mathematical topic is terrible quality because it assumes you already know Calculus. Edit: Also, the author mentions right at the start of the article: > In this series of blogposts, I will document my experience of porting the game that I am currently working on... I’ve planned on doing this for a long time and want to keep some sort of record of my expectations, the journey and the result. The point of this article is very clear from the beginning. It's simply documenting the authors journey. They don't need to provide any value to you. You didn't pay to read this article and they may want to keep a record for personal reasons. Putting it out their for the world is great since it allows other people to have a glance at how the experience went, and I hate when people leave discouraging comments because some article/video/entertainment doesn't cater to their every need.
- QuadrupleA 4y ago
  Geez, just do some web searches. Or ignore it, and find an article on subjects you already know. I've been following D and Jai for years and found this really well written and interesting. Just depends on your background knowledge.
- 4y ago
johnisgood 4y ago
I hope the game will not end up like Limit Theory (https://www.kickstarter.com/projects/joshparnell/limit-theory-an-infinite-procedural-space-game https://www.kickstarter.com/projects/joshparnell/limit-theor...). They are similar, apart from the 3D and 2D part.
SevenNation 4y ago
> The largest elephant in the room to address is probably Rust. ... Breaking this down, I can only find two practical problems the author has with Rust: - long compile times - the ownership model ("the borrow checker") The rest of this paragraph appears to be much more general in nature. Given that the project is only 58,000 lines of D/C++, it's hard to believe that compile time alone is so bad as to drive a decision toward an experimental language like Jai. So it appears that the main problem the author has is the ownership model ("the borrow checker"). It would be interesting to know more, but the author does not elaborate. AFAICT, the Rust compiler can be viewed as enforcing the good practices that C++ developers already recognize. So how can this be an issue at all, especially given the ability break out of the ownership model into unsafe Rust (or use other tricks) if the situation calls for it?
SilentLambda 4y ago
"That said, D definitely has some advantages over C++, like more powerful metaprogramming, no need for headers, no uninitialized values and more. But unfortunately, these upsides are outweighed by the downsides." Yes, after working with D for far too long, I have decided to delete it off my computer and I'm going back to C++, where a class is still a first-class 'type', and a 'value' type at that (at least by default). In D, a class is a reference type only and worse, the D language has no means of declaring, let alone enforcing at compile time, a perimeter around such a type, within the so called 'D module'. The entire D module is within the perimeter of your class type, at all times!
shp0ngle 4y ago
Where can I actually see more about Jai? All I keep finding are youtube videos and twitch streams, which are the worst way for me to learn about a programming language. I guess it’s still not generally available?
conaclos 4y ago
Fast compilation seems very appealing. It is one of the main reason why I am interested into Go and Zig. I recently started working with Rust for contributing to projects like Rome/tools [1] and deno_lint [2]. The compilation and IDE experience is frustrating. Compilation is slow. I am afraid that this is rooted to the inherent complexity of Rust. [1] https://github.com/rome/tools https://github.com/rome/tools [2] https://github.com/denoland/deno_lint https://github.com/denoland/deno_lint

212 points by Growtika 4y ago

I heard about this "promising" language as a youngster and never heard of it again, it doesn't seem to have taken off after all these years, and probably it's too later for it ever take off.

dmitriid 4y ago

Can't wait to see how this progresses. So far the only source of knowledge have been Blow's streams, but he's the author, and he knows his language inside and out. It will be immensely interesting to see other people's experiences.

throwaway17_17 4y ago
Tsoding, coding on Twitch and with archives on YouTube has a few streams from after he got into the beta. They give a decent overview of using the language in its current state, and I find Tsoding petty entertaining as well.
- dmitriid 4y ago
  Thanks! I'll check them out!

chromatin 4y ago

I find the "slow compile time" complaint of D interesting; overall I find ldc2 to be one of the faster compilers out there. I suppose the combination of features and metaprogramming used by the author is responsible for the 3-minute compile times in debug mode. While jai is evidently expected to be substantially faster (author estimates 5 sec for this codebase???), other choices like Rust would be far, far slower than D.

dmitriid 4y ago
> author estimates 5 sec for this codebase??? This expectation comes from jai compile times for the game Jonathan Blow is developing: https://www.twitch.tv/j_blow https://www.twitch.tv/j_blow It takes about 2.5 seconds to compile and link over 168k lines of code in debug mode. See e.g. https://www.twitch.tv/videos/1659970118 https://www.twitch.tv/videos/1659970118 at around 30 seconds in
- Kukumber 4y ago
  My game+engine wich is composed of 37k lines of D + 22k of C code (with D's ImportC), fully recompiles in 1.2 seconds with DMD https://bpa.st/raw/WCVQ https://bpa.st/raw/WCVQ
p0nce 4y ago
To get best compile times with D it's important not go over board with CTFE and templates.

gavinray 4y ago

It's worth mentioning that DMD is a reference compiler and not meant as a production tool. DMD is for fast compile time during dev cycles and testing new language features early. You've got the LLVM and GCC compilers for production. The problem with lack of debug info on Windows in D is solved by using LDC with LLDB/GDB to debug You can also ask the compiler to emit CodeView debugging info if you want to stick with Microsoft stuff

mattarm 4y ago
I don't think "DMD is a reference compiler and not meant as a production tool" is a great way to characterize it. There are certainly valid reasons for choosing LLVM and GCC for "production", but also valid reasons for choosing DMD.
- anta40 4y ago
  As far as I know, DMD doesn't offer much optimisations. So you use DMD for development, and once the testing is done, built the production release with LDC or GDC.
  - WalterBright 4y ago
    DMD does do all the basic data flow analysis optimizations, along with register allocation by coloring, etc. The problem is there are just a ton of special cases needed to generate good code, especially with the x86 instruction set which is nothing but special cases <g>. Doing these takes a lot of developer time, one by one.
Xorlev 4y ago
The author's chief complaint appears to be that dmd produces incorrect debug info on Windows, something I'd expect a reference compiler to do. A reference compiler doesn't have to be fast, but it does need to be correct.
- mhh__ 4y ago
  Debug info is one of the backend things that does get priority with dmd. It has gotten some performance increases in recent years but these are mostly just a nicety. Personally I have fixed some debug info generation problems so it is being worked on.

aeturnum 4y ago

I'm really interested to see Jai in the wild, but I have to say I find the Jai v.s. Rust comparison here...unconvincing to say to least. It's particularly funny to be excited about "being able to do anything during compilation" using Jai and then complain to that the #1 problem in programming is "culturally tolerated and even encouraged complexity."

mjburgess 4y ago
Complexity often arises from restrictions on "anything", rather than people actually doing anything.
- owenpalmer 4y ago
  Excellent point!
- aeturnum 4y ago
  I'm curious - what you think of as a good example? I think of C++ templates & macros, which I would not describe as arising from restrictions?
  - ablob 4y ago
    Template metaprogramming in the absence of constexpr/concepts would be my guess.
    - Ar-Curunir 4y ago
      Traits are more restricted in what they can do than Templates, so the original quote does not apply here
      - mort96 4y ago
        Template metaprogramming is the idea of executing arbitrary logic at compile time using the template system as your programming language. The replacement for template metaprogramming is a solid compile-time execution model, not a traits system.

dymk 4y ago

I don’t understand how this language is planning on ever seeing adoption when it’s been restricted from open use since it’s inception. Seems like vaporware.

diimdeep 4y ago
See https://github.com/Jai-Community/Jai-Community-Library/wiki/Overview#history https://github.com/Jai-Community/Jai-Community-Library/wiki/... and https://github.com/Ivo-Balbaert/The_Way_to_Jai https://github.com/Ivo-Balbaert/The_Way_to_Jai
- DannyBee 4y ago
  Your links back up the statement. The first says the compiler is proprietary and unavailable outside of some beta testers, but may be open in the great and nebulous future.
- p0nce 4y ago
  > Supports x86 architecture only No native macOS then? > No virtual functions > Jai is less dogmatic okay
  - Cyberdog 4y ago
    > No native macOS then? Or Pi, or iOS, or practically all Android hardware… Kind of a non-starter IMO. Maybe this was marginally acceptable when the language started in '14 but it becomes less so every year.
    - imran0 4y ago
      It has an LLVM backend so adding support for other architectures is not difficult. JB already ported the compiler to Nintendo Switch several years ago. https://old.reddit.com/r/NintendoSwitch/comments/cixsoy/jonathan_blows_jai_game_programming_language_now/ https://old.reddit.com/r/NintendoSwitch/comments/cixsoy/jona...

rizky05 4y ago

andai 4y ago

“In the beginning I hinted at expecting the porting process to go somewhat smoothly (which I’m sure I won’t regret later). The reason for this is that I have two systems in my game that will hopefully be immensely helpful in this endeavor: - The game records the inputs (HID, loaded files, network, …) of a play-session into a file and replay them later. When replaying, feeding recorded input into the deterministic game loop leads to the exact same state, down to the bit. - The game hashes game state at various points during execution and saves these hashes to a different file. When replaying, the contents of this file can be used to check that the execution of the replayed session exactly matches the original execution.”

tonetheman 4y ago

Hard to get much out of this since the language is not available to everyone.

Kukumber 4y ago

> ldc2 is awfully slow to compile5 LDC = LLVM backend it's slow for D, for C++, for Zig and also slow for Jai The reference compiler "DMD" is what you use to get fast iteration time > (jai) Reducing compile times from about 60s right now to under 5s, hopefully around 1s Try to compile the DMD compiler and notice how fast it is at compilation So that sounds like it's a "your code" problem, my game engine takes 1.2 seconds to fully recompile with DMD > the documentation is lacking https://dlang.org/phobos/index.html https://dlang.org/phobos/index.html Looks complete with exhaustive code examples that you can run online > Ok, so I don’t like D and C++ and jai looks good. But what about all the other languages? Ok so that explains everything

stephc_int13 4y ago

I'd love to see more about Jai. I don't understand why they are not working in the open, this is a tool, not a game, input for a large community is extremely valuable. But as brilliant as Jon Blow is, he is at least equally as stubborn. Fast compilation and nice, easy to read syntax with good default is exactly what I am looking for. I can understand the hype about metaprogramming, and its potential usefulness, but I also think this is a pandora box. Pretty much like C macros and C++ templates, overuse can lead to a very messy codebase, hard to read, hard to debug, with a lot of unexpected side effects.

hbn 4y ago
I don't see why a language can't be one guy's passion project. Even if he releases it and it turns out he's the only person that likes it, so what? It's not like he's had millions in VC or government investment dumped into it or something. He's got some ideas and he's implementing them. I'm interested to see how it turns out. Golang has had success and was primarily made in private by 3 guys and hasn't strayed too far from the original founding principles they came up with.
- stephc_int13 4y ago
  Yeah, of course he can do whatever he wants. The think is that quite a few people have been convinced by his talks and want to use Jai for real projects.
  - googlryas 4y ago
    If there's enough actual interest, some group of people will just clone the language in an open manner since enough is known about it. But that doesn't seem to be happening at the moment.
    - cyber_kinetist 4y ago
      I thought Odin was one of the clone languages at the start (although later it diverged from Jai in many aspects, and is far ahead Jai in the aspect that it's open-source and is actually being used in production). But really I would like Jai to take some time to mature over the years, instead of rushing out for an immediate public release. It has some very ambitious ideas that would absolutely be killer features, but I think it needs ample time to get polished. (Off to writing C++ code during the time then...)

StreamBright 4y ago

I am surprised by the design of jai. The author has very interesting view on typical programming problems (RAII, exceptions).

poulpy123 4y ago

I'm not sure that if passing from a widely used language to a marginal language wasn't a success why going from a marginal language to a totally obscure one will be

mariusor 4y ago
An obscure language custom built for the purpose of writing computer games and by a developer with razor sharp focus on making it better who is very responsive about bugs submitted by the people with access to the beta.

hesdeadjim 4y ago

I’m in the private beta and things are changing constantly. I’m not sure why anyone would pick a language that won’t ever have a real ecosystem of libraries. Coming from C++ and need to remain low-level? Pick Rust, done.

throwaway17_17 4y ago
With the almost total incompatibility in mindset and preferences between Jai and Rust, I’m not sure if this is actually practical ‘advice’. I can’t help but think most devs that want to use Jai are avoiding Rust purposefully so advising them to skip Jai as a potential replacement for their low level development needs and use Rust is just not going to happen.
- Ar-Curunir 4y ago
  Who is avoiding Rust (an open source, publicly accessible, and relatively widespread language) for Jai (a closed vapourware language that is apparently not anywhere near release) for anything but toy projects?
  - throwaway17_17 4y ago
    I was not implying that people were specifically avoiding Rust in favor of Jai. My comment was more to the point that people who would be interested in Jai are most likely the same devs that are not interested in Rust. The reasons devs give for avoiding Rust are mixed but tend to focus on the restrictiveness inherent in Rust’s chosen memory model or the appearance of continual expansion of the language and its complexity or a strong aversion to the manner of dealing with and heavy usage of third party dependency, along with some other topics (syntax, functional-adjacent styling, insistence on idiomatic code at any cost, etc). While some or all of those reasons may be overblown, some devs just do not want Rust but are looking for an alternative to C or C++.
protoster 4y ago
>I’m not sure why anyone would pick a language that won’t ever have a real ecosystem of libraries Could you elaborate?

didip 4y ago

This person might as well said that he is not planning to finish his game. Screwing around with tooling and languages seems to be a common pitfall to not finishing a game. Why not just use the best ecosystem like Unreal/Unity and get the game out of the door fast?

markdog12 4y ago
> And the most important reason: I just feel like it!
naikrovek 4y ago
> Why not just use the best ecosystem like Unreal/Unity laughable. absolutely laughable. I'm not even talking about politics of using an engine versus writing one; unity and unreal are complex tools with their own problems. if you value what you are producing, and if you value quality and you want your game to be just the way you want it, Unity and Unreal will fight you just as hard as D or C++ have been for the blog author. Unity/Unreal are also simply not capable of a lot of things. They are definitely not a pure win for someone making a game.
- nomel 4y ago
  Could you give an examples of a game that used a custom game engines that couldn’t achieve what they wanted in Unreal or Unity? Edit: The only style I would think would be infinite/strange geometry, like Manifold garden (great game!). But, it’s Unity.
  - jessermeyer 4y ago
    I've worked on several. You haven't heard of them because none of them shipped due to issues with commercial engines (in particular Unreal Engine 4). They are very difficult to use if the game play semantics are complicated and require lots of interaction with world state or world geometry. If you're making a common FPS, they are great.
    - deleted 4y ago

cleandreams 4y ago

I wrote an extremely hard and complex program in which I had to use c++ metaprogramming throughout. Not my decision. I do not understand why you could possibly need more metaprogramming than C++ provides. This is a problematic feature to begin with. Not worthless but not worth much. The debugging was difficult. Many problems show up buried in literally thousands of compiler error messages.

P5fRxh5kUvp2th 4y ago
C++ started with templates as a type polymorphism mechanisms. Then it was shown to be turing complete so people started using it as such. The over time it started getting more and more things constexpr such that it's easier to execute during compile time (because it's turing complete). Jai just made the decision that everything is constexpr by default. And the answer to your question is that it's easier to do in Jai (in theory) than C++.
gallier2 4y ago
That's the issue with C++ metaprogramming. It's so unwieldy and complex that the benefits are difficult to see and the occasions to be worthwhile limited. When the language has better (i.e. simpler) facilities for meta-programming like D, it will open big venues where it makes sense to use it. It is like programming object oriented in C, it is possible to do but so unwieldy and requiring discipline and verbose boiler plate that there's rarely an occasion where it would be interesting to use. While it is much easier to do in C++ or D or Java (etc.) as they provide the abstractions that makes it useable. C++'s meta-programming facilities are not good enough.
- bjourne 4y ago
  I disagree. Object-oriented programming is very easy in C even though the syntax itself doesn't guide you. Essentially, all C code I write is object-oriented. Perhaps you are thinking about dynamic dispatch which indeed is very annoying to implement correctly in C?
  - deleted 4y ago

unwind 4y ago

As someone who is aware of jai but not at all following the development, do the terms of the closed beta the OP is in permit code to be shown? I guess "yes", it seems so pointless otherwise. Would be very interesting to see e.g. "here is how this (gnarly) C++ turned into beatiful jai", zoomed-in to actual code level.

vinyl7 4y ago
Yea, theres no NDA aside from not distributing the compiler. Theres a bunch of code on github, though it can be hard to find
- unwind 4y ago
  Okay, great. That totally makes sense, but on the other hand Mr Blow seems to have some rather unusual ideas regarding how to develop this new language (and is a game developer, which sometimes brings its own set of secrecy-first ideas [1]). [1]: For reasons, of course. Been there.
sbarre 4y ago
Blow regularly streams himself hacking on jai on Twitch, so I don't think there's any concern about hiding the code.

bachmeier 4y ago

> there’s also the general feeling that the D creators’ vision of what a fixed C++ looks like is just vastly different from mine I have emphasized for years that D is not C++. It can be used as a replacement for C++, in the sense that it does the same things, but it is not accurate to call it a "fixed C++". So many times I have seen C++ programmers disappointed that D is not C++. On the other hand, if you're a C programmer, you'll probably be comfortable writing D code. I think of D and C++ as incompatible forks of C.

WalterBright 4y ago
> So many times I have seen C++ programmers disappointed that D is not C++. More generally, every time I learn a new language, it is always frustrating, because I cannot write the same thing in the new language that I was used to writing in the old. It takes time until one starts thinking in terms of the new language, and only then will the frustration fade.
SilentLambda 4y ago
Yes, we can agree that D is not like C++ - not at all. The reason C++ programmers are usually disappointed, is because D gets marketed as such (it's the bait you need to get them to look at D afterall). In fact, even on the basic class type, C++ and D are really miles apart. Make no mistake about this - they are miles apart, even on this one construct (the same contruct for there being a C++ in the first place). It is clear when you use D, that it wasn't designed to be like C++. D does have a subset that is C like, and that's primarly because it is, in essence, C. So D is D. D is not at all like C++. A subset of D is like C (cause it is - more or less - C).

ibiza 4y ago

> a big chunk of these vulnerabilities would not exist if C and C++ [...] simply didn’t have zero-terminated string, initialized values by default, had a proper pointer+length type thus replacing 90% of pointer arithmetic with easily bounds-checkable code, and had established a culture that discouraged the prevalent ad-hoc style of memory management. This is Rust's calling-card, so I find this plea for a better lang / eco rather jarring after dismissing Rust for somehow "making the wrong tradeoffs".

winter_blue 4y ago
> initialized values by default Doesn't the Linux kernel zero out memory pages that are accessed by a process, before they've been written to? I guess it doesn't matter, since the larger concern is that the process could accidentally read its own freed blocks....
vvanders 4y ago
Yeah, I think Bevy[1] is also a great example of something that fits in well with a lot of Rust's strengths while also showing that it's possible to lean in heavily to very gamedev centric approaches with how it approached ECS. I think there's is some truth around try to avoid unsafe, that said the times I've dropped down into it I've found myself chasing heap corruption or use-after-free on more than one occasion :). Some of Jai's AoS/SoA transforms look neat and certainly interested to see what it looks like once it starts opening up more. [1] https://bevyengine.org/ https://bevyengine.org/
wk_end 4y ago
If I'm reading OP right, their issue with Rust is the borrow checker - none of those features require one. They're asking for a language that's safer than C, not as safe as Rust, but simpler and easier to ram things through. I don't necessarily agree, but I think that's what they're saying.
BenFrantzDale 4y ago
This is also C++20’s calling card. It has `std::span`.

haolez 4y ago

> All information for building a program is contained within the source code of the program. Thus there is no need for a make command or project files to build a Jai program. Ok, now I'm curious. Will have to check it out.

smegsicle 4y ago

> Porting 58k lines of D to C++ in Jail and they call sprinkling a bit of water "cruel and unusual"

pietmichal 4y ago

Very interesting! Apparently a lot of information on the Internet about the Jai language is outdated which makes me curious about the follow up post! I don't know if OP is the blog owner. If yes, I'd love to have an option to subscribe to an email list. I've installed an RSS feed but email is still my preference. Thanks!

zengid 4y ago

> In my eyes, the most important ones are faster compilation and allowing metaprogramming via unrestricted compile time execution. I'm fascinated by languages that are adding more compile-time programming features, in the context of a low-level performance-oriented use case. Would love to know more about the history and state of the art of this area.

Corendos 4y ago
That actually makes a lot of sense to improve performance on specific parts. Think about generating lookup tables at compile time for example. Instead of having a separate script generating them, you can keep the generation close to the place you use it, written in the same language and always up to date.

pizza234 4y ago

Putting together the available document "JayPrimer" (which may be obsolete) with the blog post, Jay it seems a language designed by somebody who's was fed up with C++, and deciced to write their own, tailored language; this looks great when seen through the lens of C++, but there isn't anything particularly innovative in the context of modern languages. In particular, I'm personally neutral to Zig, but there seems to be little reason to prefer Jay over it. From the primer: > Arbitrary Compile-Time Code Execution This is the big selling point, but, brought to the extreme, it's not necessarily a good thing. The examples in the primer are intended to look great: - Insert build time data - Download the OpenGL spec and build the most recent gl.h header file - Contact a build server and retrieve/send build data but they're the type of things that turn a build into a monster. I guess comptime execution is big in the gamedev area (I can't say, I have little experience), but I suppose Zig fits the typical gamedev use cases (curious to hear devs with hands-on experience). > Code Refactoring The example presented seems to be "extract to function", which sufficiently advanced IDEs should support. It's also unclear if it's currently implemented. > Integrated Build Process I don't see this as a good thing. It's good from the perspective of old programming languages, whose build tools are a mess. But having a separate tool is actually an advantage, as long as it's standardized and well integrated (I suppose modern languages have this support). > SOA AND AOS This seems to be a very niche feature. > Reflection and Run-Time Type Information This is very convenient, but again, nothing unique. > FUNCTION POLYMORPHISM It seems to be an odd (flexible/inferred) generics implementation. One of the language objectives is to never perform automated type casting, but in this example, it is performed. > THE ANY TYPE I guess this is a polarizing feature. > STRUCT POINTER OWNERSHIP Is this syntactic sugar for a C++ destructor? > Other Cool Stuff > Specific data types for 8, 16, and 32 bit integers No 64/128? :^) Regarding the post, the non-trivial selling points are described as: > Reducing compile times from about 60s right now to under 5s, hopefully around 1s This is certainly very appealing. > having debuggers work Uh? That's based on the bad D experience. > Replacing build-scripts with jai code > Catching more errors by introducing custom compilation checks using metaprogramming > Replacing complex metaprogramming code with simpler, imperative code See Zig. The rest are trivialities.

fefe23 4y ago

Please choose more descriptive titles for your submissions here, people. Some dude I don't know is porting a game I know nothing about to a language I know nothing about and his reason is "because I feel like it". He has grievances about the current state of things and the port hasn't gotten anywhere yet. I have a hard time thinking of a blog post that would be less useful to me. No lessons learned, no "why you should consider this obscure programming language you have never heard anything about". Also you need to go to wikipedia to find out the programming language is targeted at games. Programming games is not one of my interests. You'd think they at least explain why jai is good for programming games. But they don't. The blog post can't, in fact, because he hasn't gotten far enough yet to draw any conclusions. At least make it look like you are making an effort to not waste my time.

_gabe_ 4y ago
You seem to just not be the target audience for this blog post. I enjoyed it because it fits in with a lot of the subjects that I've been looking into and it resonated with me. Your comment is the equivalent of saying that some research paper about a mathematical topic is terrible quality because it assumes you already know Calculus. Edit: Also, the author mentions right at the start of the article: > In this series of blogposts, I will document my experience of porting the game that I am currently working on... I’ve planned on doing this for a long time and want to keep some sort of record of my expectations, the journey and the result. The point of this article is very clear from the beginning. It's simply documenting the authors journey. They don't need to provide any value to you. You didn't pay to read this article and they may want to keep a record for personal reasons. Putting it out their for the world is great since it allows other people to have a glance at how the experience went, and I hate when people leave discouraging comments because some article/video/entertainment doesn't cater to their every need.
QuadrupleA 4y ago
Geez, just do some web searches. Or ignore it, and find an article on subjects you already know. I've been following D and Jai for years and found this really well written and interesting. Just depends on your background knowledge.
4y ago

johnisgood 4y ago

I hope the game will not end up like Limit Theory (https://www.kickstarter.com/projects/joshparnell/limit-theory-an-infinite-procedural-space-game https://www.kickstarter.com/projects/joshparnell/limit-theor...). They are similar, apart from the 3D and 2D part.

SevenNation 4y ago

> The largest elephant in the room to address is probably Rust. ... Breaking this down, I can only find two practical problems the author has with Rust: - long compile times - the ownership model ("the borrow checker") The rest of this paragraph appears to be much more general in nature. Given that the project is only 58,000 lines of D/C++, it's hard to believe that compile time alone is so bad as to drive a decision toward an experimental language like Jai. So it appears that the main problem the author has is the ownership model ("the borrow checker"). It would be interesting to know more, but the author does not elaborate. AFAICT, the Rust compiler can be viewed as enforcing the good practices that C++ developers already recognize. So how can this be an issue at all, especially given the ability break out of the ownership model into unsafe Rust (or use other tricks) if the situation calls for it?

SilentLambda 4y ago

"That said, D definitely has some advantages over C++, like more powerful metaprogramming, no need for headers, no uninitialized values and more. But unfortunately, these upsides are outweighed by the downsides." Yes, after working with D for far too long, I have decided to delete it off my computer and I'm going back to C++, where a class is still a first-class 'type', and a 'value' type at that (at least by default). In D, a class is a reference type only and worse, the D language has no means of declaring, let alone enforcing at compile time, a perimeter around such a type, within the so called 'D module'. The entire D module is within the perimeter of your class type, at all times!

shp0ngle 4y ago

Where can I actually see more about Jai? All I keep finding are youtube videos and twitch streams, which are the worst way for me to learn about a programming language. I guess it’s still not generally available?

conaclos 4y ago

Fast compilation seems very appealing. It is one of the main reason why I am interested into Go and Zig. I recently started working with Rust for contributing to projects like Rome/tools [1] and deno_lint [2]. The compilation and IDE experience is frustrating. Compilation is slow. I am afraid that this is rooted to the inherent complexity of Rust. [1] https://github.com/rome/tools https://github.com/rome/tools [2] https://github.com/denoland/deno_lint https://github.com/denoland/deno_lint