7 ms·

Porting the Unity Engine to .NET CoreCLR

194 points by benaadams 9y ago

drawkbox 9y ago
CoreCLR will be nice for Unity with the 2x performance improvement. I wish Microsoft had moved to it quicker and Unity wouldn't have had that lull on Mono 2.x .NET and had to build IL2CPP. When IL2CPP was being rolled out it was quite impacting to build size and actually making iOS builds for a while in 2015 from like March to summer was not workable for months. Now they have Android on it and it works well but keeping all that up to date is cumbersome for a smaller tech/engine team. I still would love a C++ engine/lib like they were going to do back when Apple was forcing AOT instead of JIT languages around 2010 [1], but having a C# engine that can be comparable in performance is killer. As long as this update is smooth and is truly faster that would be welcome. It was harder to trust Microsoft before during the Mono/.NET disconnected period so Unity treaded carefully but after Microsoft bought Xamarin and went .NET standard/core it seems like they are embracing and not extinguishing. Microsoft's new focus is Azure and .NET core/standard help expand that as well as its use by Unity (always a C# champion) and even Unreal Engine possibly which some have attempted [2]. If Unity switches to .NET core/standard possibly Unreal Engine will be compelled to add it natively. [1] https://blogs.unity3d.com/2010/07/02/unity-and-ios-4-0-update-iii/ https://blogs.unity3d.com/2010/07/02/unity-and-ios-4-0-updat... [2] https://mono-ue.github.io/ https://mono-ue.github.io/ or https://github.com/xiongfang/UnrealCS https://github.com/xiongfang/UnrealCS
- lostmsu 9y ago
  CoreCLR so far does not work on either Android or iOS AFAIK.
  - aras_p 9y ago
    Correct, right now it's only Windows/Mac/Linux. That leaves the whole mobile & console platforms without it, so it's not like our IL2CPP stuff is "not needed anymore".
    - drawkbox 9y ago
      Hey Aras you guys are doing a good job. So I guess CoreCLR will cause more work for Unity engine teams in the interim. IL2CPP is great right now and gets past all AOT issues as the initial pain is over. Ralph Hauwert and those guys did a good job. It also gets compilation AOT down to C++ so it cannot be messed with by Apple. But I guess CoreCLR will be another standard/lib to manage and suddenly the standard xkcd on standards becomes relevant [1]. I can still dream of Unity Plan B from 2010 though with a fully C++ API that mimics the C# one. [1] https://xkcd.com/927/ https://xkcd.com/927/
    - sebazzz 9y ago
      I wonder then if CoreRT will be able to play a role then.
- gameswithgo 9y ago
  IL2CPP will still be needed for IOS as far as I know.
  - xoofx 9y ago
    IL2CPP is going to stay for sure, but imo, it should be reserved to platforms where we don't have a more efficient way of having GC aware codegen (because it has not been ported yet, typically for a new platform). CoreRT could perfectly work at some point on iOS, and more efficiently than IL2CPP. But this will require lots of work, so indeed, this is not going to happen anytime soon, maybe never, but we should be aware of all the options and why they each matter.
  - pjmlp 9y ago
    Also game consoles and Web Assembly.
- gameswithgo 9y ago
  > is truly faster With Mike Acton involved you can count on things getting truly faster, and not just because of .NET core.
  - reificator 9y ago
    Oh really? I'm suddenly interested, because the way he generally pushes things to work is the way I like them to work. I've always been uncomfortable with Unity's "pretend to be OO and pretend to be component based" approach, especially since they (IMO) excel at neither. If Mike Acton is in charge, I'm going to start paying more attention to new developments there now. Watching the talk at the bottom of the post, and I'm just at job scheduling. This already seems like a good move forward for Unity. EDIT: Finished watching the talk. I'm floored. I figured there would be some changes but I didn't realize so much was already done to move to an ECS system. ECS fits the way I think better, performs well, and (IMO) lowers the content creation barriers for the kinds of games I like to make and play. This is great news, as soon as I can download this, I'm in.
    - pjmlp 9y ago
      There are lots of other C# goodies and known devs joining Unity, there are other GDC 2018 talks with related content. Also the C# 7.x low level improvements not only had feedback from previous experience on Singularity and Midori, they also got input from Unity's work on HPC#.
      - polskibus 9y ago
        What low level improvements in C# 7 do you mean? Ref returns, etc? Anything besides syntactic sugar?
        oblio 9y ago
        Maybe these? https://github.com/dotnet/corefxlab/blob/master/docs/specs/span.md https://github.com/dotnet/corefxlab/blob/master/docs/specs/s...
- doomlaser 9y ago
  They put up their editor source code on Github recently (https://github.com/Unity-Technologies/UnityCsReference https://github.com/Unity-Technologies/UnityCsReference), and originally it also included the full source to their runtime garbage collector, which ended up being the standard open source C++ boehm collector (https://github.com/ivmai/bdwgc https://github.com/ivmai/bdwgc) from the 90s. Unity should replace it with a reference counting object management system, or update to a newer garbage collection technique to minimize the inadvertent sweeps that cause huge choppy frame rate drops.
  - pcwalton 9y ago
    I hope they don't use naive reference counting. It would reduce throughput and would not eliminate pauses. A standard incremental or concurrent generational tracing GC would be a better choice. In games you have a lot of knowledge about timing that you can use to help schedule GC: you always know (or should know anyway) how much time there is until the next VBLANK, so you can use that to bound pause times for your major collections.
    - doomlaser 9y ago
      Large parts of iOS and its ecosystem use automatic reference counting, and it has a reputation for the interface being noticeably smoother than Android, which uses Java's garbage collector.
      - pcwalton 9y ago
        Android doesn't use the HotSpot garbage collector.
  - merlish 9y ago
    They're planning both. Reference counting/bump pointer allocation in-frame for temp stuff & no GC & manual memory management for the upcoming high-performance restricted C# subset data-oriented jobs code, and for full-fat real C# code they're slowly working to clean up their (C++ coded) runtime so that a non-conservative garbage collector can work. Re. Boehm, that was the garbage collector Mono shipped until they wrote SGen some years back. I'm quite excited by the work they're doing on performance. It looks great.
- AdeptusAquinas 9y ago
  The other great thing about CoreCLR, though probably a bit niche, is it would allow other languages to be more easily used for Unity dev. F# for example, is part of dotnet core by default.
- coldacid 9y ago
  IL2CPP had to be built anyway because of Apple's rules about interpreters in iOS apps. Unity can't just ignore that platform.
  - carussell 9y ago
    Xamarin ships for iOS. It does AOT compilation from CIL to native.
    - bananaboy 9y ago
      That's what Unity did before IL2CPP as well (not surprising of course since it was Mono). I think they didn't want to have to keep writing new native code generators for the AOT process for different architectures, which is why they went the IL2CPP route. That way they can generate the C++ code without worrying about what platform it's going to be running on and also take advantage of the C++ compiler's optimiser.
- DubiousPusher 9y ago
  That the divide existed at all is mind-blowing to me. In the middle of the deepest ebb in Microsoft's popularity with developers a group of people were like, "hey we love this tech so much we're going to reverse engineer and reimplement it ourselves from nothing." And Microsoft was like, "cool, good luck with that."
  - zamalek 9y ago
    They even handed out unit tests for the Moonlight team. It was certainly the start of the Microsoft renaissance, which is very strange considering it happened under Balmer.
  - SomeHacker44 9y ago
    "reverse engineer and reimplement it ourselves from nothing" :)
    - DubiousPusher 9y ago
      Fair enough. Nothing as in $0. For a lot more than nothing in terms of personal satisfaction.
sanxiyn 9y ago
> But also, you will notice lots of work related to ARM32 and ARM64, including corporate work from Samsung for the Tizen platform It's nice to see this recognized. Source: I worked on open source at Samsung.
- voltagex_ 9y ago
  Hopefully I can use some of the postmarketOS work to run Tizen on some more devices. The ghost of the Nokia N900 lives on!
  - pjmlp 9y ago
    Actual Tizen has nothing to do with Nokia N900 spirit, other than using a Linux kernel.
disordinary 9y ago
I wonder if Unity regrets using Mono / C# as their scripting language, it seems like they have so many workarounds and hoops that they jump through that it must be a nightmare to maintain.
- fsloth 9y ago
  I don't think so. The tooling and mindshare one gets with C# should be tremendously valuable. There are not that many options for performant languages which provide all the things C# ecosystem provides out of the box.
- kevingadd 9y ago
  I don't think they had any better options available to them at the time. Embedding and integration for an engine like Unity is just a tremendous challenge no matter what stack you use. I've done this sort of integration w/ various languages (python, lua, vbscript, c#, actionscript, javascript, etc) and C# is one of the most reasonable choices, if only because it was designed with native interop scenarios in mind. There are pointers, there are structs, etc. Also keep in mind Unity did offer multiple scripting solutions - they had Boo for a while (eventually dropped), and there's UnityScript, though I'm not super familiar with how it works... C# is the one that won.
  - coldacid 9y ago
    Wasn't UnityScript just Unity-flavoured JS, like ActionScript was for Flash?
- pjmlp 9y ago
  Not at all, actually they are doubling down on C#. Developed a new compiler for a subset of C# that they call High Performance C#, and are porting engine code from C++ to HPC#. Unity was also one of the contributors to the new C# 7.x safer low level programming features. Lots of GDC 2018 talks about how it all works.
- SomeHacker44 9y ago
  I've tried many languages for the last 35 years and C# is, at least of the C-style OO-focused languages, one of the nicer ones if not the nicest one. Microsoft has been doing very nice things with it the last several years, especially. I, for one, am very happy with C# in Unity. I would still prefer a LISP (Arcadia/Clojure comes to mind) as a first class citizen, and support for a truly functional language (such as F#) would be nice, but I can live with C#, especially with the more-functional C#6 and .NET 4.6/.7 support in the latest Unity versions. If Unity supported CoreCLR and all CLR languages natively, as well as NuGet, along with Rider (or VS/VSCode) it would be a near-perfect development environment for games, IMO. Not necessarily for everyone and everything, but enough to be good for many things and many people, including amateurs and learners.

194 points by benaadams 9y ago

CoreCLR will be nice for Unity with the 2x performance improvement. I wish Microsoft had moved to it quicker and Unity wouldn't have had that lull on Mono 2.x .NET and had to build IL2CPP. When IL2CPP was being rolled out it was quite impacting to build size and actually making iOS builds for a while in 2015 from like March to summer was not workable for months. Now they have Android on it and it works well but keeping all that up to date is cumbersome for a smaller tech/engine team. I still would love a C++ engine/lib like they were going to do back when Apple was forcing AOT instead of JIT languages around 2010 [1], but having a C# engine that can be comparable in performance is killer. As long as this update is smooth and is truly faster that would be welcome. It was harder to trust Microsoft before during the Mono/.NET disconnected period so Unity treaded carefully but after Microsoft bought Xamarin and went .NET standard/core it seems like they are embracing and not extinguishing. Microsoft's new focus is Azure and .NET core/standard help expand that as well as its use by Unity (always a C# champion) and even Unreal Engine possibly which some have attempted [2]. If Unity switches to .NET core/standard possibly Unreal Engine will be compelled to add it natively. [1] https://blogs.unity3d.com/2010/07/02/unity-and-ios-4-0-update-iii/ https://blogs.unity3d.com/2010/07/02/unity-and-ios-4-0-updat... [2] https://mono-ue.github.io/ https://mono-ue.github.io/ or https://github.com/xiongfang/UnrealCS https://github.com/xiongfang/UnrealCS

lostmsu 9y ago
CoreCLR so far does not work on either Android or iOS AFAIK.
- aras_p 9y ago
  Correct, right now it's only Windows/Mac/Linux. That leaves the whole mobile & console platforms without it, so it's not like our IL2CPP stuff is "not needed anymore".
  - drawkbox 9y ago
    Hey Aras you guys are doing a good job. So I guess CoreCLR will cause more work for Unity engine teams in the interim. IL2CPP is great right now and gets past all AOT issues as the initial pain is over. Ralph Hauwert and those guys did a good job. It also gets compilation AOT down to C++ so it cannot be messed with by Apple. But I guess CoreCLR will be another standard/lib to manage and suddenly the standard xkcd on standards becomes relevant [1]. I can still dream of Unity Plan B from 2010 though with a fully C++ API that mimics the C# one. [1] https://xkcd.com/927/ https://xkcd.com/927/
  - sebazzz 9y ago
    I wonder then if CoreRT will be able to play a role then.
gameswithgo 9y ago
IL2CPP will still be needed for IOS as far as I know.
- xoofx 9y ago
  IL2CPP is going to stay for sure, but imo, it should be reserved to platforms where we don't have a more efficient way of having GC aware codegen (because it has not been ported yet, typically for a new platform). CoreRT could perfectly work at some point on iOS, and more efficiently than IL2CPP. But this will require lots of work, so indeed, this is not going to happen anytime soon, maybe never, but we should be aware of all the options and why they each matter.
- pjmlp 9y ago
  Also game consoles and Web Assembly.
gameswithgo 9y ago
> is truly faster With Mike Acton involved you can count on things getting truly faster, and not just because of .NET core.
- reificator 9y ago
  Oh really? I'm suddenly interested, because the way he generally pushes things to work is the way I like them to work. I've always been uncomfortable with Unity's "pretend to be OO and pretend to be component based" approach, especially since they (IMO) excel at neither. If Mike Acton is in charge, I'm going to start paying more attention to new developments there now. Watching the talk at the bottom of the post, and I'm just at job scheduling. This already seems like a good move forward for Unity. EDIT: Finished watching the talk. I'm floored. I figured there would be some changes but I didn't realize so much was already done to move to an ECS system. ECS fits the way I think better, performs well, and (IMO) lowers the content creation barriers for the kinds of games I like to make and play. This is great news, as soon as I can download this, I'm in.
  - pjmlp 9y ago
    There are lots of other C# goodies and known devs joining Unity, there are other GDC 2018 talks with related content. Also the C# 7.x low level improvements not only had feedback from previous experience on Singularity and Midori, they also got input from Unity's work on HPC#.
    - polskibus 9y ago
      What low level improvements in C# 7 do you mean? Ref returns, etc? Anything besides syntactic sugar?
      - oblio 9y ago
        Maybe these? https://github.com/dotnet/corefxlab/blob/master/docs/specs/span.md https://github.com/dotnet/corefxlab/blob/master/docs/specs/s...
doomlaser 9y ago
They put up their editor source code on Github recently (https://github.com/Unity-Technologies/UnityCsReference https://github.com/Unity-Technologies/UnityCsReference), and originally it also included the full source to their runtime garbage collector, which ended up being the standard open source C++ boehm collector (https://github.com/ivmai/bdwgc https://github.com/ivmai/bdwgc) from the 90s. Unity should replace it with a reference counting object management system, or update to a newer garbage collection technique to minimize the inadvertent sweeps that cause huge choppy frame rate drops.
- pcwalton 9y ago
  I hope they don't use naive reference counting. It would reduce throughput and would not eliminate pauses. A standard incremental or concurrent generational tracing GC would be a better choice. In games you have a lot of knowledge about timing that you can use to help schedule GC: you always know (or should know anyway) how much time there is until the next VBLANK, so you can use that to bound pause times for your major collections.
  - doomlaser 9y ago
    Large parts of iOS and its ecosystem use automatic reference counting, and it has a reputation for the interface being noticeably smoother than Android, which uses Java's garbage collector.
    - pcwalton 9y ago
      Android doesn't use the HotSpot garbage collector.
- merlish 9y ago
  They're planning both. Reference counting/bump pointer allocation in-frame for temp stuff & no GC & manual memory management for the upcoming high-performance restricted C# subset data-oriented jobs code, and for full-fat real C# code they're slowly working to clean up their (C++ coded) runtime so that a non-conservative garbage collector can work. Re. Boehm, that was the garbage collector Mono shipped until they wrote SGen some years back. I'm quite excited by the work they're doing on performance. It looks great.
AdeptusAquinas 9y ago
The other great thing about CoreCLR, though probably a bit niche, is it would allow other languages to be more easily used for Unity dev. F# for example, is part of dotnet core by default.
coldacid 9y ago
IL2CPP had to be built anyway because of Apple's rules about interpreters in iOS apps. Unity can't just ignore that platform.
- carussell 9y ago
  Xamarin ships for iOS. It does AOT compilation from CIL to native.
  - bananaboy 9y ago
    That's what Unity did before IL2CPP as well (not surprising of course since it was Mono). I think they didn't want to have to keep writing new native code generators for the AOT process for different architectures, which is why they went the IL2CPP route. That way they can generate the C++ code without worrying about what platform it's going to be running on and also take advantage of the C++ compiler's optimiser.
DubiousPusher 9y ago
That the divide existed at all is mind-blowing to me. In the middle of the deepest ebb in Microsoft's popularity with developers a group of people were like, "hey we love this tech so much we're going to reverse engineer and reimplement it ourselves from nothing." And Microsoft was like, "cool, good luck with that."
- zamalek 9y ago
  They even handed out unit tests for the Moonlight team. It was certainly the start of the Microsoft renaissance, which is very strange considering it happened under Balmer.
- SomeHacker44 9y ago
  "reverse engineer and reimplement it ourselves from nothing" :)
  - DubiousPusher 9y ago
    Fair enough. Nothing as in $0. For a lot more than nothing in terms of personal satisfaction.

sanxiyn 9y ago

> But also, you will notice lots of work related to ARM32 and ARM64, including corporate work from Samsung for the Tizen platform It's nice to see this recognized. Source: I worked on open source at Samsung.

voltagex_ 9y ago
Hopefully I can use some of the postmarketOS work to run Tizen on some more devices. The ghost of the Nokia N900 lives on!
- pjmlp 9y ago
  Actual Tizen has nothing to do with Nokia N900 spirit, other than using a Linux kernel.

disordinary 9y ago

I wonder if Unity regrets using Mono / C# as their scripting language, it seems like they have so many workarounds and hoops that they jump through that it must be a nightmare to maintain.

fsloth 9y ago
I don't think so. The tooling and mindshare one gets with C# should be tremendously valuable. There are not that many options for performant languages which provide all the things C# ecosystem provides out of the box.
kevingadd 9y ago
I don't think they had any better options available to them at the time. Embedding and integration for an engine like Unity is just a tremendous challenge no matter what stack you use. I've done this sort of integration w/ various languages (python, lua, vbscript, c#, actionscript, javascript, etc) and C# is one of the most reasonable choices, if only because it was designed with native interop scenarios in mind. There are pointers, there are structs, etc. Also keep in mind Unity did offer multiple scripting solutions - they had Boo for a while (eventually dropped), and there's UnityScript, though I'm not super familiar with how it works... C# is the one that won.
- coldacid 9y ago
  Wasn't UnityScript just Unity-flavoured JS, like ActionScript was for Flash?
pjmlp 9y ago
Not at all, actually they are doubling down on C#. Developed a new compiler for a subset of C# that they call High Performance C#, and are porting engine code from C++ to HPC#. Unity was also one of the contributors to the new C# 7.x safer low level programming features. Lots of GDC 2018 talks about how it all works.
SomeHacker44 9y ago
I've tried many languages for the last 35 years and C# is, at least of the C-style OO-focused languages, one of the nicer ones if not the nicest one. Microsoft has been doing very nice things with it the last several years, especially. I, for one, am very happy with C# in Unity. I would still prefer a LISP (Arcadia/Clojure comes to mind) as a first class citizen, and support for a truly functional language (such as F#) would be nice, but I can live with C#, especially with the more-functional C#6 and .NET 4.6/.7 support in the latest Unity versions. If Unity supported CoreCLR and all CLR languages natively, as well as NuGet, along with Rider (or VS/VSCode) it would be a near-perfect development environment for games, IMO. Not necessarily for everyone and everything, but enough to be good for many things and many people, including amateurs and learners.