5 ms·

> was written in Common Lisp and automatically translated to C++ That's the reason the .NET GC source code is still in one single massive .cpp file. I mean, l

> was written in Common Lisp and automatically translated to C++ That's the reason the .NET GC source code is still in one single massive .cpp file. I mean, look at it [1]. Github even refuses to display it. [1] https://github.com/dotnet/runtime/blob/master/src/coreclr/src/gc/gc.cpp https://github.com/dotnet/runtime/blob/master/src/coreclr/sr...

thu2111 6y ago
Well, sort of. That may explain why it started that way. The real reason appears to be that the maintainer of it is kind of crazy and appears to not be properly managed. She's actually rejected community pull requests to break it up in the past on the grounds that she just prefers it in one giant file, and doesn't believe it makes any difference to comprehension or codebase usability. https://github.com/dotnet/runtime/issues/4024#issuecomment-77922359 https://github.com/dotnet/runtime/issues/4024#issuecomment-7... https://github.com/dotnet/runtime/issues/4024#issuecomment-227222339 https://github.com/dotnet/runtime/issues/4024#issuecomment-2... Honestly the .NET code is kind of a mess compared to the OpenJDK code. The developers really don't seem quite as sharp. It's impossible to imagine HotSpot containing a source file that's 35,000 lines long, let alone dismissing attempts to improve it with ridiculous justifications like "it'll cause merge conflicts". The lines themselves are about as high quality as you might guess from something converted from Lisp and then hacked on for a decade or so by people who prefer it all in one file - enormous amounts of #ifdefs all over the place, it starts by defining its own random number generator, a mishmash of different coding styles, dead code, lots of undocumented and untunable magic numbers ... it's really a disaster zone. Behold: https://raw.githubusercontent.com/dotnet/runtime/master/src/coreclr/src/gc/gc.cpp https://raw.githubusercontent.com/dotnet/runtime/master/src/... And compare to the HotSpot GCs: https://github.com/openjdk/jdk/tree/master/src/hotspot/share/gc https://github.com/openjdk/jdk/tree/master/src/hotspot/share... (multiple algorithms for throughput vs latency, "epsilon" is the smallest GC implementation possible that doesn't actually collect anything - it's useful for learning and cases where you know your process won't last long enough to need a collection) When the CLR was first open sourced I was very curious to see if OpenJDK might now have some stronger competition when .NET was ported to other systems and made more UNIX friendly. After reading through the code and seeing how the developers acted I concluded they'd never displace the JVM.
- flocial 6y ago
  Well, her github profile pic is "I ️ 35000 lines in gc.cpp". She's been working on it successfully for 15 years so I guess she's untouchable even within the org (I can't imagine the amount of stress she went through over the years). Aside from issues with a 35K source file, being a deep domain expert with a singular focus seems awesome. https://github.com/Maoni0 https://github.com/Maoni0 https://www.youtube.com/watch?v=ujkSnko0JNQ https://www.youtube.com/watch?v=ujkSnko0JNQ
- nn3 6y ago
  Isn't that rant a bit over the top just for a large source file? If the source file size matters you're really not using your editor right. With a modern editor it shouldn't really matter where something is, you'll always just directly jump to the symbols. And that github source view isn't really that great anyways. For any serious reading better clone it. Apparently the billions of VC capital weren't able to buy a proper source code xref setup where you can actually jump to symbols directly, which is like the most important feature for any code review. I guess the maintainer was just not very interested in trivial submissions. Yes splitting up source files is trivial.
  - flocial 6y ago
    Also, considering the source's long history it would be a huge break for stylistic reasons.
  - foepys 6y ago
    > Yes splitting up source files is trivial. It's not trivial. Have you ever had to maintain a project across multiple versions? .NET is even crazier, it supports multiple versions across multiple projects with a lot of shared code that lives in different repositories. If you split the file you cannot apply patches to older versions or different projects anymore. Merging will be a nightmare and the change history will, while not technically lost, be very hard to follow. Try it for yourself. Make a git repository, a large file, two or three LTS version branches, and then split the file up. Now try to apply a patch you made in master to the LTS branches.
why_only_15 6y ago
Haha the only comparable file I'm aware of is UIView.m, which I believe is 900kb
neppo 6y ago
The file is 38k lines, in case anyone wondered
raverbashing 6y ago
The translator seems to be the most interesting part of it, but which one is it?
rb808 6y ago
Well it works much better than the JVM one so they must be doing something right. I never saw processes GC lock up like JVM until I worked on a big Java project.
- vips7L 6y ago
  Which JVM one? CMS? Parallel? G1? ZGC? Shenandoah?
  - rb808 6y ago
    You're right, its getting better. Main problems were older JREs, G1 in 11 is better but still locks up. We haven't used ZGC in prod yet. The main thing is dotnet had this figured out 10 years ago while Java is still struggling.
    - mumblemumble 6y ago
      There are two sides to that coin. .NET, by putting all their efforts into just one GC, has absolutely reaped a lot of benefits in terms of working well without any fuss. .NET also has better ergonomics around memory management. For example, it has much clearer and stronger guarantees about what state the runtime environment will be in after an out-of-memory error. On the other hand, I'm sure that making these guarantees have at times constrained the things that .NET can do with its garbage collector. Java, by making the GC a plugin, has certainly spread its efforts thinner. It's also turned the Java-facing side of the memory management subsystem into a somewhat scary and mysterious black box, since that very swappability means that the application itself can assume almost nothing about its behavior. On the other hand, if ops wants to put the effort into tuning it, Java lets them have a lot more ability to control an application's memory and performance characteristics in production. It also leaves a lot more room for boutique JVMs with their own special memory managers. Which approach to prefer is, I think, as much a matter of personal or organizational values as it is about technical merit.
- whateveracct 6y ago
  What about throughput?

thu2111 6y ago

Well, sort of. That may explain why it started that way. The real reason appears to be that the maintainer of it is kind of crazy and appears to not be properly managed. She's actually rejected community pull requests to break it up in the past on the grounds that she just prefers it in one giant file, and doesn't believe it makes any difference to comprehension or codebase usability. https://github.com/dotnet/runtime/issues/4024#issuecomment-77922359 https://github.com/dotnet/runtime/issues/4024#issuecomment-7... https://github.com/dotnet/runtime/issues/4024#issuecomment-227222339 https://github.com/dotnet/runtime/issues/4024#issuecomment-2... Honestly the .NET code is kind of a mess compared to the OpenJDK code. The developers really don't seem quite as sharp. It's impossible to imagine HotSpot containing a source file that's 35,000 lines long, let alone dismissing attempts to improve it with ridiculous justifications like "it'll cause merge conflicts". The lines themselves are about as high quality as you might guess from something converted from Lisp and then hacked on for a decade or so by people who prefer it all in one file - enormous amounts of #ifdefs all over the place, it starts by defining its own random number generator, a mishmash of different coding styles, dead code, lots of undocumented and untunable magic numbers ... it's really a disaster zone. Behold: https://raw.githubusercontent.com/dotnet/runtime/master/src/coreclr/src/gc/gc.cpp https://raw.githubusercontent.com/dotnet/runtime/master/src/... And compare to the HotSpot GCs: https://github.com/openjdk/jdk/tree/master/src/hotspot/share/gc https://github.com/openjdk/jdk/tree/master/src/hotspot/share... (multiple algorithms for throughput vs latency, "epsilon" is the smallest GC implementation possible that doesn't actually collect anything - it's useful for learning and cases where you know your process won't last long enough to need a collection) When the CLR was first open sourced I was very curious to see if OpenJDK might now have some stronger competition when .NET was ported to other systems and made more UNIX friendly. After reading through the code and seeing how the developers acted I concluded they'd never displace the JVM.

flocial 6y ago
Well, her github profile pic is "I ️ 35000 lines in gc.cpp". She's been working on it successfully for 15 years so I guess she's untouchable even within the org (I can't imagine the amount of stress she went through over the years). Aside from issues with a 35K source file, being a deep domain expert with a singular focus seems awesome. https://github.com/Maoni0 https://github.com/Maoni0 https://www.youtube.com/watch?v=ujkSnko0JNQ https://www.youtube.com/watch?v=ujkSnko0JNQ
nn3 6y ago
Isn't that rant a bit over the top just for a large source file? If the source file size matters you're really not using your editor right. With a modern editor it shouldn't really matter where something is, you'll always just directly jump to the symbols. And that github source view isn't really that great anyways. For any serious reading better clone it. Apparently the billions of VC capital weren't able to buy a proper source code xref setup where you can actually jump to symbols directly, which is like the most important feature for any code review. I guess the maintainer was just not very interested in trivial submissions. Yes splitting up source files is trivial.
- flocial 6y ago
  Also, considering the source's long history it would be a huge break for stylistic reasons.
- foepys 6y ago
  > Yes splitting up source files is trivial. It's not trivial. Have you ever had to maintain a project across multiple versions? .NET is even crazier, it supports multiple versions across multiple projects with a lot of shared code that lives in different repositories. If you split the file you cannot apply patches to older versions or different projects anymore. Merging will be a nightmare and the change history will, while not technically lost, be very hard to follow. Try it for yourself. Make a git repository, a large file, two or three LTS version branches, and then split the file up. Now try to apply a patch you made in master to the LTS branches.

why_only_15 6y ago

Haha the only comparable file I'm aware of is UIView.m, which I believe is 900kb

neppo 6y ago

The file is 38k lines, in case anyone wondered

raverbashing 6y ago

The translator seems to be the most interesting part of it, but which one is it?

rb808 6y ago

Well it works much better than the JVM one so they must be doing something right. I never saw processes GC lock up like JVM until I worked on a big Java project.

vips7L 6y ago
Which JVM one? CMS? Parallel? G1? ZGC? Shenandoah?
- rb808 6y ago
  You're right, its getting better. Main problems were older JREs, G1 in 11 is better but still locks up. We haven't used ZGC in prod yet. The main thing is dotnet had this figured out 10 years ago while Java is still struggling.
  - mumblemumble 6y ago
    There are two sides to that coin. .NET, by putting all their efforts into just one GC, has absolutely reaped a lot of benefits in terms of working well without any fuss. .NET also has better ergonomics around memory management. For example, it has much clearer and stronger guarantees about what state the runtime environment will be in after an out-of-memory error. On the other hand, I'm sure that making these guarantees have at times constrained the things that .NET can do with its garbage collector. Java, by making the GC a plugin, has certainly spread its efforts thinner. It's also turned the Java-facing side of the memory management subsystem into a somewhat scary and mysterious black box, since that very swappability means that the application itself can assume almost nothing about its behavior. On the other hand, if ops wants to put the effort into tuning it, Java lets them have a lot more ability to control an application's memory and performance characteristics in production. It also leaves a lot more room for boutique JVMs with their own special memory managers. Which approach to prefer is, I think, as much a matter of personal or organizational values as it is about technical merit.
whateveracct 6y ago
What about throughput?