4 ms·

FreeBSD has a(nother) new C compiler: Intel oneAPI DPC++/C++

92 points by ingve 3y ago

stabbles 3y ago
Good luck with that, cause there are bugs, as the article also points out. Previous release miscompiled Python [1] Current release miscompiles bison [2] It's also the type of compiler that defaults to -ffast-math to look good in benchmarks (well, all vendor compilers do). That breaks many packages that rely on signalling NaNs, or use e.g. Kahan summation for stable math with IEEE floats. It's caught in the Spack package manager because it's probably one of the few package managers that lets users pick their own compiler for a full software stack. [1] https://github.com/spack/spack/issues/38724 https://github.com/spack/spack/issues/38724 and https://github.com/python/cpython/issues/106424 https://github.com/python/cpython/issues/106424 [2] https://github.com/spack/spack/issues/37172#issuecomment-1810420500 https://github.com/spack/spack/issues/37172#issuecomment-181...
- repelsteeltje 3y ago
  So it's an LLVM front end + Intel backend, relatively new and not in common use. Not surprised there are issues, but wouldn't it be fair to attribute those to "growing pains"? First issue was resolved, the bison issue is still pending. > It was a compiler bug. The test now passes when cpython is built with an internal build. The next compiler release (likely 2024.0) should have the fix. So I wouldn't recommend running FreeBSD on oneAPI in production just yet, but I give them some credit & celebrations for reaching a milestone.
  - stabbles 3y ago
    It does not look like correctness is their main concern, they mostly care about performance of the vectorizer ;) Also note that before Intel oneAPI there was the now deprecated Intel Classic Compiler (icc), which also had bugs, since 2003. In my opinion it's unlikely that a closed source vendor compiler will ever be on the same level as gcc and clang. It will never be tested as much, and it's hard to make actionable bug reports for closed source software. Being based on LLVM is of course a good choice... hopefully fewer bugs.
    - repelsteeltje 3y ago
      My money's on llvm/clang and gcc as well, though Microsoft Visual C++ is still hanging in there. Fwiw icc once had a reputation for being fast but I never thought high of their software. Only hands on experience I had was with Intel Media SDK where even basic stuff, like finding a legal download, hardware requirements and installation guidance was tough. And then each iteration both software and hardware was deprecated faster than Google can dream to abandon their products.
      - CoastalCoder 3y ago
        > Fwiw icc once had a reputation for being fast but I never thought high of their software. Aside from some GUI scaling issues on Linux, I was really impressed by Intel Vtune. Also, some of their low level libraries like Xed and Pin Tools are pretty cool, and seemed (last I looked) well-maintained.
        repelsteeltje 3y ago
        Ah yes. VTune. I had forgotton that one, pretty good stuff indeed.
        jsheard 3y ago
        Their Embree raytracing library is at the heart of pretty much every high-end production CGI renderer, it's the gold standard at what it does. Intel does have some gems.
    - JonChesterfield 3y ago
      A really high performance vectorizer is dramatically easier to write if you hand wave some of the correctness ideas. It's an interesting point in the design space. Classically compilers considered miscompilation to be a serious problem - you disable the 'optimisation' while you work out who got the maths wrong. But maybe today you're better off with a compiler that produces fast wrong code where the customer doesn't seem to be upset about the wrong part. ffast-math is popular, enabling that by default is a plausible thing to do. Plenty of bugs in gcc and llvm. I'm really sure a closed source compiler could be higher quality. Probably not in the C++ world, but I expect there's an ADA compiler out there that gets the answer right whatever the source code is.
- RuggedPineapple 3y ago
  It works at least well enough to build a working FreeBSD kernel, as the article notes that was built and then booted as a test. I have to imagine if the thing can build that its doing pretty well. That isn't a trivial piece of software. Makes me suspect any teething issues will be sorted and that it's not fundamentally compromised.
- Twirrim 3y ago
  > It's also the type of compiler that defaults to -ffast-math to look good in benchmarks Not sure if they're still doing that, but in the early days of Clear Linux they were using -ffast-math on everything they compiled, as a way to give their distro a big performance boost over competition, too.
  - stabbles 3y ago
    I think they do a lot of good stuff, like LTO and PGO. But in benchmarks you sometimes see like a 4x speedup compared to ubuntu, which is obviously not due to superior compilers. For example: https://github.com/phoronix-test-suite/phoronix-test-suite/issues/633 https://github.com/phoronix-test-suite/phoronix-test-suite/i...
angiosperm 3y ago
One might almost guess from the name that it is a C++ compiler, and not so limited as to compile only the legacy C subset.
jxy 3y ago
> There is a workaround: I installed the compiler using the same command on my WSL Actually you only need to have a relatively complete linux under /compat/linux. You could even chroot into that and do the oneapi install. The trillion dollar question is: can you get the intel discrete gpu to work, compiling for and running on the gpu?
- canucker2016 3y ago
  And the billion dollar question is: does the code generator treat AMD / Intel CPUs as equivalent when generating code or does it only look at Intel CPU-specific information for generating code? see https://techreport.com/news/does-intels-compiler-cripple-amd-performance/ https://techreport.com/news/does-intels-compiler-cripple-amd... [update] and https://old.reddit.com/r/Amd/comments/440ze8/amd_cpus_and_intel_compilers/ https://old.reddit.com/r/Amd/comments/440ze8/amd_cpus_and_in...
VagabundoP 3y ago
Serious question, why do we need yet another C compiler?
- blacklion 3y ago
  Different compilers rely in different UBs for code optimization. So, it is very useful to compile OSS projects with different compilers, it helps find more bugs and make sources more standard-complaint and portable
- musicale 3y ago
  Eventually we may get one that works properly and supports useful features like bounds checking, stacks that grow upward rather than downward, trapping on undefined behavior, etc. We might even get a new and more robust libc - maybe one that even supports static compilation.
mise_en_place 3y ago
I haven't tried it myself, but you can use the Zig toolchain as a C and C++ compiler[0]. In this day and age, is it really necessary to have yet another C compiler? I believe you can do the same with D as well. [0]https://medium.com/@edlyuu/zig-c-c-compiler-wtf-is-zig-c-790d9ad8d85b https://medium.com/@edlyuu/zig-c-c-compiler-wtf-is-zig-c-790...
- deleted 3y ago
  [deleted]
- jxy 3y ago
  The value of Intel's oneAPI tool chain is that you can write code that runs on their GPU, much like Nvidia's cuda or nvhpc.
intothemild 3y ago
The year is 2024, and I'm using another newer, buggier C compiler.
- b5n 3y ago
  The year is 2074, I'm still using gcc and emacs :)
  - exikyut 3y ago
    !remindme 50 years
PresidentZippy 3y ago
Not to crap on someone's hard work, but usually reinventing something is only warranted if you can do it >2x better than the best available solution. Whether that means requiring half as much hardware to do the same job or requiring half as much manpower to use, usually it's a waste of time and money to make something that's only marginally better at best. How much benefit did Intel expect to reap from rewriting an LLVM x64 compiler backend from the ground up?
- JonChesterfield 3y ago
  Isn't this a GPU backend, not an x64 one?
  - PresidentZippy 3y ago
    The backend builds both x86_64 machine code, and GPU kernels/FPGA microcode according to their documentation, but that just begs another question: How is this significantly better than the existing tools for compiling C into GPU and FPGA code? It would help if Intel published some kind of chalk talk with a live demo showing how much faster you can build HPC applications using their new toolkit. I'm not summarily writing it off, but I need a little convincing before I put 20+ hours into trying it out myself.
    - JonChesterfield 3y ago
      Yes, though the llvm x64 backend may turn out to be very similar to the upstream one. Existing tools for compiling C to Intel's GPU are this tool, there are no others.
      - PresidentZippy 3y ago
        Is there anything more to it than that? If not, the documentation would be a lot more helpful if it lead with something straight to the point. Here's something that could go directly under the title of the README: "This compiler consists of a custom LLVM frontend and backend. The backend compiles LLVM IR code into machine code consisting of x86_64 instructions and Intel GPU code. The frontend works in conjunction with the backend to compile C and C++ code with special optimization which when enabled, compiles OpenMP routines into hardware-accelerated code targeting Intel GPUs, FPGAs, or AMD and NVIDIA GPUs." As someone who only used OpenMP academically, I don't see much of a point in that. In the post C++11 world, where we can write type-safe compile-time code, preprocessor macro definitions should stay in C code. Until Intel GPUs are at least competitive with the big boys, interop with their products doesn't concern me a whole hell of a lot. I'm not going to plan my scientific computing applications around the integrated graphics found on cheap Wintel consumer devices.

92 points by ingve 3y ago

Good luck with that, cause there are bugs, as the article also points out. Previous release miscompiled Python [1] Current release miscompiles bison [2] It's also the type of compiler that defaults to -ffast-math to look good in benchmarks (well, all vendor compilers do). That breaks many packages that rely on signalling NaNs, or use e.g. Kahan summation for stable math with IEEE floats. It's caught in the Spack package manager because it's probably one of the few package managers that lets users pick their own compiler for a full software stack. [1] https://github.com/spack/spack/issues/38724 https://github.com/spack/spack/issues/38724 and https://github.com/python/cpython/issues/106424 https://github.com/python/cpython/issues/106424 [2] https://github.com/spack/spack/issues/37172#issuecomment-1810420500 https://github.com/spack/spack/issues/37172#issuecomment-181...

repelsteeltje 3y ago
So it's an LLVM front end + Intel backend, relatively new and not in common use. Not surprised there are issues, but wouldn't it be fair to attribute those to "growing pains"? First issue was resolved, the bison issue is still pending. > It was a compiler bug. The test now passes when cpython is built with an internal build. The next compiler release (likely 2024.0) should have the fix. So I wouldn't recommend running FreeBSD on oneAPI in production just yet, but I give them some credit & celebrations for reaching a milestone.
- stabbles 3y ago
  It does not look like correctness is their main concern, they mostly care about performance of the vectorizer ;) Also note that before Intel oneAPI there was the now deprecated Intel Classic Compiler (icc), which also had bugs, since 2003. In my opinion it's unlikely that a closed source vendor compiler will ever be on the same level as gcc and clang. It will never be tested as much, and it's hard to make actionable bug reports for closed source software. Being based on LLVM is of course a good choice... hopefully fewer bugs.
  - repelsteeltje 3y ago
    My money's on llvm/clang and gcc as well, though Microsoft Visual C++ is still hanging in there. Fwiw icc once had a reputation for being fast but I never thought high of their software. Only hands on experience I had was with Intel Media SDK where even basic stuff, like finding a legal download, hardware requirements and installation guidance was tough. And then each iteration both software and hardware was deprecated faster than Google can dream to abandon their products.
    - CoastalCoder 3y ago
      > Fwiw icc once had a reputation for being fast but I never thought high of their software. Aside from some GUI scaling issues on Linux, I was really impressed by Intel Vtune. Also, some of their low level libraries like Xed and Pin Tools are pretty cool, and seemed (last I looked) well-maintained.
      - repelsteeltje 3y ago
        Ah yes. VTune. I had forgotton that one, pretty good stuff indeed.
      - jsheard 3y ago
        Their Embree raytracing library is at the heart of pretty much every high-end production CGI renderer, it's the gold standard at what it does. Intel does have some gems.
  - JonChesterfield 3y ago
    A really high performance vectorizer is dramatically easier to write if you hand wave some of the correctness ideas. It's an interesting point in the design space. Classically compilers considered miscompilation to be a serious problem - you disable the 'optimisation' while you work out who got the maths wrong. But maybe today you're better off with a compiler that produces fast wrong code where the customer doesn't seem to be upset about the wrong part. ffast-math is popular, enabling that by default is a plausible thing to do. Plenty of bugs in gcc and llvm. I'm really sure a closed source compiler could be higher quality. Probably not in the C++ world, but I expect there's an ADA compiler out there that gets the answer right whatever the source code is.
RuggedPineapple 3y ago
It works at least well enough to build a working FreeBSD kernel, as the article notes that was built and then booted as a test. I have to imagine if the thing can build that its doing pretty well. That isn't a trivial piece of software. Makes me suspect any teething issues will be sorted and that it's not fundamentally compromised.
Twirrim 3y ago
> It's also the type of compiler that defaults to -ffast-math to look good in benchmarks Not sure if they're still doing that, but in the early days of Clear Linux they were using -ffast-math on everything they compiled, as a way to give their distro a big performance boost over competition, too.
- stabbles 3y ago
  I think they do a lot of good stuff, like LTO and PGO. But in benchmarks you sometimes see like a 4x speedup compared to ubuntu, which is obviously not due to superior compilers. For example: https://github.com/phoronix-test-suite/phoronix-test-suite/issues/633 https://github.com/phoronix-test-suite/phoronix-test-suite/i...

angiosperm 3y ago

One might almost guess from the name that it is a C++ compiler, and not so limited as to compile only the legacy C subset.

jxy 3y ago

> There is a workaround: I installed the compiler using the same command on my WSL Actually you only need to have a relatively complete linux under /compat/linux. You could even chroot into that and do the oneapi install. The trillion dollar question is: can you get the intel discrete gpu to work, compiling for and running on the gpu?

canucker2016 3y ago
And the billion dollar question is: does the code generator treat AMD / Intel CPUs as equivalent when generating code or does it only look at Intel CPU-specific information for generating code? see https://techreport.com/news/does-intels-compiler-cripple-amd-performance/ https://techreport.com/news/does-intels-compiler-cripple-amd... [update] and https://old.reddit.com/r/Amd/comments/440ze8/amd_cpus_and_intel_compilers/ https://old.reddit.com/r/Amd/comments/440ze8/amd_cpus_and_in...

VagabundoP 3y ago

Serious question, why do we need yet another C compiler?

blacklion 3y ago
Different compilers rely in different UBs for code optimization. So, it is very useful to compile OSS projects with different compilers, it helps find more bugs and make sources more standard-complaint and portable
musicale 3y ago
Eventually we may get one that works properly and supports useful features like bounds checking, stacks that grow upward rather than downward, trapping on undefined behavior, etc. We might even get a new and more robust libc - maybe one that even supports static compilation.

mise_en_place 3y ago

I haven't tried it myself, but you can use the Zig toolchain as a C and C++ compiler[0]. In this day and age, is it really necessary to have yet another C compiler? I believe you can do the same with D as well. [0]https://medium.com/@edlyuu/zig-c-c-compiler-wtf-is-zig-c-790d9ad8d85b https://medium.com/@edlyuu/zig-c-c-compiler-wtf-is-zig-c-790...

deleted 3y ago
[deleted]
jxy 3y ago
The value of Intel's oneAPI tool chain is that you can write code that runs on their GPU, much like Nvidia's cuda or nvhpc.

intothemild 3y ago

The year is 2024, and I'm using another newer, buggier C compiler.

b5n 3y ago
The year is 2074, I'm still using gcc and emacs :)
- exikyut 3y ago
  !remindme 50 years

PresidentZippy 3y ago

Not to crap on someone's hard work, but usually reinventing something is only warranted if you can do it >2x better than the best available solution. Whether that means requiring half as much hardware to do the same job or requiring half as much manpower to use, usually it's a waste of time and money to make something that's only marginally better at best. How much benefit did Intel expect to reap from rewriting an LLVM x64 compiler backend from the ground up?

JonChesterfield 3y ago
Isn't this a GPU backend, not an x64 one?
- PresidentZippy 3y ago
  The backend builds both x86_64 machine code, and GPU kernels/FPGA microcode according to their documentation, but that just begs another question: How is this significantly better than the existing tools for compiling C into GPU and FPGA code? It would help if Intel published some kind of chalk talk with a live demo showing how much faster you can build HPC applications using their new toolkit. I'm not summarily writing it off, but I need a little convincing before I put 20+ hours into trying it out myself.
  - JonChesterfield 3y ago
    Yes, though the llvm x64 backend may turn out to be very similar to the upstream one. Existing tools for compiling C to Intel's GPU are this tool, there are no others.
    - PresidentZippy 3y ago
      Is there anything more to it than that? If not, the documentation would be a lot more helpful if it lead with something straight to the point. Here's something that could go directly under the title of the README: "This compiler consists of a custom LLVM frontend and backend. The backend compiles LLVM IR code into machine code consisting of x86_64 instructions and Intel GPU code. The frontend works in conjunction with the backend to compile C and C++ code with special optimization which when enabled, compiles OpenMP routines into hardware-accelerated code targeting Intel GPUs, FPGAs, or AMD and NVIDIA GPUs." As someone who only used OpenMP academically, I don't see much of a point in that. In the post C++11 world, where we can write type-safe compile-time code, preprocessor macro definitions should stay in C code. Until Intel GPUs are at least competitive with the big boys, interop with their products doesn't concern me a whole hell of a lot. I'm not going to plan my scientific computing applications around the integrated graphics found on cheap Wintel consumer devices.