5 ms·

Got any standardized methods that could be automated/scripted under a clean Windows environment and doesn't require internet access / licensing? Give me a shout

Got any standardized methods that could be automated/scripted under a clean Windows environment and doesn't require internet access / licensing? Give me a shout - ian@anandtech.com

fierarul 7y ago
Maybe anandtech.com could create / pay for such tests?
hajile 7y ago
Linux Kernel would probably be good. WSL could probably do it without much trouble though I don't know how WSL affects performance compared to native Linux. Compiling Visual Studio Code Typescript could be an interesting target too. There should be a large open-source C# and/or Java project out there that can be compiled as well.
- scns 7y ago
  Is Typescript compilation multithreaded?
  - lliamander 7y ago
    A good question, though I will point out that either way benchmarking would still be useful.
  - mtone 7y ago
    The source code is in .ts, and appears to use Node child processes [0], so probably yes. I'd be curious to see the kind of resource usage is involved building TSC. My limited experience using Node worker threads (to compile small React apps with WebPack in parallel) showed lots of overhead and memory usage for just starting the build (Node and library instances, can't say in what proportion), although it is still worth the expense. [0] https://github.com/microsoft/TypeScript/blob/master/scripts/build/utils.js https://github.com/microsoft/TypeScript/blob/master/scripts/...
lliamander 7y ago
> Got any standardized methods that could be automated/scripted I'm sorry, I wish I did! Interest in hardware performance is only a relatively recent interest of mine. I especially think the IDE tests would be a bit challenging as I'm not even sure how to instrument or measure those. I might reach out to the Intellij folks to see if they have any ideas.. > under a clean Windows environment Ah, shucks. I haven't touched Windows in years :). That's the tricky thing: according to Stack Overflow's surveys, only about half of software developers use Windows. The other half are split evenly between Mac and Linux. Covering all three platforms would likely be a challenge. > Give me a shout Thanks! I'll do some research, and if I have something more concrete I'll be sure to pass it along.
manigandham 7y ago
.NET Core is open-source and free, with an offline SDK installer. [1] There are plenty of large open C# code bases, like the ASP.NET web framework [2], that you can use to compile and get a good performance score. And like the other comment said, you can probably get complimentary copies of the IDEs from Jetbrains to run benchmarks with. They're also scriptable which helps. 1 .https://dotnet.microsoft.com/download/visual-studio-sdks https://dotnet.microsoft.com/download/visual-studio-sdks 2. https://github.com/aspnet/AspNetCore https://github.com/aspnet/AspNetCore 3. https://www.jetbrains.com/ https://www.jetbrains.com/
Bayart 7y ago
Honestly something like a Linux Kernel compilation would be the most straightforward, static and consistent method to test it out. Just have a basic environment without overhead (like Alpine Linux) with GCC or LLVM/Clang sitting on a USB key, use GNU time to measure the execution of your command, redirect to a log file and you're done. user@host$ (time make -j[insert number of threads]) 2>logfile.txt

Got any standardized methods that could be automated/scripted under a clean Windows environment and doesn't require internet access / licensing? Give me a shout - ian@anandtech.com

fierarul 7y ago

Maybe anandtech.com could create / pay for such tests?

hajile 7y ago

Linux Kernel would probably be good. WSL could probably do it without much trouble though I don't know how WSL affects performance compared to native Linux. Compiling Visual Studio Code Typescript could be an interesting target too. There should be a large open-source C# and/or Java project out there that can be compiled as well.

scns 7y ago
Is Typescript compilation multithreaded?
- lliamander 7y ago
  A good question, though I will point out that either way benchmarking would still be useful.
- mtone 7y ago
  The source code is in .ts, and appears to use Node child processes [0], so probably yes. I'd be curious to see the kind of resource usage is involved building TSC. My limited experience using Node worker threads (to compile small React apps with WebPack in parallel) showed lots of overhead and memory usage for just starting the build (Node and library instances, can't say in what proportion), although it is still worth the expense. [0] https://github.com/microsoft/TypeScript/blob/master/scripts/build/utils.js https://github.com/microsoft/TypeScript/blob/master/scripts/...

lliamander 7y ago

> Got any standardized methods that could be automated/scripted I'm sorry, I wish I did! Interest in hardware performance is only a relatively recent interest of mine. I especially think the IDE tests would be a bit challenging as I'm not even sure how to instrument or measure those. I might reach out to the Intellij folks to see if they have any ideas.. > under a clean Windows environment Ah, shucks. I haven't touched Windows in years :). That's the tricky thing: according to Stack Overflow's surveys, only about half of software developers use Windows. The other half are split evenly between Mac and Linux. Covering all three platforms would likely be a challenge. > Give me a shout Thanks! I'll do some research, and if I have something more concrete I'll be sure to pass it along.

manigandham 7y ago

.NET Core is open-source and free, with an offline SDK installer. [1] There are plenty of large open C# code bases, like the ASP.NET web framework [2], that you can use to compile and get a good performance score. And like the other comment said, you can probably get complimentary copies of the IDEs from Jetbrains to run benchmarks with. They're also scriptable which helps. 1 .https://dotnet.microsoft.com/download/visual-studio-sdks https://dotnet.microsoft.com/download/visual-studio-sdks 2. https://github.com/aspnet/AspNetCore https://github.com/aspnet/AspNetCore 3. https://www.jetbrains.com/ https://www.jetbrains.com/

Bayart 7y ago

Honestly something like a Linux Kernel compilation would be the most straightforward, static and consistent method to test it out. Just have a basic environment without overhead (like Alpine Linux) with GCC or LLVM/Clang sitting on a USB key, use GNU time to measure the execution of your command, redirect to a log file and you're done. user@host$ (time make -j[insert number of threads]) 2>logfile.txt