4 ms·

One of the things that I wish IntelliJ had was some way of doing some "real world" benchmarks. It'd be great to know exactly what sort of speed improvements th

One of the things that I wish IntelliJ had was some way of doing some "real world" benchmarks. It'd be great to know exactly what sort of speed improvements this might bring (or if it's really faster at all). It _seems_ like IJ could be scripted using the plugin system to do things like: - Load and apply formatting to a 1 MB source file. - Index an entire project. - Run a regex find/replace on a code base. - Load 50 code files and time how long it takes for each to be ready to display.

AtlasBarfed 6y ago
Mac meta-scripting / macros / virtual keyboard/mouse scripting exists.
random5634 6y ago
Great idea!
deleted 6y ago
[deleted]
skybrian 6y ago
This would be difficult to make meaningful because projects vary by orders of magnitude in performance-relevant metrics, as does user behavior. IntelliJ has an enormous number of features and people use their own subset of them. Not to mention things like plugins and other options for customization. It's a huge space.
- mdaniel 6y ago
  But as far as making perf improvements to IJ itself, that actually would be meaningful: it would allow them to say "commit eeffaa takes 2m16.555ms" to perform the steps outlined by nvahalik, but "commit aabbff takes 2m01.222ms" and thus it's getting better for that metric I wholeheartedly agree there will always be very weird setups on very weird source, but if they can make the 80/20 run faster, hopefully a rising tide will lift all boats kind of thing
  - skybrian 6y ago
    It's not just "very weird setups." Totally normal setups differ greatly. One example of variation is that the users might be writing code using different languages. The parser and static analysis for a project written in Go will be very different than a project written in Java. All the language-specific stuff is different code that has different performance characteristics. As an analogy, it's sort of like testing browser performance for loading an "average" web page, when each web site has different performance, depending on how it was written. Load times for Hacker News aren't all that relevant for Gmail or the New York Times. Yes, you can make a benchmark and optimize it, but you might not even be running the same code that real users are running, due to plugins.

AtlasBarfed 6y ago

Mac meta-scripting / macros / virtual keyboard/mouse scripting exists.

random5634 6y ago

Great idea!

deleted 6y ago

[deleted]

skybrian 6y ago

This would be difficult to make meaningful because projects vary by orders of magnitude in performance-relevant metrics, as does user behavior. IntelliJ has an enormous number of features and people use their own subset of them. Not to mention things like plugins and other options for customization. It's a huge space.

mdaniel 6y ago
But as far as making perf improvements to IJ itself, that actually would be meaningful: it would allow them to say "commit eeffaa takes 2m16.555ms" to perform the steps outlined by nvahalik, but "commit aabbff takes 2m01.222ms" and thus it's getting better for that metric I wholeheartedly agree there will always be very weird setups on very weird source, but if they can make the 80/20 run faster, hopefully a rising tide will lift all boats kind of thing
- skybrian 6y ago
  It's not just "very weird setups." Totally normal setups differ greatly. One example of variation is that the users might be writing code using different languages. The parser and static analysis for a project written in Go will be very different than a project written in Java. All the language-specific stuff is different code that has different performance characteristics. As an analogy, it's sort of like testing browser performance for loading an "average" web page, when each web site has different performance, depending on how it was written. Load times for Hacker News aren't all that relevant for Gmail or the New York Times. Yes, you can make a benchmark and optimize it, but you might not even be running the same code that real users are running, due to plugins.