4 ms·
> All benchmarks were ran on an X1 Carbon 2016 (i7 U6600 + 16 GB) in Node.js 11.9.0. They should test on average hardware and not on the top one. Take a 2013 y
by codedokode 8y ago
> All benchmarks were ran on an X1 Carbon 2016 (i7 U6600 + 16 GB) in Node.js 11.9.0.
They should test on average hardware and not on the top one. Take a 2013 year Celeron or Atom with 2 Gb of RAM and HDD and test on it.
Regarding privacy, I think the code that blocks network requests could run in an isolated environment so that it cannot send the information about requests outside.
- barrkel 8y agoI think it should be on a desktop class machine. I'm less concerned about absolute numbers and more with variance. Laptops generally cannot dissipate all the heat from full CPU usage for more than a minute or two. Benchmark runs, if they exercise the CPU hard for any length of time, get thermally limited and the results end up with substantially more variance. Even things as simple as build times, I've seen vary by 20+%.
- pythux 8y agoBenchmarking is always a hard problem - no such thing as spherical chickens in a vacuum. That said we'd love to at least standardize the setup for other people running the benchmarks; that is why we opened all the code and data, as a starting point. For the study, measurements were run with one of our personal laptops (an X1 Carbon from 2016 with an i7 U6600 CPU and 16 GB of Ram, which is indeed a pretty powerful machine). We tried very hard to limit the impact of frequency throttling due to limited thermal dissipation of the device for the long-running benchmarks. In fact, for the measurements we put the laptop outside at 0 degrees Celsius, and we could observe that the CPU temperature did not go beyond 60 degrees (which is pretty low). Do you have any suggestions on how we could improve this setup? We welcome all contributions.