3 ms·

This has less to do with bad engineers than bad policies and procedures. You don't performance test before shipping to prod? you get what you get.

by soperj 2mo ago

RSHEPP 2mo ago
Startups have limited resources, we prioritize the best we can. As a staff engineer it's my responsibility to bring this to my company, but it's not an overnight process.
- soperj 2mo ago
  why wouldn't you use those limited resources to test things before they go to production, instead of creating more crappy things that go to production?
  - rhdunn 2mo ago
    What do you test? How much time/effort do you spend testing each of those things? How do you know those will be performance issues? There's only so much you can do -- making reasonable assumptions, choosing suitable data structures, database indices, testing various workloads, etc. -- in a limited test environment. You may spend a week optimizing a feature that only 10 people use or that never gets to the point where it becomes an issue. Or you may have something that can't easily be tested at scale, such as various counts or other dynamic data that are determined by complex queries that you may (likely) find you need to cache but don't necessarily know which values will become issues until you start using the system.
    - soperj 2mo ago
      We have a test team that runs various test suites(automated) and manual testers that test functionality. They spend their entire working day, every day testing.
- fn-mote 2mo ago
  From this argument, it sounds like your company is making a rational choice to go with what you’re seeing. Even if you don’t like it.
cpgxiii 2mo ago
To a first approximation, basically no one in the industry does. Users regularly report performance regressions in basically every major piece of software, most of which would have been caught by performance testing.
- rhdunn 2mo ago
  It can sometimes be difficult to predict what will cause performance regressions when developing an application or web server with test data. So you do your best with what you know at the time of release, and then deal with performance issues when you know what is causing regressions (e.g. saved searches) and can then make measurements and profiling to see what exactly is causing the issue. Likewise, improving performance pushes the limit at which performance regressions happen allowing more data (documents, triangles/pixels, etc.) to be processed. This allows things like more complex game graphics. That in turn makes it harder to improve performance for the next round (more advanced triangle/face culling and pre-processing). There can also be trade-offs with things like data layout, memory usage (caching and memoization), or implementation. For example, when processing XML/HTML data you could use DOM (more memory and upfront parsing, but easier to perform complex queries across the data), SAX (less memory, but more complex to process due to tracking state), or reader API (similar to SAX but a different processing model). There can be challenges with various features, such as type checking with higher-order generic types. Others add various levels of overhead, such as parsing a program, constructing an AST (Abstract Syntax Tree), generating an IR (Intermediate Representation), the optimization passes and final code generation. JavaScript evaluation for example is complex. Before Chrome the approach was to use a slow interpreter. IIRC, Chrome was the first browser to introduce JIT (Just-in-Time) compilation, leading to faster JavaScript and eventually more complex applications running in that language. Modern browser JavaScript pipelines are complex in order to achieve and maintain performance: 1. start interpreting the code on the AST so it is run immediately (or only rely on (2)); 2. generate a machine code equivalent of that interpreted code (for faster baseline performance); 3. run more aggressive optimizations on known types based on profiling/analysis for even faster performance, including special handling of things like asm.js. [1] https://v8.dev/blog/ignition-interpreter https://v8.dev/blog/ignition-interpreter [2] https://benediktmeurer.de/2016/11/25/v8-behind-the-scenes-november-edition/ https://benediktmeurer.de/2016/11/25/v8-behind-the-scenes-no... [3] https://www.cs.cornell.edu/courses/cs6120/2020fa/blog/tracemonkey/ https://www.cs.cornell.edu/courses/cs6120/2020fa/blog/tracem... [4] https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/ https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/

This has less to do with bad engineers than bad policies and procedures. You don't performance test before shipping to prod? you get what you get.

by soperj 2mo ago

This has less to do with bad engineers than bad policies and procedures. You don't performance test before shipping to prod? you get what you get.

RSHEPP 2mo ago

Startups have limited resources, we prioritize the best we can. As a staff engineer it's my responsibility to bring this to my company, but it's not an overnight process.

soperj 2mo ago
why wouldn't you use those limited resources to test things before they go to production, instead of creating more crappy things that go to production?
- rhdunn 2mo ago
  What do you test? How much time/effort do you spend testing each of those things? How do you know those will be performance issues? There's only so much you can do -- making reasonable assumptions, choosing suitable data structures, database indices, testing various workloads, etc. -- in a limited test environment. You may spend a week optimizing a feature that only 10 people use or that never gets to the point where it becomes an issue. Or you may have something that can't easily be tested at scale, such as various counts or other dynamic data that are determined by complex queries that you may (likely) find you need to cache but don't necessarily know which values will become issues until you start using the system.
  - soperj 2mo ago
    We have a test team that runs various test suites(automated) and manual testers that test functionality. They spend their entire working day, every day testing.
fn-mote 2mo ago
From this argument, it sounds like your company is making a rational choice to go with what you’re seeing. Even if you don’t like it.

cpgxiii 2mo ago

To a first approximation, basically no one in the industry does. Users regularly report performance regressions in basically every major piece of software, most of which would have been caught by performance testing.

rhdunn 2mo ago
It can sometimes be difficult to predict what will cause performance regressions when developing an application or web server with test data. So you do your best with what you know at the time of release, and then deal with performance issues when you know what is causing regressions (e.g. saved searches) and can then make measurements and profiling to see what exactly is causing the issue. Likewise, improving performance pushes the limit at which performance regressions happen allowing more data (documents, triangles/pixels, etc.) to be processed. This allows things like more complex game graphics. That in turn makes it harder to improve performance for the next round (more advanced triangle/face culling and pre-processing). There can also be trade-offs with things like data layout, memory usage (caching and memoization), or implementation. For example, when processing XML/HTML data you could use DOM (more memory and upfront parsing, but easier to perform complex queries across the data), SAX (less memory, but more complex to process due to tracking state), or reader API (similar to SAX but a different processing model). There can be challenges with various features, such as type checking with higher-order generic types. Others add various levels of overhead, such as parsing a program, constructing an AST (Abstract Syntax Tree), generating an IR (Intermediate Representation), the optimization passes and final code generation. JavaScript evaluation for example is complex. Before Chrome the approach was to use a slow interpreter. IIRC, Chrome was the first browser to introduce JIT (Just-in-Time) compilation, leading to faster JavaScript and eventually more complex applications running in that language. Modern browser JavaScript pipelines are complex in order to achieve and maintain performance: 1. start interpreting the code on the AST so it is run immediately (or only rely on (2)); 2. generate a machine code equivalent of that interpreted code (for faster baseline performance); 3. run more aggressive optimizations on known types based on profiling/analysis for even faster performance, including special handling of things like asm.js. [1] https://v8.dev/blog/ignition-interpreter https://v8.dev/blog/ignition-interpreter [2] https://benediktmeurer.de/2016/11/25/v8-behind-the-scenes-november-edition/ https://benediktmeurer.de/2016/11/25/v8-behind-the-scenes-no... [3] https://www.cs.cornell.edu/courses/cs6120/2020fa/blog/tracemonkey/ https://www.cs.cornell.edu/courses/cs6120/2020fa/blog/tracem... [4] https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/ https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/