5 ms·

I enabled ubsan on a local build of our code. It found some issues, but good god the build time was atrocious! I doubt it's something you could enable on a CI p

I enabled ubsan on a local build of our code. It found some issues, but good god the build time was atrocious! I doubt it's something you could enable on a CI pipeline.

wffurr 5y ago
We run sanitizers nightly, but not as part of presubmit checks. It strikes a good balance between productivity and safety.
- vlovich123 5y ago
  I’ve always run sanitized builds as presubmit without any meaningful issue. It’s always nicer to have these issues caught before code gets merged into a shared repo. The long part of things is code review anyway so waiting on a build isn’t the biggest deal in the world.
  - wffurr 5y ago
    That's great if it's fast enough. For some builds, like the GP and mine, it's extremely slow.
    - vlovich123 5y ago
      My point is I've run really big builds (i.e. taking about an hour to build non-sanitized) & we still did sanitized builds pre-submit. How big is the disparity you're seeing between sanitized & non-sanitized builds?
      - wffurr 5y ago
        An hour, ouch. That's pretty long. We try to keep our presubmit under 10 minutes, and that's already frustrating. I can't figure out how to dig the numbers out of our CI infrastructure for the nightly sanitizer runs.
        vlovich123 5y ago
        I guess my point was more that it depends on how you structure your workflow. As long as you can do reviews and kick off asynchronous merge requests, the actual time it takes starts to be come largely irrelevant in my experience (aside from the impact it has on local iteration). These days I'm on a different project and CI ASAN builds are 33 minutes vs normal builds of 21 minutes. I'd say that extra 10 minutes is fine because sanitized builds run in parallel to normal builds & this is clean builds vs incremental.
- renox 5y ago
  Sure but then you waste time bisecting the commit which introduced the issue.. I wonder if the bissection couldn't be automated also in the CI.
  - wffurr 5y ago
    The system we use at work does indeed bisect a build breakage automatically.
gpderetta 5y ago
I haven't found the sanitzer builds to be particularly slow. You mean running the test suite? Even that it wasn't particularly bad.

I enabled ubsan on a local build of our code. It found some issues, but good god the build time was atrocious! I doubt it's something you could enable on a CI pipeline.

wffurr 5y ago

We run sanitizers nightly, but not as part of presubmit checks. It strikes a good balance between productivity and safety.

vlovich123 5y ago
I’ve always run sanitized builds as presubmit without any meaningful issue. It’s always nicer to have these issues caught before code gets merged into a shared repo. The long part of things is code review anyway so waiting on a build isn’t the biggest deal in the world.
- wffurr 5y ago
  That's great if it's fast enough. For some builds, like the GP and mine, it's extremely slow.
  - vlovich123 5y ago
    My point is I've run really big builds (i.e. taking about an hour to build non-sanitized) & we still did sanitized builds pre-submit. How big is the disparity you're seeing between sanitized & non-sanitized builds?
    - wffurr 5y ago
      An hour, ouch. That's pretty long. We try to keep our presubmit under 10 minutes, and that's already frustrating. I can't figure out how to dig the numbers out of our CI infrastructure for the nightly sanitizer runs.
      - vlovich123 5y ago
        I guess my point was more that it depends on how you structure your workflow. As long as you can do reviews and kick off asynchronous merge requests, the actual time it takes starts to be come largely irrelevant in my experience (aside from the impact it has on local iteration). These days I'm on a different project and CI ASAN builds are 33 minutes vs normal builds of 21 minutes. I'd say that extra 10 minutes is fine because sanitized builds run in parallel to normal builds & this is clean builds vs incremental.
renox 5y ago
Sure but then you waste time bisecting the commit which introduced the issue.. I wonder if the bissection couldn't be automated also in the CI.
- wffurr 5y ago
  The system we use at work does indeed bisect a build breakage automatically.

gpderetta 5y ago

I haven't found the sanitzer builds to be particularly slow. You mean running the test suite? Even that it wasn't particularly bad.