6 ms·

I love using Gitlab SCM, but I'm surprised that Gitlab CI is described as "Lovable" while major bug fixes are gated for months on an uncertain future feature re

I love using Gitlab SCM, but I'm surprised that Gitlab CI is described as "Lovable" while major bug fixes are gated for months on an uncertain future feature release: "Job marked as success when job terminate midway in Kubernetes" https://gitlab.com/gitlab-org/gitlab-runner/issues/4119 https://gitlab.com/gitlab-org/gitlab-runner/issues/4119 IMO a production CI bug that falsely reports success merits a high priority patch release but Gitlab doesn't seem to see it that way.

sytse 7y ago
This bug took too long to fix. We are not happy how we prioritized the last few releases for CI. In 12.4 we'll focus on fixing 8 of outstanding CI bugs https://gitlab.com/groups/gitlab-org/-/issues?scope=all&utf8=%E2%9C%93&state=opened&milestone_title=12.4&label_name[]=Category%3AContinuous%20Integration&label_name[]=bug https://gitlab.com/groups/gitlab-org/-/issues?scope=all&utf8...
- Operyl 7y ago
  So, can you shed some light on how the actual prioritization works? There are some bugs, one specifically I've mentioned elsewhere, that just get infinitely shoved into the void.
  - sytse 7y ago
    The Product Manager (in the case Jason Lenny who is out of office today) prioritizes between bugs, vulnerabilities, new features, and tech debt. He or she uses customer input, user input, and company input. Read more on https://about.gitlab.com/handbook/product/#prioritization https://about.gitlab.com/handbook/product/#prioritization To find the relevant Product Manager see https://about.gitlab.com/handbook/product/categories/ https://about.gitlab.com/handbook/product/categories/
    - Operyl 7y ago
      So, the unfortunate reality of prioritizing customer wants like this is that you're always going to focus on big shiny new features, and not quite so much the smaller (while still important) bug fixes that really need consideration I feel. I felt like I got shoved into a black hole, and it does not inspire faith in your product anymore. Even if I did become a customer, which I was seriously considering before this, we're going to be "too small" to make a dent compared to your bigger customer base.
      - sytse 7y ago
        Existing customers tend to prioritize stability (fixing bugs) over new features. Please note that even if you're not a customer our issue trackers are open so you can @mention the Product Manager to help them understand the severity and help to diagnose and fix things if you're so inclined.
        Operyl 7y ago
        As seen in https://gitlab.com/gitlab-org/gitlab-runner/issues/4029 https://gitlab.com/gitlab-org/gitlab-runner/issues/4029, a few people did show frustration, and why. And then began the never ending push back after push back.
        sytse 7y ago
        I agree this issue hasn't been prioritized as high as it should. I meant to indicate that with "This bug took too long to fix." but I should have said it took to long to schedule. We concluded the same internally recently and hence the focus on bugs and wider community contributions in 12.4. We overly focussed on getting the DAG https://docs.gitlab.com/ee/ci/directed_acyclic_graph/ https://docs.gitlab.com/ee/ci/directed_acyclic_graph/ and Merge Trains https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/ https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipeli... out the door and missed a bad bug, we're sorry.
        sooheon 7y ago
    - jl-gitlab 7y ago
      Hey, I'm Jason - the PM director for this area. I really do appreciate the pings - you can @ me at my username jlenny in issues that are important to you. It's true in general, even for features, but especially for issues a heads up helps us make sure we are prioritizing things in the right order.
      - Operyl 7y ago
        You had already been involved in the issue, from what I recall. I also did state my dissatisfaction when it just continued to get pushed back. I get it, bug fixes aren’t glamorous and it’s not fun to work on them, but they still need to happen.
    - rossmohax 7y ago
      PM optimizes for his own KPIs, which are at exec level. Given the history of ignoring bugfixes and becoming famous for it, clearly direction leadership sets gives no incentive to fix customers problems.
arianvanp 7y ago
Getting Gitlab CI to work has been a major PITA for me. I would maybe give it a quarter circle in its current state. Not a heart. The runner self-modifies the config file and uses it as state which makes it basically impossible to deploy it in an immutable way (Like on nixos). I found out that most config options (but not all of them) are also avaible as environment variables and cli flags so that's how I hacked around the issue but it means I can't support all features that the runner has because the environment variables do not expose all the available config options. The fact that it's so hard to automate a piece of (in theory) stateless software is really annoying :( https://gitlab.com/arianvp/nixos-gitlab-runner https://gitlab.com/arianvp/nixos-gitlab-runner https://gitlab.com/gitlab-org/gitlab-runner/issues/1932 https://gitlab.com/gitlab-org/gitlab-runner/issues/1932 https://gitlab.com/arianvp/nixos-gitlab-runner/issues/2 https://gitlab.com/arianvp/nixos-gitlab-runner/issues/2
- rossmohax 7y ago
  Luckily they started to adapt Kubernetes themselves and some good improvements are underway from this doogfooding. In particular UX of a runner registration was improved with config template feature https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/1263 https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/1... There is still a problem, that self modifying config option doesn't fit nicely with Nix though
igornadj 7y ago
It's described as Lovable due to the measurements they make on real user behaviour. So people must be using it and sticking with it.

sytse 7y ago

This bug took too long to fix. We are not happy how we prioritized the last few releases for CI. In 12.4 we'll focus on fixing 8 of outstanding CI bugs https://gitlab.com/groups/gitlab-org/-/issues?scope=all&utf8=%E2%9C%93&state=opened&milestone_title=12.4&label_name[]=Category%3AContinuous%20Integration&label_name[]=bug https://gitlab.com/groups/gitlab-org/-/issues?scope=all&utf8...

Operyl 7y ago
So, can you shed some light on how the actual prioritization works? There are some bugs, one specifically I've mentioned elsewhere, that just get infinitely shoved into the void.
- sytse 7y ago
  The Product Manager (in the case Jason Lenny who is out of office today) prioritizes between bugs, vulnerabilities, new features, and tech debt. He or she uses customer input, user input, and company input. Read more on https://about.gitlab.com/handbook/product/#prioritization https://about.gitlab.com/handbook/product/#prioritization To find the relevant Product Manager see https://about.gitlab.com/handbook/product/categories/ https://about.gitlab.com/handbook/product/categories/
  - Operyl 7y ago
    So, the unfortunate reality of prioritizing customer wants like this is that you're always going to focus on big shiny new features, and not quite so much the smaller (while still important) bug fixes that really need consideration I feel. I felt like I got shoved into a black hole, and it does not inspire faith in your product anymore. Even if I did become a customer, which I was seriously considering before this, we're going to be "too small" to make a dent compared to your bigger customer base.
    - sytse 7y ago
      Existing customers tend to prioritize stability (fixing bugs) over new features. Please note that even if you're not a customer our issue trackers are open so you can @mention the Product Manager to help them understand the severity and help to diagnose and fix things if you're so inclined.
      - Operyl 7y ago
        As seen in https://gitlab.com/gitlab-org/gitlab-runner/issues/4029 https://gitlab.com/gitlab-org/gitlab-runner/issues/4029, a few people did show frustration, and why. And then began the never ending push back after push back.
        sytse 7y ago
        I agree this issue hasn't been prioritized as high as it should. I meant to indicate that with "This bug took too long to fix." but I should have said it took to long to schedule. We concluded the same internally recently and hence the focus on bugs and wider community contributions in 12.4. We overly focussed on getting the DAG https://docs.gitlab.com/ee/ci/directed_acyclic_graph/ https://docs.gitlab.com/ee/ci/directed_acyclic_graph/ and Merge Trains https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/ https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipeli... out the door and missed a bad bug, we're sorry.
        sooheon 7y ago
  - jl-gitlab 7y ago
    Hey, I'm Jason - the PM director for this area. I really do appreciate the pings - you can @ me at my username jlenny in issues that are important to you. It's true in general, even for features, but especially for issues a heads up helps us make sure we are prioritizing things in the right order.
    - Operyl 7y ago
      You had already been involved in the issue, from what I recall. I also did state my dissatisfaction when it just continued to get pushed back. I get it, bug fixes aren’t glamorous and it’s not fun to work on them, but they still need to happen.
  - rossmohax 7y ago
    PM optimizes for his own KPIs, which are at exec level. Given the history of ignoring bugfixes and becoming famous for it, clearly direction leadership sets gives no incentive to fix customers problems.

arianvanp 7y ago

Getting Gitlab CI to work has been a major PITA for me. I would maybe give it a quarter circle in its current state. Not a heart. The runner self-modifies the config file and uses it as state which makes it basically impossible to deploy it in an immutable way (Like on nixos). I found out that most config options (but not all of them) are also avaible as environment variables and cli flags so that's how I hacked around the issue but it means I can't support all features that the runner has because the environment variables do not expose all the available config options. The fact that it's so hard to automate a piece of (in theory) stateless software is really annoying :( https://gitlab.com/arianvp/nixos-gitlab-runner https://gitlab.com/arianvp/nixos-gitlab-runner https://gitlab.com/gitlab-org/gitlab-runner/issues/1932 https://gitlab.com/gitlab-org/gitlab-runner/issues/1932 https://gitlab.com/arianvp/nixos-gitlab-runner/issues/2 https://gitlab.com/arianvp/nixos-gitlab-runner/issues/2

rossmohax 7y ago
Luckily they started to adapt Kubernetes themselves and some good improvements are underway from this doogfooding. In particular UX of a runner registration was improved with config template feature https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/1263 https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/1... There is still a problem, that self modifying config option doesn't fit nicely with Nix though

igornadj 7y ago

It's described as Lovable due to the measurements they make on real user behaviour. So people must be using it and sticking with it.