4 ms·
> And they weren't allowed to pick which tasks they used the AI on. They were allowed to pick whether or not to use AI on a subset of tasks. They weren't force
by sumeno 1y ago
> And they weren't allowed to pick which tasks they used the AI on.
They were allowed to pick whether or not to use AI on a subset of tasks. They weren't forced to use AI on tasks that don't make sense for AI
- rosspackard 1y agoHalf the tasks they were not allowed to use AI.
- sumeno 1y agoYes, and the other half they had the option to use AI. That's why I said they were allowed to pick whether or not to use AI on a subset of tasks. On the other subset they were not allowed to use AI.
- throwaway284927 1y agoThat is not true, usage of AI was decided randomly. From the paper: "To directly measure the impact of AI tools on developer productivity, we conduct a randomized controlled trial by having 16 developers complete 246 tasks (2.0 hours on average) on well-known open-source repositories (23,000 stars on average) they regularly contribute to. Each task is randomly assigned to allow or disallow AI usage, and we measure how long it takes developers to complete tasks in each condition."
- sumeno 1y agoDirectly from the paper: > If AI is allowed, developers can use any AI tools or models they choose, including no AI tooling if they expect it to not be helpful. If AI is not allowed, no generative AI tooling can be used. AI is allowed not required
- throwaway284927 1y agoTrue, my bad, I didn't read you correctly. What you said was true. I do believe however that it's important to emphasize the fact that they didn't got to choose in general, though, which I think your wording (even though it is correct) does not make evident.