4 ms·

Why do you want them to write code at all, then? Why not just task them with writing user stories around the parameterized functions they need, and just let you

Why do you want them to write code at all, then? Why not just task them with writing user stories around the parameterized functions they need, and just let you figure out how to implement it all?

duped 4y ago
You usually have to evaluate the model as you develop it, it's not something that a story driven model will fit
Tcepsa 4y ago
I expect that it makes sense to have the modelers do at least enough coding to test the common use cases/expected inputs for the model to validate it before handing it off to be "properly" implemented.
p1necone 4y ago
Assuming you're writing actually detailed and complete user stories and those user stories are for functionality that basically amounts to "just math" (i.e. no UI or API standards or whatever to worry about alongside the business logic) I imagine it's not much more difficult or time consuming to just write the code rather than describing it in a user story. Having one person write user stories and a second person, who understands the domain less than the first person write the actual code would be twice as much work for a worse result.
lmm 4y ago
What's the best way to write those user stories? Certainly not plain English; you'd probably want to use some kind of pseudocode. Ah, but it would be good if you could run that pseudocode and see what the results are in the happy path, without worrying too much about what happens when something goes wrong. We have that, it's called Python, so that tends to be how these user stories are expressed.
bobbruno 4y ago
The idea that creating a model can be a “user story written in math” seems to me a variation on a common misunderstanding about what model creation, particularly the role of coding in it is. Data scientists, statisticians, modellers, don’t go in knowing what the model is and just coding or specifying it. They use code as a exploratory tool to test several hypotheses until they find one that seems to hold - the code, the data, the algorithm, the statistical test, the exploration and the reasoning are all tools in the process of discovering the right model. It simply can’t be specified in advance, it has to be run and tested. Having som spaghetti code is a natural consequence of this exploratory, iterative process. Applying good SW engineering practices to this exploratory endeavour is just a natural consequence of the process when you don’t know if your code will be of any use before you finish running and checking the test results. Why would you bother modularising and doing test coverage on something that is very likely to be thrown away after one or 2 runs? I say this from 28 years of experience both as a data engineer and data scientist. I am a good python developer, and I can write production grade code. But I won’t refactor my code into that until I know that is the code that generates the right model. And I certainly can’t specify this particular code before writing some dirty version of it, testing it and confirming that the model it trains passes some statistical tests, at least. Basically, the code is not the product - the product is the result of applying some transformations on data and running that through some ML or statistical algorithm to generate a model. Transformations and algorithm being unknown to be useful until tested, hence specification being unknown until coded, run and tested.

Why do you want them to write code at all, then? Why not just task them with writing user stories around the parameterized functions they need, and just let you figure out how to implement it all?

duped 4y ago

You usually have to evaluate the model as you develop it, it's not something that a story driven model will fit

Tcepsa 4y ago

I expect that it makes sense to have the modelers do at least enough coding to test the common use cases/expected inputs for the model to validate it before handing it off to be "properly" implemented.

p1necone 4y ago

Assuming you're writing actually detailed and complete user stories and those user stories are for functionality that basically amounts to "just math" (i.e. no UI or API standards or whatever to worry about alongside the business logic) I imagine it's not much more difficult or time consuming to just write the code rather than describing it in a user story. Having one person write user stories and a second person, who understands the domain less than the first person write the actual code would be twice as much work for a worse result.

lmm 4y ago

What's the best way to write those user stories? Certainly not plain English; you'd probably want to use some kind of pseudocode. Ah, but it would be good if you could run that pseudocode and see what the results are in the happy path, without worrying too much about what happens when something goes wrong. We have that, it's called Python, so that tends to be how these user stories are expressed.

bobbruno 4y ago

The idea that creating a model can be a “user story written in math” seems to me a variation on a common misunderstanding about what model creation, particularly the role of coding in it is. Data scientists, statisticians, modellers, don’t go in knowing what the model is and just coding or specifying it. They use code as a exploratory tool to test several hypotheses until they find one that seems to hold - the code, the data, the algorithm, the statistical test, the exploration and the reasoning are all tools in the process of discovering the right model. It simply can’t be specified in advance, it has to be run and tested. Having som spaghetti code is a natural consequence of this exploratory, iterative process. Applying good SW engineering practices to this exploratory endeavour is just a natural consequence of the process when you don’t know if your code will be of any use before you finish running and checking the test results. Why would you bother modularising and doing test coverage on something that is very likely to be thrown away after one or 2 runs? I say this from 28 years of experience both as a data engineer and data scientist. I am a good python developer, and I can write production grade code. But I won’t refactor my code into that until I know that is the code that generates the right model. And I certainly can’t specify this particular code before writing some dirty version of it, testing it and confirming that the model it trains passes some statistical tests, at least. Basically, the code is not the product - the product is the result of applying some transformations on data and running that through some ML or statistical algorithm to generate a model. Transformations and algorithm being unknown to be useful until tested, hence specification being unknown until coded, run and tested.