3 ms·
I've been a data scientist for over a decade and have delivered on a wide range of high-impact projects and products. I don't recognize this as meaningful or he
by civilized 3y ago
I've been a data scientist for over a decade and have delivered on a wide range of high-impact projects and products. I don't recognize this as meaningful or helpful to my field. More of a me-too from an inexperienced perspective.
The bulleted list at the top seems to be a random collection of irrelevant, almost unintelligible thoughts, stuffed cargo cult-style into the template of the agile manifesto. What does "APIs over databases" mean? It sounds like "oranges over apples" to me.
The numbered list of principles is better, but still not always that helpful. There's a strange spirit of perfectionism and inflexibility, e.g. in #3 and #5. Maybe some data scientists have time to automate everything they do, but I don't think it's a good general principle. #4, 6, and 7 are better. But overall there is a disorganized, random, unmotivated feel to both lists.
Perhaps something better could be developed, but for now, I think data scientists looking for a manifesto would be best served by going back to the original agile manifesto (https://agilemanifesto.org/ https://agilemanifesto.org/) and reflecting on how it applies to our field. Which it doesn't, at least not universally. But it didn't apply universally to software engineering either. Just replace "software" with "data analysis" or "predictive analytics" or what have you and it all carries over pretty well.
- noduerme 3y agoThanks. I was about to ask what "APIs over databases" meant. I design DBs and write APIs and I was struggling to imagine a scenario in which a data consumer - or I as a consumer - wouldn't want to understand both. I'm not even sure to whom they're interchangeable terms. Possibly to a front end coder who doesn't want to craft queries? I don't call myself a data scientist or even a software engineer (though I'd be well within my rights to, under the current understanding of those descriptions). I'm a coder who writes a lot of business logic and analyzes results. But I have noticed a creeping tendency recently for people who futz around with large data dashboards or LLM conversational strategies to try to distance themselves from the people who write code, as if somehow we who actually think through the logistics of performance and structure are just road workers while they're driving the shiny Tesla of Science, coming to the Big Answers. It's ridiculous, because anyone who's built a NN or a large DB knows how to query one, and is usually moderately bored by the results.
- epgui 3y agoI’m a biochemist and software engineer, and FWIW I agree with you. This article, in its current form, is nonsense.
- ACow_Adonis 3y agoI'm in the same boat (in terms of both field and experience) and I find the original post bordering on absolutely nonsensical gibberish. It's like a technical version of an astrology column.
- wodenokoto 3y agoI don't know what the author thought, but to me it sounds like adding layer for the purpose of adding layers. E.g, Your data pipeline shouldn't be doing a SQL query. It should be asking an API to do the query for you.
- TeMPOraL 3y agoWhich is funny, because SQL is an API. Security notwithstanding, it's often better than whatever API you can throw in front of it for the sake of "clean architecture" or whatever.
- pvdoom 3y agoHell, you could argue that SQL directly is clean architecture, and adding APIs is making it less clean and more complicated.
- jerzyt 3y agoYeah, I'd expand on the "API over DBs" based on my experience. I could make it nest much deeper: DBs over Excel spreadsheets, and Excel spreadsheets over PDFs or scraped HTMLs, yada, yada, yada. But before I waste any time creating an API, my client wants to know if the project is even viable.