This is really interesting. It "had" me at >Through this selection bias, past research often excludes the broader, lesser-resourced ML community -- for example

This is really interesting. It "had" me at >Through this selection bias, past research often excludes the broader, lesser-resourced ML community -- for example, practitioners working at startups, at non-tech companies, and in the public sector. As a tiny boutique consultancy, we've worked for organizations in different sectors: the public sector for employment and transportation, energy sector, "non-tech" as in office supplies and luxury cosmetics, telecom companies, banks, etc. What we found is that the "maturity level" is a spectrum: one one end, there's the "out-of-the-loop" organization who wants to use "AI". We accompany these organizations and hold their hand. We give a lay of the land and help to understand what machine learning is about, what the pre-requisites in their organization should be, what we can and cannot do with machine learning and clarifying expectations, the ethical questions to address, the necessity to frame the problem to solve very well, the necessity for us to interact with their domain experts and the people we will build the product for "If you want to cut someone's hair, they need to be in the room". On the other end of the spectrum, there's the R&D team at the large tech company that has an internal machine learning team but needs help on something specific. We get to brass tax quickly. This difference in maturity is not present only in our clients, but we have matured over the years as well. We continually refine two aspects. The first aspect is the way we do consulting in terms of scoping, qualification, leveraging our experience curve, the questions we ask, the early inclusion of everyone who has a stake, and making that into repeatable products so we can reduce our clients' "time to value". It used to take us many, many, meeting calls and emails to accompany our clients, and that takes time from everyone, and the project, whether we green-light it or not. The second aspect we have worked on is technical: the tooling. Over the years, we have worked on so many projects in different sectors and industries for different problems and functions, using different approaches, that some patterns have emerged. We built an internal machine learning platform[0] to help us get data products right, and help our tiny team. We do this refinement for reasons that may be so obvious. For example, after some rough changes in our company, we pushed for remote work around 2018/2019 because commute is so horrible. It was frustrating to see colleagues look through the window to see if the train is here or late, or if they'd have to improvize a way to go home. It sucked not to have the last bus at 1600. It also sucked to spend 3 hours in the morning to get to the office either because of traffic, or because a train was late, or something like that. That uncertainty is draining. Therefore, we needed to be able to work remotely, which lead us to equip ourselves with the necessary tools, and change the way we were doing things as a tiny team. The proverbial "big powerful workstation with GPU and RAM", or SSHing into things. Our PhD colleagues who kept us at the state of the art still needed to tap on someone's shoulder ask them to troubleshoot driver issues on their "powerful laptop", or set up their environments, or deploy their models. We wated to spare them that, and enable them to be more autonomous (like deploy their own models). These anecdotal issues that lead to changing practices are funny, and pretty much widespread. What's interesting is that there are large technology organizations with a strong "AI" activity, but they are so large that there are departments withing that very tech "AI" organization that do not reap the reward of other departments' experience, haven't talked to them, or aren't aware of their experience curve or their tooling. It's fascinating. - [0]: https://news.ycombinator.com/item?id=28777589 https://news.ycombinator.com/item?id=28777589