4 ms·
It would be very enjoyable to talk further about this. One reason I am hesitant to invest too much time into a prediction-focused side project is that I believe
by p4wnc6 11y ago
It would be very enjoyable to talk further about this. One reason I am hesitant to invest too much time into a prediction-focused side project is that I believe many organizations that purportedly use machine learning actually do not care about prediction, as opposed to marketing or demoware.
A major influence on me is a 2011 Cato Unbound article called "Who Cares About Forecast Accuracy" by Robin Hanson [0].
I actually only found the article a few months ago, but it crystallizes the experiences I've lived through for the past 5 years perfectly.
Since the social status concerns of executives are typically the primary concerns within any organization, even if a better forecast can add value in some ways, if it risks taking value away in the social status realm, everyone will veto it.
As a result, people are often hired purely for credential (this has happened to me before). Then you are asked to do things that are directly damaging to your own career because they take you so far away from your area of expertise or your specialization that they make it harder for you to be hired by anyone else.
I have not yet encountered any organization which actually engages in fundamental research about improving forecast accuracy or truly solving problems with A.I. or machine learning.
Instead, I've only encountered places that put a thick wrapper of marketing around a thin kernel of simplistic technology, and then spend all their efforts on scaling up the marketing and sales apparatuses around the fundamentally-simple thing. (Simple things are often perfectly fine, of course. But in these situations, the simplicity was not chosen because it was a cost-effective way to meet business needs. Rather, it was chosen because it was cheap and easier to politically control. And after making the choice, no further effort was ever spent investigating whether any type of alternative could improve upon the simple thing in a cost-saving way.)
For a long time, I was very bitter about this. But then I realized that if this is the only mode of employment for machine learning researchers and engineers, it might still be OK. If the other aspects of the job afford you ways of meeting other goals, then perhaps it's OK to sacrifice the aspect of intellectual curiosity.
Unfortunately for me, the other goals I have include having adequate privacy at work, focusing on software craftsmanship and being proud of things that I make professionally, and maintaining a healthy work/life balance. And it seems these things, like intellectual curiosity, are also mostly impossible to find in modern technology jobs.
At any rate, I am not convinced that it would be a wise investment of time to conduct this type of side project. One bad outcome would be for it to embody sort of cargo-cult data journalism practices, like mining for t-stats or something and milking a misrepresentative headline for a bunch of blog hits. An even worse outcome would be to actually solve a problem really well, parlay it into a job offer, and find you have been hired purely as office decoration, to produce demoware or increase value for a potential acqui-hire or whatever.
[0] http://www.cato-unbound.org/2011/07/13/robin-hanson/who-cares-about-forecast-accuracy http://www.cato-unbound.org/2011/07/13/robin-hanson/who-care...
- partisan 11y agoI don't see you throwing yourself a pity party or anything so don't misunderstand me, but I feel like you have painted yourself into the corner in which you find yourself. You are absolutely right about what you observe in the world of technology. Is this an insoluble problem? Probably not, but I think you might find that a bit of flexibility will not only improve your chances of being hired, but also improve your chances of finding a place that will motivate you and bring you some level of satisfaction. Compromise on some things in the short term to join an organization where you _could_ one day be happy. Then, bring change in a positive way so that your co-workers embrace it. It has to start small and it takes time, but one day, you just might look back and realize that you were actually satisfied with what you were working on, with whom you were working, and why you were doing it. Sadly, that is probably at around the time, or after you decide you need more and then you can leave and do it again. I found myself in a similar situation with short stints on my resume just like yours. I eventually learned that there was no perfect job, but that I had created a pretty good one over time.
- p4wnc6 11y agoI am not sure how to approach this. I have worked in a variety of conditions and I have some idea of which things are merely just not my favorite, but easy to deal with, vs. which things are outright dealbreakers. For example, I remember the joy I felt when I first read the passage of Paul Graham's essay "Great Hackers" in which he describes the notion of placing software developers in a noisy open area as similar to making painters work in a factory where the air is full of soot. For me personally, working in a noisy, non-private space is simply not possible. I'd like to compromise, but it's borderline physical pathology with me. Wearing headphones doesn't work. The most significant thing I need in order to provide the business with the productivity they hired me to give them is privacy. But at the same time, I don't like working from home. I love collaborating and having an "open door policy" to talk with and assist coworkers with anything I can. I just need a door that shuts so I can protect some of my time to use for the more thought-intensive and creativity-focused aspects of my work. For me, there's no way to compromise on this. I've worked in cubicle banks, open-plan shared desks, shared bullpens, etc. etc. Even though I still get high marks from my boss, it drives me mad and causes me to inescapably resent showing up at work every day. Almost to the point where I'm starting to wonder if I have misophonia. Anyway, this singular issue (finding a private office) seems like it is the single hardest thing in the software industry. I speculate this is because businesses want to confer lowered status on the activities of engineers, but whatever it is, it makes the process of interviewing pretty maddening. I've had several places get to the point of making me a job offer, and then tell me that it is impossible for them to negotiate a private office for me. They have offered to raise the pay, add more vacation, and many other things. But I have to turn them down, for my own health. But it really begs the question... why from a business point of view is it so hard to provide some employees (who plausibly need it) with privacy. Why is employment for our field so overwhelmingly dominated by open-plan spaces, that people are willing to offer more money to attract someone but not at all willing to compromise on the type of physical space you can inhabit while working? Anyway, for me is suggests that it's not so easy as "just compromise." There's a lot that I'd be happy to compromise on. I'll work in a wide range of languages. I can work in Windows. I can work long hours. I'm willing to live pretty much anywhere. But what I can't compromise is the need for private space ... which seems to be the one thing that essentially no one provides.