7 ms·

On Software Tooling

113 points by jugjug 8y ago

mateus1 8y ago
I like this. I've gone really far with just Google Query or a simple SQL query. Another idea I like is not to "fall in love" with your analysis, per se. I've seen plenty of extrapolations and wonky charts that end up just being a waste of time.
arnon 8y ago
A nice poem, and it may be true in some cases, but not a universal truth. When you're a telecom, and you have more than 40 million subscribers, a whiteboard may be a bit limiting.
- 131012 8y ago
  Yeah I feel it is thought provoking, but mostly false. There's no way you can replicate (efficiently) a whiteboard.
- voidhorse 8y ago
  Good point. The piece is sound in the abstract and is good advice to recall when you consider the essence of your endeavors, but tooling does count when you consider the material situation—some approaches become untenable at certain magnitudes. Funily enough, the best we can do is often trade a big problem for a smaller one—« how do we serve X million people » becomes « how do we teach x thousand employees to use the tool that enables us to serve X million people «
- Cthulhu_ 8y ago
  Well yeah, but before you invest millions in a BI tool, you should decide what you want to do with it first - you can work out what data and statistics you'd like to have in a smaller scale on a whiteboard first. Investing in a tool without first doing the task by hand is wasteful. No I'm not saying you should do it with all 40 million subscribers, just start with getting one or two and a whiteboard, then a few dozen or hundred in an excel sheet.
- onion2k 8y ago
  When you're a telecom, and you have more than 40 million subscribers, a whiteboard may be a bit limiting. The idea that your problem is too big for a whiteboard shows that you're trying to solve too much of the problem at once.
- beat 8y ago
  Don't conflate data with process. It's fair to say you can't process 40 million bits of data on a whiteboard. On the other hand, if you have 40 million bits of data, but can't work out how to answer the question you're asking of it (or in many cases, what question you're trying to ask!), then it's useless. This is where whiteboards, paper, and other physical solutions really shine. You can make notes for yourself or communicate with those in your shared physical space in a very immediate, speed-of-thought way. In general, software (even BI software) is limited to doing what it is programmed to do. We necessarily adapt our thinking to the tools we have to express the ideas we have. Dancing about architecture is difficult, as Steve Martin pointed out.
safgasCVS 8y ago
Earlier this morning I spent 45 minutes in a meeting discussing the replacement for a $4 million big data system we bought 2 years ago because no one was using it. The solution discussed wasnt to ask the analysts why no one was using it but to spend $20k on POCs for two competing systems each costing a further $5mil as their on-paper specs are marginally superior. DAMMIT THEY WILL BE MADE TO READ THIS LINK
hangtwenty 8y ago
My head is nodding at the intention here, but it's important to remember this reactionary attitude can be damaging as well. That is missing from the post. Specifically, reactionary conservatism about tools and services can be a competitive disadvantage. In several workplaces I have seen this attitude co-mingle with pride in original code, and result in a counter-productive NIH (Not Invented Here)[0] attitude (reinventing wheels). Macho/heroism makes it worse. While trend-following can be inefficient, if the engineers are both excited and humble and ready to weigh many trade-offs, this inefficiency might be "cheaper" than reinventing wheels for all problems. Cue platitude: it takes balance :) [0]: https://en.wikipedia.org/wiki/Not_invented_here https://en.wikipedia.org/wiki/Not_invented_here --------------------------------------------------------- EDIT: To clarify I'm talking especially about _uninteresting problems,_ for which there may be an obvious off-the-shelf solution, such as when I've seen engineers invent a build system from scratch (and then cost _lots_ over years as that custom thing needs maintenance) ... when their needs could have been met by the language's standard tooling and using its plugin architecture. Or, when "just use sticky notes and a whiteboard" gets so stubborn that a growing team stays in chaos for lack of organization. On the other hand, it is quite bad when trend-following causes not just churn (switching tools), but overkill that introduces tons of cost and risk. i.e. distributed systems where you didn't need one. Relevant on this point: You Are Not Google by Ozan Onay https://blog.bradfieldcs.com/you-are-not-google-84912cf44afb https://blog.bradfieldcs.com/you-are-not-google-84912cf44afb
- jmull 8y ago
  I took this maybe in a different way. I don't think the point is, "no need for so much software tooling" but rather, "software tooling by itself isn't sufficient." (And that "underlying hard-work, the focus on stuff that matters, and courage to make a decision" are much more significant.)
- nine_k 8y ago
  The point of the text is that a need, and an understanding precede the use of any tools. The use of a tool without a need or an understanding does not help at best, and is harmful pretty often. You do not become a photographer by buying a lot of gear. You train your eye, study, and maybe buy gear eventually when you know why you need it. You do not become a software developer buy buying a bunch of supercomputers. You study, you start with smallest things, and maybe you buy hardware eventually, or deploy a lot of cloud instances, when you know why you need it. You do not become a successful company by quickly raising tons of capital. You study, learn about the markets and the needs of your prospective customers, start with building rather simple things, and maybe eventually you take hefty investments, when you know why you need it. You can go on and on. The old adage is to use the right tool for the job. This begins with understanding the job, and understanding the need of tools. Then you can choose the tools. Starting with tools, especially expensive tools, and believing that the use of these tools will automatically help achieve success, is a mistake.
teilo 8y ago
Process first. Then Automation. Put another way: Commitment and accountability to a process must precede the tool that implements said process.
- Jtsummers 8y ago
  If you charge enough, maybe I can get you to come in and consult with my bosses. I've been saying that for years and they keep mixing up the order. Now they've automated the wrong processes using custom in-house tooling and don't know why they're getting strange numbers (people fake the data to fit the tool, but do something totally different).
  - teilo 8y ago
    I wish it were that easy. But unfortunately, if the bosses aren't already asking these questions, they probably can't be consulted into it. They might bring a consultant in, and nod away at his analysis. They may write some memos, talk about it in the manager meetings. But that's it. Nothing changes. Management defines culture. Culture is a function of personality. Personality rarely changes. That is why leadership matters so much. That is why we have seen companies stuck in a rut for years, but then radically transformed when a new CEO is brought in. Or vice versa, companies that are doing well, and then tank under new leadership. But hey, if you can talk them into flying a middle-market CIO in to give them a come-to-jesus speech, I'll be happy to take the money.
- petermcneeley 8y ago
  Perhaps your experience is different but to me the emphasis on process is a focus on symptoms. What I have seen is process is the attempted cure to bad employees or incoherent culture. Good employees just do the right things.
  - kthejoker2 8y ago
    I could literally rewrite your comment swapping "process" for "people" and it would resonate with others. People, process, technology. They are 3 pillars. You cannot build without all 3 working in sync. Good employees can only do the right things with the right technologies - which may be a whiteboard and a notepad - and the right processes so that they know what the "right thing" is.
marcinzm 8y ago
I agree in principle but I've first hand seen what happens when a company that has grown doesn't get the tools it needs. A ticketing system, for example, is how you create a unified view of what matters and what should be worked on. Otherwise it becomes a game of politics and social cunning down to the IC level which is not how I want to spend 8 hours of my day.
draw_down 8y ago
Okay.
DesHacker 8y ago
Pragmatism v/s Idealism. We live in the real world. Your 'picture-perfect life on FB' is not perfect at all.
alangpierce 8y ago
I think maybe there's some nuance here to how to think about tooling. A data science team without any sort of data warehouse is probably going to be ineffective. Maybe one way to think about it: some tools try to model and streamline the things you're already doing, and some tools expand the set of things you're able to do. I think the first type is the one to be more skeptical of. Face-to-face conversations, whiteboards, and Excel sheets are all extremely flexible and powerful. It's possible for tools to improve on these, but it's a high bar.
nlawalker 8y ago
From a different perspective: Important companies don't do data science with pen and paper. Important data-science teams don't work on whiteboards. Important studies aren't presented in notepad. What counts is perception, the focus on image, and the courage to spend big to prove value.
- s3m4j 8y ago
  It's the same as "resume driven development" but at enterprise scale ?
  - deleted 8y ago
    [deleted]

113 points by jugjug 8y ago

I like this. I've gone really far with just Google Query or a simple SQL query. Another idea I like is not to "fall in love" with your analysis, per se. I've seen plenty of extrapolations and wonky charts that end up just being a waste of time.

arnon 8y ago

A nice poem, and it may be true in some cases, but not a universal truth. When you're a telecom, and you have more than 40 million subscribers, a whiteboard may be a bit limiting.

131012 8y ago
Yeah I feel it is thought provoking, but mostly false. There's no way you can replicate (efficiently) a whiteboard.
voidhorse 8y ago
Good point. The piece is sound in the abstract and is good advice to recall when you consider the essence of your endeavors, but tooling does count when you consider the material situation—some approaches become untenable at certain magnitudes. Funily enough, the best we can do is often trade a big problem for a smaller one—« how do we serve X million people » becomes « how do we teach x thousand employees to use the tool that enables us to serve X million people «
Cthulhu_ 8y ago
Well yeah, but before you invest millions in a BI tool, you should decide what you want to do with it first - you can work out what data and statistics you'd like to have in a smaller scale on a whiteboard first. Investing in a tool without first doing the task by hand is wasteful. No I'm not saying you should do it with all 40 million subscribers, just start with getting one or two and a whiteboard, then a few dozen or hundred in an excel sheet.
onion2k 8y ago
When you're a telecom, and you have more than 40 million subscribers, a whiteboard may be a bit limiting. The idea that your problem is too big for a whiteboard shows that you're trying to solve too much of the problem at once.
beat 8y ago
Don't conflate data with process. It's fair to say you can't process 40 million bits of data on a whiteboard. On the other hand, if you have 40 million bits of data, but can't work out how to answer the question you're asking of it (or in many cases, what question you're trying to ask!), then it's useless. This is where whiteboards, paper, and other physical solutions really shine. You can make notes for yourself or communicate with those in your shared physical space in a very immediate, speed-of-thought way. In general, software (even BI software) is limited to doing what it is programmed to do. We necessarily adapt our thinking to the tools we have to express the ideas we have. Dancing about architecture is difficult, as Steve Martin pointed out.

safgasCVS 8y ago

Earlier this morning I spent 45 minutes in a meeting discussing the replacement for a $4 million big data system we bought 2 years ago because no one was using it. The solution discussed wasnt to ask the analysts why no one was using it but to spend $20k on POCs for two competing systems each costing a further $5mil as their on-paper specs are marginally superior. DAMMIT THEY WILL BE MADE TO READ THIS LINK

hangtwenty 8y ago

My head is nodding at the intention here, but it's important to remember this reactionary attitude can be damaging as well. That is missing from the post. Specifically, reactionary conservatism about tools and services can be a competitive disadvantage. In several workplaces I have seen this attitude co-mingle with pride in original code, and result in a counter-productive NIH (Not Invented Here)[0] attitude (reinventing wheels). Macho/heroism makes it worse. While trend-following can be inefficient, if the engineers are both excited and humble and ready to weigh many trade-offs, this inefficiency might be "cheaper" than reinventing wheels for all problems. Cue platitude: it takes balance :) [0]: https://en.wikipedia.org/wiki/Not_invented_here https://en.wikipedia.org/wiki/Not_invented_here --------------------------------------------------------- EDIT: To clarify I'm talking especially about _uninteresting problems,_ for which there may be an obvious off-the-shelf solution, such as when I've seen engineers invent a build system from scratch (and then cost _lots_ over years as that custom thing needs maintenance) ... when their needs could have been met by the language's standard tooling and using its plugin architecture. Or, when "just use sticky notes and a whiteboard" gets so stubborn that a growing team stays in chaos for lack of organization. On the other hand, it is quite bad when trend-following causes not just churn (switching tools), but overkill that introduces tons of cost and risk. i.e. distributed systems where you didn't need one. Relevant on this point: You Are Not Google by Ozan Onay https://blog.bradfieldcs.com/you-are-not-google-84912cf44afb https://blog.bradfieldcs.com/you-are-not-google-84912cf44afb

jmull 8y ago
I took this maybe in a different way. I don't think the point is, "no need for so much software tooling" but rather, "software tooling by itself isn't sufficient." (And that "underlying hard-work, the focus on stuff that matters, and courage to make a decision" are much more significant.)
nine_k 8y ago
The point of the text is that a need, and an understanding precede the use of any tools. The use of a tool without a need or an understanding does not help at best, and is harmful pretty often. You do not become a photographer by buying a lot of gear. You train your eye, study, and maybe buy gear eventually when you know why you need it. You do not become a software developer buy buying a bunch of supercomputers. You study, you start with smallest things, and maybe you buy hardware eventually, or deploy a lot of cloud instances, when you know why you need it. You do not become a successful company by quickly raising tons of capital. You study, learn about the markets and the needs of your prospective customers, start with building rather simple things, and maybe eventually you take hefty investments, when you know why you need it. You can go on and on. The old adage is to use the right tool for the job. This begins with understanding the job, and understanding the need of tools. Then you can choose the tools. Starting with tools, especially expensive tools, and believing that the use of these tools will automatically help achieve success, is a mistake.

teilo 8y ago

Process first. Then Automation. Put another way: Commitment and accountability to a process must precede the tool that implements said process.

Jtsummers 8y ago
If you charge enough, maybe I can get you to come in and consult with my bosses. I've been saying that for years and they keep mixing up the order. Now they've automated the wrong processes using custom in-house tooling and don't know why they're getting strange numbers (people fake the data to fit the tool, but do something totally different).
- teilo 8y ago
  I wish it were that easy. But unfortunately, if the bosses aren't already asking these questions, they probably can't be consulted into it. They might bring a consultant in, and nod away at his analysis. They may write some memos, talk about it in the manager meetings. But that's it. Nothing changes. Management defines culture. Culture is a function of personality. Personality rarely changes. That is why leadership matters so much. That is why we have seen companies stuck in a rut for years, but then radically transformed when a new CEO is brought in. Or vice versa, companies that are doing well, and then tank under new leadership. But hey, if you can talk them into flying a middle-market CIO in to give them a come-to-jesus speech, I'll be happy to take the money.
petermcneeley 8y ago
Perhaps your experience is different but to me the emphasis on process is a focus on symptoms. What I have seen is process is the attempted cure to bad employees or incoherent culture. Good employees just do the right things.
- kthejoker2 8y ago
  I could literally rewrite your comment swapping "process" for "people" and it would resonate with others. People, process, technology. They are 3 pillars. You cannot build without all 3 working in sync. Good employees can only do the right things with the right technologies - which may be a whiteboard and a notepad - and the right processes so that they know what the "right thing" is.

marcinzm 8y ago

I agree in principle but I've first hand seen what happens when a company that has grown doesn't get the tools it needs. A ticketing system, for example, is how you create a unified view of what matters and what should be worked on. Otherwise it becomes a game of politics and social cunning down to the IC level which is not how I want to spend 8 hours of my day.

draw_down 8y ago

Okay.

DesHacker 8y ago

Pragmatism v/s Idealism. We live in the real world. Your 'picture-perfect life on FB' is not perfect at all.

alangpierce 8y ago

I think maybe there's some nuance here to how to think about tooling. A data science team without any sort of data warehouse is probably going to be ineffective. Maybe one way to think about it: some tools try to model and streamline the things you're already doing, and some tools expand the set of things you're able to do. I think the first type is the one to be more skeptical of. Face-to-face conversations, whiteboards, and Excel sheets are all extremely flexible and powerful. It's possible for tools to improve on these, but it's a high bar.

nlawalker 8y ago

From a different perspective: Important companies don't do data science with pen and paper. Important data-science teams don't work on whiteboards. Important studies aren't presented in notepad. What counts is perception, the focus on image, and the courage to spend big to prove value.

s3m4j 8y ago
It's the same as "resume driven development" but at enterprise scale ?
- deleted 8y ago
  [deleted]