6 ms·
I was once the CTO of a financial services firm that had been founded by a non-technical person. When I was hired the code base was in poor shape. With a bit of
by browningstreet 2y ago
I was once the CTO of a financial services firm that had been founded by a non-technical person. When I was hired the code base was in poor shape. With a bit of guidance we overhauled the codebase in 3 months and swept clean about 15 years of technical debt. They were using a 13 year old version of the compiler.
New clients were requiring audits and attestations we wouldn’t have been able to pass.
I was able to hire a few seasoned engineers to begin working on the next pass over the monolithic core design. We needed to prepare for a container and cloud based future, given where our clients were going.
The founder just complained and complained. He couldn’t accept that we were working on things the client couldn’t see. I spent a lot of time negotiating for a percentage of total velocity for core design changes.. the rest of our time and effort would be focused on things the customers were demanding and which the founder thought we should focus on.
We had increased velocity about 2000% already. He wanted 4000%. We needed sprints for back end redeployments. Founder was visibly angry when he overheard us taking about refactoring back end things like deployment scripts and data architecture.
Senior engineer gave him a copy of The Cathedral and the Bazaar, just to share something that could lead to some kind of common vocabulary. Founder spent weekend marking up his copy and on Monday firmly announced that he rejected the text in whole.
Senior engineer quit on the spot and walked out.
- systems 2y ago"put the donkey, where the owner of the donkey wants you to put the donkey" - egyptian proverb i dont know how much money this company was making, and why it couldnt invest in both backend and client facing communications but, on the other hand, being client focused and continuously delivering client facing feature, might be the thing that keeps the revenu streams coming in other words, i used to always take the dev side in stories like this, but overtime i learn to be more sympathetic to the clients, i have been in situations where the devs overestimated the value of cleaner architecture, its a financial service, not a software company, i think the real solution would have been to completely outsource the software development
- gavinhoward 2y agoI think you are both right. But I don't think GP was ignoring clients. If anything, he was doing what those clients wanted by moving to containers and a cloud architecture. And I say that as someone that doesn't like those things.
- evilduck 2y agoI've never seen outsourcing of a company's core product result in anything but litigation. If a team legitimately made a 2000% leap in throughput and it wasn't just from manipulating metrics, and then that still wasn't good enough then nothing will be satisfactory in the eyes of that owner, you rightfully should give up that battle and find greener pastures. Expecting someone to come in and wave a magic wand of 4000% productivity is absurd. Expecting that you can outsource that sort of demand is plain stupid.
- rqtwteye 2y agoThere are a lot of managers who believe you should always put a squeeze on devs and keep the org in constant panic mode. Nothing will ever be good enough. If you achieve one goal, you get hit with another crazy deadline. It never ends.
- bluefirebrand 2y ago> but, on the other hand, being client focused and continuously delivering client facing feature, might be the thing that keeps the revenu streams coming I firmly reject this. If you can't build quality then you don't deserve to be in business I'm so sick of the fact that everyone is ok accepting "just barely passable" It has lead to everything new being built is crap. New housing is crap, new furniture is crap, new software is crap I get that building nice things is expensive and takes time, but I want to live in a world of nice things instead of this constant race to the bottom, produce endless quantities of crap world that we are in right now Doesn't anyone else feel like we're being crushed under all this crap we're building?
- 2y ago
- bmitc 2y agoWhat is The Cathedral and the Bazaar about? I couldn't quite tell from a cursory glance.
- gavinhoward 2y agoDisclaimer: been a while since I read it. It compares two software development styles. It compares typical software dev in a company to building a cathedral: directions come from on high, and you execute your task. Then it compares typical Open Source development, such as with the Linux kernel, and compares it to a bazaar: everyone is doing their own thing, but somehow, things get done. It's old, and maybe a little outdated now that the corporate world has taken over Open Source, but I would still suggest reading it.
- esafak 2y agohttps://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar
- bmitc 2y agoYea, but I guess I was looking for something deeper, because at the outset, I'd be a little offput by the suggestion to read it and any disagreement about my disagreement on it. And I fully understand the issues described by the above parent. The book comes off as a bit opinionated with the blinders of a Linux kernel development and 80-90s attitude about open source software. That's a pretty narrow focus of software development. For example: > Every good work of software starts by scratching a developer's personal itch. I've written good software professionally, but none of it was kickstarted by my own personal itch.
- spit2wind 2y agoCatB is a fine book. That feels like such add choice for the situation, aside from just seeing if the person was willing to engage with ideas within software. I'm curious what texts people would suggest for a non-developer to get some insight. The ones that come to my mind are the Mythical Man Month, Peopleware, Facts and Fallacies of Software Engineering by Glass, or maybe even the 1968 NATO report on software engineering.
- bluefirebrand 2y agoEvery now and then I have to ask my PM at work "If I give you a second copy of the Mythical Man Month, could you read it twice as fast"? He doesn't like that question, but it does get the point across
- jiggawatts 2y agoI’m stealing that quote for my own purposes. I hope you don’t mind.
- q7xvh97o2pDhNrh 2y agoWhat you really need to do is schedule daily status meetings, a longer weekly status meeting, and -- this is the real velocity trick -- monthly OKR reviews as well as quarterly slide presentations. It's important the slide deck be polished for all the people who want to come learn about the project, so it makes sense to spend at least a month preparing it. And the neat thing is that all this work will get you lots of suggestions from executives about how you could address all the velocity issues that your project seems to have all the time! So you'd better budget 2-4 weeks after the presentation to follow up, have stakeholder meetings, and make sure everyone really feels heard. With just these few simple techniques, you too can get your engineers moving at the blinding speed of an average FAANG engineering team.
- whilenot-dev 2y ago6 years ago I gave a C-level guy a fresh copy of Perform or Else[0] on my way out. I don't know if he even read it. [0]: https://www.routledge.com/Perform-or-Else-From-Discipline-to-Performance/McKenzie/p/book/9780415247689 https://www.routledge.com/Perform-or-Else-From-Discipline-to...
- mattm 2y agoHow did you measure velocity?
- cowanon44 2y agoI find Microsoft's 5 Pillars in their Well-Formed Architecture to be a useful way to explain this to non-technical people. At the start of a project, fully focusing on your features makes sense. But as a project grows, non-functional concerns can sabotage the business model if not addressed. - Reliability - Security - Cost optimization - Performance Efficiency - Operational Excellence The first four have a direct impact on the customer. Buggy or insecure software, or excess costs, will all affect the customer directly. The fifth point is about devops, and making sure that we can continuously (as a process) meet the customer's needs in all of these areas. I've found that if I can fit refactoring into one of these pillars, it is much easier to explain to non-technical people. Of course, some people just can't be helped, and aren't open to alternate ways to think about software projects.
- cess11 2y agoI think that was a good move by the senior engineer. In part because I've done something similar and want to believe it's the right thing to do. Did it change anything? Did the founder manage to keep things going?
- a_c 2y agoIt works in everyone’s favour. Founder got what his culture sets out to build. Customer got their features faster. Senior engineer clearly knows it is not the place for him and moved on. Everyone wins.
- jprete 2y agoProbably feature development slowed to a crawl over a few years. Code debt isn't actually an abstract thing, it's just hard to explain to non-coders the thousand little slowdowns it injects into every project.
- a_c 2y agoWhat I meant was the founder got what he asked for. Short term gain at the expense of sustainability. It is a feature rather than a bug. For a company looking for product market fit, "good code" might take a back seat. Andy Grove mentioned that an executive job is to, paraphrased, maintain the company culture. His context was about larger established companies. But I think the point still holds. It is the founder's job to sell his vision to his employees. It is the market's job to determine whether such vision thrives.
- chriscbr 2y agoThis is blatant conservatism, and it's a mentality which leads to dead players. The job of an executive is to make good decisions; it's in the name: execut-ive. To decide always in the direction of the existent culture is a very peculiar bias which is only successful in certain situations.
- jsrcout 2y agoA friend calls it the Logjam Principle. There are always a few logs floating down the river and that's not really a problem. But a small logjam of just a few logs will grow over time until the entire river is obstructed. It's better not to let things get to that state.
- naasking 2y ago> Customer got their features faster. Did they though?
- FrustratedMonky 2y agoShouldn't the free market drive companies with these type of leaders down ? They should become swamped by the faster more agile competitors? But sadly, it seems the inertia keeps some pretty ignorant organizations functioning for decades.
- jprete 2y agoThe problem is that the free market only really works when all information is available to all parties and there's low switching cost. So it works decently for commodities, but no business ever wants to sell a commodity. In fact most business advice I've seen is about how to get away from selling commodified products into a free market and instead sell opaquely differentiated products to a locked-in market with high switching costs and barriers to entry.
- esafak 2y agoPeter Thiel: "Competition is for losers."
- randomdata 2y agoPerhaps, but there is nothing free market about financial services.
- jimbokun 2y agoYes. This is why you should immediately start interviewing when you observe this kind of behavior coming from management.
- mcguire 2y agoEventually, yes. Unless they sell to another company and the technical debt goes into the trash with the company's product. On the other hand, "the market can remain irrational longer than you can remain solvent."
- datadrivenangel 2y agoWould it be ethical in a situation like this to hide all the technical improvements and just show the increased speed of client facing delivery? Seems like everyone would be happier.
- HeyLaughingBoy 2y agoUsing the word "hide" in any scenario should immediately get your spidey-sense tingling.
- arcbyte 2y agoI'm always telling my developers to stop talking to the customer about technical details. Do you think carpenters talk to their customers about types of nails and dimensional lumber? Or they talk curves, size, and color?
- jimbokun 2y agoThis puts me in mind of a home owner on one of those renovations shows where the workers discover a structural flaw mid way through the job, and explain this means they don't have as much in the budget now for added features. The equivalent of the boss in this story would demand to leave the structural issues unaddressed to keep the marble counter tops in the budget, and then the house would cave in a year later.
- eurekin 2y agoThat reminds me of one case, where an author of software saw any improvement or genuine feedback as a personal attack - directed at him and his "legacy". Can't go into specifics, but only features directly or indirectly praising the original software were allowed. I was student/intern at the time and learned about that the hard way. Still, one of most important teaching moment to this day.
- avmich 2y ago> Senior engineer gave him a copy of The Cathedral and the Bazaar, just to share something that could lead to some kind of common vocabulary. Founder spent weekend marking up his copy and on Monday I'd quite like to read those markings. Do you have them?
- ak_111 2y agoI have a meta question here, am always curious when seemingly competent engineer such as yourself put up with this kind of management. It is unbearable to work for someone who doesn't appreciate your work let alone belittles truly hard-earned achievements, for example that statement when he demanded you get 4000% rather than 2000% that you achieved would have made me quit on the spot. Further from the sounds of it this sounds like a random finance company and with your skills you could have easily walked into any tech/finance company so I am curious why you remained at this company and didn't quit sooner?
- ryandrake 2y agoNot OP, but I worked for a similar founder. The only work he wanted software engineers to do was writing code, and not just that... only 1. code that added a customer visible feature or 2. code that fixed a major, customer-facing bug that a customer actually complained about. Fixes for bugs not discovered by customers? No. Performance improvements? No, not even visible ones. Refactorings? No. Technical debt cleanup? No. Build speedups? No. Update the code to work on a recent compiler? No. It gets better! Version control? No. Bug tracker? No. Unit tests? No. I had to implement the above three on my free time. Standing up dedicated build and test infrastructure? LOL get real. Eventually, I was able to argue for a few of these, but it was always a fight. The reason I stayed was that the hiring market is not always that great, and it tends to be much harder to find a new job than HN would have you believe. This idea that most of us can simply "walk into any" company and get a job is fiction in all but the hottest job markets.
- Kache 2y agoI don't understand these kinds of leadership/management. What they want clearly isn't a tech company. They don't want technical wisdom, insight, nor vision. Why bother hiring in-house SWE's at all? They should just stick with consultants and contactors.
- browningstreet 2y agoKeeping it slightly abstract because enough people can figure out company/time/place from what I wrote… I had something to accomplish before I left. I made a mark in the niche we were operating in. Beyond that I’m reluctant to be more specific.
- swozey 2y agoAhh yes, reading this as an SRE. This is the average Ops team experience, having to find ways to prove that we're not a cost center (I cut out AWS bill down 40% last year migrating to ARM64) because your work isn't blasted on the front page of a product changelog. And things... "just work," Until they don't. And they did because of the front loaded work we did correctly to keep things working.