9 ms·

I Failed to Transform the Enterprise

151 points by anonyfox 5y ago

ask_b123 5y ago
So, if I'm understanding correctly, the author used Lisp in this story?
- tartoran 5y ago
  I figure you’ve changed tabs and read a different story. No mentions of Lisp here
  - gknoy 5y ago
    One of the few hyperlinks is to an article by PG about how using Lisp enabled his small team to build amazing stuff that ran circles around larger teams that were using other languages. Based on that link, with no other knowledge, I also assumed that meant the author might have used Lisp, but it also makes sense in the "higher order language", or even just "writing custom code to solve problems" sense.
- htrp 5y ago
  I think it's more related to the language power continuum.... he was probably coding in a "more powerful language" while the rest of the org was coding in a less powerful one. If I had to guess Rust vs Java/C/Perl?
- Floegipoky 5y ago
  The article mentions Phoenix, so probably Elixir.
sqldba 5y ago
So true. Seen situations so much like this.
darkerside 5y ago
Interesting tone. The tone says, all these people are idiots, and they failed to understand my beautiful creations. But he then outlines all his failings at the bottom, which I agree with, so he clearly knows better. It's it possible that this tone deaf attitude was a primary culprit in the downfall of the organization he and his handpicked successor built?
- marcinzm 5y ago
  You can't build something in a vacuum when there's other people with power in the organization. You need to treat the system as a whole including the politics involved. Sometimes that mean a less optimal solution to gain some influence with key stakeholders. This is one reason technologists can make bad managers. They don't think about the politics or don't want to play it.
- rocqua 5y ago
  His tone isn't "you are all idiots". His tone is "you all don't care about real progress, just selfish politics". The exasperation comes from total breakdown of the internal story "we are all here to make the company better". Whilst at the same time people keep repeating that line.
  - darkerside 5y ago
    Yes, you state it better, and my characterization was a bit facetious. The common thread is that, he implies, or explicitly states, that he has the organization's best interests at heart and others don't. That might be true or not, but you can't operate with that mindset and expect people to come along with you. Whether or not it's true, you need to presume positive intent. Perhaps they see aspects of their jobs that won't automate well that you do not. Is that stubborn self interest, or standing up to a naive new leader who doesn't get it?
    - mkr-hn 5y ago
      The hardest and most important mindset shift for me was when I realized almost every person who supports terrible policies/politicians does so thinking they do it because they care, and they do from their particular perspective. I can't usually bridge the gap with someone who commits or supports horrors out of "concern for souls[like mine!]," for example, but most situations aren't at such a timeless impasse. Every family member who ever turned into a whiny baby while trying to do free IT for them saw themselves as the victim, or the person under threat. That's the kind of person with power in most organizations, and you have to find some way to meet them in the middle without compromising everything to get anything done. It's not enough to be right.
agomez314 5y ago
this link is returning me a 404
- michaelcampbell 5y ago
  Working fine for me.
  - agomez314 5y ago
    working now for me!
- ask_b123 5y ago
  https://archive.is/brKSf https://archive.is/brKSf
SuoDuanDao 5y ago
Wow, that sounds frustrating. I'm curious what the company was, sounds like a place to avoid.
- antoineMoPa 5y ago
  Based on his CV, probably Aida.de: https://anonyfox.com/cv https://anonyfox.com/cv
- jrochkind1 5y ago
  I think it could literally be just about any large company that's at least 15 years old and doesn't sell software.
michaelcampbell 5y ago
> The problem: to achieve the desired level of UX, we needed to convince stakeholders that a single-page-app approach would be needed, including some APIs to fuel them. I'd wager a SPA is a hammer/nail reaction here.
- anyonecancode 5y ago
  At one job I ran into this pretty badly. The team I joined had an internal app that was, basically, a bunch of forms connected to a database. It had a lot of problems -- the data models didn't actually map to what the business needed, the UI was unintuitive. My manager insisted that what we needed to do was rewrite the entire thing as a React SPA. I told him, well, we're just going to end up with the exact same problems, only now implemented in a different framework. This won't solve any of the actual problems. He insisted, though, saying that React was "modern." Thankfully, the depth of mismanagement here was pretty exceptional in my experience. In most cases, even if I'm personally skeptical of the need for an SPA I can see a reasonable argument in favor. The cartoonish straight out admitting that the sole purpose was to be more "modern" with literally zero regard for actually serving any user needs is thankfully not something I've run across too often.
  - mkr-hn 5y ago
    There are markets where being seen as old-fashioned is fatal, so this might not be irrational. Just because it's internal doesn't mean they're not worried about the wrong people seeing it in an old-fashioned state. People who worry about that (for good reason or not) don't always have a boundary on that concern, and sometimes internals leak.
    - anyonecancode 5y ago
      In fairness, his audience was his manager, not the app's users. So it was perfectly rationale for him to push for the relatively easy re-write instead of worrying about the app's users, given that his manager was not a user of the app. As an engineer who finds value in making things people actually find useful, though, it was a pretty dispiriting experience.
reilly3000 5y ago
One thing that vendors have that internal teams don’t is million dollar marketing budgets. Selling the sizzle of successful internal projects is its own kind if work, but if neglected even the greatest of projects will succumb to the onslaught of vendors. Good technical leaders recognize this inherently and celebrate their teams creations, non-technical leaders don’t and take pride in buying from big brands and the fancy luncheons that come along with them.
- netcan 5y ago
  Underrated comment. If you break down what most vendors and consultants do, in terms of either dollars or hours, you'll usually find that they mostly do sales, PR, client relationship, etc. Projects are designed and managed to make the right people look good to the right people, manage expectations, etc. They'll spend massive time giving non-tech executive buyers easy, fun choices.. let them feel strategic, visionary and empowered. They'll communicate the hell out of every piece of progress. Whoever hired them also has a massive incentive to buy into and sell onwards, that the project is a massive success. IE... if your job is to make a clients' Salesforce instance do a thing, 80% you your job is stuff other than making a clients' Salesforce instance do the thing. Internal dev teams will rarely have this pageantry. Meanwhile, no one outside that team knows what they do, how well they do it or such.
- ed_elliott_asc 5y ago
  This is one of the most insightful things I have ever read on HN.
  - jollybean 5y ago
    If you want the 'aha' moment have a look at most software vendors incomes statements. They spend 2x more on selling than they do on R&D. So when that Oracle Sales Guy is giving you the pretty talk, and taking you to lunch remember that all that money your sending to Oracle is literally mostly going to him! The product makers just get 'a cut'. Now consider how much of high tech is just that, sales and marketing, conferences and luncheons. That's a staggering waste for a developed economy. We are very inefficient with these things, we need better approach to IT.
yardie 5y ago
> Big solutions to the big problems. Exactly what consultancies sell. It was obvious to them that having any inhouse-developers is the biggest business risk at all. I will never, ever, ever, do in-house software development ever again. Non-software companies treat their inhouse developers like crap. They don't get it. I don't know when non-software businesses thought it was a good idea to move software developers in house. Most of the company don't want them there. Or deal with them begrudgingly. It's hard to quantify their revenue so they are treated as an expense not cost saving or revenue generating. And they are the first on the chopping block when things turn down. In my experience, if you are recruited by a company that doesn't sell software or software services you should tread very carefully, ask lots of questions. Sometimes you'll find out a 100 year old furniture company only has a dev team because the grandson thought it was a good idea. And then you run very far away.
- ProAm 5y ago
  > Big solutions to the big problems. Exactly what consultancies sell. The problem is most consultants are not good at what they do. It why the sales guys for the consulting company are not the devs. Most the time consultants come in for 3-6-12 months, cause some trouble and then just leave the mess to the company to address later. There are some good consultants but most are just people that couldn't work in house because they are not up to par for long term engagements of products that need to stand the test of time to add legitimate value. > It's hard to quantify their revenue so they are treated as an expense not cost saving or revenue generating. This depends on the business, most the time, unless you make software you are an expense. The company makes their money in different departments and making money is the name of the game. It's takes a different set of skills (albeit better/stronger, industry knowledge and communication skills) to be an in-house developer vs a contractor.
  - alex_anglin 5y ago
    >The company makes their money in different departments and making money is the name of the game. The challenge here is that under this model the different departments get credit for the revenue they generate. What their relationship is with the in-house dev team is very much something that one should be aware of if they choose to be in that situation. It's not too dissimilar a situation to every single 'change management' model that has executive commitment as the number one success factor.
lowercased 5y ago
Reminded of a couple of projects: 1. I worked with a single internal guy to build a web-based ordering system, in the days where people faxed in orders. This replaced weeks of data entry, dozens of part time data entry people, and enabled a big change in materials ordering and production (orders were all in a database within a few hours, vs literally weeks of keyed-in faxes from before). The one guy that was in charge of all this suddenly had a lot more power internally, and it pissed off a lot of other folks who suddenly had less influence. 2. More recently, worked with a team of external contractors, and internal folks were resistant to help any of us set up external dev systems. They all worked in the physical offices, and had no need for 'run a dev system on a laptop'. One of our guys spent a lot of time (and kept getting chastised by the client's mgt) for getting and documenting a build process for external contractors to use. The pandemic hits, office buildings are shut down, and internal folks have a much harder time figuring out... how to do anything from home. In both cases, under-the-radar skunkworks projects ended up having large (and in one case transformative) impact and value, but were ignored or fought by those with entrenched power.
- specialist 5y ago
  > ...fought by those with entrenched power. For decades now, per Alistair Cockburn, I assumed people just really, really hate change. There's some research which shows our selfish little brains have an immunological type reaction to change. Anything that threatens our internal stable equilibrium is squashed like an invader. No matter how much we say we want to change. No matter how much we may need to change. While wholly correct, turns out Cockburn was an optimist. We now know that some fraction of people would rather watch the whole world burn than give up one iota of power.
- sergius 5y ago
  A case of 'Innovator's Dilemma' perhaps?
wayoutthere 5y ago
There was an article earlier this week about why every engineer should do a stint in consulting. This is why. The reason companies like this use consulting companies is because the VPs want total control. Internal IT is purely a cost center, and unless your company (read: the business) is already on board with the product lifecycle op model, there is no point in trying to do any of this from within IT. Consultants work at the VP level and in a large enough company, we find a technical area (usually data as it has the biggest business returns) to modernize, then leverage success there build out the business case for the rest of the company (data is a good place to start building API gateways). And selfishly, the business is willing to pay higher hourly rates than internal IT (which we generally leave to the offshore MSPs). When I sell transformational solutions to a company, I know it won’t be successful without top-down buy-in. Consultants are also better positioned to do these transformations because we do them all the time; and the exercises around stakeholder management and consensus building are extremely important. They’re the kinds of things internal teams are never resourced or trained for, and they don’t happen quickly.
- htrp 5y ago
  The problem in consulting is that you almost always end up in an internal turf war. As mercs, you can pack up and leave once the paycheck runs out.....
  - pjmlp 5y ago
    Indeed, however for me personally being a merc is much more enjoyable, specially because each project is a different stack, so it easier to change than being labeled "X Developer". Plus thanks to those political games there are lots of soft skills to learn as well.
jf22 5y ago
> So while many team members were arguing about theoretical things, I onboarded important developers on to the new system and they were productive refactoring the old things we produced into the new technology stack. I question the wisdom of somebody who started refactoring and replacing systems right after massive layoffs in an industry that was crippled by the pandemic. Businesses in survival mode can't spend resources like this. The author points out the executives became enraged when the found this out and I agree with the executives.
- andix 5y ago
  The pandemic was also a chance to implement new stuff, as there was anyway no real revenue generated. Everything was chaotic, so if you made mistakes you could just brush them off with "because of covid".
- wsc981 5y ago
  > I question the wisdom of somebody who started refactoring and replacing systems right after massive layoffs in an industry that was crippled by the pandemic. Well wisdom is gained from experience and it seems the author learned a few lessons.
- lowercased 5y ago
  And... when everything failed except the new system, which seemed to keep the company operational despite the executives' views... you still agree with them? You agree with their 'enraged' state? You think it was justified? Even with hindsight, or just at that moment in time?
  - villasv 5y ago
    Refactoring by definition doesn’t fix things though so parent comment is still right
    - BoorishBears 5y ago
      What? By literal definition refactoring is fixing things without changing functionality... > refactoring the old things we produced into the new technology stack. You don't see how that could fix things?
Aeolun 5y ago
This is such a pain in the ass. You’d think that just providing a better solution would make people happy, but noooo. Do not encroach upon my power, you cretin! It doesn’t matter if the business dies because of my stupidity, as long as I go down captain of this ship!
- MattGaiser 5y ago
  I think a lot of it is that managers aren't really the captain, or at least lack the same expectations. They are more a contracted operations manager who can hop in a lifeboat first or can get paid more steadily if the ship runs aground and therefore they are more needed to clean up the mess.
- wsc981 5y ago
  Just from the point of view of some manager: perhaps worked many years at a company with the ultimate goal of entering C-suite. Maybe gave up other good opportunities for this goal. In such situations it must suck a lot to see your power and influence wane. You might have made many sacrifices and then someone starts some projects that could ruin everything you worked so hard for.
  - bmeski 5y ago
    Then learn to code. Edit: It's a legitimate solution. I've survived 3 reorgs at my company because I could code. I read up on the new incoming system and became effective.
    - deleted 5y ago
      [deleted]
  - Aeolun 5y ago
    > You might have made many sacrifices and then someone starts some projects that could ruin everything you worked so hard for. As annoying as that is, I kind of feel you should be above that. You have all right to be annoyed and skeptical, but not to sabotage the effort.
SPBS 5y ago
> And then the obvious thing happened: a complete breakdown of all important systems, like a house of cards. And: most vendors were not working due to lockdowns or other important issues during the pandemic. > The only system not affected by the complete meltdown was: our new one, which was designed to withstand outages from backend systems below. I don't understand, how can their SPA frontend still be functional when the backend is totally down? You can display things to the user, but you cannot do anything meaningful from it?
- jdlshore 5y ago
  Depends on what the backend systems were doing. They may have been data feeds that could be cached, with the “front-end” actually being a whole app with its own back end and database. Or perhaps taking orders and queueing them up for later delivery when the back-end systems came back on line.
Gunax 5y ago
Where's the ceo and board in all of this. Someone must actually care about the bottom line.
- mbrodersen 5y ago
  CEO’s care about their yearly bonus, making the majority share holders happy, and the next job opportunity. Everything else is secondary.
fridif 5y ago
I've been both in-house and consultant to a variety of big corps in a variety of industries, and I can say that invariably every single one is messed up and inefficient. Yet still somehow customers buy the products and (more often than not) the company delivers a product that is half working enough to be tolerated and used.
jmull 5y ago
Meh. When it seems like everyone is stupid but you... you are probably the one who is on the wrong track. This is all very vague, so who knows. But at the very least, I would guess the OP's attitude of I-know-better-than-everything-that-came-before arrogance plus ignorance of why things were the way they were made big failures inevitable. It's a real limitation -- though a common one -- to need everyone to see things from your perspective, rather than being able to understand their perspective.
- ikiris 5y ago
  a vast amount of people think horse dewormer is a viable strategy. If anything there is a crisis of "my ignorance is just as valuable as your experience", and not the other way around.
  - hobs 5y ago
    Absolutely, but if your technical solution was so definitely brilliant then it would have solved the problems you purport it magically solved. Generally its great, but not a panacea, and while it would have been improvements it would have suffered fun new problems. Source: been the young guy who thought if only they listened, and then they did.
  - imwillofficial 5y ago
    You mean like India and their hundreds of thousands of cured people?
    - scruffyherder 5y ago
      No he means when it killed Joe Rogan. RIP
      - deleted 5y ago
        [deleted]
      - imwillofficial 5y ago
        This must be awkward for you to read: 33 districts in Uttar Pradesh are now Covid-free: State govt Overall, the state has a total of 199 active cases, while the positivity rate came down to less than 0.01 per cent. In an area with a statistically zero vaccination rate. https://www.hindustantimes.com/cities/lucknow-news/33-districts-in-uttar-pradesh-are-now-covid-free-state-govt-101631267966925.html https://www.hindustantimes.com/cities/lucknow-news/33-distri... I await your heartfelt apology due to new facts. /s
w0mbat 5y ago
I was expecting a Star Trek article and am very disappointed.
majormajor 5y ago
I agree with the author that the big failure point was the "triumph" that resulted in a new fully-separate department, before any of the covid-triggered failures or new development. You've lost at that point. You can't make it me against you, in-house software vs everybody else, you have to share the pain and credit. Otherwise only you feel the pain and any of your "wins" will be improvements that require time and expense from other departments without them getting anything tangible comp- or recognition-wise in exchange. Let alone getting into the "this means the jobs of people I employ to do it manually in the old system will be gone" aspect, which the author touches on too. (And I imagine that even the big failures during Covid maybe weren't as terribly critical to them as to the dev team because of how low-volume the industry was.) "When there just is no interest in actually improving things, I should leave it at that. " Yep, if you can't convince them to care about and drive the improvement themselves, you're gonna have a hard time if driving it becomes the most important thing to you.
babesh 5y ago
Several observations: This is just how company politics works. Those executives wouldn't be in their positions if they didn't look out for themselves first. Also it is turtles most of the way down. This experience should be a good lesson. Only the larger of non-tech companies can get away long term with internally built tech. This is because only they have the money to build and maintain teams and build and maintain the technical expertise. Sometimes, it still takes a massive bite in the ass for these giants to wisen up. For example, see Home Depot and their credit card scandal and then their subsequent investment in self checkout, online sales, and lockers. They have done really well for themselves. Smaller companies eventually either get swallowed up or destroyed by bigger companies, by disruptive new players, or get a new lease on life due to the creation of a service layer below them. Nowadays those new disruptive players are often tech plays. Also the new service layer is often tech. One example of a tech service layer is Shopify. AirBnB and Uber are examples of industries getting swallowed up by tech. Expedia is another example of disruption from the previous generation. Home Depot swallowing up more of the construction supply industry is an example of bigger players crushing smaller players. Bye OSH.
Apofis 5y ago
First guy over here! Call Ellison.
wilsynet 5y ago
> I started talks with upper leadership to move all the people currently working on building/improving products internally into a dedicated business unit. New things go with old things. The people who maintain a product ideally should own the improvement or replacement of that product. That's how you get everyone on board. And if you can't manage to do that, then you'll probably end up in some kind war.
deleted 5y ago
[deleted]
postfacto 5y ago
Deming’s First Theorem: “Nobody gives a hoot about profit”.
TedDoesntTalk 5y ago
> I even pitched who the new VP should be, a charismatic domain expert So you’re a developer AND you’re nominating executives? Something about this story or the author stinks.
- speed_spread 5y ago
  Well, if he was hired as a high-level architect, it would make sense for him to also have some things to say about management.
  - TedDoesntTalk 5y ago
    Early in his post he says he wrote the core product. I read that as him being a developer not a high level architect.
tomnipotent 5y ago
> Other executives were afraid of the new VP. Not because he was inherently evil, quite the contrary, but the pure existence of his new role meant intermeddling with their own topics to them. Something executives tend to dislike. How could you possibly know that they were "afraid"? More likely, a group of people just had an abrupt re-org with little-to-no input and now had a new poorly-defined internal "digital" group that failed to provide clear direction on how it was going to replace an existing 3rd-party process. If worked suddenly ground to a halt and the new department was the only thing that changed, maybe existing leadership wasn't the problem? > Without any real support from any other executive, our poor VP was tasked the impossible: digitalize the enterprise... but without any decision making power outside of his own department. What about the other poor VP's? How did this new digital VP engage with them, individually and as a leadership team, to ensure that this new department had a well-structured process by which it provided value to the company? Did this new department try to lead the agenda, or simply steer it? You should be aiming to provide a service, not expect service from others. > Executives went full enrage when they heard about what we did there Well, yeah? This reinforces my opinion that the new department completely failed to engage the company. I'm sure the executive leadership team had more important things it wanted accomplished. How did adding Phoenix LiveView to an existing mess of a stack add value to the company in a way that aligns with existing objectives? Was the digital VP even consulted on this decision? It's also possible leadership at the top, a CEO/founder, didn't know how to manage an executive leadership team and the dysfunction went too deep for a VP to fix. If the leadership team isn't meeting regularly and on the same page and collaborating with the CEO to determine a roadmap, then something is very wrong. You can't fix bad communication between leadership stakeholders that can't decide amongst themselves what priorities are. > they wouldn't even come up with such ideas on thir own You're telling me that a group of people that had been successfully making demands of 3rd-party contractors and companies couldn't do the same of an internal team? I don't buy it. How did they get to such a tangled mess in the first place? > there are sometimes no other options than doing it as we did Maybe so, but how did you bring along other stakeholders in this decision-making process? I wouldn't tolerate this attitude from a contractor/freelancer, or a full-time employee. A big part of leadership is convincing others of the value of a certain path, and failure to do that isn't the fault of the other party. > Without toplevel commitment, best efforts and even results don't matter. Yes. In the vacuum of clear leadership direction, other people will find opportunity to fill the empty void with work. But separate leaders working in isolation, even creating one-hit-wonders, will never be successful as a team collaborating on even a mediocre agenda. > When there just is no interest in actually improving things, I should leave it at that. I don't believe this for a second. Either there was complete absence of top-level leadership, or the new department wasn't structured to service the business and failed to build the confidence of other stakeholders that it could deliver value. I see a lot of blaming the other guy, and very little other actual introspection other than "without including all people affected by this".
gchokov 5y ago
You don’t transform the enterprise. The enterprise transforms you..
jrochkind1 5y ago
This is so profoundly universal.
rdiddly 5y ago
Anyone your work makes look good will be on your side. Anyone who looks bad as a result of your work will be against you. Simple. Making yourself look good has limited effect, since you're already presumably on your own side.
- michaelcampbell 5y ago
  This is brilliant.

151 points by anonyfox 5y ago

ask_b123 5y ago

So, if I'm understanding correctly, the author used Lisp in this story?

tartoran 5y ago
I figure you’ve changed tabs and read a different story. No mentions of Lisp here
- gknoy 5y ago
  One of the few hyperlinks is to an article by PG about how using Lisp enabled his small team to build amazing stuff that ran circles around larger teams that were using other languages. Based on that link, with no other knowledge, I also assumed that meant the author might have used Lisp, but it also makes sense in the "higher order language", or even just "writing custom code to solve problems" sense.
htrp 5y ago
I think it's more related to the language power continuum.... he was probably coding in a "more powerful language" while the rest of the org was coding in a less powerful one. If I had to guess Rust vs Java/C/Perl?
Floegipoky 5y ago
The article mentions Phoenix, so probably Elixir.

sqldba 5y ago

So true. Seen situations so much like this.

darkerside 5y ago

Interesting tone. The tone says, all these people are idiots, and they failed to understand my beautiful creations. But he then outlines all his failings at the bottom, which I agree with, so he clearly knows better. It's it possible that this tone deaf attitude was a primary culprit in the downfall of the organization he and his handpicked successor built?

marcinzm 5y ago
You can't build something in a vacuum when there's other people with power in the organization. You need to treat the system as a whole including the politics involved. Sometimes that mean a less optimal solution to gain some influence with key stakeholders. This is one reason technologists can make bad managers. They don't think about the politics or don't want to play it.
rocqua 5y ago
His tone isn't "you are all idiots". His tone is "you all don't care about real progress, just selfish politics". The exasperation comes from total breakdown of the internal story "we are all here to make the company better". Whilst at the same time people keep repeating that line.
- darkerside 5y ago
  Yes, you state it better, and my characterization was a bit facetious. The common thread is that, he implies, or explicitly states, that he has the organization's best interests at heart and others don't. That might be true or not, but you can't operate with that mindset and expect people to come along with you. Whether or not it's true, you need to presume positive intent. Perhaps they see aspects of their jobs that won't automate well that you do not. Is that stubborn self interest, or standing up to a naive new leader who doesn't get it?
  - mkr-hn 5y ago
    The hardest and most important mindset shift for me was when I realized almost every person who supports terrible policies/politicians does so thinking they do it because they care, and they do from their particular perspective. I can't usually bridge the gap with someone who commits or supports horrors out of "concern for souls[like mine!]," for example, but most situations aren't at such a timeless impasse. Every family member who ever turned into a whiny baby while trying to do free IT for them saw themselves as the victim, or the person under threat. That's the kind of person with power in most organizations, and you have to find some way to meet them in the middle without compromising everything to get anything done. It's not enough to be right.

agomez314 5y ago

this link is returning me a 404

michaelcampbell 5y ago
Working fine for me.
- agomez314 5y ago
  working now for me!
ask_b123 5y ago
https://archive.is/brKSf https://archive.is/brKSf

SuoDuanDao 5y ago

Wow, that sounds frustrating. I'm curious what the company was, sounds like a place to avoid.

antoineMoPa 5y ago
Based on his CV, probably Aida.de: https://anonyfox.com/cv https://anonyfox.com/cv
jrochkind1 5y ago
I think it could literally be just about any large company that's at least 15 years old and doesn't sell software.

michaelcampbell 5y ago

> The problem: to achieve the desired level of UX, we needed to convince stakeholders that a single-page-app approach would be needed, including some APIs to fuel them. I'd wager a SPA is a hammer/nail reaction here.

anyonecancode 5y ago
At one job I ran into this pretty badly. The team I joined had an internal app that was, basically, a bunch of forms connected to a database. It had a lot of problems -- the data models didn't actually map to what the business needed, the UI was unintuitive. My manager insisted that what we needed to do was rewrite the entire thing as a React SPA. I told him, well, we're just going to end up with the exact same problems, only now implemented in a different framework. This won't solve any of the actual problems. He insisted, though, saying that React was "modern." Thankfully, the depth of mismanagement here was pretty exceptional in my experience. In most cases, even if I'm personally skeptical of the need for an SPA I can see a reasonable argument in favor. The cartoonish straight out admitting that the sole purpose was to be more "modern" with literally zero regard for actually serving any user needs is thankfully not something I've run across too often.
- mkr-hn 5y ago
  There are markets where being seen as old-fashioned is fatal, so this might not be irrational. Just because it's internal doesn't mean they're not worried about the wrong people seeing it in an old-fashioned state. People who worry about that (for good reason or not) don't always have a boundary on that concern, and sometimes internals leak.
  - anyonecancode 5y ago
    In fairness, his audience was his manager, not the app's users. So it was perfectly rationale for him to push for the relatively easy re-write instead of worrying about the app's users, given that his manager was not a user of the app. As an engineer who finds value in making things people actually find useful, though, it was a pretty dispiriting experience.

reilly3000 5y ago

One thing that vendors have that internal teams don’t is million dollar marketing budgets. Selling the sizzle of successful internal projects is its own kind if work, but if neglected even the greatest of projects will succumb to the onslaught of vendors. Good technical leaders recognize this inherently and celebrate their teams creations, non-technical leaders don’t and take pride in buying from big brands and the fancy luncheons that come along with them.

netcan 5y ago
Underrated comment. If you break down what most vendors and consultants do, in terms of either dollars or hours, you'll usually find that they mostly do sales, PR, client relationship, etc. Projects are designed and managed to make the right people look good to the right people, manage expectations, etc. They'll spend massive time giving non-tech executive buyers easy, fun choices.. let them feel strategic, visionary and empowered. They'll communicate the hell out of every piece of progress. Whoever hired them also has a massive incentive to buy into and sell onwards, that the project is a massive success. IE... if your job is to make a clients' Salesforce instance do a thing, 80% you your job is stuff other than making a clients' Salesforce instance do the thing. Internal dev teams will rarely have this pageantry. Meanwhile, no one outside that team knows what they do, how well they do it or such.
ed_elliott_asc 5y ago
This is one of the most insightful things I have ever read on HN.
- jollybean 5y ago
  If you want the 'aha' moment have a look at most software vendors incomes statements. They spend 2x more on selling than they do on R&D. So when that Oracle Sales Guy is giving you the pretty talk, and taking you to lunch remember that all that money your sending to Oracle is literally mostly going to him! The product makers just get 'a cut'. Now consider how much of high tech is just that, sales and marketing, conferences and luncheons. That's a staggering waste for a developed economy. We are very inefficient with these things, we need better approach to IT.

yardie 5y ago

> Big solutions to the big problems. Exactly what consultancies sell. It was obvious to them that having any inhouse-developers is the biggest business risk at all. I will never, ever, ever, do in-house software development ever again. Non-software companies treat their inhouse developers like crap. They don't get it. I don't know when non-software businesses thought it was a good idea to move software developers in house. Most of the company don't want them there. Or deal with them begrudgingly. It's hard to quantify their revenue so they are treated as an expense not cost saving or revenue generating. And they are the first on the chopping block when things turn down. In my experience, if you are recruited by a company that doesn't sell software or software services you should tread very carefully, ask lots of questions. Sometimes you'll find out a 100 year old furniture company only has a dev team because the grandson thought it was a good idea. And then you run very far away.

ProAm 5y ago
> Big solutions to the big problems. Exactly what consultancies sell. The problem is most consultants are not good at what they do. It why the sales guys for the consulting company are not the devs. Most the time consultants come in for 3-6-12 months, cause some trouble and then just leave the mess to the company to address later. There are some good consultants but most are just people that couldn't work in house because they are not up to par for long term engagements of products that need to stand the test of time to add legitimate value. > It's hard to quantify their revenue so they are treated as an expense not cost saving or revenue generating. This depends on the business, most the time, unless you make software you are an expense. The company makes their money in different departments and making money is the name of the game. It's takes a different set of skills (albeit better/stronger, industry knowledge and communication skills) to be an in-house developer vs a contractor.
- alex_anglin 5y ago
  >The company makes their money in different departments and making money is the name of the game. The challenge here is that under this model the different departments get credit for the revenue they generate. What their relationship is with the in-house dev team is very much something that one should be aware of if they choose to be in that situation. It's not too dissimilar a situation to every single 'change management' model that has executive commitment as the number one success factor.

lowercased 5y ago

Reminded of a couple of projects: 1. I worked with a single internal guy to build a web-based ordering system, in the days where people faxed in orders. This replaced weeks of data entry, dozens of part time data entry people, and enabled a big change in materials ordering and production (orders were all in a database within a few hours, vs literally weeks of keyed-in faxes from before). The one guy that was in charge of all this suddenly had a lot more power internally, and it pissed off a lot of other folks who suddenly had less influence. 2. More recently, worked with a team of external contractors, and internal folks were resistant to help any of us set up external dev systems. They all worked in the physical offices, and had no need for 'run a dev system on a laptop'. One of our guys spent a lot of time (and kept getting chastised by the client's mgt) for getting and documenting a build process for external contractors to use. The pandemic hits, office buildings are shut down, and internal folks have a much harder time figuring out... how to do anything from home. In both cases, under-the-radar skunkworks projects ended up having large (and in one case transformative) impact and value, but were ignored or fought by those with entrenched power.

specialist 5y ago
> ...fought by those with entrenched power. For decades now, per Alistair Cockburn, I assumed people just really, really hate change. There's some research which shows our selfish little brains have an immunological type reaction to change. Anything that threatens our internal stable equilibrium is squashed like an invader. No matter how much we say we want to change. No matter how much we may need to change. While wholly correct, turns out Cockburn was an optimist. We now know that some fraction of people would rather watch the whole world burn than give up one iota of power.
sergius 5y ago
A case of 'Innovator's Dilemma' perhaps?

wayoutthere 5y ago

There was an article earlier this week about why every engineer should do a stint in consulting. This is why. The reason companies like this use consulting companies is because the VPs want total control. Internal IT is purely a cost center, and unless your company (read: the business) is already on board with the product lifecycle op model, there is no point in trying to do any of this from within IT. Consultants work at the VP level and in a large enough company, we find a technical area (usually data as it has the biggest business returns) to modernize, then leverage success there build out the business case for the rest of the company (data is a good place to start building API gateways). And selfishly, the business is willing to pay higher hourly rates than internal IT (which we generally leave to the offshore MSPs). When I sell transformational solutions to a company, I know it won’t be successful without top-down buy-in. Consultants are also better positioned to do these transformations because we do them all the time; and the exercises around stakeholder management and consensus building are extremely important. They’re the kinds of things internal teams are never resourced or trained for, and they don’t happen quickly.

htrp 5y ago
The problem in consulting is that you almost always end up in an internal turf war. As mercs, you can pack up and leave once the paycheck runs out.....
- pjmlp 5y ago
  Indeed, however for me personally being a merc is much more enjoyable, specially because each project is a different stack, so it easier to change than being labeled "X Developer". Plus thanks to those political games there are lots of soft skills to learn as well.

jf22 5y ago

> So while many team members were arguing about theoretical things, I onboarded important developers on to the new system and they were productive refactoring the old things we produced into the new technology stack. I question the wisdom of somebody who started refactoring and replacing systems right after massive layoffs in an industry that was crippled by the pandemic. Businesses in survival mode can't spend resources like this. The author points out the executives became enraged when the found this out and I agree with the executives.

andix 5y ago
The pandemic was also a chance to implement new stuff, as there was anyway no real revenue generated. Everything was chaotic, so if you made mistakes you could just brush them off with "because of covid".
wsc981 5y ago
> I question the wisdom of somebody who started refactoring and replacing systems right after massive layoffs in an industry that was crippled by the pandemic. Well wisdom is gained from experience and it seems the author learned a few lessons.
lowercased 5y ago
And... when everything failed except the new system, which seemed to keep the company operational despite the executives' views... you still agree with them? You agree with their 'enraged' state? You think it was justified? Even with hindsight, or just at that moment in time?
- villasv 5y ago
  Refactoring by definition doesn’t fix things though so parent comment is still right
  - BoorishBears 5y ago
    What? By literal definition refactoring is fixing things without changing functionality... > refactoring the old things we produced into the new technology stack. You don't see how that could fix things?

Aeolun 5y ago

This is such a pain in the ass. You’d think that just providing a better solution would make people happy, but noooo. Do not encroach upon my power, you cretin! It doesn’t matter if the business dies because of my stupidity, as long as I go down captain of this ship!

MattGaiser 5y ago
I think a lot of it is that managers aren't really the captain, or at least lack the same expectations. They are more a contracted operations manager who can hop in a lifeboat first or can get paid more steadily if the ship runs aground and therefore they are more needed to clean up the mess.
wsc981 5y ago
Just from the point of view of some manager: perhaps worked many years at a company with the ultimate goal of entering C-suite. Maybe gave up other good opportunities for this goal. In such situations it must suck a lot to see your power and influence wane. You might have made many sacrifices and then someone starts some projects that could ruin everything you worked so hard for.
- bmeski 5y ago
  Then learn to code. Edit: It's a legitimate solution. I've survived 3 reorgs at my company because I could code. I read up on the new incoming system and became effective.
  - deleted 5y ago
    [deleted]
- Aeolun 5y ago
  > You might have made many sacrifices and then someone starts some projects that could ruin everything you worked so hard for. As annoying as that is, I kind of feel you should be above that. You have all right to be annoyed and skeptical, but not to sabotage the effort.

SPBS 5y ago

> And then the obvious thing happened: a complete breakdown of all important systems, like a house of cards. And: most vendors were not working due to lockdowns or other important issues during the pandemic. > The only system not affected by the complete meltdown was: our new one, which was designed to withstand outages from backend systems below. I don't understand, how can their SPA frontend still be functional when the backend is totally down? You can display things to the user, but you cannot do anything meaningful from it?

jdlshore 5y ago
Depends on what the backend systems were doing. They may have been data feeds that could be cached, with the “front-end” actually being a whole app with its own back end and database. Or perhaps taking orders and queueing them up for later delivery when the back-end systems came back on line.

Gunax 5y ago

Where's the ceo and board in all of this. Someone must actually care about the bottom line.

mbrodersen 5y ago
CEO’s care about their yearly bonus, making the majority share holders happy, and the next job opportunity. Everything else is secondary.

fridif 5y ago

I've been both in-house and consultant to a variety of big corps in a variety of industries, and I can say that invariably every single one is messed up and inefficient. Yet still somehow customers buy the products and (more often than not) the company delivers a product that is half working enough to be tolerated and used.

jmull 5y ago

Meh. When it seems like everyone is stupid but you... you are probably the one who is on the wrong track. This is all very vague, so who knows. But at the very least, I would guess the OP's attitude of I-know-better-than-everything-that-came-before arrogance plus ignorance of why things were the way they were made big failures inevitable. It's a real limitation -- though a common one -- to need everyone to see things from your perspective, rather than being able to understand their perspective.

ikiris 5y ago
a vast amount of people think horse dewormer is a viable strategy. If anything there is a crisis of "my ignorance is just as valuable as your experience", and not the other way around.
- hobs 5y ago
  Absolutely, but if your technical solution was so definitely brilliant then it would have solved the problems you purport it magically solved. Generally its great, but not a panacea, and while it would have been improvements it would have suffered fun new problems. Source: been the young guy who thought if only they listened, and then they did.
- imwillofficial 5y ago
  You mean like India and their hundreds of thousands of cured people?
  - scruffyherder 5y ago
    No he means when it killed Joe Rogan. RIP
    - deleted 5y ago
      [deleted]
    - imwillofficial 5y ago
      This must be awkward for you to read: 33 districts in Uttar Pradesh are now Covid-free: State govt Overall, the state has a total of 199 active cases, while the positivity rate came down to less than 0.01 per cent. In an area with a statistically zero vaccination rate. https://www.hindustantimes.com/cities/lucknow-news/33-districts-in-uttar-pradesh-are-now-covid-free-state-govt-101631267966925.html https://www.hindustantimes.com/cities/lucknow-news/33-distri... I await your heartfelt apology due to new facts. /s

w0mbat 5y ago

I was expecting a Star Trek article and am very disappointed.

majormajor 5y ago

I agree with the author that the big failure point was the "triumph" that resulted in a new fully-separate department, before any of the covid-triggered failures or new development. You've lost at that point. You can't make it me against you, in-house software vs everybody else, you have to share the pain and credit. Otherwise only you feel the pain and any of your "wins" will be improvements that require time and expense from other departments without them getting anything tangible comp- or recognition-wise in exchange. Let alone getting into the "this means the jobs of people I employ to do it manually in the old system will be gone" aspect, which the author touches on too. (And I imagine that even the big failures during Covid maybe weren't as terribly critical to them as to the dev team because of how low-volume the industry was.) "When there just is no interest in actually improving things, I should leave it at that. " Yep, if you can't convince them to care about and drive the improvement themselves, you're gonna have a hard time if driving it becomes the most important thing to you.

babesh 5y ago

Several observations: This is just how company politics works. Those executives wouldn't be in their positions if they didn't look out for themselves first. Also it is turtles most of the way down. This experience should be a good lesson. Only the larger of non-tech companies can get away long term with internally built tech. This is because only they have the money to build and maintain teams and build and maintain the technical expertise. Sometimes, it still takes a massive bite in the ass for these giants to wisen up. For example, see Home Depot and their credit card scandal and then their subsequent investment in self checkout, online sales, and lockers. They have done really well for themselves. Smaller companies eventually either get swallowed up or destroyed by bigger companies, by disruptive new players, or get a new lease on life due to the creation of a service layer below them. Nowadays those new disruptive players are often tech plays. Also the new service layer is often tech. One example of a tech service layer is Shopify. AirBnB and Uber are examples of industries getting swallowed up by tech. Expedia is another example of disruption from the previous generation. Home Depot swallowing up more of the construction supply industry is an example of bigger players crushing smaller players. Bye OSH.

Apofis 5y ago

First guy over here! Call Ellison.

wilsynet 5y ago

> I started talks with upper leadership to move all the people currently working on building/improving products internally into a dedicated business unit. New things go with old things. The people who maintain a product ideally should own the improvement or replacement of that product. That's how you get everyone on board. And if you can't manage to do that, then you'll probably end up in some kind war.

deleted 5y ago

[deleted]

postfacto 5y ago

Deming’s First Theorem: “Nobody gives a hoot about profit”.

TedDoesntTalk 5y ago

> I even pitched who the new VP should be, a charismatic domain expert So you’re a developer AND you’re nominating executives? Something about this story or the author stinks.

speed_spread 5y ago
Well, if he was hired as a high-level architect, it would make sense for him to also have some things to say about management.
- TedDoesntTalk 5y ago
  Early in his post he says he wrote the core product. I read that as him being a developer not a high level architect.

tomnipotent 5y ago

> Other executives were afraid of the new VP. Not because he was inherently evil, quite the contrary, but the pure existence of his new role meant intermeddling with their own topics to them. Something executives tend to dislike. How could you possibly know that they were "afraid"? More likely, a group of people just had an abrupt re-org with little-to-no input and now had a new poorly-defined internal "digital" group that failed to provide clear direction on how it was going to replace an existing 3rd-party process. If worked suddenly ground to a halt and the new department was the only thing that changed, maybe existing leadership wasn't the problem? > Without any real support from any other executive, our poor VP was tasked the impossible: digitalize the enterprise... but without any decision making power outside of his own department. What about the other poor VP's? How did this new digital VP engage with them, individually and as a leadership team, to ensure that this new department had a well-structured process by which it provided value to the company? Did this new department try to lead the agenda, or simply steer it? You should be aiming to provide a service, not expect service from others. > Executives went full enrage when they heard about what we did there Well, yeah? This reinforces my opinion that the new department completely failed to engage the company. I'm sure the executive leadership team had more important things it wanted accomplished. How did adding Phoenix LiveView to an existing mess of a stack add value to the company in a way that aligns with existing objectives? Was the digital VP even consulted on this decision? It's also possible leadership at the top, a CEO/founder, didn't know how to manage an executive leadership team and the dysfunction went too deep for a VP to fix. If the leadership team isn't meeting regularly and on the same page and collaborating with the CEO to determine a roadmap, then something is very wrong. You can't fix bad communication between leadership stakeholders that can't decide amongst themselves what priorities are. > they wouldn't even come up with such ideas on thir own You're telling me that a group of people that had been successfully making demands of 3rd-party contractors and companies couldn't do the same of an internal team? I don't buy it. How did they get to such a tangled mess in the first place? > there are sometimes no other options than doing it as we did Maybe so, but how did you bring along other stakeholders in this decision-making process? I wouldn't tolerate this attitude from a contractor/freelancer, or a full-time employee. A big part of leadership is convincing others of the value of a certain path, and failure to do that isn't the fault of the other party. > Without toplevel commitment, best efforts and even results don't matter. Yes. In the vacuum of clear leadership direction, other people will find opportunity to fill the empty void with work. But separate leaders working in isolation, even creating one-hit-wonders, will never be successful as a team collaborating on even a mediocre agenda. > When there just is no interest in actually improving things, I should leave it at that. I don't believe this for a second. Either there was complete absence of top-level leadership, or the new department wasn't structured to service the business and failed to build the confidence of other stakeholders that it could deliver value. I see a lot of blaming the other guy, and very little other actual introspection other than "without including all people affected by this".

gchokov 5y ago

You don’t transform the enterprise. The enterprise transforms you..

jrochkind1 5y ago

This is so profoundly universal.

rdiddly 5y ago

Anyone your work makes look good will be on your side. Anyone who looks bad as a result of your work will be against you. Simple. Making yourself look good has limited effect, since you're already presumably on your own side.

michaelcampbell 5y ago
This is brilliant.