On HN its always big bad management who is the cause of every security problem or shoddy piece of engineering. If only that pesky management would screw off then we could do things "properly".
You'd be suprieed at how many incompetent engineers there are out there. If "engineer" Alice in your story was actually competent they would never agree to implement the proposed "fix". Its not a fix. To pass it off as one would be malpractice for a real Engineer.
And I might add: especially at big companies where being yes men is a higher valued characteristic than being technical
You live in a nice world, because usually the actual power (and duty) of a dev team member is to advise, not to agree or refuse.
You can always quit. Or force them to fire you for refusing to implement a non fix.
In reality though I doubt this narrative even occurred. Some incompetent engineer likely proposed this fix thinking that it was actually a fix.
Edit: I see I've been downvoted for this comment. If we were real engineers working on things like cars and bridges we'd actually be held accountable. Take some pride in your work people, this is one of the most in demand professions in today's economy.
Quitting assumes the possibility of finding a job around the corner and not having people that depend on you.
You might be surprised, but if you move to Europe, it is actually so! Move to Berlin of Frankfurt. There is always work for Software Developer here, it is hard not to get one and you get spammed by recruiters all the time.
You don't have to take shit from management, you can just go elsewhere. Everything is also very near.
I live in Germany, now try to convince the family to move along.
Well as a European I find the "uproot your life for a better job" mentality completely alien. Short of a war I'm not sure what would make me leave the place I call home.
> Quitting assumes the possibility of finding a job around the corner and not having people that depend on you.
Quitting assumes the possibility of finding a job after spending 4 months doing leetcode 6 hours a day and not having people that depend on you.
You suggest it was just one lone engineer being incompetent. Then you respond to downvotes with the non-sequitur "take some pride in your work" - are you suggesting pride is a fix for incompetence? What?
There are two points.
1) Any engineer involved in implementing this "fix" is incompetent.
2) If you are not incompetent and management tries to force you to implement this "fix" then you should take pride in your work and refuse to do it, even if it means quitting your job (or forcing the company to fire you).
It's easy to opine and judge from on high when it's not your nuts in the vice.
1) is obviously false, since an engineer could be competent and evil. Or competent and under duress - needing to keep their job to keep their visa, family health insurance, etc. Or competent but misled - "Implement the suggested fix to show how easy it is to work around" "OK, here it is" "Now work on something else", while it ships.
2) Why should you? Unless you're willing to be homeless and leave society, you're not going to find a way to live in a capitalist society which is pure idealism and no compromises, and you can't effect change in a system from without, only from within. Quit, and someone else will implement this fix - you feel good, which is worthless, and the fix ships, which is bad, and Cisco doesn't change, which is bad. Pick your battles and you could use this result in future - to build other people's trust in your recommendations, to widen the review process before future fixes are agreed upon, to have a proven record of the cost of doing things twice.
If there were actual accountability, software engineers would have a much better lever against management. For some software this is case. If you write safety critical software you can be held personally responsible for accidents. In those industries engineer pushback is much more effective.
My background is safety-critical and -- speaking from first-hand experience -- it was truly incredible how much push-back we'd get when raising safety issues.
System tick timer rolled over after 2^32 1ms clock ticks (a little over seven weeks) and the software mishandled it by rebooting and losing control of the process. It wasn't until QA got involved (it turned out they saw it on a long-term test) that we were cleared to actually fix it. It was a two-character change in an inline function!
In some places "cover it up, don't discuss it in email" is the natural response. It really depends on the management and the organisational culture. You can have management who support the engineering team, or you can have management who micromanage and overrule and end up turning a two-day hotfix cycle into ten months of "try soldering a resistor to every single pin on the PCB".
I like your spirit, but there's another conclusion we can draw: very few of us outside of a few special areas in embedded are "engineers". We tinker with bits and bytes to make business systems. Sometimes we bump up against some general engineering fundamentals. But we're just technicians meeting business goals.
Yep, and the same can be said for “real engineers”. Most EEs, civil engineers, etc work on boring stuff as well with negligible risk of doing any harm.
At a large organisation I worked at years ago we used to call it Coder Shopping - when a manager would do the rounds trying to find someone who would say "yes" in a situation like this.
Invariably there would be someone who would, and if you were the person who had initially refused you probably wouldn't even be aware that it had happened until the bug reports came in...
So whose fault is the cancerous heap of flaming dogshit that is nodejs, then? Some manager managed it into existence?
No, it's the brainchild of an engineer. Somebody actually thought nodejs was a good idea. Face it, there are lots of really bad engineers out there.
I honestly envy the sort of person who can post something like this. Not liking nodejs is fine, but to extrapolate from "I don't like nodejs" to "Thousands of engineers at companies like Google are wrong and I am right" must require such a level of myopic, ignorant self-belief that is completely alien to me. I just find myself respecting other people's work too much, even if I don't like it.
Invoking "thousands of engineers at companies like Google" is a logical fallacy. There are tons of incompetent people out there writing code, and lots of them work for FAANG.
>I just find myself respecting other people's work too much, even if I don't like it.
Maybe that's the difference between us, then. I don't feel that we should hand out Participant ribbons. I'm sure the nodejs people work very hard, but the fact is their product is just objectively awful.
> Thousands of engineers at companies like Google
That's an appeal to authority. It's also worth noting that nodejs is not a Google project and it is barely used internally at all. Only for small toy projects. Google uses python for scripting, or java for real servers.
That's an appeal to authority.
It is, but it's also shorthand for "people who are likely to be good based on the fact they've been through a rigorous hiring process designed to filter out the worst engineers". Sometimes, maybe, you should let the principle of charity win over the rigorous rhetorical logic you can use to dismiss an argument because it broke The Rules you learned on LessWrong.
Absolutely: Cisco made this happen, not Engineer Alice. If your boss asks you to do something within my ethical boundaries and you have 5 tired co-workers in favour of shipping it, eventually you'll break and say "fine". It doesn't make you responsible or the one that pulled the trigger.
deleted 8y ago
[deleted]
"Engineer" conveys image of middle class white-collar job with relatively high status, good education and responsibilities. That word now used for everyone doing programming related jobs inside office space for no good reason.
I think the word "tehnician" should be used to describe most grey-collar ICT jobs, including most programmers. Their responsibility and scope of their work is limited. Many programming jobs are closer to blue collar factory level mechanic than engineer.
https://en.wikipedia.org/wiki/Grey-collar https://en.wikipedia.org/wiki/Grey-collar
https://en.wikipedia.org/wiki/Technician https://en.wikipedia.org/wiki/Technician
Please bitch "associate professional"
Thankfully there are countries where Engineer is still a proper word, not something that you are allowed to call yourself after a 6 month bootcamp.
And still others have, though it may not be appreciated in all industries, _Chartered_ Engineers.
Yup, in the UK it bugs me when I keep meeting people who introduce themselves as an 'engineer'. When I ask them if they do mechanical or civil engineering, then I usually get to say "ah, so you're a programmer, just like me".
One thing I did sometimes like about being in Austria is the obsession with academic titles they have there... if somebody is a qualified engineer, they invariably use "Ing" as a title in place of "Mr" or "Ms"; people with two doctorates (not uncommon) really do call themselves Dr Dr So-and-so.
There are plenty of poor engineers, but this “fix” has to go through a process defined and enforced by managers. In a good process, multiple engineers would review the code, and if the office culture is good, then objections will be brought up and listened to, and if a fix is insufficient, management will understand and assign resources to make it right. None of those things depend on any one engineer or manager. They’re a culture issue. But the only people with the power to fix a broken culture is management. Engineer Alice doesn’t have that power herself.
In large orgs there would also be some kind of security team which would have input on the CVEs. Cisco does have them. It sounds like they were not involved at all here for some reason.
The only way, and I mean the only way, this gets blocked is because a manager (not a dev) assesses the risk of improper fixing to be higher than the reward of not doing a shit ton of work justifies.
A significant portion of that is making sure the manager is fully cognisant of what the quick fix means. Alice's job is to ensure that happens. The managers job is to advocate to his bosses that this has to be done and is a net win to do it properly.
It is as likely manager person failed in advocacy as it is she failed in information transfer, but there are plenty of gradients here.
Who hired the incompetent engineers?
It really is amazing the kind of mental gymnastics HN commenters will display in order to shift the blame to everybody else, isn't it?
The question here, is if it would stop 90% of the attacks, would it be a worthwile way to spend 5 minutes.
And the answer to that question is almost invariably yes.
If you're a company the size of Cisco, and you have two widely publicised vulnerabilities like this, then this sort of failure is _always_ management's responsibility.
That responsibility might come in the form of "big bad management" chasing the cheapest possible "fix" against the advice of the the hero engineers, but it can take other shapes too. It might be poor oversight and QA processes, or a failure to hire and/or assign competent engineers for the task.
We can "No True Scotsman" this all day long.
It is bad management to hire incompetent engineers.
It is bad management to fire competent engineers that push back.
It is bad management to make employees feel as though they would be fired for standing up in any way to management.
Take away the rhetorical shortcuts, and responsibility still always falls at the feet of the ones that have the power to make the decision. As long as engineers are hierarchically bound to obey managers upon pain of sudden and unplanned unemployment, the managers are ultimately responsible for everything the company does. If the company does not have tenured engineers that have stop work authority, the managers are fully to blame. And if they do, management still might be partially at fault.
We have this headcanon where a competent engineer could stand up, sweep away the slide deck on screen with a wave of the hand, and offer ironclad mathematical proof that the quick fix is no fix at all, and that the long, painful, but correct fix is what is Right for the Company (tm). The manager would then do the slow clap and tell that engineer to fix it right this time. It is a fantasy that is a great comfort to those of us doing maintenance work on a massive crock of shit, making the payments on technical debt that we are expressly forbidden from making any attempts to retire. But it is a fantasy. A real manager would cut the engineer off in mid-proof and say that the quick fix is happening, and that would be the end of the story.
The only way for an engineer that is "at will" and without a contract to push back is by falling on their own sword. They can quit, at great cost to themselves, to create a very minor inconvenience to the company, who will end up doing what it wanted anyway.
Being in this situation myself, you're damned right I have no responsibility for the crap code that goes out, because I have almost no autonomy in the manner in which I do my job. I cannot work outside the scope of the maintenance tickets I am assigned. And my company rents me out by the hour to the customer, so it has negative incentive to allow me to build up infrastructural capital that would allow me to do the same work in less time. I will blame management all day long, and then a bit more on evenings and weekends.
If you read the parent post and think "bad management" I don't think the message was clear.
Yes, but this is software engineering. Where there is no such thing as malpractice, and where managers can not be prosecuted for criminal negligence even when their overriding of the engineers professional position gets dozens of people killed. In this delightful world, managers rule the roost and engineers do not have the ability to refuse to do anything.
Big bad management gets to keep their jobs when they make bad product decisions, so let's call it a wash.
If management let this important security fix get executed and deployed without a competent engineer in the loop, that's a horrible mistake in itself, but it's far less likely than management choosing this alternative based on accurate information from engineers.
Stereotypes are valid first order approximations.