13 ms·

Am I stuck in a local maximum?

136 points by dustinmoris 5y ago

seibelj 5y ago
I've worked with people who were TDD maximalists, Agile / Scrum maximalists, microservice maximalists, Gang of Four maximalists, and even a MongoDB maximalist. What has become clear to me is that the silver bullet doesn't exist. Pick the tool for the job, and accept that exceptions exist for every rule. The maximalist is too rigid, and in the rigidity is arrogance and ultimately failure. I would even say that if you take someone who is obsessed with something like TDD, and then ban all engineers from using TDD, it is quite possible you get a better product. You don't want TDD for every problem - sometimes it simply is a barrier to success.
daxfohl 5y ago
Try going all-in for a year on the other viewpoint, assuming you've reached a local maximum on your favored approach. At worst you'll be slightly less efficient for a year. At best you'll be more efficient and come out of it with a whole new dimension in your skillset.
- oreally 5y ago
  I wonder how much he'd hate his heroes if he tries something like that. Or maybe he'd decide it's not worth his time and money and stay stuck in his own bubble.
- AdrianB1 5y ago
  Just try to listen to the other viewpoint from someone competent, it may be enough.
shannifin 5y ago
Nice to see a recommendation for Petzold's The Annotated Turing in the post, it's a great read. I'm not sure I really understand the writer's concern though. The debates he mentions (functional versus object-oriented programming, dynamic versus static typing, oral versus written collaboration) don't seem like issues with right or wrong answers in and of themselves, but rather issues for which to compare and contrast benefits and tradeoffs according to one's goals, priorities, and preferences in a given situation. Disagreement with heroes in such a context seems a complete non-issue. > "could I be stuck in a local maximum?" I perhaps don't quite understand his meaning of the question, but as perfection is infinitely precise and necessarily unattainable, everyone (who's at least honestly doing their best) is always stuck at a personal local maximum, and will be until they find a new maximum through continued learning. And I think there'll always be enough variety of experiences and preferences among individuals that we'll never all completely converge to the same maximum.
timClicks 5y ago
Many of the author's heroes became prominent because it benefited them as consultants pushing their methodology. Those methodologies are designed (in part) to make it easier for younger, less experienced programmers to contribute to large software projects. I would say that it's okay to disagree with them over time as you gain experience and understanding.
- daxfohl 5y ago
  Actually, I think this is the correct response. @everyone please ignore my response below/above/wherever.
- zozbot234 5y ago
  Agile methods are not designed for "younger, less experienced programmers". They work best for enabling successful, experienced hackers.
  - barrkel 5y ago
    In practice, Agile is often Scrum, which in practice breaks down tasks into bite-sized blobs of functionality on a conveyor belt of work, to be implemented by a team of interchangeable human resources.
    - FearNotDaniel 5y ago
      That would be Kanban... And in practice, many orgs are performing a pseudo-Scrum which is really just a timeboxed Kanban with a lot of cargo-culting of the Scrum ceremonies.
  - timClicks 5y ago
    If you watch Robert C. ("Uncle Bob") Martin's talks, he spends a lot of time talking about how the first generation of programmers didn't need agile because they were already professionals from other fields. The agile methodologies were created because the industry decided to hire younger, less experienced, and less diligent, staff.
Apofis 5y ago
This may have been incidental but I think I was just triggered for the first time in my life. The far right is now statically typed and dynamic typing is now the left? /slight tang of sarcasm
- KSteffensen 5y ago
  Makes sense. The right is usually the law-and-order side while the left is the live-and-let-live hippies. :-)
random314 5y ago
His heroes are the agile manifesto folks? Yeah, it is a good thing he disagrees with them. Bob Martin has been posting howlers for a long time. One of the problems with never formally studying CS is that you might never discover real computer scientists who could be your heroes. Who could disagree with Lamport, peyton jones, Hinton?
- Animats 5y ago
  Agile is a tool for feature-dominated webcrap where "sort of works" is good enough. It's not useful for, say, database internals or hard real-time. If you just need to put ads on a page, a deep knowledge of Knuth and Dijkstra will just make you unhappy with the mess that HTML/CSS/Javascript has become.
  - hoseja 5y ago
    Sadly, this sort of software seems to dominate discussion nowadays.
    - deleted 5y ago
      [deleted]
  - zozbot234 5y ago
    > Agile is a tool for feature-dominated webcrap where "sort of works" is good enough. It's not useful for, say, database internals or hard real-time. Have to disagree there. The basic point of "agile" methods is to promote small-scale teams (with the fabled "two-pizza team" at the largest end) doing heavily-iterative development with an emphasis on refactoring and continuous delivery ("release early and often"). There's no reason why this could not work for these sorts of projects.
Animats 5y ago
"After I read the book, I've come to understand that general-purpose static type system can never prove unequivocally that a generic program works. That's what Church, Turing, and Gödel proved." Oh, not that again. If a program is close to undecidable, it's broken. Microsoft ran into this with the Static Driver Verifier. If they couldn't prove that a driver could not blow up the kernel after the prover had cranked for a few minutes, the driver is rejected. Really, if you don't have easily proven decidabilty in your device driver, it has no business going into production.
- infogulch 5y ago
  I'd like to see the industry move towards proof-carrying code. This takes decideability checking one step farther: not only does your driver have to be decideable, you have to prove that ahead of time and include a proof that can be checked in linear-time right along with the binary. At that point even signatures become redundant; you don't need to trust any central authority if you have proof of the programs validity right there.
  - Animats 5y ago
    You still need some criterion to verify against. The Static Driver Verifier is successful because its criterion is just "will not blither all over memory" or make an incorrect API call. Passing the check doesn't mean the driver will drive the device properly, but it shouldn't crash the kernel. Or offer buffer overflow exploits.
    - Joker_vD 5y ago
      Yes, those exactly are criteria that PCC was developed in mind with: "examples of safety properties that are covered in this paper are memory safety and compliance with data access policies, resource usage bounds, and data abstraction boundaries" (from the abstract of [0]). That's what safety is, after all: that something bad does not happen. [0] George C. Necula, Peter Lee. Safe, Untrusted Agents using Proof-Carrying Code. 1998. http://www.cs.cmu.edu/afs/cs/project/pop-10/member/petel/www/publications/lncs98.pdf http://www.cs.cmu.edu/afs/cs/project/pop-10/member/petel/www...
aliswe 5y ago
> I don't have a formal degree in computer science That ploeh does not have this is to me a big shock. It would not be a big stretch to say that every .NET systems architect worth their salt has at least stumbled upon his content once in their life.
- flohofwoe 5y ago
  What has one to do with the other though? Writing (and "architecting") good software is mainly craftmanship which is learned via experience, not something that requires a "formal degree". The years at university are probably a good place to get some of that experience, but "computer science" is mostly tangential to the actual problems of programming.
  - aliswe 5y ago
    yeah, ik its kinda irrational, but he is a very academic and scientific type of guy. oh, and for the record I dont have a degree either.
darawk 5y ago
> To the right, in this context, means more statically typed. While the notion is natural, the sentence is uninformed. When I wrote the article, I hadn't yet read Charles Petzold's excellent Annotated Turing. Although I had heard about the halting problem before reading the book, I hadn't internalised it. I wasn't able to draw inferences based on that labelled concept. > After I read the book, I've come to understand that general-purpose static type system can never prove unequivocally that a generic program works. That's what Church, Turing, and Gödel proved. This is not exactly the right inference. You can have a language that has the property "if it compiles, it works", that language just cannot also be Turing complete.
- naasking 5y ago
  > This is not exactly the right inference. You can have a language that has the property "if it compiles, it works", that language just cannot also be Turing complete. Not quite. It can be Turing complete as long as you fill in the proof steps the compiler can't infer for itself. Coq, Idris and other theorem provers can all produce programs with arbitrary looping behaviour given valid proofs are generated.
  - dllthomas 5y ago
    This seems wrong. If you require a proof that a program halts for all input (or of any halting equivalent property), then there will be programs you cannot write - those that possess the property in question but for which no proof exists, and those that actually lack the property. In particular, you obviously cannot write a universal Turing machine in a system that requires a proof of halting for all input, because it by definition won't halt if it is fed a program that does not halt. A system that cannot build a UTM is not Turing complete.
    - naasking 5y ago
      I'm not sure what exactly is not right. Type-based theorem provers exist (Coq, Idris, Agda, etc.). They can be used to write Turing complete programs (eg. CompCert). If you encode the properties that need to be satisfied and the compiler successfully compiles your program, then those properties are satisfied. I suspect you've simply misunderstood the claims that were made. In particular, the above does not entail that you can write any program. Some programs simply have no valid type.
nodejs_rulez_1 5y ago
> In a recent discussion, some of my heroes expressed the opinion that they don't need fancy functional-programming concepts and features to write good code. I'm sure that they don't. It's OK to disagree with Bob Martin - many do. The days of TDD silver bullet training gold rush are over.
- fredrikholm 5y ago
  Not necessarily a Robert Martin fan, but he is writing Clojure (functional Lisp) as his primary language these days.
- pjmlp 5y ago
  My favourite TDD question to presenters was to show how to design GUIs according to specific UI/UX criteria, while following the TDD religion.
seanwilson 5y ago
> "To the far right, we have a hypothetical language with such a strong type system that, indeed, if it compiles, it works." > For good measure, despite my failure to understand the implications of the halting problem, I'm otherwise happy with the article series Types + Properties = Software. You shouldn't consider this particular example a general condemnation of it. It's just an example of a mistake I made. This time, I'm aware of it, but there are bound to be plenty of other examples where I don't even realise it. I don't see what the mistake is. Halting and Godel's incompleteness theorems only say there exists problems where you don't know if there's a solution or not - that's very different from saying you can't come up with solutions for specific problems. For example, you can't tell if any arbitrary crazy program halts but most loops people code in practice have trivial proofs of halting because it's usually simple loops from 0 to the fixed length of a collection. If some code is full of loops where you're nervous about them halting a lot, there's probably something very wrong with the design that would get flagged in integration tests and during use too. It's been demonstrated you can statically prove the correctness of something as complex as an operating system for example (see https://sel4.systems/About/ https://sel4.systems/About/) so the above isn't much of a barrier to proving interesting program properties: seL4's implementation is formally (mathematically) proven correct (bug-free) against its specification, has been proved to enforce strong security properties, and if configured correctly its operations have proven safe upper bounds on their worst-case execution times
talkingtab 5y ago
A great post! Thanks. For me, it is the meta issue of how we think about the examples that is most interesting rather than the examples. One contribution I have is that we humans are biased toward action as we must be. And biased toward results. These are survival skills. If acting requires me to adopt some belief, I do so - even if that belief is incomplete or even false. And if acting on that belief produces some result it is grows stronger. Long, long ago I had a friend who had an old car and not much much money. The car would often not start on the first try, so he would get out, open the hood and wrap some exposed wires in electrical tape, then get back in the car and it would usually start right up. I offered to replace the exposed wires. I was bewildered to find that the wires were not connected at either end! I had watched the process where wrapping the wires fixed the problems, yet one had to be insane to accept that as the cause. I finally understood that the float on the carburetor was defective (acting like the float in a toilet). The time delay in wrapping the wire allowed the gas in the flooded engine to evaporate enough to start. I replaced the float and that fixed the problem. At least that is what I now tell myself, but perhaps my own belief system is as flawed as wrapping the wires? Perhaps it was not the float but a gasket or that it fixed an interaction between some unknown type of subatomic particles which in the future will be used for near instantaneous space travel? So in answer to the question "Am I stuck in a local maximum" the answer is "always". (edit for clarity)
open-source-ux 5y ago
I don't have any heroes in the field of computer science (hero is too strong a word for me), but there are many experienced people I admire who express ideas or opinions I find appealing. An example: I admire Niklaus Wirth (creator of Pascal, Modula-2, Oberon) and in particular his preference for strongly-types programming languages that are small in size. He's spent a lifetime in the computer science field, but he disagrees with the ideas of others - and others disagree with him. Wirth is critical of functional programming as a programming paradigm, for example. Overall, can anyone argue that someone is emphatically right or wrong in this field? Everyone settles on a philosophy that shapes their outlook based on their experience and research. And that outlook might change over time. That accrued wisdom and experience (or inexperience!) will naturally cause someone to gravitate to one philosophy, or disagree with another.
brianmcc 5y ago
I really enjoyed this and felt it resonates quite a lot with my own views. I think a lot about there not being a "right" or "wrong" approach, absolutely, rather approaches which work for some teams and individuals that don't succeed for others. As well as being too easily distracted by shiny new tech toys, our industry's other big dysfunction is a ready insistence by "thought leaders" about how to do things right, the One True Way.
lmilcin 5y ago
I found that no single "pure" paradigm is a best response to development needs. By pure I mean pure OOP, FP, TDD, etc. Instead I am currently working on mixing various paradigms together to achieve better result. Some examples: -- I use OOP to flesh out the domain model of my application, but use FP for most of the infrastructure and application around it. Some processes within OOP model are implemented using FP. I use OOP for its strength in bringing meaning and clarity to the most important part of the model. -- I use unit tests only for the important, non-trivial constraints on the tricky parts of the code. For the rest I prefer functional tests -- focusing on testing external application behavior lets me refactor the code easily without spending time on a huge mass of unit tests that need to be brought to shape after every refactoring. I still test the application -- the test suite ensures I will get notified if application misbehaves, but now it is not tied to actual internal implementation.
naasking 5y ago
> After I read the book, I've come to understand that general-purpose static type system can never prove unequivocally that a generic program works. This sentence seems confused. What is the meaning of "works"? The most general interpretation seems to be "works" = "satisfies some proposition P". But, "general-purpose static type system can never prove unequivocally that a generic program satisfies some proposition P" is false. In fact, static type systems unequivocally prove many propositions about general programs. The generality of the propositions depend on the expressive power of the type system. If you dial static typing up to 11 (dependent types), then P can be literally any proposition about the program. This sometimes requires programmers to provide the right type annotations, but this is no different in principle from needing to program behaviour, ie. a computer needs you to write out the program describing what you want it to do, and a type checker needs you to write out what you want it to check. You wouldn't expect to just wave your hand and your computer create the program you have in your mind for you? Or write out the kind of program you want in ambiguous, natural English and have the computer figure out exactly what you meant?
IceDane 5y ago
My theory is that these career evangelists have spent too much of their careers doing just that: evangelizing. They haven't actually been writing code or building software. They've just been telling everyone how they think they should be building software. I have arrived at the same conclusions as you wrt. static vs dynamic typing. A sufficiently powerful static type system entirely subsumes dynamic typing, and there is quite literally nothing you can accomplish with dynamic typing that you cannot accomplish with such a static type system. After writing software using both dynamically and statically typed languages, this is the conclusion I have arrived at, and it feels like a no-brainer to me. This is so much of a no-brainer to me that I can't help but be biased when someone disagrees. I want to think that I am open to having my mind changed, but I have yet to see any evidence that sways me. I do think perhaps a lack of external demand for code quality and correctness might sometimes be at fault. When all you need to do is write the code for the happy path correctly, maybe you just feel static types get in the way. I've converted several javascript projects to typescript, and in every single one of those projects, the conversion exposed multiple(in the tens or sometimes hundreds) cases of incorrect handling of null/undefined or just type mismatches. I know this has to do with culture and the specific company and engineering discipline, but what I'm trying to get at is that I think a lot of people out there are writing code this way: just hack away until it works, and fix it if it ever crashes. When this is all you want, maybe proving correctness to the compiler is too bothersome. But I don't and won't work this way.
- 1337shadow 5y ago
  > there is quite literally nothing you can accomplish with dynamic typing that you cannot accomplish with such a static type system. Exactly, and all you need is Any > a lot of people out there are writing code this way: just hack away until it works, and fix it if it ever crashes. I'm willing to agree that Non-TDD hackers should go for static typing.
  - uDontKnowMe 5y ago
    >Exactly, and all you need is Any If you're using Any, then your program is (at least in part) dynamically typed. I don't think it's fair to say "look! Static typing is better than dynamic typing in any and all situations! (so long as it can use dynamic typing)"
jbjbjbjb 5y ago
I find the “hero” wording a little grating and I think it’s the problem here. They’re just people expressing mostly subjective opinions. The problem is developers put these people on a pedestal and take their word as gospel. In reality you need good judgement to understand and appreciate the trade offs. I do also think developers look for silver bullets and happy to accept anyone with some authority telling them it exists and they needn’t think for themselves.
- h2odragon 5y ago
  article says: "I find myself disagreeing with my heroes on a regular basis, and that makes me uncomfortable." Why should you agree with your heroes? You are not in the circumstances they were whenever they became your hero.
hyperman1 5y ago
I first read this problem: micro-management, red tape, overly bureaucratic processes, and a lack of developer autonomy and then this: essentially have no process. Everything is ad hoc, nothing is written down, deployment is a manual process, and there are meetings and interruptions all the time They weird thing about enterprise is: I see both happen at the same time. We have an issue tracker, but its a bad product and the metrics are heavily politicized. So we track our issues in excel. No way to know on whose excel your name appears. We have a document sharing platform, but the fine grained access control makes it impossible to let the relevant people read the document. So we mail different versions around and forget to update the central truth. We have architecture documents, but they have nothing to do with what runs in reality. We can't document what runs in reality because it would conflict with the offical architecture documents. We have a release process that takes a week or a month, and many paper is written about risk and impact, test plans, deployment plans, resource availability, etc... But the actual risks must stay unspoken lest the release is cancelled. Tests were done on an environment that is forbidden to resemble production. People ignore the deployment plan and do whatever they like, and thank god because the deployment plan is unusable. People are not available when things go wrong, because the deployment plan booked them for 2 days for doing 5 minutes of work and they decided their part was done. The worst problem: Instituting any real process is impossible, because officially there already is one.
- scns 5y ago
  This may be the worst thing i read on HN ever.
  - hyperman1 5y ago
    In what way? Language? Content? World view?
    - scns 5y ago
      The horror i felt imagining how the people who work in such an environment might feel, if they haven't checked out allready. (Edit) So it isn't actually that dramatic.
bjornsing 5y ago
Interesting questions, but I think one aspect is sorely missing from the analysis: personality. The author sees himself as an introvert. He likes typed languages. He reads books about Turing, Church and Gödel. He seems to like understanding things. Sounds to me like he has a cognitive style where what Kahneman calls System 2 is dominant. I’m not familiar with all his “heroes”, but I would assume most people become “heroes” because they are somewhat extroverted. They like talking to people. They like going to conferences. They like being in the limelight. With this often comes a different cognitive style, closer to what Kahneman calls System 1. People are incredibly different when it comes to cognition. I think the answer to the question in the title is: Your thinking is based on a faulty premise. There simply is no single objective function.
- davidgl 5y ago
  What a brilliant comment, great insight
jrexilius 5y ago
Setting aside the specific topic of typing, the article is a great read simply as an inspiration in analysis and introspection. A great example of a "healthy sense of doubt", applied internally and externally, pushing along learning. Good piece.
ozim 5y ago
I think that is an important blog post. My idea for approach is to discuss those things on forums on the internet - at work on coffee breaks. Just to challenge own beliefs and see other arguments. While starting new project don't spend too much time discussing or blocking people. Because a lot of projects will just roll and arguing about starting project in functional paradigm is most likely just cost time and brings nothing. Best tool for the job most often is the tool that team knows best.
- scns 5y ago
  I have to disagree and risk getting labeled a member of the Rust Evangelism Strike Force. The service at NPM and that interview with that developer at Qovery paint a different picture. (Edit) Bleacher Report replacing 150 Ruby on Rails instances by 8 Phoenix on Elixir instances would be another example.
  - ozim 5y ago
    Trying not to be totally contrarian :) so with following expressions I am not a smart ass, only setting up sentences in such way to convey what I think is hard problem in delivering software. But good luck if you have 5 X(lang) developers to make that decision to go with Rust. If you have to deliver v1 of a product and have limited budget, good luck hiring 5 brand new Rust developers. What I learned as boy scout - first you have to have people that want to do the work - having loads of money can help but most of the time we don't have unlimited supply of money. Having people who want to do the job beats everything. People that are deep in tech X(or lang) rarely switch to something else and then you still have to take into account that they will write X in Rust which might not be the best way of doing things. On the other hand if we assume spherical cows or unlimited budgets/unlimited time frames for a project - we can discuss getting X devs to write in Y language.
zvrba 5y ago
Or maybe you have just outgrown your heroes and it's THEY who're stuck in their local maximum.

136 points by dustinmoris 5y ago

I've worked with people who were TDD maximalists, Agile / Scrum maximalists, microservice maximalists, Gang of Four maximalists, and even a MongoDB maximalist. What has become clear to me is that the silver bullet doesn't exist. Pick the tool for the job, and accept that exceptions exist for every rule. The maximalist is too rigid, and in the rigidity is arrogance and ultimately failure. I would even say that if you take someone who is obsessed with something like TDD, and then ban all engineers from using TDD, it is quite possible you get a better product. You don't want TDD for every problem - sometimes it simply is a barrier to success.

daxfohl 5y ago

Try going all-in for a year on the other viewpoint, assuming you've reached a local maximum on your favored approach. At worst you'll be slightly less efficient for a year. At best you'll be more efficient and come out of it with a whole new dimension in your skillset.

oreally 5y ago
I wonder how much he'd hate his heroes if he tries something like that. Or maybe he'd decide it's not worth his time and money and stay stuck in his own bubble.
AdrianB1 5y ago
Just try to listen to the other viewpoint from someone competent, it may be enough.

shannifin 5y ago

Nice to see a recommendation for Petzold's The Annotated Turing in the post, it's a great read. I'm not sure I really understand the writer's concern though. The debates he mentions (functional versus object-oriented programming, dynamic versus static typing, oral versus written collaboration) don't seem like issues with right or wrong answers in and of themselves, but rather issues for which to compare and contrast benefits and tradeoffs according to one's goals, priorities, and preferences in a given situation. Disagreement with heroes in such a context seems a complete non-issue. > "could I be stuck in a local maximum?" I perhaps don't quite understand his meaning of the question, but as perfection is infinitely precise and necessarily unattainable, everyone (who's at least honestly doing their best) is always stuck at a personal local maximum, and will be until they find a new maximum through continued learning. And I think there'll always be enough variety of experiences and preferences among individuals that we'll never all completely converge to the same maximum.

timClicks 5y ago

Many of the author's heroes became prominent because it benefited them as consultants pushing their methodology. Those methodologies are designed (in part) to make it easier for younger, less experienced programmers to contribute to large software projects. I would say that it's okay to disagree with them over time as you gain experience and understanding.

daxfohl 5y ago
Actually, I think this is the correct response. @everyone please ignore my response below/above/wherever.
zozbot234 5y ago
Agile methods are not designed for "younger, less experienced programmers". They work best for enabling successful, experienced hackers.
- barrkel 5y ago
  In practice, Agile is often Scrum, which in practice breaks down tasks into bite-sized blobs of functionality on a conveyor belt of work, to be implemented by a team of interchangeable human resources.
  - FearNotDaniel 5y ago
    That would be Kanban... And in practice, many orgs are performing a pseudo-Scrum which is really just a timeboxed Kanban with a lot of cargo-culting of the Scrum ceremonies.
- timClicks 5y ago
  If you watch Robert C. ("Uncle Bob") Martin's talks, he spends a lot of time talking about how the first generation of programmers didn't need agile because they were already professionals from other fields. The agile methodologies were created because the industry decided to hire younger, less experienced, and less diligent, staff.

Apofis 5y ago

This may have been incidental but I think I was just triggered for the first time in my life. The far right is now statically typed and dynamic typing is now the left? /slight tang of sarcasm

KSteffensen 5y ago
Makes sense. The right is usually the law-and-order side while the left is the live-and-let-live hippies. :-)

random314 5y ago

His heroes are the agile manifesto folks? Yeah, it is a good thing he disagrees with them. Bob Martin has been posting howlers for a long time. One of the problems with never formally studying CS is that you might never discover real computer scientists who could be your heroes. Who could disagree with Lamport, peyton jones, Hinton?

Animats 5y ago
Agile is a tool for feature-dominated webcrap where "sort of works" is good enough. It's not useful for, say, database internals or hard real-time. If you just need to put ads on a page, a deep knowledge of Knuth and Dijkstra will just make you unhappy with the mess that HTML/CSS/Javascript has become.
- hoseja 5y ago
  Sadly, this sort of software seems to dominate discussion nowadays.
  - deleted 5y ago
    [deleted]
- zozbot234 5y ago
  > Agile is a tool for feature-dominated webcrap where "sort of works" is good enough. It's not useful for, say, database internals or hard real-time. Have to disagree there. The basic point of "agile" methods is to promote small-scale teams (with the fabled "two-pizza team" at the largest end) doing heavily-iterative development with an emphasis on refactoring and continuous delivery ("release early and often"). There's no reason why this could not work for these sorts of projects.

Animats 5y ago

"After I read the book, I've come to understand that general-purpose static type system can never prove unequivocally that a generic program works. That's what Church, Turing, and Gödel proved." Oh, not that again. If a program is close to undecidable, it's broken. Microsoft ran into this with the Static Driver Verifier. If they couldn't prove that a driver could not blow up the kernel after the prover had cranked for a few minutes, the driver is rejected. Really, if you don't have easily proven decidabilty in your device driver, it has no business going into production.

infogulch 5y ago
I'd like to see the industry move towards proof-carrying code. This takes decideability checking one step farther: not only does your driver have to be decideable, you have to prove that ahead of time and include a proof that can be checked in linear-time right along with the binary. At that point even signatures become redundant; you don't need to trust any central authority if you have proof of the programs validity right there.
- Animats 5y ago
  You still need some criterion to verify against. The Static Driver Verifier is successful because its criterion is just "will not blither all over memory" or make an incorrect API call. Passing the check doesn't mean the driver will drive the device properly, but it shouldn't crash the kernel. Or offer buffer overflow exploits.
  - Joker_vD 5y ago
    Yes, those exactly are criteria that PCC was developed in mind with: "examples of safety properties that are covered in this paper are memory safety and compliance with data access policies, resource usage bounds, and data abstraction boundaries" (from the abstract of [0]). That's what safety is, after all: that something bad does not happen. [0] George C. Necula, Peter Lee. Safe, Untrusted Agents using Proof-Carrying Code. 1998. http://www.cs.cmu.edu/afs/cs/project/pop-10/member/petel/www/publications/lncs98.pdf http://www.cs.cmu.edu/afs/cs/project/pop-10/member/petel/www...

aliswe 5y ago

> I don't have a formal degree in computer science That ploeh does not have this is to me a big shock. It would not be a big stretch to say that every .NET systems architect worth their salt has at least stumbled upon his content once in their life.

flohofwoe 5y ago
What has one to do with the other though? Writing (and "architecting") good software is mainly craftmanship which is learned via experience, not something that requires a "formal degree". The years at university are probably a good place to get some of that experience, but "computer science" is mostly tangential to the actual problems of programming.
- aliswe 5y ago
  yeah, ik its kinda irrational, but he is a very academic and scientific type of guy. oh, and for the record I dont have a degree either.

darawk 5y ago

> To the right, in this context, means more statically typed. While the notion is natural, the sentence is uninformed. When I wrote the article, I hadn't yet read Charles Petzold's excellent Annotated Turing. Although I had heard about the halting problem before reading the book, I hadn't internalised it. I wasn't able to draw inferences based on that labelled concept. > After I read the book, I've come to understand that general-purpose static type system can never prove unequivocally that a generic program works. That's what Church, Turing, and Gödel proved. This is not exactly the right inference. You can have a language that has the property "if it compiles, it works", that language just cannot also be Turing complete.

naasking 5y ago
> This is not exactly the right inference. You can have a language that has the property "if it compiles, it works", that language just cannot also be Turing complete. Not quite. It can be Turing complete as long as you fill in the proof steps the compiler can't infer for itself. Coq, Idris and other theorem provers can all produce programs with arbitrary looping behaviour given valid proofs are generated.
- dllthomas 5y ago
  This seems wrong. If you require a proof that a program halts for all input (or of any halting equivalent property), then there will be programs you cannot write - those that possess the property in question but for which no proof exists, and those that actually lack the property. In particular, you obviously cannot write a universal Turing machine in a system that requires a proof of halting for all input, because it by definition won't halt if it is fed a program that does not halt. A system that cannot build a UTM is not Turing complete.
  - naasking 5y ago
    I'm not sure what exactly is not right. Type-based theorem provers exist (Coq, Idris, Agda, etc.). They can be used to write Turing complete programs (eg. CompCert). If you encode the properties that need to be satisfied and the compiler successfully compiles your program, then those properties are satisfied. I suspect you've simply misunderstood the claims that were made. In particular, the above does not entail that you can write any program. Some programs simply have no valid type.

nodejs_rulez_1 5y ago

> In a recent discussion, some of my heroes expressed the opinion that they don't need fancy functional-programming concepts and features to write good code. I'm sure that they don't. It's OK to disagree with Bob Martin - many do. The days of TDD silver bullet training gold rush are over.

fredrikholm 5y ago
Not necessarily a Robert Martin fan, but he is writing Clojure (functional Lisp) as his primary language these days.
pjmlp 5y ago
My favourite TDD question to presenters was to show how to design GUIs according to specific UI/UX criteria, while following the TDD religion.

seanwilson 5y ago

> "To the far right, we have a hypothetical language with such a strong type system that, indeed, if it compiles, it works." > For good measure, despite my failure to understand the implications of the halting problem, I'm otherwise happy with the article series Types + Properties = Software. You shouldn't consider this particular example a general condemnation of it. It's just an example of a mistake I made. This time, I'm aware of it, but there are bound to be plenty of other examples where I don't even realise it. I don't see what the mistake is. Halting and Godel's incompleteness theorems only say there exists problems where you don't know if there's a solution or not - that's very different from saying you can't come up with solutions for specific problems. For example, you can't tell if any arbitrary crazy program halts but most loops people code in practice have trivial proofs of halting because it's usually simple loops from 0 to the fixed length of a collection. If some code is full of loops where you're nervous about them halting a lot, there's probably something very wrong with the design that would get flagged in integration tests and during use too. It's been demonstrated you can statically prove the correctness of something as complex as an operating system for example (see https://sel4.systems/About/ https://sel4.systems/About/) so the above isn't much of a barrier to proving interesting program properties: seL4's implementation is formally (mathematically) proven correct (bug-free) against its specification, has been proved to enforce strong security properties, and if configured correctly its operations have proven safe upper bounds on their worst-case execution times

talkingtab 5y ago

A great post! Thanks. For me, it is the meta issue of how we think about the examples that is most interesting rather than the examples. One contribution I have is that we humans are biased toward action as we must be. And biased toward results. These are survival skills. If acting requires me to adopt some belief, I do so - even if that belief is incomplete or even false. And if acting on that belief produces some result it is grows stronger. Long, long ago I had a friend who had an old car and not much much money. The car would often not start on the first try, so he would get out, open the hood and wrap some exposed wires in electrical tape, then get back in the car and it would usually start right up. I offered to replace the exposed wires. I was bewildered to find that the wires were not connected at either end! I had watched the process where wrapping the wires fixed the problems, yet one had to be insane to accept that as the cause. I finally understood that the float on the carburetor was defective (acting like the float in a toilet). The time delay in wrapping the wire allowed the gas in the flooded engine to evaporate enough to start. I replaced the float and that fixed the problem. At least that is what I now tell myself, but perhaps my own belief system is as flawed as wrapping the wires? Perhaps it was not the float but a gasket or that it fixed an interaction between some unknown type of subatomic particles which in the future will be used for near instantaneous space travel? So in answer to the question "Am I stuck in a local maximum" the answer is "always". (edit for clarity)

open-source-ux 5y ago

I don't have any heroes in the field of computer science (hero is too strong a word for me), but there are many experienced people I admire who express ideas or opinions I find appealing. An example: I admire Niklaus Wirth (creator of Pascal, Modula-2, Oberon) and in particular his preference for strongly-types programming languages that are small in size. He's spent a lifetime in the computer science field, but he disagrees with the ideas of others - and others disagree with him. Wirth is critical of functional programming as a programming paradigm, for example. Overall, can anyone argue that someone is emphatically right or wrong in this field? Everyone settles on a philosophy that shapes their outlook based on their experience and research. And that outlook might change over time. That accrued wisdom and experience (or inexperience!) will naturally cause someone to gravitate to one philosophy, or disagree with another.

brianmcc 5y ago

I really enjoyed this and felt it resonates quite a lot with my own views. I think a lot about there not being a "right" or "wrong" approach, absolutely, rather approaches which work for some teams and individuals that don't succeed for others. As well as being too easily distracted by shiny new tech toys, our industry's other big dysfunction is a ready insistence by "thought leaders" about how to do things right, the One True Way.

lmilcin 5y ago

I found that no single "pure" paradigm is a best response to development needs. By pure I mean pure OOP, FP, TDD, etc. Instead I am currently working on mixing various paradigms together to achieve better result. Some examples: -- I use OOP to flesh out the domain model of my application, but use FP for most of the infrastructure and application around it. Some processes within OOP model are implemented using FP. I use OOP for its strength in bringing meaning and clarity to the most important part of the model. -- I use unit tests only for the important, non-trivial constraints on the tricky parts of the code. For the rest I prefer functional tests -- focusing on testing external application behavior lets me refactor the code easily without spending time on a huge mass of unit tests that need to be brought to shape after every refactoring. I still test the application -- the test suite ensures I will get notified if application misbehaves, but now it is not tied to actual internal implementation.

naasking 5y ago

> After I read the book, I've come to understand that general-purpose static type system can never prove unequivocally that a generic program works. This sentence seems confused. What is the meaning of "works"? The most general interpretation seems to be "works" = "satisfies some proposition P". But, "general-purpose static type system can never prove unequivocally that a generic program satisfies some proposition P" is false. In fact, static type systems unequivocally prove many propositions about general programs. The generality of the propositions depend on the expressive power of the type system. If you dial static typing up to 11 (dependent types), then P can be literally any proposition about the program. This sometimes requires programmers to provide the right type annotations, but this is no different in principle from needing to program behaviour, ie. a computer needs you to write out the program describing what you want it to do, and a type checker needs you to write out what you want it to check. You wouldn't expect to just wave your hand and your computer create the program you have in your mind for you? Or write out the kind of program you want in ambiguous, natural English and have the computer figure out exactly what you meant?

IceDane 5y ago

My theory is that these career evangelists have spent too much of their careers doing just that: evangelizing. They haven't actually been writing code or building software. They've just been telling everyone how they think they should be building software. I have arrived at the same conclusions as you wrt. static vs dynamic typing. A sufficiently powerful static type system entirely subsumes dynamic typing, and there is quite literally nothing you can accomplish with dynamic typing that you cannot accomplish with such a static type system. After writing software using both dynamically and statically typed languages, this is the conclusion I have arrived at, and it feels like a no-brainer to me. This is so much of a no-brainer to me that I can't help but be biased when someone disagrees. I want to think that I am open to having my mind changed, but I have yet to see any evidence that sways me. I do think perhaps a lack of external demand for code quality and correctness might sometimes be at fault. When all you need to do is write the code for the happy path correctly, maybe you just feel static types get in the way. I've converted several javascript projects to typescript, and in every single one of those projects, the conversion exposed multiple(in the tens or sometimes hundreds) cases of incorrect handling of null/undefined or just type mismatches. I know this has to do with culture and the specific company and engineering discipline, but what I'm trying to get at is that I think a lot of people out there are writing code this way: just hack away until it works, and fix it if it ever crashes. When this is all you want, maybe proving correctness to the compiler is too bothersome. But I don't and won't work this way.

1337shadow 5y ago
> there is quite literally nothing you can accomplish with dynamic typing that you cannot accomplish with such a static type system. Exactly, and all you need is Any > a lot of people out there are writing code this way: just hack away until it works, and fix it if it ever crashes. I'm willing to agree that Non-TDD hackers should go for static typing.
- uDontKnowMe 5y ago
  >Exactly, and all you need is Any If you're using Any, then your program is (at least in part) dynamically typed. I don't think it's fair to say "look! Static typing is better than dynamic typing in any and all situations! (so long as it can use dynamic typing)"

jbjbjbjb 5y ago

I find the “hero” wording a little grating and I think it’s the problem here. They’re just people expressing mostly subjective opinions. The problem is developers put these people on a pedestal and take their word as gospel. In reality you need good judgement to understand and appreciate the trade offs. I do also think developers look for silver bullets and happy to accept anyone with some authority telling them it exists and they needn’t think for themselves.

h2odragon 5y ago
article says: "I find myself disagreeing with my heroes on a regular basis, and that makes me uncomfortable." Why should you agree with your heroes? You are not in the circumstances they were whenever they became your hero.

hyperman1 5y ago

I first read this problem: micro-management, red tape, overly bureaucratic processes, and a lack of developer autonomy and then this: essentially have no process. Everything is ad hoc, nothing is written down, deployment is a manual process, and there are meetings and interruptions all the time They weird thing about enterprise is: I see both happen at the same time. We have an issue tracker, but its a bad product and the metrics are heavily politicized. So we track our issues in excel. No way to know on whose excel your name appears. We have a document sharing platform, but the fine grained access control makes it impossible to let the relevant people read the document. So we mail different versions around and forget to update the central truth. We have architecture documents, but they have nothing to do with what runs in reality. We can't document what runs in reality because it would conflict with the offical architecture documents. We have a release process that takes a week or a month, and many paper is written about risk and impact, test plans, deployment plans, resource availability, etc... But the actual risks must stay unspoken lest the release is cancelled. Tests were done on an environment that is forbidden to resemble production. People ignore the deployment plan and do whatever they like, and thank god because the deployment plan is unusable. People are not available when things go wrong, because the deployment plan booked them for 2 days for doing 5 minutes of work and they decided their part was done. The worst problem: Instituting any real process is impossible, because officially there already is one.

scns 5y ago
This may be the worst thing i read on HN ever.
- hyperman1 5y ago
  In what way? Language? Content? World view?
  - scns 5y ago
    The horror i felt imagining how the people who work in such an environment might feel, if they haven't checked out allready. (Edit) So it isn't actually that dramatic.

bjornsing 5y ago

Interesting questions, but I think one aspect is sorely missing from the analysis: personality. The author sees himself as an introvert. He likes typed languages. He reads books about Turing, Church and Gödel. He seems to like understanding things. Sounds to me like he has a cognitive style where what Kahneman calls System 2 is dominant. I’m not familiar with all his “heroes”, but I would assume most people become “heroes” because they are somewhat extroverted. They like talking to people. They like going to conferences. They like being in the limelight. With this often comes a different cognitive style, closer to what Kahneman calls System 1. People are incredibly different when it comes to cognition. I think the answer to the question in the title is: Your thinking is based on a faulty premise. There simply is no single objective function.

davidgl 5y ago
What a brilliant comment, great insight

jrexilius 5y ago

Setting aside the specific topic of typing, the article is a great read simply as an inspiration in analysis and introspection. A great example of a "healthy sense of doubt", applied internally and externally, pushing along learning. Good piece.

ozim 5y ago

I think that is an important blog post. My idea for approach is to discuss those things on forums on the internet - at work on coffee breaks. Just to challenge own beliefs and see other arguments. While starting new project don't spend too much time discussing or blocking people. Because a lot of projects will just roll and arguing about starting project in functional paradigm is most likely just cost time and brings nothing. Best tool for the job most often is the tool that team knows best.

scns 5y ago
I have to disagree and risk getting labeled a member of the Rust Evangelism Strike Force. The service at NPM and that interview with that developer at Qovery paint a different picture. (Edit) Bleacher Report replacing 150 Ruby on Rails instances by 8 Phoenix on Elixir instances would be another example.
- ozim 5y ago
  Trying not to be totally contrarian :) so with following expressions I am not a smart ass, only setting up sentences in such way to convey what I think is hard problem in delivering software. But good luck if you have 5 X(lang) developers to make that decision to go with Rust. If you have to deliver v1 of a product and have limited budget, good luck hiring 5 brand new Rust developers. What I learned as boy scout - first you have to have people that want to do the work - having loads of money can help but most of the time we don't have unlimited supply of money. Having people who want to do the job beats everything. People that are deep in tech X(or lang) rarely switch to something else and then you still have to take into account that they will write X in Rust which might not be the best way of doing things. On the other hand if we assume spherical cows or unlimited budgets/unlimited time frames for a project - we can discuss getting X devs to write in Y language.

zvrba 5y ago

Or maybe you have just outgrown your heroes and it's THEY who're stuck in their local maximum.