9 ms·

Programmer as wizard, programmer as engineer (2018)

166 points by matfil 8y ago

revskill 8y ago
Universal, component-based application is the answer of the "boundary" in the article. You got both wizard and engineering solution, which is simple, easy to delete and FUN.
- moltar 8y ago
  Is it what DLL files on Windows are/used to be?
  - revskill 8y ago
    Sure. As soon as we could pack independent component into portable unit, then it's the solution we want.
  - sebcat 8y ago
    My understanding is that it is more like COM, CORBA and NetBeans. I guess it is one of those things that can mean different things to different people. At its core, it's just separation of concerns, usually branded and (re-)packaged. Often times, that core idea seems to get lost somewhere around the way...
    - teknologist 8y ago
      I will assume that you meant JavaBeans (the component classes) and not NetBeans (the IDE).
      - sebcat 8y ago
        Indeed, thanks!
    - DonHopkins 8y ago
      The core idea of integrating components written in multiple languages got totally lost along the way from COM (a way for components written in different languages to interoperate while avoiding the DLL Hell problem) => ActiveX (a web-friendly marketing name for COM because you couldn't google for "COM") => Java Beans (a marketing name for vaporware to express the idea that Sun had an alternative to ActiveX, which was misleadingly positioned as a replacement for AWT, but when it was finally implemented was only useful for web servers, and certainly wouldn't ever work with anything but 100% Pure Java) => POJO (Plain Old Java Objects, which have nothing to do with user interface widgets or any other language than Java).
- bboreham 8y ago
  Brad Cox described this in “Object-oriented Programming”, a book I first read in 1989. Building applications. simply by plugging together off-the-shelf components remains a beautiful vision, and it is almost entirely unrealised 30 years on.
  - revskill 8y ago
    It's possible now.
  - marcodave 8y ago
    > “Object-oriented Programming”, [...] Building applications. simply by plugging together off-the-shelf components I've read just here on HN that microservices architecture is none other than an implementation of the original concept of OOP.
    - bboreham 8y ago
      Possibly. But my understanding of “the concept of OOP” involves a separate identity for each object. Suppose the thing you are dealing with is “Customers”, then with OOP each Customer is a separate object, while with MSOP you must communicate to the service which Customer you are talking about. The thing you talk to and the thing you are talking about have different identities.
jgoodhcg 8y ago
Cute way to put it. Just like many I'm in the midst of painful transition from quick and dirty mvp to well engineered end product. Everything Rich Hickey has been giving talks on and clojure itself seem to be the best _solution_ I've seen.
quantombone 8y ago
Using PyTorch (and the broader space of machine learning algorithms under the “deep learning” category) really makes me feel like a wizard. But the downside to being Python-dependent is that putting PyTorch stuff into products not easy. I hope PyTorch 1.0 will change that. Note: This article was not written with Machine Learning in mind, and I will have to re-read the article to better articulate my thoughts on “Machine Learning Wizardry” and juxtapose my own ideas with those of the article author. Kudos to author: The article’s main metaphor is excellent because it got my creative juices flowing (i.e., brain working at 110% for a few brief moments).
- mlboss 8y ago
  You can always use ONNX to convert PyTorch trained model to any other format. https://onnx.ai/ https://onnx.ai/
fullstackchris 8y ago
I wholeheartedly agree that software development as a whole, especially in the past few years, has been more in the spirit of 'wizardry'. But with the speed at which the sheer amount of new software 'stuff' comes out each year, there's simply hasn't been enough time to develop rigorous engineering specs or best practices for all of those tools. Perhaps we're starting to see an initial version of a universally accepted model, at least for the frontend, with the mentioned Typescript + Flux (I'll just say Redux). Many assume that Redux is only a state container, and it is at face value, but more semantically it is (when implemented correctly) a very logical boilerplate where it puts everything related to state in its proper place, so any developer can look at the code base and immediately get a general idea of where state is set, what events exist and how the state is used throughout an application. I'd like to see things like these very strict patterns emerge for other tools, like Node. For example, I could imagine a snippet of THE universally accepted express boilerplate for a login, given a specific backend... (can an email + password login really get _that_ customized?) ...argh, on second thought I suppose it can, but even then such a boilerplate could have sections with a freebie spot for customizations Eh, it's late and I wonder if such universal pattern ideas are a pipedream... I suppose only time will tell...
markmiro 8y ago
It would be interesting if programmers took "sketching" to be a valuable and necessary part of programming. It's common practice for painters to make a pencil draft first. It's common in industrial design to produce prototypes. However, when it comes to code we treat it similar to writing. We may have a first draft, but the final draft is often nothing more than a cleaned-up draft. I could be wrong. I never wrote professionally. It would be interesting if we had languages that would be great for prototyping but designed to be unusable in production. However, I'm having a hard time imagining properties that don't already exist in languages like Python and JS. You want weak typing of course, but you'd be ok with poor security. Maybe we'd some nice features that would make the language run slowly since it running in prod would be a non-goal.
- supercleanbro 8y ago
  I mean, some writers like to make up outlines and stuff to plan out books, though so do stream of consciousness style and clean it up in later drafts. I can see that planning out your code can be beneficial and honestly ideal. Just break it down in a sort of outline what all the program needs and then what each part of the code would need, etc.
- DonHopkins 8y ago
  How about all the libraries and tools turn over every two months? Oh, there's JavaScript!
  - strken 8y ago
    Node.js turns 10 in a few months. TypeScript is 6 years old. Webpack is 6 years old. React is 5 years old. Redux is 3 years old. Front end JS and TS went through a phase of rapid change 2 to 6 years ago, but I don't think that rapid change is still happening. Certainly there are new libraries, and maintainers still like to play fast and loose with backwards compatibility, but it's not obvious that you have to keep up to date with every single new framework anymore.
- taneq 8y ago
  Part of the reason I like C-style header files for declarations is that I use these as a 'sketch' / 'story map' for the piece of code I'm writing. I tend to spend much more time per line writing my headers than my code, because once I've thought through the public interface and it's close to its final form, the internal code pretty much writes itself.
kelnos 8y ago
> What do we do? For me, it's "simple": I never ever ever ever write one to throw away (and I do mean that in a literal, absolutist sense, which is rare for me). This does mean that I often can't use wizard tools (so it can be less fun to build). I never use dynamic languages, and build on the JVM (Scala or Java), because I know it will scale, and there are battle-tested libraries that do nearly everything under the sun floating out there. (If your org has a different "blessed" platform for production services, then use that.) It isn't quite as quick to build the MVP as if I was just hacking something together. But I can do it fast enough, and still end up with a maintainable, evolve-able code base. It's not a perfect process. My first version usually has only a few tests that verify behaviors that I had trouble modeling clearly in code and didn't feel confident about. Sometimes I miss error cases here and there that someone else has to find and deal with later. Also note that, because I use strongly-typed languages, I can push a decent amount of correctness verification onto the type system, so the compiler catches a ton of errors that I'd need a giant test suite to catch using many dynamic languages. The tests that I do write focus on logical correctness, not code correctness. But at the end of the day, I deliver products on time that I feel much more comfortable being robust in a production environment than build-one-to-throw-away prototypes. Stuff that I'm fine holding a pager for if I need to. I have several "prototypes" that are still running in production several years after my first release, maintained by other people after I've moved on. And by and large they still contain a lot of the original code, and the design remains close to (and/or continues to be heavily influenced by) the original design. On the flip side, I've had to deal with code that's been thrown together with the expectation that it could be thrown away later (of course it never can be), and it's incredibly difficult to bring it up to a robustness level that would be deemed acceptable for a generally-available product. These code bases constantly set off pagers for dubious reasons and write unactionable crap to logging systems... and it doesn't have to be that way!
- jrs95 8y ago
  In my experience even things which are explicitly prototypes can be dangerous, as it can appear to be in the short term interest of the business to take a prototype and modify it as little as possible to get it shipped. I've seen this result in massive headaches and thousands of wasted man hours, and for what? Shipping an "MVP" that can't be effectively iterated on 2 weeks sooner? The worst case of this I saw, the technical aspects of this were bad enough that it was (in my opinion) what caused the product to fail.
dmichulke 8y ago
A really good alternative to Python and C: Clojure + Java Mostly because - Clojure is very very terse - Java has the battle-tested libs - they run on the same (J)VM - so no FFI required in your code
- zimablue 8y ago
  I think the problem with this is, if I'm going to say "completely rewrite this [function/class/module] into another language", then there's a good chance you want that language to be about as fast as possible. I guess because a lot of problems fall straight from "speed absolutely doesn't matter" to "this is the bottleneck of the whole thing". I think that is a reason why there's a lot of python/c++ in hedge fund land. I've written some clojure but don't know the c interop story for it.
  - dmichulke 8y ago
    > completely rewrite this into another language You just opened the box of pandora for a million reasons most of which are unbeknownst to the both of us :) But for the sake of argument I will continue under this assumption > language to be about as fast as possible If you're interested in Mathy stuff like Machine Learning, FFT, ... then maybe. But even for those you usually have JNI bindings, so it's easy to use most of those mathy C libs if necessary. But I guess that 95% of all software isn't about speed but about something else (correctness, maintainability, safety against threats, portability, ...) because costs today are usually dictated by manpower costs or those arising from safety/security incidents and much less often by hardware costs compared to say a decade ago.
    - zimablue 8y ago
      When I say completely rewrite, I mean rewrite a class, method or library which is what we're talking about (since you probably started writing in python/clojure). Sorry, it was poorly phrased.
    - mywittyname 8y ago
      My limited experience taught me that doing anything mathy in Java is painful due to types (well, more lack of math-specific built-in types). The input/output is usually going to be in primitive types while the library likely uses some home-brew custom typing(or Commons, if you're lucky) for Complex numbers, matrices, etc. So you have to do the type conversion song-and-dance. The Python libraries never seem to care. Just give them a list of a list of number-ish values and off they go. Java might have some great ML/Math libraries, but the fact that Python dominates data science suggests that my experience is a real-world pain-point.
rgoulter 8y ago
"Wizard, Engineer" reminds me of Yegge's "Software Liberal/Conservative" approach to risk. https://plus.google.com/110981030061712822816/posts/KaSKeg4vQtz https://plus.google.com/110981030061712822816/posts/KaSKeg4v... Albeit, rather than "wizards like implicit/magic, engineers prefer explicit/boilerplate/maintainability", the difference Yegge suggests was management of risk.
- matfil 8y ago
  Thanks, that's a pretty interesting take. (Also a reminder that we're probably going to lose some interesting stuff when Google+ goes kaboom...)
Adamantcheese 8y ago
A wizard then should have a spellbook, one filled with all sorts of spells written out for immediate use. Maintainable hacks if you will. Those are probably just scripts though I suppose?
- bryanrasmussen 8y ago
  The problem with spellbooks is one seen in media on the subject of wizardry, that the wizard spends a lot of time memorizing spells. This is why the most useful spells get turned into artifacts that the wizard does not have to always memorize to know how to cast correctly. What is needed is an artifact like spellbook, that the wizard when faced with a situation could describe it to the spellbook and get back the correct spell or combination of spells to solve the situation. Attempts have been made to create such an artifact, but unfortunately the resulting spellbooks still take a long time to find the correct spell and when reading the spells you often find that there are missing ingredients or a complicated set of gestures that must be performed to make the spell work, and you have to read all about these gestures in turn to figure out which ones you really need.
  - thisiszilff 8y ago
    Have you heard of hoogle (Haskell's type signature search)? It almost exactly feels like what you've described.
    - bryanrasmussen 8y ago
      no, thanks for telling me!
bitwize 8y ago
Lisp is the ultimate "wizarding" programming language. But when I miraculously was called upon to maintain an enterprise code base in Common Lisp, it was an absolute joy. Because whenever they encountered a roadblock in maintenance, the Lisp wizards who had come before me just wizarded up a solution. One of the things that stuck out was that it had its own custom test framework, that was head and shoulders above XUnit, Mocha, or any other commonly-used test framework. Adding a new test was virtually a one-liner; the test would generate test data , send it to the server, and check the server's response against an XML template provided by the test case.
zimablue 8y ago
I'm sceptical of his argument that "we've gone from dynamic being trendy back to typed(java), because 'people' had to maintain dynamic codebases". An equivalent but probably equally not-the-real-explanation argument would be that we've gone from an era of opportunity into an era of oligopoly as the internet titans have emerged and so the "coolest kids" who everyone cargocults have gone from being fast-growing startups to members of the big-5 elite. Kind of the same as his argument just with detail on who his "people" are and why, but it changes the implications. I think it's more as he touches on that the two are converging. I think eventually some sort of pluggable typing will win (the "proofs" on your system won't be a single compilation pass but maybe different typing for different parts of the system, and specific proofs run between compile and runtime as the two blur), which will look more like gradual typing.
- jondubois 8y ago
  According to the definitions provided, it seems that engineering is the wrong approach for the vast majority of systems (especially popular web-based platforms). We should be leaning more towards wizarding. Most systems change all the time. So long as a company needs executives to make decisions and steer in different directions depending on the economic conditions, companies also need the flexibility to change their code. Even the Linux Kernel which is now decades old is still being changed all the time. If you use tools which assume that every line of code you write is not going to change, then unless you're programming an aircraft or a medical hardware device you're probably using the wrong tools. You should not assume that just because some low level module is deeply nested within the code, that it means that it should not be changed or thrown out. That's why I prefer dynamically typed languages for web systems; they start with the rigth assumption about the ever-evolving nature of the project. If JavaScript was designed to be statically typed from the beginning, web browsers would not have attained the usefulness or popularity that they have today.
  - zimablue 8y ago
    I'm not sure to what extent I believe this. There are a lot of overlapping distinctions, some blurry, I think. He chooses some for wizardry vs engineering but I don't know whether they're best. For example he puts "magic"=implicit, but I would normally say highly implicit code is "engineering" because it implies that someone has understood and explored the problem enough to write a very tuned framework or library. I think reasons for putting very implicit code in with wizardry could be that it might be that the person using it doesn't understand it, and that it's somewhat in opposition to strong typesystems wizardry/engineering high uptime/downtime possible efficiency important/efficiency not important large codebase/small codebase not implicit/implicit (I disagree most with this one) problem is understood/code is exploratory (I think this is the most important distinction) high specialism/low specialism of coders changes slowly/changes quickly typed/untyped I would say websites are engineering because the problem space is well understood, they need high uptime and they tend to be written by specialists (React guy, django guy etc).
jbergens 8y ago
It sounds like optionally typed languages like Typescript should be really good if you want to start in "wizarding" mode and then change the code more into an "engineering" solution.
luord 8y ago
There are quite a few false assumptions here. > People have certainly managed to create test suites that make it harder to maintain the code. Sure, but pretending that this is the norm is just not true. And arguing against an extreme case can be done against anything . > I get the impression Google has been able to migrate a lot of C++ and Python to Go using this approach. Not only is this heavily suspect (I'd love to read a citation for this) but if there's a language that represents the "engineering" side (?) pretty clearly, it is C++. > Gradual type systems has started to garner a lot more interest. No they haven't. Those have existed for over five decades . Another problem with these articles is how they seem oddly ignorant of the history of computer science. This is specially odd coming from a PhD (assuming that I looked up the right person in google). And, last but not least, some of the best engineered pieces of code are precisely shells, dynamic languages and frameworks. It's things like these what makes these articles seem like they were written by java developers annoyed because java is no longer the trendiest toy.
- iseeyoubydesign 8y ago
  >People have certainly managed to create test suites that make it harder to maintain the code. example? these test suites are they hard to maintain or is the code its testing harder???? >And, last but not least, some of the best engineered pieces of code are precisely shells, dynamic languages and frameworks. I don think its realized that it might not be wizardy or engineering but something more like art. architecture
- marcosdumay 8y ago
  > No they haven't. Those have existed for over five decades. There is nothing in "started to garner more interest" that implies something is new. It's even the other way around.
casper345 8y ago
"Python + C" I am taking this approach working/learning with the raspberry pi. Although it can do both python and C, I know in electronics working with low level C will be a lot beneficial for my learning and code itself. When I migrate to more advance topics to scripting and ML, I will use both PYthon and C to implement what is needed.

166 points by matfil 8y ago

Universal, component-based application is the answer of the "boundary" in the article. You got both wizard and engineering solution, which is simple, easy to delete and FUN.

moltar 8y ago
Is it what DLL files on Windows are/used to be?
- revskill 8y ago
  Sure. As soon as we could pack independent component into portable unit, then it's the solution we want.
- sebcat 8y ago
  My understanding is that it is more like COM, CORBA and NetBeans. I guess it is one of those things that can mean different things to different people. At its core, it's just separation of concerns, usually branded and (re-)packaged. Often times, that core idea seems to get lost somewhere around the way...
  - teknologist 8y ago
    I will assume that you meant JavaBeans (the component classes) and not NetBeans (the IDE).
    - sebcat 8y ago
      Indeed, thanks!
  - DonHopkins 8y ago
    The core idea of integrating components written in multiple languages got totally lost along the way from COM (a way for components written in different languages to interoperate while avoiding the DLL Hell problem) => ActiveX (a web-friendly marketing name for COM because you couldn't google for "COM") => Java Beans (a marketing name for vaporware to express the idea that Sun had an alternative to ActiveX, which was misleadingly positioned as a replacement for AWT, but when it was finally implemented was only useful for web servers, and certainly wouldn't ever work with anything but 100% Pure Java) => POJO (Plain Old Java Objects, which have nothing to do with user interface widgets or any other language than Java).
bboreham 8y ago
Brad Cox described this in “Object-oriented Programming”, a book I first read in 1989. Building applications. simply by plugging together off-the-shelf components remains a beautiful vision, and it is almost entirely unrealised 30 years on.
- revskill 8y ago
  It's possible now.
- marcodave 8y ago
  > “Object-oriented Programming”, [...] Building applications. simply by plugging together off-the-shelf components I've read just here on HN that microservices architecture is none other than an implementation of the original concept of OOP.
  - bboreham 8y ago
    Possibly. But my understanding of “the concept of OOP” involves a separate identity for each object. Suppose the thing you are dealing with is “Customers”, then with OOP each Customer is a separate object, while with MSOP you must communicate to the service which Customer you are talking about. The thing you talk to and the thing you are talking about have different identities.

jgoodhcg 8y ago

Cute way to put it. Just like many I'm in the midst of painful transition from quick and dirty mvp to well engineered end product. Everything Rich Hickey has been giving talks on and clojure itself seem to be the best _solution_ I've seen.

quantombone 8y ago

Using PyTorch (and the broader space of machine learning algorithms under the “deep learning” category) really makes me feel like a wizard. But the downside to being Python-dependent is that putting PyTorch stuff into products not easy. I hope PyTorch 1.0 will change that. Note: This article was not written with Machine Learning in mind, and I will have to re-read the article to better articulate my thoughts on “Machine Learning Wizardry” and juxtapose my own ideas with those of the article author. Kudos to author: The article’s main metaphor is excellent because it got my creative juices flowing (i.e., brain working at 110% for a few brief moments).

mlboss 8y ago
You can always use ONNX to convert PyTorch trained model to any other format. https://onnx.ai/ https://onnx.ai/

fullstackchris 8y ago

I wholeheartedly agree that software development as a whole, especially in the past few years, has been more in the spirit of 'wizardry'. But with the speed at which the sheer amount of new software 'stuff' comes out each year, there's simply hasn't been enough time to develop rigorous engineering specs or best practices for all of those tools. Perhaps we're starting to see an initial version of a universally accepted model, at least for the frontend, with the mentioned Typescript + Flux (I'll just say Redux). Many assume that Redux is only a state container, and it is at face value, but more semantically it is (when implemented correctly) a very logical boilerplate where it puts everything related to state in its proper place, so any developer can look at the code base and immediately get a general idea of where state is set, what events exist and how the state is used throughout an application. I'd like to see things like these very strict patterns emerge for other tools, like Node. For example, I could imagine a snippet of THE universally accepted express boilerplate for a login, given a specific backend... (can an email + password login really get _that_ customized?) ...argh, on second thought I suppose it can, but even then such a boilerplate could have sections with a freebie spot for customizations Eh, it's late and I wonder if such universal pattern ideas are a pipedream... I suppose only time will tell...

markmiro 8y ago

It would be interesting if programmers took "sketching" to be a valuable and necessary part of programming. It's common practice for painters to make a pencil draft first. It's common in industrial design to produce prototypes. However, when it comes to code we treat it similar to writing. We may have a first draft, but the final draft is often nothing more than a cleaned-up draft. I could be wrong. I never wrote professionally. It would be interesting if we had languages that would be great for prototyping but designed to be unusable in production. However, I'm having a hard time imagining properties that don't already exist in languages like Python and JS. You want weak typing of course, but you'd be ok with poor security. Maybe we'd some nice features that would make the language run slowly since it running in prod would be a non-goal.

supercleanbro 8y ago
I mean, some writers like to make up outlines and stuff to plan out books, though so do stream of consciousness style and clean it up in later drafts. I can see that planning out your code can be beneficial and honestly ideal. Just break it down in a sort of outline what all the program needs and then what each part of the code would need, etc.
DonHopkins 8y ago
How about all the libraries and tools turn over every two months? Oh, there's JavaScript!
- strken 8y ago
  Node.js turns 10 in a few months. TypeScript is 6 years old. Webpack is 6 years old. React is 5 years old. Redux is 3 years old. Front end JS and TS went through a phase of rapid change 2 to 6 years ago, but I don't think that rapid change is still happening. Certainly there are new libraries, and maintainers still like to play fast and loose with backwards compatibility, but it's not obvious that you have to keep up to date with every single new framework anymore.
taneq 8y ago
Part of the reason I like C-style header files for declarations is that I use these as a 'sketch' / 'story map' for the piece of code I'm writing. I tend to spend much more time per line writing my headers than my code, because once I've thought through the public interface and it's close to its final form, the internal code pretty much writes itself.

kelnos 8y ago

> What do we do? For me, it's "simple": I never ever ever ever write one to throw away (and I do mean that in a literal, absolutist sense, which is rare for me). This does mean that I often can't use wizard tools (so it can be less fun to build). I never use dynamic languages, and build on the JVM (Scala or Java), because I know it will scale, and there are battle-tested libraries that do nearly everything under the sun floating out there. (If your org has a different "blessed" platform for production services, then use that.) It isn't quite as quick to build the MVP as if I was just hacking something together. But I can do it fast enough, and still end up with a maintainable, evolve-able code base. It's not a perfect process. My first version usually has only a few tests that verify behaviors that I had trouble modeling clearly in code and didn't feel confident about. Sometimes I miss error cases here and there that someone else has to find and deal with later. Also note that, because I use strongly-typed languages, I can push a decent amount of correctness verification onto the type system, so the compiler catches a ton of errors that I'd need a giant test suite to catch using many dynamic languages. The tests that I do write focus on logical correctness, not code correctness. But at the end of the day, I deliver products on time that I feel much more comfortable being robust in a production environment than build-one-to-throw-away prototypes. Stuff that I'm fine holding a pager for if I need to. I have several "prototypes" that are still running in production several years after my first release, maintained by other people after I've moved on. And by and large they still contain a lot of the original code, and the design remains close to (and/or continues to be heavily influenced by) the original design. On the flip side, I've had to deal with code that's been thrown together with the expectation that it could be thrown away later (of course it never can be), and it's incredibly difficult to bring it up to a robustness level that would be deemed acceptable for a generally-available product. These code bases constantly set off pagers for dubious reasons and write unactionable crap to logging systems... and it doesn't have to be that way!

jrs95 8y ago
In my experience even things which are explicitly prototypes can be dangerous, as it can appear to be in the short term interest of the business to take a prototype and modify it as little as possible to get it shipped. I've seen this result in massive headaches and thousands of wasted man hours, and for what? Shipping an "MVP" that can't be effectively iterated on 2 weeks sooner? The worst case of this I saw, the technical aspects of this were bad enough that it was (in my opinion) what caused the product to fail.

dmichulke 8y ago

A really good alternative to Python and C: Clojure + Java Mostly because - Clojure is very very terse - Java has the battle-tested libs - they run on the same (J)VM - so no FFI required in your code

zimablue 8y ago
I think the problem with this is, if I'm going to say "completely rewrite this [function/class/module] into another language", then there's a good chance you want that language to be about as fast as possible. I guess because a lot of problems fall straight from "speed absolutely doesn't matter" to "this is the bottleneck of the whole thing". I think that is a reason why there's a lot of python/c++ in hedge fund land. I've written some clojure but don't know the c interop story for it.
- dmichulke 8y ago
  > completely rewrite this into another language You just opened the box of pandora for a million reasons most of which are unbeknownst to the both of us :) But for the sake of argument I will continue under this assumption > language to be about as fast as possible If you're interested in Mathy stuff like Machine Learning, FFT, ... then maybe. But even for those you usually have JNI bindings, so it's easy to use most of those mathy C libs if necessary. But I guess that 95% of all software isn't about speed but about something else (correctness, maintainability, safety against threats, portability, ...) because costs today are usually dictated by manpower costs or those arising from safety/security incidents and much less often by hardware costs compared to say a decade ago.
  - zimablue 8y ago
    When I say completely rewrite, I mean rewrite a class, method or library which is what we're talking about (since you probably started writing in python/clojure). Sorry, it was poorly phrased.
  - mywittyname 8y ago
    My limited experience taught me that doing anything mathy in Java is painful due to types (well, more lack of math-specific built-in types). The input/output is usually going to be in primitive types while the library likely uses some home-brew custom typing(or Commons, if you're lucky) for Complex numbers, matrices, etc. So you have to do the type conversion song-and-dance. The Python libraries never seem to care. Just give them a list of a list of number-ish values and off they go. Java might have some great ML/Math libraries, but the fact that Python dominates data science suggests that my experience is a real-world pain-point.

rgoulter 8y ago

"Wizard, Engineer" reminds me of Yegge's "Software Liberal/Conservative" approach to risk. https://plus.google.com/110981030061712822816/posts/KaSKeg4vQtz https://plus.google.com/110981030061712822816/posts/KaSKeg4v... Albeit, rather than "wizards like implicit/magic, engineers prefer explicit/boilerplate/maintainability", the difference Yegge suggests was management of risk.

matfil 8y ago
Thanks, that's a pretty interesting take. (Also a reminder that we're probably going to lose some interesting stuff when Google+ goes kaboom...)

Adamantcheese 8y ago

A wizard then should have a spellbook, one filled with all sorts of spells written out for immediate use. Maintainable hacks if you will. Those are probably just scripts though I suppose?

bryanrasmussen 8y ago
The problem with spellbooks is one seen in media on the subject of wizardry, that the wizard spends a lot of time memorizing spells. This is why the most useful spells get turned into artifacts that the wizard does not have to always memorize to know how to cast correctly. What is needed is an artifact like spellbook, that the wizard when faced with a situation could describe it to the spellbook and get back the correct spell or combination of spells to solve the situation. Attempts have been made to create such an artifact, but unfortunately the resulting spellbooks still take a long time to find the correct spell and when reading the spells you often find that there are missing ingredients or a complicated set of gestures that must be performed to make the spell work, and you have to read all about these gestures in turn to figure out which ones you really need.
- thisiszilff 8y ago
  Have you heard of hoogle (Haskell's type signature search)? It almost exactly feels like what you've described.
  - bryanrasmussen 8y ago
    no, thanks for telling me!

bitwize 8y ago

Lisp is the ultimate "wizarding" programming language. But when I miraculously was called upon to maintain an enterprise code base in Common Lisp, it was an absolute joy. Because whenever they encountered a roadblock in maintenance, the Lisp wizards who had come before me just wizarded up a solution. One of the things that stuck out was that it had its own custom test framework, that was head and shoulders above XUnit, Mocha, or any other commonly-used test framework. Adding a new test was virtually a one-liner; the test would generate test data , send it to the server, and check the server's response against an XML template provided by the test case.

zimablue 8y ago

I'm sceptical of his argument that "we've gone from dynamic being trendy back to typed(java), because 'people' had to maintain dynamic codebases". An equivalent but probably equally not-the-real-explanation argument would be that we've gone from an era of opportunity into an era of oligopoly as the internet titans have emerged and so the "coolest kids" who everyone cargocults have gone from being fast-growing startups to members of the big-5 elite. Kind of the same as his argument just with detail on who his "people" are and why, but it changes the implications. I think it's more as he touches on that the two are converging. I think eventually some sort of pluggable typing will win (the "proofs" on your system won't be a single compilation pass but maybe different typing for different parts of the system, and specific proofs run between compile and runtime as the two blur), which will look more like gradual typing.

jondubois 8y ago
According to the definitions provided, it seems that engineering is the wrong approach for the vast majority of systems (especially popular web-based platforms). We should be leaning more towards wizarding. Most systems change all the time. So long as a company needs executives to make decisions and steer in different directions depending on the economic conditions, companies also need the flexibility to change their code. Even the Linux Kernel which is now decades old is still being changed all the time. If you use tools which assume that every line of code you write is not going to change, then unless you're programming an aircraft or a medical hardware device you're probably using the wrong tools. You should not assume that just because some low level module is deeply nested within the code, that it means that it should not be changed or thrown out. That's why I prefer dynamically typed languages for web systems; they start with the rigth assumption about the ever-evolving nature of the project. If JavaScript was designed to be statically typed from the beginning, web browsers would not have attained the usefulness or popularity that they have today.
- zimablue 8y ago
  I'm not sure to what extent I believe this. There are a lot of overlapping distinctions, some blurry, I think. He chooses some for wizardry vs engineering but I don't know whether they're best. For example he puts "magic"=implicit, but I would normally say highly implicit code is "engineering" because it implies that someone has understood and explored the problem enough to write a very tuned framework or library. I think reasons for putting very implicit code in with wizardry could be that it might be that the person using it doesn't understand it, and that it's somewhat in opposition to strong typesystems wizardry/engineering high uptime/downtime possible efficiency important/efficiency not important large codebase/small codebase not implicit/implicit (I disagree most with this one) problem is understood/code is exploratory (I think this is the most important distinction) high specialism/low specialism of coders changes slowly/changes quickly typed/untyped I would say websites are engineering because the problem space is well understood, they need high uptime and they tend to be written by specialists (React guy, django guy etc).

jbergens 8y ago

It sounds like optionally typed languages like Typescript should be really good if you want to start in "wizarding" mode and then change the code more into an "engineering" solution.

luord 8y ago

There are quite a few false assumptions here. > People have certainly managed to create test suites that make it harder to maintain the code. Sure, but pretending that this is the norm is just not true. And arguing against an extreme case can be done against anything . > I get the impression Google has been able to migrate a lot of C++ and Python to Go using this approach. Not only is this heavily suspect (I'd love to read a citation for this) but if there's a language that represents the "engineering" side (?) pretty clearly, it is C++. > Gradual type systems has started to garner a lot more interest. No they haven't. Those have existed for over five decades . Another problem with these articles is how they seem oddly ignorant of the history of computer science. This is specially odd coming from a PhD (assuming that I looked up the right person in google). And, last but not least, some of the best engineered pieces of code are precisely shells, dynamic languages and frameworks. It's things like these what makes these articles seem like they were written by java developers annoyed because java is no longer the trendiest toy.

iseeyoubydesign 8y ago
>People have certainly managed to create test suites that make it harder to maintain the code. example? these test suites are they hard to maintain or is the code its testing harder???? >And, last but not least, some of the best engineered pieces of code are precisely shells, dynamic languages and frameworks. I don think its realized that it might not be wizardy or engineering but something more like art. architecture
marcosdumay 8y ago
> No they haven't. Those have existed for over five decades. There is nothing in "started to garner more interest" that implies something is new. It's even the other way around.

casper345 8y ago

"Python + C" I am taking this approach working/learning with the raspberry pi. Although it can do both python and C, I know in electronics working with low level C will be a lot beneficial for my learning and code itself. When I migrate to more advance topics to scripting and ML, I will use both PYthon and C to implement what is needed.