13 ms·

Ability to see expertise is a milestone worth aiming for (2022)

261 points by nickwritesit 3y ago

jiggawatts 3y ago
This is an extremely important article to read for professionals of all sorts, not just freelancers. It's an issue that I struggle with nearly daily. I can't convince customers to pay me what I'm worth, because they can't judge my true value. Meanwhile, they'll happily pay Deloitte or Accenture 3x as much as what I charge, and then take the misguided ramblings of a junior-sold-as-senior at face value, not knowing the difference. > I’ve noticed that in nearly every skill tree I’ve climbed, there comes a point where you begin to notice certain nuances of skill that novices not only don’t notice, but that you cannot articulate using the commonly accepted vocabulary of the skill. This happened to me just over the last month or so. I was trying to explain to a CTO how horrifically wrong it is to plumb their entire cloud network through just a pair of virtual machines for "security". I just could not make him understand that 5 milliseconds of additional latency in the era where microseconds are an eternity is a disaster for a veritable smorgasbord of upcoming projects. It just. Would. Not. Sink. In. To some people, "micro" and "milli" sound the same. Latency just doesn't compute as a thing to be concerned about. I've spent my career performance tuning, but then how do I explain this to people who think a 60-second timeout is a performance problem best solved by buying another million dollars worth of kit? The article jumps between martial arts and the world of finance, so I'll jump to another example: AI playing games. When I watched AlphaZero play chess, I was just a tiny bit impressed. I watched it beat the Go grandmaster and I was not very impressed. Then I watched it play Dota 2 -- in which I've sunk several thousand hours -- and holy crap. Wow! It was suddenly terrifying how good it was, because I could judge the nuance of what it was doing.
- marginalia_nu 3y ago
  > To some people, "micro" and "milli" sound the same To be fair, it almost is. Both units are used to talk about times of about the same size. Using both in the same sentence is like mixing meters and feet. It's much clearer to say e.g. 1500us vs 3us; or 1.5ms vs 0.003ms. Don't mix units when making a comparison.
  - chaxor 3y ago
    In the era of Electron, most things take minutes anyway, so milliseconds is an upgrade.
    - maccard 3y ago
      The reason things take minutes is because people ignore order of magnitude diffences when they're insignificant, (it doesn't matter if X takes 10ms instead of 0.1ms), and then over time someone builds on that and makes the same assumption (it doesn't matter if it takes 100ms, when it could take 1ms, followed by it doesn't matter if it takes 10s). Avoiding these order of magnitude differencdz early on is the best way to ensure you don't end up with a SPA that takes 15 seconds to populate a list with 20 items in it.
  - nooby12345 3y ago
    A factor of 1000 actually
    - marginalia_nu 3y ago
      Hmm, I'm clearly not awake enough to SI. But the point still stands. Don't mix units.
      - jiggawatts 3y ago
        The lack of expertise here is confirming the point of the article. I'm not mixing units. Milliseconds and microseconds are both measured in units of seconds. The milli- and micro- prefixes are orders of magnitude. They're scales, not units. When a scientist starts talking about measurements accurate to attoseconds, I sit up and take notice. The number in front doesn't make a difference. It's the "atto" that raises my eyebrows. Back to computing: I had a debate here with someone who claimed that 128-bit floats are required for some chip layout codes. I questioned that, because a 32-bit number can subdivide the side-length of a chip down to sub-atomic resolution, and a 64-bit float can model the wiggles of the hadrons making up the atomic nuclei! To some people, 128-bit sounds "twice as good as 64-bit". To experts, it's... umm... a sign you've made a mistake.
- hardware2win 3y ago
  >. I just could not make him understand that 5 milliseconds of additional latency in the era where microseconds are an eternity Context matters, not everywhere 5ms more makes difference.
  - bryanrasmussen 3y ago
    I felt the context was really far down the stack, so services that lots of other services would be depending on was how I read it. The places I want fast, high volume and never failing cause so much depends on a red wheelbarrow.
  - jiggawatts 3y ago
    At the core of a data centre, between VMs, it always makes a difference. An enormous one.
  - nmcela 3y ago
    As a gamedev, that sentence nearly gave me a heart attack. :) I get where you are coming from, but the amount of work you can do in 5ms is mindblowing.
- ZephyrBlu 3y ago
  It sounds like you were just telling the CTO "this is wrong" instead of offering an alternative solution? If that's the case it's not surprising it didn't sink in. You're just giving him another problem. > I've spent my career performance tuning, but then how do I explain this to people who think a 60-second timeout is a performance problem best solved by buying another million dollars worth of kit? I'm surprised you've spent your career doing this and you still haven't figured out how to get people to listen to you. Making your work legible to others feels as important as the work itself.
  - jiggawatts 3y ago
    For reference, I did offer alternative solutions. > I'm surprised you've spent your career doing this and you still haven't figured out how to get people to listen to you. This is the point of the article, which you appear to have missed. There's a point where it becomes essentially impossible to articulate the differences to the lay person. Or for that matter, someone actually quite skilled, but not a "master". It's not a lack of skill on the part of the expert, it's the lack of skill on the part of everybody else.
    - thesumofall 3y ago
      But is this really the point of the article? The article argues that it is difficult to identify expertise. However, the situation you describe is one where (if I read you correctly) not your expertise was questioned but your recommendation. If there wasn’t any tangible difference between the CTO’s and your suggested course of action on a metric that he as a customer cared about, then maybe his proposal was indeed the better one? And yes, probably he didn’t care about the exact latencies in the system, but he would probably care about customer retention, system availability, etc.?
      - jiggawatts 3y ago
        I'm finding this entire line of discussion incredibly curious because not only does it confirm my original argument and the that of the article, but it hammers the point home. Everything you've just said is gibberish to a master. It's like making a straight-faced argument that setting the speed limit to 1 mph on a motorway might make perfect sense, and the traffic engineer is actually at fault for not making it clear that this is nonsense. I get that you don't see the ridiculousness of 50 microseconds vs 5 milliseconds, but that's the point. You don't see it, and you start making random arguments untethered from reality. The fact that you are blind to it, unlike being blind to 1 mph vs 65 mph is the entire point. You may have driven thousands of hours. You have don't have thousands of hours of network performance tuning expertise. Both of us wouldn't be able to even begin to understand the complaints Ayrton Senna might make about how the steering is too "crunchy" at the extremes or that the brake strength imbalance is shifting the car about. There's levels of expertise, and if two people are too far apart, meaningful communication between them becomes impossible.
- dumpsterdiver 3y ago
  If two nearly identical wallets (very minor, but noticeable differences) are placed on a counter and one is significantly more expensive... how would you expect those wallets to sell? I would expect both of them to sell out. The people looking for a higher quality product will immediately go for the more expensive product, and the remaining wallets would sell to people who just need a wallet.
  - joshuaissac 3y ago
    > The people looking for a higher quality product will immediately go for the more expensive product It is more complicated than that. There is no guarantee that the more expensive wallet is also the higher-quality product. The people looking for a higher-quality product can be classified into those who can measure the quality well (perhaps by observing those minor differences), and those who have to use less effective proxies of quality (like price).
    - detourdog 3y ago
      This day and age we can make our own inflation. We are all free to pay as much or as little for wallet as we would like. So we all shop by price, quality, or availability.
- ZephyrBlu 3y ago
  > When I watched AlphaZero play chess, I was just a tiny bit impressed. I watched it beat the Go grandmaster and I was not very impressed. Then I watched it play Dota 2 -- in which I've sunk several thousand hours -- and holy crap. Wow! It was suddenly terrifying how good it was, because I could judge the nuance of what it was doing Funny, my game of choice is StarCraft 2 and when I watched AlphaStar beat StarCraft 2 pros I was significantly less impressed than when I watched AlphaGo beat Lee Sedol at Go.
  - jiggawatts 3y ago
    You should be more impressed knowing that that AlphaStar was handicapped. Its clicking was rate-limited and its inputs were delayed by something like 150 milliseconds to make it "fair" for the humans. If you remove those limits, it would absolutely wipe the floor with the twitching corpse of any human player. I saw it play Warcraft 2 without the limiters, and it was just ridiculous. It would dance the units around and keep using them with 1 hp left for many minutes and then heal them back up juuuuust enough to take exactly one hit from an enemy.
    - whateveracct 3y ago
      But not having human reaction time in a real-time game isn't interesting. And the strategies that arise from that aren't either. If anything, human-level I/O constraints should make the AI more impressive because it's actually playing the same game.
    - ZephyrBlu 3y ago
      The handicapping was very ineffective. AlphaStar had 250-300+ APM and it was basically 100% effective APM. Pros have 300-400+ APM, but their effective APM is much lower due to spam patterns and checking production among other things. Almost everyone in the community seemed to agree AlphaStar was very unfair. Things like a 3 way surround and performing blink micro on individual Stalkers on opposite sides of the screen are completely inhuman. This is also what made the Dota 2 bots feel somewhat disappointing. Of course perfect mechanical skill beats humans. Watching the SF mid 1v1 matches illustrated this very clearly to me. The AI chipped the player down with perfect AAs, then finished them off with perfect distance razes. The bots weren't 100% mechanical skill, but it was a large enough proportion that it was uninteresting.
- ianpurton 3y ago
  > Meanwhile, they'll happily pay Deloitte or Accenture 3x as much as what I charge, and then take the misguided ramblings of a junior-sold-as-senior at face value, not knowing the difference. This is so true. It's also very difficult to stop as the consultancy gets the project and the client doesn't get to interview the people who actually turn up.
- 3abiton 3y ago
  > Meanwhile, they'll happily pay Deloitte or Accenture 3x as much as what I charge, and then take the misguided ramblings of a junior-sold-as-senior at face value, not knowing the difference. The main selling point is that big 4/8/16 have standard process, that does not really work, but gives the impression of "high quality" to clients, who are not technical.
- VoodooJuJu 3y ago
  >I’ve noticed that in nearly every skill tree I’ve climbed, there comes a point where you begin to notice certain nuances of skill that novices not only don’t notice, but that you cannot articulate using the commonly accepted vocabulary of the skill. I've encountered this, and what works is re-framing it in terms they'll understand. Things that are universally understood are: time, money, and competition. In this case, the "micro" and "milli" pitch is technical minutiae and is not what I'm referring to when I say "time". These are more effective frames: >If we do this, it will save/cost us months/years/dollars... >If we do this, the performance hit will knock out our competitive advantage, which we'll have to make up for it with something else, which will cost time/money... >This thing is insufficient for our customer's needs, which will cause our customers to seek out competitors, which will cause us to lose money... >This thing is too complex and will require additional training and is error-prone, which will cost us time and money... In addition to those, for every decision you refute, recommend a superior alternative, and again, frame it in terms of time and money. Now, this won't be a one-sided thing, you'll likely get into a conversation, and a lot of "why's" will try to extract that technical minutiae answer from you, which you might want to mention at some point in the exchange, but don't give it up easily, and do not emphasize it. Make sure you continue to emphasize the more convincing frames: time and money.
- giantg2 3y ago
  "take the misguided ramblings of a junior-sold-as-senior at face value, not knowing the difference" Based on my experiences, I'm throughly convinced that most of the time the consultants will tell management what they want to hear. If management doesn't like the answer then they go to someone else. How likely it is that they will accept an unwanted opinion from a consultant is solely based on prestige.
  - rvba 3y ago
    One thing are presentations, other thing is actuall stuff build by 'juniors sold as seniors'. Read all the horror stories of shitty software or ERP implementations. Often it is the consultant's fault - since they are technically weak.
- darkerside 3y ago
  Isn't it possible that you may not understand the CTO's priorities? Security is clearly essential to him, and performance is essential to you. I don't doubt there are other ways to satisfy both criteria, but frankly, it just doesn't sound like the CTO cared about those milliseconds.
  - jiggawatts 3y ago
    That came up too. The security aspect of the product does less than nothing: not only does it fail to encrypt data end-to-end, it uses public ip addresses for both the control and data plane. Those public endpoints had an unauthenticated RCE vulnerability recently. It’s a staggeringly bad product and it blew my mind now it was impossible to convince anyone of this. We used pictures, for crying out loud. We did all the things everyone here has been helpfully suggesting. Talking about business consequences (dire), costs (enormous), risks (massive), etc… The response was: “the sales guy from the vendor assured me it’s all fine. This is just your opinion.” The causal chain of a->b->c was just not sinking in. The CTO he could not judge anyone’s expertise not just mine — he also couldn’t judge the lack of expertise by the vendor and their sales team. So he chose based on who has the nicer suit, the shiniest smile, etc…
- xyzelement 3y ago
  // I just could not make him understand that 5 milliseconds of additional latency in the era where microseconds are an eternity is a disaster for a veritable smorgasbord of upcoming projects. It just. Would. Not. Sink. In. Does this company do something where this latency would be noticeable? Can you give an example of business impact that would result from it? If you can, why don't you? If you can't, then sounds like it doesn't matter. The CTO didn't get to be the CTO by optimizing the irrelevant.
  - cgeier 3y ago
    Ideally, estimate the business values in local currency (and be able to defend those estimates).
    - xyzelement 3y ago
      I agree that's ideal but probably wouldn't expect a latency optimization expert to understand my business well enough to do that and that's fine. But I -would- expect them to say something on the level of: "this is the kinda latency that will prevent you from ever moving to a cloud desktop" or "at this level of latency, you are going to suffer X second delay every time you interact with a complex web app like Jira" or "you are an HFT firm and I know that your competitors are at about 1% of your latency." Then let your client decide if this is something that matters to them. Quite different than "wow the CTO is too dumb to understand my genius"
  - heyoni 3y ago
    That was my exact reaction too. There’s a reason python and other “slow” languages are dominating. Very few industries require that kind speed, and even fewer require it at the cost of simplifying code or architecture.
  - girvo 3y ago
    In my experience the CTO got to be CTO because they’re friends with the CEO, the board, etc, rather than their own technical ability being superior than the seniors/leads under them. And that goes for me too, when I was CTO. It’s a position, it doesn’t have any inherent guarantee of ability to avoid optimising the irrelevant: some will, some won’t.
- thrw4wy-d2 3y ago
  > Then I watched it play Dota 2 -- in which I've sunk several thousand hours -- and holy crap Fascinating. I’ve never professionally worked in what it sounds like you do, so I was moderately agreeing with you until this. I’ve played Dota likely as long as you in addition to playing at a semi pro level (placing well in a couple official pro qualifying tournaments as well as playing pubs with the players the AI played against) and I couldn’t disagree more. It was impressive, but it’s main advantage came from its inhuman reaction times. They added some buffer but still didn’t account for 5 players working in perfect unison. That kind of coordination is an enormous advantage in a game like Dota. I left roughly feeling like I do now about AI - very impressive, but way over hyped and only sold to people that don’t truly get Dota.
- danielmarkbruce 3y ago
  Branding is real.
- msp26 3y ago
  It's interesting that you bring up Dota because I find that it's a ready example of the vocab point failing. When you listen to pros talk, they misuse words and concepts all the time but it doesn't matter because the other pros listening get the 'feel' of what they're saying. (I'm not a pro or anything, just high rank, but to use myself as an example) I try to make an effort to use the correct words when explaining things to friends. But even I find myself saying things like 'just look at the [lane] creeps' to explain what I want my laning partner to do and then I realise how unhelpful it is. Because when they see the creeps they don't understand what I see.
- Corrado 3y ago
  I have a couple of similar experiences. I've spent >1,500 hours playing Team Fortress 2 and can recognize things that other gamer friends just can't see. I'm still not good at TF2, but I can recognize great playing. Years ago I was active in Tae Kwon Do and achieved black-belt status. One time, during a sparing match with my teacher, I was astounded to be able to spot a hole in his defense. In fact, I was so distracted by my perception that I completely froze and he scored a point. I like to think that my seeing that hole gave me a glimpse into what it might be like to be good at TDK. I still think about that experience to this day.
a_c 3y ago
The trick I use for myself is, if all I could see is binary (this is too easy, how hard can it be, or it is impossible), then I'm either too novice or I'm an true expert in this particular subject matter. As Bruce Lee put it, a punch is a punch [1]. I'm still hoping to be the latter 1. https://brucelee.com/podcast-blog/2017/6/21/51-the-three-stages https://brucelee.com/podcast-blog/2017/6/21/51-the-three-sta...
fidotron 3y ago
There is clearly a major problem with this in SV. A lot of people seem to confuse confidence with expertise, and fail to grasp that a scary proportion of the time the individual is so confident because they don’t understand the problems with what they are doing. Furthermore, modern software management practice doesn’t want this because it wants replaceable cogs. If you can recognize expertise you have to appreciate that teams where everyone is good at everything are nonsensical, and that you need actual balanced mixes of personalities and specialists, which makes hiring a far harder problem. As such recognizing expertise is very much not the order of the day.
- 3abiton 3y ago
  Dunning-kruger strikes again
  - dustingetz 3y ago
    it's not DK, it's sales. It exploits substitution bias: https://en.wikipedia.org/wiki/Attribute_substitution https://en.wikipedia.org/wiki/Attribute_substitution "Thinking Fast and Slow" has a whole chapter on this with ample evidence
- wesapien 3y ago
  Wouldn't it be better to hire a mixture of specialists and generalists especially today when diversity should be focused in thought rather than looks. Someone not me said that specialists are a master of one and connector of none.
  - nradov 3y ago
    Sure, ideally. In the real world hiring managers have to take what they can get from the applications that come in. And in a rapidly changing industry the specialists that you need today might be useless in two years. One approach that can work in some organizations is to hire mostly generalists as employees, and bring in specialists when needed as temporary contractors.
- ModernMech 3y ago
hardware2win 3y ago
This is why arguing with people from industry and reading a lot is very useful, because it allows you to build your nomenclature In software world such a word for me was "modeling" It no longer was "coding", "programming", I started modeling real world
- someweirdperson 3y ago
  But that's only for code that's painted using the mouse, not for code typed on the keyboard.
  - taeric 3y ago
    I'm curious what you mean? Code is very much a symbolic modeling of a problem domain.
    - someweirdperson 3y ago
      There are people who consider software written, I mean painted, or as they say modelled, in the language of e.g. Simulink to be of higher value than other representations of code. The reason for this is probably different use of the same terms. Some code might be tested within an implementation of a model of the real environment. That is a frequent use-case used to sell graphical languages. The term model is used for two different things: The model of the real world to allow testing and for the implementation of the solution in a graphical language. Of course developers in those environments are Klingons like we all are, no real effort for tests, and only the graphical solution remains. But it is perceived as if it had all the positive attributes of a solution tested in a simulated reality. And therefore something modelled is better, because it is modelled.
      - taeric 3y ago
        I think this is still doing it a disservice. You don't test a code with a model of the real environment. You execute the code, which was modeling something. Consider, "F=ma" is a mathematical model relating force to mass and acceleration. You can test this by dropping several things each from the same height and seeing that the force linearly increases based on the mass of what you are dropping. Similarly, you could test the model by accelerating something to a halt at different speeds and seeing how much force is imparted into it. Is that not a "model" because it is not necessarily graphical? So, similarly, much CRUD software can be seen as modeling different agents in an overall system and what happens when they interact in certain ways. Edit: I want to add that I think I see where you are coming from. Many treatments of graphical models like to hold them up as a special form of modeling.
candiddevmike 3y ago
Is expertise still important in the age of ChatGPT?
- ido 3y ago
  yes
- SanderNL 3y ago
  If anything it’s more important. Without it you cannot separate its confabulation from its useful output. Knowing when to use what tool, how to do it properly and to what extent takes expertise as well. Carpenters and craftsmen in general are infinitely more productive with woodworking and related tools than I will ever be. My destiny is assembling IKEA cupboards. The same principle will apply with AI tools. Sure you can whip up something, but a specialist will do it infinitely better in 1/100th of the time with the same tools. Guess who will be employed to make use of said tools?
  - candiddevmike 3y ago
    Probably not the best comparison, as a carpenter or craftsmen can't tell a LLM something to build X out of wood and have it come out mostly OK (yet?). Anyone can tell ChatGPT to write some code to do X, not just expert programmers. It may not be perfect, but it's only going to get better, no?
    - SanderNL 3y ago
      This metaphor is not about the tools so much as it is about the nature of expertise and its effect on the results of said tools. Someone with expertise will use any tools, including GPT, in ways a novice can’t even understand. Same tools, different outputs. Not the tools, because basically anyone has access, but the one using it will determine the outcome. In a capitalist society you will have to compete otherwise known as staying employed. Someone with expertise can hook up AIs in ways and with a velocity that will leave amateurs in the dust. The amateurs will still be fiddling around with “prompts” and Googling or ChatGPT’ing what “latent” means while the expert has working systems ready to go. I don’t see this chasm getting smaller. These tools magnify your inherent (in)competency. Try to compete with a professional artist competent with Stable Diffusion if you want a taste of that.
swyx 3y ago
great article and well supported (loved the tennis example). i can't help but notice that Cedric (author of commoncog, and a friend) is accumulating a good body of work exploring knowledge and expertise acquisition. is this something that is generically applicable or are we reading convincing-sounding text generated by a very advanced human expertise language model? I guess the only way to know for sure is for Cedric to open up a coaching practice and see the results?
- shawntan 3y ago
  great substack and podcast (love the guests you have on) i can't help but notice that Swyx (author of latent space) is accumulating a body of work exploring LLMs and covering AI progress. is this something that is analysis by an expert or are we reading convincing-sounding text generated by a very advanced AI hype language model? I guess the only way to know for sure is for Swyx to train an LLM or do deep learning research and see the results?
  - swyx 3y ago
    lmao.. not either of those things but i do have something smol i am working on
FailMore 3y ago
Really enjoyed. Very nice to have a new concept inserted into your brain! And well written to boot
the-mitr 3y ago
also related is Marvin Minsky's idea of negative expertise https://web.media.mit.edu/~minsky/papers/NegExp.mss.txt https://web.media.mit.edu/~minsky/papers/NegExp.mss.txt
deleted 3y ago
[deleted]
someweirdperson 3y ago
(2022) https://news.ycombinator.com/item?id=30930985 https://news.ycombinator.com/item?id=30930985
- dang 3y ago
  Thanks! Macroexpanded: Ability to see expertise is a milestone worth aiming for - https://news.ycombinator.com/item?id=30930985 https://news.ycombinator.com/item?id=30930985 - April 2022 (18 comments)
kashunstva 3y ago
This relates closely to Anders Ericsson's theory of expert performance through deliberate practice. The article discusses the role of nuanced domain-specific vocabulary to describe elements of expertise before pivoting to subsume it all under the rubric of _models_. This is exactly how Ericsson sees deliberate practice working - by building extraordinarily detailed and nuanced neurocognitive models and being able to draw on them readily (and often unconsciously) in performance.
lapcat 3y ago
The problem with the author's tennis anecdote about Roger Federer's 2017 career resurgence is that in reality it had more to do with the top player Novak Djokovic's elbow injury than with Federer's backhand. "In 2017, Djokovic suffered from an elbow injury that weakened his results until the 2018 Wimbledon Championships, where he won the title while ranked No. 21 in the world." https://en.wikipedia.org/wiki/Novak_Djokovic#Career_statistics https://en.wikipedia.org/wiki/Novak_Djokovic#Career_statisti...
derbOac 3y ago
I understand the intent, but this article seems to be arguing for an Emperor's New Clothes scenario where no one can identify the value of something except for a vanishingly small number of people. It's some economic variant of the tree falling in the forest analogy: if a service has quality but no one can recognize it does it have value? It strikes me as a recipe for regulatory capture and overcredentialing. Value ultimately lies in the purchaser, not the producer. Maybe it depends on the market but this seemed to be arguing for something dangerous to me.
- WanderPanda 3y ago
  I would go further and say it is not even with the purchaser but with the consumer (who by definition "destroys" that value)
- blackbear_ 3y ago
  > Value ultimately lies in the purchaser, not the producer. I'm not entirely convinced that this holds, in practice. It makes sense for rational purchasers with perfect and complete information, but that is never true in reality, and it ignored all cognitive biases that humans use to make decisions, including being misled by advertisment. For example, the tobacco industry can scream all they want that smoking does not cause cancer, but being loud about it does not make it true. Initially, people didn't even like smoking because of the bad taste, but massive advertising campaigns including paying celebrities to smoke in public changed the perception of the value of smoking. In a similar manner, after acquiring all major sources of diamonds and inflating their prices, advertisement convinced the public that a man absolutely needs a ring with a diamond to "prove" their love to their bride. Same for the food industry and sugar, you know where I am going. All of these are examples of "value" created by producers by misleading customers that their product is good and not harmful for them. But facts don't become true or false just because everybody believes in them. People can believe false things, and no matter what they do or say, those things remain false. Not to mention that immediate value for individuals can have considerable negative value in the long-term for society. Think at the huge resources we spend on finding cures for cancer and obesity, just because we sell addictive stuff and pretend that people "want" those. If you consider society as a whole, this appears to be an extremely inefficient use of resources.
shadowsun7 3y ago
Author here. Previous time this was on the front page: https://news.ycombinator.com/item?id=30930985 https://news.ycombinator.com/item?id=30930985 As an update, I temporarily relocated to a different city and spent four months training Judo for five hours every day and have since reached the vocab point for the sport. That experiment was covered here: https://commoncog.com/expertise-acceleration-experiment-judo/ https://commoncog.com/expertise-acceleration-experiment-judo... and https://commoncog.com/mental-strength-judo-life/ https://commoncog.com/mental-strength-judo-life/
- Swizec 3y ago
  Wow! How did you make time for 5 hours per day? I workout a lot and time is one of my biggest challenges.
- ssivark 3y ago
  Loved the post. Just to add to the principal observation about the vocab point — One thing I’ve noticed is that experts who can communicate/teach well are very effective at getting you to the vocab point (to the extent possible without experience) quickly! And once you reach the vocab point, the field is legible enough that you have the means to organize deliberate practice and interpret experience (if you want to go towards mastery) or delegate details (if you don’t need them). I wonder if this provokes any more thoughts from you.
- lcnPylGDnU4H9OF 3y ago
  I'm glad you were able to articulate this idea so clearly. I think similar advice which maybe isn't so clear is to try to avoid "being the smartest person in the room". On the topic, I'm reminded of coworkers talking about the most recent Super Bowl (I think, I don't follow sports) which included commentary from players who played as quarterbacks in the same season. One of the things my coworkers mentioned was how interesting it was to hear them talk about the game because they used unfamiliar terms.
nativecoinc 3y ago
There are some teachers of Samatha meditation, a Buddhist concentration meditation, that teach that you at any point are meditating at one of ten stages. This is both about skill as well as about factors like how agitated your mind is in general. And these stages seem to have very well-defined descriptions: for example one stage might be differentiated from the previous one by overcoming drowsiness. This is very helpful since some people teach meditation by just telling you how to do it but not (in fact sometimes actively avoiding) how you can evaluate where you are and how you are doing. Then the practice becomes just about “being present” and other slogans that might be wholly non-actionable. It’s also very difficult to protest this kind of approach (edit: the non-goal approach) since merely mentioning things like "goal" or "evaluation" will trigger someone’s knee-jerk you’re-doing-it-wrong reaction, even though what they practice might be completely different to what you are doing or trying to achieve.
xyzelement 3y ago
Not directly related to the article but I had two amusing encounters with expertise recently. I sat next to an Econ professor (at an Ivy) on a flight. I told him I had worked at a hedge fund, and he was like "wow, I could never put myself in a situation where my living depended on my economic theories being right." A good friend of mine is a kinda-famous evolutionary biologist. He's in his mid-40s now, unmarried and doesn't have kids, and likely none on the horizon. In the economist case, what does it mean that in his expert opinion, he wouldn't trust his own expert opinion! In the biologist case, what does it mean to be one of the world's leading experts on "evolutionary biology" and yet completely fail to pass your own genes? What's most interesting to me is that neither of these scenarios strike us as "super wrong." I can imagine most people wouldn't hesitate to attend a class taught by either one of these guys, while we'd avoid analogous situations (eg: we wouldn't use a broke financial advisor or fly on an airplane whose designer refuses to set foot on it.) What's the difference?
- selykg 3y ago
  As far as the economist. I suspect he understands that they're theories, thus, to some extent guesses. Some data might support it, but does it hold true in every case? Probably not. Does it hold true in most cases? Maybe not. Would he bet money, effectively what a hedge fund does in my very limited experience here? Sounds like no. This to me sounds like a simple example of he understands the limitations of his work! As for your evolutionary biologist friend. Being interested in evolutionary biology does not mean you are inherently someone that wants to be a parent. These are very very different things. Besides, evolutionary biology impacts more than just humans, it impacts every species. Being a parent comes with a great responsibility. Someone could just as easily be a really big fan of dogs, but not want one because of the responsibility or the costs, or similar.
  - xyzelement 3y ago
    Yup, your explanation resonates with what I think are the superficial reasons society is accepting of both of these cases. You have to think a little bit deeper to feel why this might be problematic. If the economist weighs the risk to not put too much stock in these theories (no pun intended actually), why does he not likewise hesitate to teach these theories to his students? Like "I don't believe 100% that this is how the world works enough to bet my own money, but I am going to tell hundreds of students every year this is how the world works." The biologist story is subtler. One could argue that failure to become a parent signals that you deeply, fundamentally, don't get biology - despite your superficial interest and theoretical knowledge? Like, you may talk about zygotes all day but you failed to make one... Maybe the difference between economists and biologists vs financial advisors and engineers is that we expect the later to have skin in the game but for some reason allow the former to not have to.
parpfish 3y ago
This seems like a factor in Dunning-Krueger. Experts can see all the gradients in ability and will extrapolate how far they are from the top, but all novices know is how far they are from the bottom. If those distance estimates are biased, it’d lead to novices overestimating ability and experts underestimating
- bena 3y ago
  Dunning-Kruger is more of a statistical effect than a psychological one. The same graphs can be reproduced using random noise.
  - taeric 3y ago
    I really regret how hard of a hold the Dunning-Kruger report has had on our generation. Worse, it seems every reference trying to point out that it is, in fact, not saying what people think it says just reinforces folks believe that it says what they want it to say. :(
xyzt 3y ago
for me, path to expertise is knowing the "language", "practicing maturity," and system 1 mastery of a topic. i think expertise is "mastery of system 1(ref Daniel Kahneman) of a topic." which means identifying more "patterns" through "practice and observation." the patterns capture both the rights and the wrongs. the entry to any subject is "learning the language" of the subject. what comes next is to become "mature." maturity is based on amount of practice in a big and unbiased search space, the ability to play with subject and identifying primitive constructs, ability to criticize and seeing the weaknesses, the measuring method for maturity level is amount of "simplicity," "freedom", "flexibility", "speed", and "ability to identify the essence" that the system one adheres to bring. excerpt from Saunders Mac Lane "category theory for working mathematician." "many properties of mathematical construction can be identified using universal mapping properties." see how simple the definition is?
xyzelement 3y ago
// Customers tends not to be experts. So after a certain level of skill has been achieved, any improvements that are made through experience are invisible to the people you’re selling to. The relevant question is - does it matter? The Formula 1 pit crew is more expert than the mechanics at the Toyota dealership in ways I can't appreciate - but does it matter for my Highlander's oil change? The answer might be "Yes, the F1 crew will do it in a way that prolongs the life of your engine" (I care about that!) or "Yes, the F1 crew will change your oil in 20% of the time" (I may or may not care about saving a few minutes every 6 months) or "No, you won't notice a difference but they are just more expert" (in which case I totally don't care!) Part of expertise is ability to articulate its relevance to a given situation. Your customer may need help in validating your expertise, but at least start with the claim of what you can do.
- nradov 3y ago
  F1 pit crew members are highly trained athletes who specialize in fast tire changes. Many of them are recruited from other sports based on strength, speed, and agility. They often have no real experience as automotive technicians. F1 teams do have world class mechanics but most of them aren't part of the pit crews.
  - Etheryte 3y ago
    Are you familiar with the expression missing the forest for the trees?
  - Cipater 3y ago
    >Many of them are recruited from other sports based on strength, speed, and agility. This is not true at all. F1 pit crews are mechanics who have worked with cars in other motorsport racing series/feeder series and worked their way up to an F1 team.
- ianmcgowan 3y ago
  I think the point is when reaching higher levels of expertise is not to try and convince existing customers to move up the ladder with you, it's that you need to find an entirely new set of customers who need that level of expertise and appreciate the difference. It's not about explaining - there's a quantum leap between levels. It might be trite, but as a contractor walking my rate up over 10 years, I found it interesting.
atleastoptimal 3y ago
I'd wager a large portion social embarrassment is when a person ostensibly believes they have competence in a certain realm but do not. Being aware of yourself and how your peaks and valleys are perceived is the foundation of charisma.
deleted 3y ago
[deleted]
kulikalov 3y ago
If it quacks like a duck - it must be a duck. Not really. > In every profession there’s a point where further improvements in skills are so subtle that the difference between one level and another can only be appreciated by another expert Goal defines the benchmark for proficiency. Better tennis player scores. Better software engineer writes more space/time efficient code or solves business problem faster (whichever the objective). Better writer gets more views or inbound sales or whatever the goal is. If you get better at achieving ever greater goals, then your expertise is growing. No amount of luck can compensate for a lack of expertise.
Corrado 3y ago
I've experienced "tacit..feelings" personally on many development projects. I'm not sure why this path is better than the other path, but it just feels better to me. One example is where to put new code. Do you put it in the existing code (monolith) or do you create a "service" for the code? Sure, some things are pretty straightforward but sometimes there really is a toss up of where this new thing should live. The problem is when someone asks me why, I can't really explain it and it gives less weight to the decision. However, looking back I can say that I've been more right than wrong, so I guess I have that going for me.
kybernetikos 3y ago
Enjoyable read that definitely touches on something real. I was struck by the anecdote that Herbie Hancock tells in the linked video though. He misheard some advice from someone he respected, took it to heart and it changed the way he played. I notice that this seems to work in opposition to the idea of a vocab point where high level practitioners are talking about real concrete experiences they are having that the lay man can't understand until they level up. The way I think it worked in this case, is that as you go up the ladder of skill, you start seeing and thinking in more abstract, metaphorical terms. For a beginning student, you're focused on the mechanics, for the master, you're focused on what it means and how it relates to the human experience. This means that a master can apply an a metaphor that the beginner can't even understand. Indeed, Herbie Hancock can apply a metaphor that nobody else could even imagine had any relevance at all. Perhaps rather than 'vocab point' we should talk about 'metaphor point'.
grumpy_coder 3y ago
There is a huge problem with this idea, it suggests that a competent critic has usable skill in an area. This is obviously false. Moreover I see it as a huge problem in our current environment of easy access to Youtube explainers of everything. In my areas of expertise I come across many more people that 'talk the talk' than can actually 'walk the walk'. A chef must be judged on dishes they produce, a dancer on their performances, and a software developer on working code.

261 points by nickwritesit 3y ago

This is an extremely important article to read for professionals of all sorts, not just freelancers. It's an issue that I struggle with nearly daily. I can't convince customers to pay me what I'm worth, because they can't judge my true value. Meanwhile, they'll happily pay Deloitte or Accenture 3x as much as what I charge, and then take the misguided ramblings of a junior-sold-as-senior at face value, not knowing the difference. > I’ve noticed that in nearly every skill tree I’ve climbed, there comes a point where you begin to notice certain nuances of skill that novices not only don’t notice, but that you cannot articulate using the commonly accepted vocabulary of the skill. This happened to me just over the last month or so. I was trying to explain to a CTO how horrifically wrong it is to plumb their entire cloud network through just a pair of virtual machines for "security". I just could not make him understand that 5 milliseconds of additional latency in the era where microseconds are an eternity is a disaster for a veritable smorgasbord of upcoming projects. It just. Would. Not. Sink. In. To some people, "micro" and "milli" sound the same. Latency just doesn't compute as a thing to be concerned about. I've spent my career performance tuning, but then how do I explain this to people who think a 60-second timeout is a performance problem best solved by buying another million dollars worth of kit? The article jumps between martial arts and the world of finance, so I'll jump to another example: AI playing games. When I watched AlphaZero play chess, I was just a tiny bit impressed. I watched it beat the Go grandmaster and I was not very impressed. Then I watched it play Dota 2 -- in which I've sunk several thousand hours -- and holy crap. Wow! It was suddenly terrifying how good it was, because I could judge the nuance of what it was doing.

marginalia_nu 3y ago
> To some people, "micro" and "milli" sound the same To be fair, it almost is. Both units are used to talk about times of about the same size. Using both in the same sentence is like mixing meters and feet. It's much clearer to say e.g. 1500us vs 3us; or 1.5ms vs 0.003ms. Don't mix units when making a comparison.
- chaxor 3y ago
  In the era of Electron, most things take minutes anyway, so milliseconds is an upgrade.
  - maccard 3y ago
    The reason things take minutes is because people ignore order of magnitude diffences when they're insignificant, (it doesn't matter if X takes 10ms instead of 0.1ms), and then over time someone builds on that and makes the same assumption (it doesn't matter if it takes 100ms, when it could take 1ms, followed by it doesn't matter if it takes 10s). Avoiding these order of magnitude differencdz early on is the best way to ensure you don't end up with a SPA that takes 15 seconds to populate a list with 20 items in it.
- nooby12345 3y ago
  A factor of 1000 actually
  - marginalia_nu 3y ago
    Hmm, I'm clearly not awake enough to SI. But the point still stands. Don't mix units.
    - jiggawatts 3y ago
      The lack of expertise here is confirming the point of the article. I'm not mixing units. Milliseconds and microseconds are both measured in units of seconds. The milli- and micro- prefixes are orders of magnitude. They're scales, not units. When a scientist starts talking about measurements accurate to attoseconds, I sit up and take notice. The number in front doesn't make a difference. It's the "atto" that raises my eyebrows. Back to computing: I had a debate here with someone who claimed that 128-bit floats are required for some chip layout codes. I questioned that, because a 32-bit number can subdivide the side-length of a chip down to sub-atomic resolution, and a 64-bit float can model the wiggles of the hadrons making up the atomic nuclei! To some people, 128-bit sounds "twice as good as 64-bit". To experts, it's... umm... a sign you've made a mistake.
hardware2win 3y ago
>. I just could not make him understand that 5 milliseconds of additional latency in the era where microseconds are an eternity Context matters, not everywhere 5ms more makes difference.
- bryanrasmussen 3y ago
  I felt the context was really far down the stack, so services that lots of other services would be depending on was how I read it. The places I want fast, high volume and never failing cause so much depends on a red wheelbarrow.
- jiggawatts 3y ago
  At the core of a data centre, between VMs, it always makes a difference. An enormous one.
- nmcela 3y ago
  As a gamedev, that sentence nearly gave me a heart attack. :) I get where you are coming from, but the amount of work you can do in 5ms is mindblowing.
ZephyrBlu 3y ago
It sounds like you were just telling the CTO "this is wrong" instead of offering an alternative solution? If that's the case it's not surprising it didn't sink in. You're just giving him another problem. > I've spent my career performance tuning, but then how do I explain this to people who think a 60-second timeout is a performance problem best solved by buying another million dollars worth of kit? I'm surprised you've spent your career doing this and you still haven't figured out how to get people to listen to you. Making your work legible to others feels as important as the work itself.
- jiggawatts 3y ago
  For reference, I did offer alternative solutions. > I'm surprised you've spent your career doing this and you still haven't figured out how to get people to listen to you. This is the point of the article, which you appear to have missed. There's a point where it becomes essentially impossible to articulate the differences to the lay person. Or for that matter, someone actually quite skilled, but not a "master". It's not a lack of skill on the part of the expert, it's the lack of skill on the part of everybody else.
  - thesumofall 3y ago
    But is this really the point of the article? The article argues that it is difficult to identify expertise. However, the situation you describe is one where (if I read you correctly) not your expertise was questioned but your recommendation. If there wasn’t any tangible difference between the CTO’s and your suggested course of action on a metric that he as a customer cared about, then maybe his proposal was indeed the better one? And yes, probably he didn’t care about the exact latencies in the system, but he would probably care about customer retention, system availability, etc.?
    - jiggawatts 3y ago
      I'm finding this entire line of discussion incredibly curious because not only does it confirm my original argument and the that of the article, but it hammers the point home. Everything you've just said is gibberish to a master. It's like making a straight-faced argument that setting the speed limit to 1 mph on a motorway might make perfect sense, and the traffic engineer is actually at fault for not making it clear that this is nonsense. I get that you don't see the ridiculousness of 50 microseconds vs 5 milliseconds, but that's the point. You don't see it, and you start making random arguments untethered from reality. The fact that you are blind to it, unlike being blind to 1 mph vs 65 mph is the entire point. You may have driven thousands of hours. You have don't have thousands of hours of network performance tuning expertise. Both of us wouldn't be able to even begin to understand the complaints Ayrton Senna might make about how the steering is too "crunchy" at the extremes or that the brake strength imbalance is shifting the car about. There's levels of expertise, and if two people are too far apart, meaningful communication between them becomes impossible.
dumpsterdiver 3y ago
If two nearly identical wallets (very minor, but noticeable differences) are placed on a counter and one is significantly more expensive... how would you expect those wallets to sell? I would expect both of them to sell out. The people looking for a higher quality product will immediately go for the more expensive product, and the remaining wallets would sell to people who just need a wallet.
- joshuaissac 3y ago
  > The people looking for a higher quality product will immediately go for the more expensive product It is more complicated than that. There is no guarantee that the more expensive wallet is also the higher-quality product. The people looking for a higher-quality product can be classified into those who can measure the quality well (perhaps by observing those minor differences), and those who have to use less effective proxies of quality (like price).
  - detourdog 3y ago
    This day and age we can make our own inflation. We are all free to pay as much or as little for wallet as we would like. So we all shop by price, quality, or availability.
ZephyrBlu 3y ago
> When I watched AlphaZero play chess, I was just a tiny bit impressed. I watched it beat the Go grandmaster and I was not very impressed. Then I watched it play Dota 2 -- in which I've sunk several thousand hours -- and holy crap. Wow! It was suddenly terrifying how good it was, because I could judge the nuance of what it was doing Funny, my game of choice is StarCraft 2 and when I watched AlphaStar beat StarCraft 2 pros I was significantly less impressed than when I watched AlphaGo beat Lee Sedol at Go.
- jiggawatts 3y ago
  You should be more impressed knowing that that AlphaStar was handicapped. Its clicking was rate-limited and its inputs were delayed by something like 150 milliseconds to make it "fair" for the humans. If you remove those limits, it would absolutely wipe the floor with the twitching corpse of any human player. I saw it play Warcraft 2 without the limiters, and it was just ridiculous. It would dance the units around and keep using them with 1 hp left for many minutes and then heal them back up juuuuust enough to take exactly one hit from an enemy.
  - whateveracct 3y ago
    But not having human reaction time in a real-time game isn't interesting. And the strategies that arise from that aren't either. If anything, human-level I/O constraints should make the AI more impressive because it's actually playing the same game.
  - ZephyrBlu 3y ago
    The handicapping was very ineffective. AlphaStar had 250-300+ APM and it was basically 100% effective APM. Pros have 300-400+ APM, but their effective APM is much lower due to spam patterns and checking production among other things. Almost everyone in the community seemed to agree AlphaStar was very unfair. Things like a 3 way surround and performing blink micro on individual Stalkers on opposite sides of the screen are completely inhuman. This is also what made the Dota 2 bots feel somewhat disappointing. Of course perfect mechanical skill beats humans. Watching the SF mid 1v1 matches illustrated this very clearly to me. The AI chipped the player down with perfect AAs, then finished them off with perfect distance razes. The bots weren't 100% mechanical skill, but it was a large enough proportion that it was uninteresting.
ianpurton 3y ago
> Meanwhile, they'll happily pay Deloitte or Accenture 3x as much as what I charge, and then take the misguided ramblings of a junior-sold-as-senior at face value, not knowing the difference. This is so true. It's also very difficult to stop as the consultancy gets the project and the client doesn't get to interview the people who actually turn up.
3abiton 3y ago
> Meanwhile, they'll happily pay Deloitte or Accenture 3x as much as what I charge, and then take the misguided ramblings of a junior-sold-as-senior at face value, not knowing the difference. The main selling point is that big 4/8/16 have standard process, that does not really work, but gives the impression of "high quality" to clients, who are not technical.
VoodooJuJu 3y ago
>I’ve noticed that in nearly every skill tree I’ve climbed, there comes a point where you begin to notice certain nuances of skill that novices not only don’t notice, but that you cannot articulate using the commonly accepted vocabulary of the skill. I've encountered this, and what works is re-framing it in terms they'll understand. Things that are universally understood are: time, money, and competition. In this case, the "micro" and "milli" pitch is technical minutiae and is not what I'm referring to when I say "time". These are more effective frames: >If we do this, it will save/cost us months/years/dollars... >If we do this, the performance hit will knock out our competitive advantage, which we'll have to make up for it with something else, which will cost time/money... >This thing is insufficient for our customer's needs, which will cause our customers to seek out competitors, which will cause us to lose money... >This thing is too complex and will require additional training and is error-prone, which will cost us time and money... In addition to those, for every decision you refute, recommend a superior alternative, and again, frame it in terms of time and money. Now, this won't be a one-sided thing, you'll likely get into a conversation, and a lot of "why's" will try to extract that technical minutiae answer from you, which you might want to mention at some point in the exchange, but don't give it up easily, and do not emphasize it. Make sure you continue to emphasize the more convincing frames: time and money.
giantg2 3y ago
"take the misguided ramblings of a junior-sold-as-senior at face value, not knowing the difference" Based on my experiences, I'm throughly convinced that most of the time the consultants will tell management what they want to hear. If management doesn't like the answer then they go to someone else. How likely it is that they will accept an unwanted opinion from a consultant is solely based on prestige.
- rvba 3y ago
  One thing are presentations, other thing is actuall stuff build by 'juniors sold as seniors'. Read all the horror stories of shitty software or ERP implementations. Often it is the consultant's fault - since they are technically weak.
darkerside 3y ago
Isn't it possible that you may not understand the CTO's priorities? Security is clearly essential to him, and performance is essential to you. I don't doubt there are other ways to satisfy both criteria, but frankly, it just doesn't sound like the CTO cared about those milliseconds.
- jiggawatts 3y ago
  That came up too. The security aspect of the product does less than nothing: not only does it fail to encrypt data end-to-end, it uses public ip addresses for both the control and data plane. Those public endpoints had an unauthenticated RCE vulnerability recently. It’s a staggeringly bad product and it blew my mind now it was impossible to convince anyone of this. We used pictures, for crying out loud. We did all the things everyone here has been helpfully suggesting. Talking about business consequences (dire), costs (enormous), risks (massive), etc… The response was: “the sales guy from the vendor assured me it’s all fine. This is just your opinion.” The causal chain of a->b->c was just not sinking in. The CTO he could not judge anyone’s expertise not just mine — he also couldn’t judge the lack of expertise by the vendor and their sales team. So he chose based on who has the nicer suit, the shiniest smile, etc…
xyzelement 3y ago
// I just could not make him understand that 5 milliseconds of additional latency in the era where microseconds are an eternity is a disaster for a veritable smorgasbord of upcoming projects. It just. Would. Not. Sink. In. Does this company do something where this latency would be noticeable? Can you give an example of business impact that would result from it? If you can, why don't you? If you can't, then sounds like it doesn't matter. The CTO didn't get to be the CTO by optimizing the irrelevant.
- cgeier 3y ago
  Ideally, estimate the business values in local currency (and be able to defend those estimates).
  - xyzelement 3y ago
    I agree that's ideal but probably wouldn't expect a latency optimization expert to understand my business well enough to do that and that's fine. But I -would- expect them to say something on the level of: "this is the kinda latency that will prevent you from ever moving to a cloud desktop" or "at this level of latency, you are going to suffer X second delay every time you interact with a complex web app like Jira" or "you are an HFT firm and I know that your competitors are at about 1% of your latency." Then let your client decide if this is something that matters to them. Quite different than "wow the CTO is too dumb to understand my genius"
- heyoni 3y ago
  That was my exact reaction too. There’s a reason python and other “slow” languages are dominating. Very few industries require that kind speed, and even fewer require it at the cost of simplifying code or architecture.
- girvo 3y ago
  In my experience the CTO got to be CTO because they’re friends with the CEO, the board, etc, rather than their own technical ability being superior than the seniors/leads under them. And that goes for me too, when I was CTO. It’s a position, it doesn’t have any inherent guarantee of ability to avoid optimising the irrelevant: some will, some won’t.
thrw4wy-d2 3y ago
> Then I watched it play Dota 2 -- in which I've sunk several thousand hours -- and holy crap Fascinating. I’ve never professionally worked in what it sounds like you do, so I was moderately agreeing with you until this. I’ve played Dota likely as long as you in addition to playing at a semi pro level (placing well in a couple official pro qualifying tournaments as well as playing pubs with the players the AI played against) and I couldn’t disagree more. It was impressive, but it’s main advantage came from its inhuman reaction times. They added some buffer but still didn’t account for 5 players working in perfect unison. That kind of coordination is an enormous advantage in a game like Dota. I left roughly feeling like I do now about AI - very impressive, but way over hyped and only sold to people that don’t truly get Dota.
danielmarkbruce 3y ago
Branding is real.
msp26 3y ago
It's interesting that you bring up Dota because I find that it's a ready example of the vocab point failing. When you listen to pros talk, they misuse words and concepts all the time but it doesn't matter because the other pros listening get the 'feel' of what they're saying. (I'm not a pro or anything, just high rank, but to use myself as an example) I try to make an effort to use the correct words when explaining things to friends. But even I find myself saying things like 'just look at the [lane] creeps' to explain what I want my laning partner to do and then I realise how unhelpful it is. Because when they see the creeps they don't understand what I see.
Corrado 3y ago
I have a couple of similar experiences. I've spent >1,500 hours playing Team Fortress 2 and can recognize things that other gamer friends just can't see. I'm still not good at TF2, but I can recognize great playing. Years ago I was active in Tae Kwon Do and achieved black-belt status. One time, during a sparing match with my teacher, I was astounded to be able to spot a hole in his defense. In fact, I was so distracted by my perception that I completely froze and he scored a point. I like to think that my seeing that hole gave me a glimpse into what it might be like to be good at TDK. I still think about that experience to this day.

a_c 3y ago

The trick I use for myself is, if all I could see is binary (this is too easy, how hard can it be, or it is impossible), then I'm either too novice or I'm an true expert in this particular subject matter. As Bruce Lee put it, a punch is a punch [1]. I'm still hoping to be the latter 1. https://brucelee.com/podcast-blog/2017/6/21/51-the-three-stages https://brucelee.com/podcast-blog/2017/6/21/51-the-three-sta...

fidotron 3y ago

There is clearly a major problem with this in SV. A lot of people seem to confuse confidence with expertise, and fail to grasp that a scary proportion of the time the individual is so confident because they don’t understand the problems with what they are doing. Furthermore, modern software management practice doesn’t want this because it wants replaceable cogs. If you can recognize expertise you have to appreciate that teams where everyone is good at everything are nonsensical, and that you need actual balanced mixes of personalities and specialists, which makes hiring a far harder problem. As such recognizing expertise is very much not the order of the day.

3abiton 3y ago
Dunning-kruger strikes again
- dustingetz 3y ago
  it's not DK, it's sales. It exploits substitution bias: https://en.wikipedia.org/wiki/Attribute_substitution https://en.wikipedia.org/wiki/Attribute_substitution "Thinking Fast and Slow" has a whole chapter on this with ample evidence
wesapien 3y ago
Wouldn't it be better to hire a mixture of specialists and generalists especially today when diversity should be focused in thought rather than looks. Someone not me said that specialists are a master of one and connector of none.
- nradov 3y ago
  Sure, ideally. In the real world hiring managers have to take what they can get from the applications that come in. And in a rapidly changing industry the specialists that you need today might be useless in two years. One approach that can work in some organizations is to hire mostly generalists as employees, and bring in specialists when needed as temporary contractors.
ModernMech 3y ago

hardware2win 3y ago

This is why arguing with people from industry and reading a lot is very useful, because it allows you to build your nomenclature In software world such a word for me was "modeling" It no longer was "coding", "programming", I started modeling real world

someweirdperson 3y ago
But that's only for code that's painted using the mouse, not for code typed on the keyboard.
- taeric 3y ago
  I'm curious what you mean? Code is very much a symbolic modeling of a problem domain.
  - someweirdperson 3y ago
    There are people who consider software written, I mean painted, or as they say modelled, in the language of e.g. Simulink to be of higher value than other representations of code. The reason for this is probably different use of the same terms. Some code might be tested within an implementation of a model of the real environment. That is a frequent use-case used to sell graphical languages. The term model is used for two different things: The model of the real world to allow testing and for the implementation of the solution in a graphical language. Of course developers in those environments are Klingons like we all are, no real effort for tests, and only the graphical solution remains. But it is perceived as if it had all the positive attributes of a solution tested in a simulated reality. And therefore something modelled is better, because it is modelled.
    - taeric 3y ago
      I think this is still doing it a disservice. You don't test a code with a model of the real environment. You execute the code, which was modeling something. Consider, "F=ma" is a mathematical model relating force to mass and acceleration. You can test this by dropping several things each from the same height and seeing that the force linearly increases based on the mass of what you are dropping. Similarly, you could test the model by accelerating something to a halt at different speeds and seeing how much force is imparted into it. Is that not a "model" because it is not necessarily graphical? So, similarly, much CRUD software can be seen as modeling different agents in an overall system and what happens when they interact in certain ways. Edit: I want to add that I think I see where you are coming from. Many treatments of graphical models like to hold them up as a special form of modeling.

candiddevmike 3y ago

Is expertise still important in the age of ChatGPT?

ido 3y ago
yes
SanderNL 3y ago
If anything it’s more important. Without it you cannot separate its confabulation from its useful output. Knowing when to use what tool, how to do it properly and to what extent takes expertise as well. Carpenters and craftsmen in general are infinitely more productive with woodworking and related tools than I will ever be. My destiny is assembling IKEA cupboards. The same principle will apply with AI tools. Sure you can whip up something, but a specialist will do it infinitely better in 1/100th of the time with the same tools. Guess who will be employed to make use of said tools?
- candiddevmike 3y ago
  Probably not the best comparison, as a carpenter or craftsmen can't tell a LLM something to build X out of wood and have it come out mostly OK (yet?). Anyone can tell ChatGPT to write some code to do X, not just expert programmers. It may not be perfect, but it's only going to get better, no?
  - SanderNL 3y ago
    This metaphor is not about the tools so much as it is about the nature of expertise and its effect on the results of said tools. Someone with expertise will use any tools, including GPT, in ways a novice can’t even understand. Same tools, different outputs. Not the tools, because basically anyone has access, but the one using it will determine the outcome. In a capitalist society you will have to compete otherwise known as staying employed. Someone with expertise can hook up AIs in ways and with a velocity that will leave amateurs in the dust. The amateurs will still be fiddling around with “prompts” and Googling or ChatGPT’ing what “latent” means while the expert has working systems ready to go. I don’t see this chasm getting smaller. These tools magnify your inherent (in)competency. Try to compete with a professional artist competent with Stable Diffusion if you want a taste of that.

swyx 3y ago

great article and well supported (loved the tennis example). i can't help but notice that Cedric (author of commoncog, and a friend) is accumulating a good body of work exploring knowledge and expertise acquisition. is this something that is generically applicable or are we reading convincing-sounding text generated by a very advanced human expertise language model? I guess the only way to know for sure is for Cedric to open up a coaching practice and see the results?

shawntan 3y ago
great substack and podcast (love the guests you have on) i can't help but notice that Swyx (author of latent space) is accumulating a body of work exploring LLMs and covering AI progress. is this something that is analysis by an expert or are we reading convincing-sounding text generated by a very advanced AI hype language model? I guess the only way to know for sure is for Swyx to train an LLM or do deep learning research and see the results?
- swyx 3y ago
  lmao.. not either of those things but i do have something smol i am working on

FailMore 3y ago

Really enjoyed. Very nice to have a new concept inserted into your brain! And well written to boot

the-mitr 3y ago

also related is Marvin Minsky's idea of negative expertise https://web.media.mit.edu/~minsky/papers/NegExp.mss.txt https://web.media.mit.edu/~minsky/papers/NegExp.mss.txt

deleted 3y ago

[deleted]

someweirdperson 3y ago

(2022) https://news.ycombinator.com/item?id=30930985 https://news.ycombinator.com/item?id=30930985

dang 3y ago
Thanks! Macroexpanded: Ability to see expertise is a milestone worth aiming for - https://news.ycombinator.com/item?id=30930985 https://news.ycombinator.com/item?id=30930985 - April 2022 (18 comments)

kashunstva 3y ago

This relates closely to Anders Ericsson's theory of expert performance through deliberate practice. The article discusses the role of nuanced domain-specific vocabulary to describe elements of expertise before pivoting to subsume it all under the rubric of _models_. This is exactly how Ericsson sees deliberate practice working - by building extraordinarily detailed and nuanced neurocognitive models and being able to draw on them readily (and often unconsciously) in performance.

lapcat 3y ago

The problem with the author's tennis anecdote about Roger Federer's 2017 career resurgence is that in reality it had more to do with the top player Novak Djokovic's elbow injury than with Federer's backhand. "In 2017, Djokovic suffered from an elbow injury that weakened his results until the 2018 Wimbledon Championships, where he won the title while ranked No. 21 in the world." https://en.wikipedia.org/wiki/Novak_Djokovic#Career_statistics https://en.wikipedia.org/wiki/Novak_Djokovic#Career_statisti...

derbOac 3y ago

I understand the intent, but this article seems to be arguing for an Emperor's New Clothes scenario where no one can identify the value of something except for a vanishingly small number of people. It's some economic variant of the tree falling in the forest analogy: if a service has quality but no one can recognize it does it have value? It strikes me as a recipe for regulatory capture and overcredentialing. Value ultimately lies in the purchaser, not the producer. Maybe it depends on the market but this seemed to be arguing for something dangerous to me.

WanderPanda 3y ago
I would go further and say it is not even with the purchaser but with the consumer (who by definition "destroys" that value)
blackbear_ 3y ago
> Value ultimately lies in the purchaser, not the producer. I'm not entirely convinced that this holds, in practice. It makes sense for rational purchasers with perfect and complete information, but that is never true in reality, and it ignored all cognitive biases that humans use to make decisions, including being misled by advertisment. For example, the tobacco industry can scream all they want that smoking does not cause cancer, but being loud about it does not make it true. Initially, people didn't even like smoking because of the bad taste, but massive advertising campaigns including paying celebrities to smoke in public changed the perception of the value of smoking. In a similar manner, after acquiring all major sources of diamonds and inflating their prices, advertisement convinced the public that a man absolutely needs a ring with a diamond to "prove" their love to their bride. Same for the food industry and sugar, you know where I am going. All of these are examples of "value" created by producers by misleading customers that their product is good and not harmful for them. But facts don't become true or false just because everybody believes in them. People can believe false things, and no matter what they do or say, those things remain false. Not to mention that immediate value for individuals can have considerable negative value in the long-term for society. Think at the huge resources we spend on finding cures for cancer and obesity, just because we sell addictive stuff and pretend that people "want" those. If you consider society as a whole, this appears to be an extremely inefficient use of resources.

shadowsun7 3y ago

Author here. Previous time this was on the front page: https://news.ycombinator.com/item?id=30930985 https://news.ycombinator.com/item?id=30930985 As an update, I temporarily relocated to a different city and spent four months training Judo for five hours every day and have since reached the vocab point for the sport. That experiment was covered here: https://commoncog.com/expertise-acceleration-experiment-judo/ https://commoncog.com/expertise-acceleration-experiment-judo... and https://commoncog.com/mental-strength-judo-life/ https://commoncog.com/mental-strength-judo-life/

Swizec 3y ago
Wow! How did you make time for 5 hours per day? I workout a lot and time is one of my biggest challenges.
ssivark 3y ago
Loved the post. Just to add to the principal observation about the vocab point — One thing I’ve noticed is that experts who can communicate/teach well are very effective at getting you to the vocab point (to the extent possible without experience) quickly! And once you reach the vocab point, the field is legible enough that you have the means to organize deliberate practice and interpret experience (if you want to go towards mastery) or delegate details (if you don’t need them). I wonder if this provokes any more thoughts from you.
lcnPylGDnU4H9OF 3y ago
I'm glad you were able to articulate this idea so clearly. I think similar advice which maybe isn't so clear is to try to avoid "being the smartest person in the room". On the topic, I'm reminded of coworkers talking about the most recent Super Bowl (I think, I don't follow sports) which included commentary from players who played as quarterbacks in the same season. One of the things my coworkers mentioned was how interesting it was to hear them talk about the game because they used unfamiliar terms.

nativecoinc 3y ago

There are some teachers of Samatha meditation, a Buddhist concentration meditation, that teach that you at any point are meditating at one of ten stages. This is both about skill as well as about factors like how agitated your mind is in general. And these stages seem to have very well-defined descriptions: for example one stage might be differentiated from the previous one by overcoming drowsiness. This is very helpful since some people teach meditation by just telling you how to do it but not (in fact sometimes actively avoiding) how you can evaluate where you are and how you are doing. Then the practice becomes just about “being present” and other slogans that might be wholly non-actionable. It’s also very difficult to protest this kind of approach (edit: the non-goal approach) since merely mentioning things like "goal" or "evaluation" will trigger someone’s knee-jerk you’re-doing-it-wrong reaction, even though what they practice might be completely different to what you are doing or trying to achieve.

xyzelement 3y ago

Not directly related to the article but I had two amusing encounters with expertise recently. I sat next to an Econ professor (at an Ivy) on a flight. I told him I had worked at a hedge fund, and he was like "wow, I could never put myself in a situation where my living depended on my economic theories being right." A good friend of mine is a kinda-famous evolutionary biologist. He's in his mid-40s now, unmarried and doesn't have kids, and likely none on the horizon. In the economist case, what does it mean that in his expert opinion, he wouldn't trust his own expert opinion! In the biologist case, what does it mean to be one of the world's leading experts on "evolutionary biology" and yet completely fail to pass your own genes? What's most interesting to me is that neither of these scenarios strike us as "super wrong." I can imagine most people wouldn't hesitate to attend a class taught by either one of these guys, while we'd avoid analogous situations (eg: we wouldn't use a broke financial advisor or fly on an airplane whose designer refuses to set foot on it.) What's the difference?

selykg 3y ago
As far as the economist. I suspect he understands that they're theories, thus, to some extent guesses. Some data might support it, but does it hold true in every case? Probably not. Does it hold true in most cases? Maybe not. Would he bet money, effectively what a hedge fund does in my very limited experience here? Sounds like no. This to me sounds like a simple example of he understands the limitations of his work! As for your evolutionary biologist friend. Being interested in evolutionary biology does not mean you are inherently someone that wants to be a parent. These are very very different things. Besides, evolutionary biology impacts more than just humans, it impacts every species. Being a parent comes with a great responsibility. Someone could just as easily be a really big fan of dogs, but not want one because of the responsibility or the costs, or similar.
- xyzelement 3y ago
  Yup, your explanation resonates with what I think are the superficial reasons society is accepting of both of these cases. You have to think a little bit deeper to feel why this might be problematic. If the economist weighs the risk to not put too much stock in these theories (no pun intended actually), why does he not likewise hesitate to teach these theories to his students? Like "I don't believe 100% that this is how the world works enough to bet my own money, but I am going to tell hundreds of students every year this is how the world works." The biologist story is subtler. One could argue that failure to become a parent signals that you deeply, fundamentally, don't get biology - despite your superficial interest and theoretical knowledge? Like, you may talk about zygotes all day but you failed to make one... Maybe the difference between economists and biologists vs financial advisors and engineers is that we expect the later to have skin in the game but for some reason allow the former to not have to.

parpfish 3y ago

This seems like a factor in Dunning-Krueger. Experts can see all the gradients in ability and will extrapolate how far they are from the top, but all novices know is how far they are from the bottom. If those distance estimates are biased, it’d lead to novices overestimating ability and experts underestimating

bena 3y ago
Dunning-Kruger is more of a statistical effect than a psychological one. The same graphs can be reproduced using random noise.
- taeric 3y ago
  I really regret how hard of a hold the Dunning-Kruger report has had on our generation. Worse, it seems every reference trying to point out that it is, in fact, not saying what people think it says just reinforces folks believe that it says what they want it to say. :(

xyzt 3y ago

for me, path to expertise is knowing the "language", "practicing maturity," and system 1 mastery of a topic. i think expertise is "mastery of system 1(ref Daniel Kahneman) of a topic." which means identifying more "patterns" through "practice and observation." the patterns capture both the rights and the wrongs. the entry to any subject is "learning the language" of the subject. what comes next is to become "mature." maturity is based on amount of practice in a big and unbiased search space, the ability to play with subject and identifying primitive constructs, ability to criticize and seeing the weaknesses, the measuring method for maturity level is amount of "simplicity," "freedom", "flexibility", "speed", and "ability to identify the essence" that the system one adheres to bring. excerpt from Saunders Mac Lane "category theory for working mathematician." "many properties of mathematical construction can be identified using universal mapping properties." see how simple the definition is?

xyzelement 3y ago

// Customers tends not to be experts. So after a certain level of skill has been achieved, any improvements that are made through experience are invisible to the people you’re selling to. The relevant question is - does it matter? The Formula 1 pit crew is more expert than the mechanics at the Toyota dealership in ways I can't appreciate - but does it matter for my Highlander's oil change? The answer might be "Yes, the F1 crew will do it in a way that prolongs the life of your engine" (I care about that!) or "Yes, the F1 crew will change your oil in 20% of the time" (I may or may not care about saving a few minutes every 6 months) or "No, you won't notice a difference but they are just more expert" (in which case I totally don't care!) Part of expertise is ability to articulate its relevance to a given situation. Your customer may need help in validating your expertise, but at least start with the claim of what you can do.

nradov 3y ago
F1 pit crew members are highly trained athletes who specialize in fast tire changes. Many of them are recruited from other sports based on strength, speed, and agility. They often have no real experience as automotive technicians. F1 teams do have world class mechanics but most of them aren't part of the pit crews.
- Etheryte 3y ago
  Are you familiar with the expression missing the forest for the trees?
- Cipater 3y ago
  >Many of them are recruited from other sports based on strength, speed, and agility. This is not true at all. F1 pit crews are mechanics who have worked with cars in other motorsport racing series/feeder series and worked their way up to an F1 team.
ianmcgowan 3y ago
I think the point is when reaching higher levels of expertise is not to try and convince existing customers to move up the ladder with you, it's that you need to find an entirely new set of customers who need that level of expertise and appreciate the difference. It's not about explaining - there's a quantum leap between levels. It might be trite, but as a contractor walking my rate up over 10 years, I found it interesting.

atleastoptimal 3y ago

I'd wager a large portion social embarrassment is when a person ostensibly believes they have competence in a certain realm but do not. Being aware of yourself and how your peaks and valleys are perceived is the foundation of charisma.

deleted 3y ago

[deleted]

kulikalov 3y ago

If it quacks like a duck - it must be a duck. Not really. > In every profession there’s a point where further improvements in skills are so subtle that the difference between one level and another can only be appreciated by another expert Goal defines the benchmark for proficiency. Better tennis player scores. Better software engineer writes more space/time efficient code or solves business problem faster (whichever the objective). Better writer gets more views or inbound sales or whatever the goal is. If you get better at achieving ever greater goals, then your expertise is growing. No amount of luck can compensate for a lack of expertise.

Corrado 3y ago

I've experienced "tacit..feelings" personally on many development projects. I'm not sure why this path is better than the other path, but it just feels better to me. One example is where to put new code. Do you put it in the existing code (monolith) or do you create a "service" for the code? Sure, some things are pretty straightforward but sometimes there really is a toss up of where this new thing should live. The problem is when someone asks me why, I can't really explain it and it gives less weight to the decision. However, looking back I can say that I've been more right than wrong, so I guess I have that going for me.

kybernetikos 3y ago

Enjoyable read that definitely touches on something real. I was struck by the anecdote that Herbie Hancock tells in the linked video though. He misheard some advice from someone he respected, took it to heart and it changed the way he played. I notice that this seems to work in opposition to the idea of a vocab point where high level practitioners are talking about real concrete experiences they are having that the lay man can't understand until they level up. The way I think it worked in this case, is that as you go up the ladder of skill, you start seeing and thinking in more abstract, metaphorical terms. For a beginning student, you're focused on the mechanics, for the master, you're focused on what it means and how it relates to the human experience. This means that a master can apply an a metaphor that the beginner can't even understand. Indeed, Herbie Hancock can apply a metaphor that nobody else could even imagine had any relevance at all. Perhaps rather than 'vocab point' we should talk about 'metaphor point'.

grumpy_coder 3y ago

There is a huge problem with this idea, it suggests that a competent critic has usable skill in an area. This is obviously false. Moreover I see it as a huge problem in our current environment of easy access to Youtube explainers of everything. In my areas of expertise I come across many more people that 'talk the talk' than can actually 'walk the walk'. A chef must be judged on dishes they produce, a dancer on their performances, and a software developer on working code.