30 ms·

95%-ile isn't that good (2020)

166 points by DantesKite 5y ago

bombcar 5y ago
A not quite obvious side effect of this - you should be aware of the vast majority of things you do that you are not in the top 5% - and don’t sweat it. Most things in life just need to be done, not done exceptionally well, so spend your effort on the things that matter. For example, I shop pretty badly at the store, no list, no plan - not worth it to me to develop those, I can spend time elsewhere on the things I can be good at.
- loloquwowndueo 5y ago
  Using a shopping list would actually save you time to spend elsewhere rather than at a crappy grocery store :)
  - ryanbrunner 5y ago
    I find I do groceries at similar speeds with or without a list (if anything, I can often be faster if I just get what seems to make sense rather than hunting out something specific from a list). Where it makes the bigger difference is in reduced food waste (probably the biggest difference) and less money spent.
    - bombcar 5y ago
      Part of it is also “do I get basically the same stuff each time” or in my case the grocery store is a block away so it’s “what five items am I grabbing now”. And even if I did have a list I’d still be bottom 50% I’m sure.
  - toolz 5y ago
    Time is rarely the limiting factor in achieving goals. The vast majority of us kill time with time sinks like playing on our phone or watching TV series - none of which are likely to be life goals. Instead optimizing for putting productive energy towards real goals and avoiding menial tasks that at best save some time seems like a more optimal strategy.
    - chii 5y ago
      > The vast majority of us kill time with time sinks like playing on our phone or watching TV series - none of which are likely to be life goals. People state a goal, but that's not a goal - that's an aspiration. The person doing the above "time wasting" is just setting the actual goal of "relax" and "hedonism", which tbh, is completely fine. What they say their goals are is irrelevant. Not very many people actually work towards the goal they say they want to achieve. Those who do are quite likely to achieve it.
  - tasha0663 5y ago
    More time to spend bikeshedding the ultimate grocery list productivity process and integrating a smart refrigerator!
    - earleybird 5y ago
      I'll see your bikeshed and raise you a yak shave. Sometimes I just need to be "good enough" at something because it's not my ultimate goal (aka the thing I want to accomplish or be good at).
  - coatmatter 5y ago
    I'll use a shopping list on the rare occasion where I really need to buy something, and very much am in tune with how to use both digital and paper lists / notes, but with the vast majority of the time, I know what I want to buy or what I want to browse, and that most of the time, having a list makes very little difference to my day to day life. Sometimes I feel that people try to over-optimise grocery shopping. Basically, my process is simple and works: 1. realise I need more food, 2. grab keys, phone (and maybe wallet) and reusable bags, 3. go to the supermarket, 4. browse the usual stuff I'm interested in and look for what's on sale, 5. grab it off the shelf if I want it. It's really not that complicated, nor at all stressful to shop without a list. Not everything in life needs a list.
- rp3 5y ago
elil17 5y ago
A video game and public speaking seem like really weird/unusual examples. A ton of people play video games, very few people do it professionally. A ton of people give talks, but it’s usually a small part of their jobs. Being even 50%-ile among professionals for either would certainly be extremely impressive. When I think about normal jobs, which are less competitive fields than professional gaming or speaking, being 95%-ile is very good (although 50%-ile is not great). Certainly, the top 5% of people at my company (200 people in my office, so 10 people I work with) are on the “top of their game”, so to speak - they always drive towards the most important objectives, rarely make errors, come up with incredible ideas, etc.
- indymike 5y ago
  > A video game and public speaking seem like really weird/unusual examples. I thought they were great examples because most people are familiar with both. Dan's not wrong: most people don't improve because they don't invest in getting better, and often getting better is easy to do. Incidentally, I watched one of my kids do this with Overwatch. She started watching the replays and seeing what she did wrong, and then that turned into what is that player doing right. Two weeks later top 50%.
  - ghaff 5y ago
    I agree. That probably do require spending a fair bit of time to build confidence and skills. But they also benefit from some deliberate practice/skills building as opposed to just repeating some of the same mistakes over and over again.
- e4e78a06 5y ago
  > Being even 50%-ile among professionals for either would certainly be extremely impressive. I think you'd be surprised at how low the bar is. And how people either 1) don't care about actively improving or 2) don't effectively learn from their mistakes. For a lot of people their career is just a way to get paid. As long as they don't get fired that's enough for them. And for other people they never developed a growth mindset and a habit of introspection, or they treat mistakes/failure as a painful memory to be avoided rather than something to be learned from.
dang 5y ago
Discussed at the time: 95th percentile isn't that hard to reach - https://news.ycombinator.com/item?id=22265197 https://news.ycombinator.com/item?id=22265197 - Feb 2020 (286 comments)
rp3 5y ago
More interesting is that reaching the 99.99th percentile is very difficult, and yet completely unremarkable. I love David Foster Wallace’s essay on this subject: “ Tennis Player Michael Joyce's Professional Artistry as a Paradigm of Certain Stuff about Choice, Freedom, Discipline, Joy, Grotesquerie, and Human Completeness.” He follows a top 200 and is impressed by how absolutely amazing he is at tennis, but ultimately knows the player will know neither fame nor fortune.
- lostdog 5y ago
  It's important to choose the right thing to be in the top 0.01 percent of. Being the 200th best bobsledder probably sucks. Being in the top 200 lawyers is probably pretty awesome.
  - thetallstick 5y ago
    And if you’re in the top, say 1/3, of engineers you’ll make a good living. If you’re in the top 1% of say, chess players, you probably won’t even get tournament invites.
    - eru 5y ago
      Though you might be able to make a living with teaching chess. Similarly for musicians: teaching music is a reasonably stable income stream.
    - ghaff 5y ago
      Tournament jobs is even a term. Look for any field that has unpaid or very lowly paid internships (or would-be actors waiting tables) because so many people will work for nothing to get a foot in the door hoping for a big break.
  - ipaddr 5y ago
    Pool sizes and opportunity vary. Once you are part of a bobsledder team you are in a limited group where bobsled track time is available. You could already be in the top 200. To be in a top 200 lawyer you would need to beat out a huge pool.
rr808 5y ago
Trivial to be top 5 percent of the general population at programming Possible to be top 5 percent of the programmers Very difficult to be in the top 5 percent of FANG programmers Nearly impossible to be in the top 5 percent of Google Principal engineers
- thisisonthetest 5y ago
  I would counter with the idea that seniority and programming ability are inversely proportional via The Gervais Principal (mentioned on hn again just recently)
  - solveit 5y ago
    Principal Engineers at Google know their stuff, a viral blogpost notwithstanding.
    - eru 5y ago
      That blogpost doesn't even contradict this. In the terms of that article, Principal Engineers would perhaps be classified as 'losers'. > The Gervais principle differs from the Peter Principle, which it superficially resembles. The Peter Principle states that all people are promoted to the level of their incompetence. It is based on the assumption that future promotions are based on past performance. The Peter Principle is wrong for the simple reason that executives aren’t that stupid, and because there isn’t that much room in an upward-narrowing pyramid. They know what it takes for a promotion candidate to perform at the to level. So if they are promoting people beyond their competence anyway, under conditions of opportunity scarcity, there must be a good reason. > Scott Adams, seeing a different flaw in the Peter Principle, proposed the Dilbert Principle: that companies tend to systematically promote their least-competent employees to middle management to limit the damage they can do. This again is untrue. The Gervais principle predicts the exact opposite: that the most competent ones will be promoted to middle management. Michael Scott was a star salesman before he become a Clueless middle manager. The least competent employees (but not all of them — only certain enlightened incompetents) will be promoted not to middle management, but fast-tracked through to senior management. To the Sociopath level. From https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-or-the-office-according-to-the-office/ https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-... To be clear: I mean to imply no opinion on whether 'The Gervais Principle' is true nor applicable. I'm just saying that 'The Gervais Principle' is perfectly compatible with Principal Engineers at Google knowing their stuff.
twofornone 5y ago
>But I think it's just the opposite: most people can become (relatively) good at most things. I think this point of view is not only blatantly false but also much more dangerous than most people realize. It sets an unrealistic standard across all disciplines, with a wide range of detrimental effects: 1. People entering majors that they are not qualified for, taking on more debt and failing later than than they otherwise would 2. Incentives for schools and colleges to inflate grades, since people are less likely to question unrealistically high graduation rates 3. 1 and 2 combining in the worst case to uncrease the number of incompetent workers across technical industries We've had decades of this sort "you can do anything if you work hard" propaganda in the west now and I fear that in addition to reducing competence across the board, it reduces the average person's ability to recognize competence, since the default assumption is that we are all endowed with the same potential, something fundamentally incompatible with human nature.
- lucasmullens 5y ago
  The article was mostly about things like table tennis and card games, not really careers (since I would consider getting an education "practicing regularly" which the article said it wasn't referring to). Most people who practice can become relatively good at table tennis, drawing, juggling, etc.
- eru 5y ago
  Your observation is perfectly compatible with the notion you quoted from the article. It's just that the 95%-ile photography major or art major etc is not gonna make money from photography or art. Nobody buys the records of a 95%-ile musician either. (Though you can probably teach music at that skill level and earn a living.) In comparison, a 95%-ile engineer can expect to make a decent living from engineering. > I think this point of view is not only blatantly false but also much more dangerous than most people realize. I partially agree about the notion being dangerous, despite probably being true. The thing is that most people never put in the conscious effort required to get to 95%-ile proficiency. Despite any exhortations to the contrary.
  - chii 5y ago
xigency 5y ago
Admittedly, I only skimmed this. But within the space of video games, I’m not sure Overwatch is a stunning example since it’s generally a casual game. I’ve been playing various iterations of Counter-Strike and StarCraft for decades. Sometimes with deliberate practice, including reading strategies, playing custom maps to practice certain skills, and watching series aimed at improving level of play. And despite the hundreds of hours invested I don’t believe I’ve ever cracked the top 50% in any leaderboard rankings. Anyone in the top 5% is definitely that good, executing complex strategies with extreme precision. There are lots of areas in life that are highly competitive — where being the best takes raw talent, a huge time investment, and a bit of luck. I wouldn’t try to diminish that.
- temp8964 5y ago
  Top 5% of leaderboard may equal to 99.99% of participants.
  - xigency 5y ago
    Leaderboard may be the wrong term. In StarCraft 2 for instance, there are Bronze, Silver, Gold, Platinum, Diamond, Masters, and Grandmasters leagues. Diamond 1 to GM might include around 5% of the ranked players.
    - temp8964 5y ago
      Ranked players maybe top 10% of all players? Purely guessing.
      - BirdieNZ 5y ago
        No, although the median of the bell curve of ranked players is probably further to the right than the median of the bell curve of unranked players.
      - xigency 5y ago
        In SC2, unranked and ranked players are actually mixed in matchmaking. So if you are playing unranked you still have an MMR, just not one you can see. As an unranked player, you can tell what division you are in based on the portrait frame of your (ranked) opponents.
XorNot 5y ago
The classic "oh actually that's a 1 in 20 occurrence rate" issue.
- sprobertson 5y ago
  "95 percentile" vs "1 in 20" is a good example of how confusing statistics are. First sounds excellent, second sounds like anyone could (try to) do it.
  - eru 5y ago
    Yes. Though I think the more interesting part of the article are when the author goes into the detailed mechanics of how that effort looks like for two examples.
  - rightbyte 5y ago
    20 people is about a school class. Being the best at something everyone does in a class is rather hard. If you suck at skipping rope (I did) you could probably climb the implicit rank quite easily by practice, till you were up against the sporty girls with aptitude.
Buttons840 5y ago
On lichess, the 95th percentile rating of active participants is 2125. I wonder if this author would agree such a rating "isn't that good"? I think it's pretty darn good.
- ummonk 5y ago
  The whole point of the article is that the average active Lichess user would be 95th percentile of people who know how to play chess.
  - Buttons840 5y ago
    So someone who only knows how to play chess would be considered a participant? Maybe this is what the author intended, I don't know. However, I find this definition of "participant" strange, because it allows us to truthfully say that someone is a participant, even though they don't play chess (they know the rules, but don't play).
    - tempestn 5y ago
      I've played chess plenty of times. Against my parents growing up, against friends, computers, etc. Probably played 50 games of chess in my life. Compared to none at all, that's a lot. But I've never actually studied the game. I don't know any theory, haven't memorized any openings, etc. I have no idea what my rating would be, but I'm sure it'd be low. I expect there are plenty of similar people out there, who "play" chess, meaning they have done so on occasion, but are not in any way students of the game.
  - Invictus0 5y ago
    That still sounds completely wrong. I might give you 75th percentile but certainly not 95th.
- coatmatter 5y ago
  Yep, I'm forever 'amazed' (read: not surprised) that this type of content continually makes it to the top of the front page of this website. With the way the author described Overwatch, it sounds like the equivalent to 95th percentile for chess would be more 50th percentile for chess, because in order to reach around that level, one would have to put in that type of work as described to purportedly reach the 95th percentile in Overwatch. Based on my understanding with the Lichess's blitz rating system (which of course can't be directly mapped to any other rating system), I expect that players rated 2175 Glicko in Lichess blitz could roughly be getting close to 1800-2000 Elo with FIDE if they played FIDE classical tournaments and were rated accordingly. For comparison, FIDE's rating floor is 1000, but newbie players would likely remain "unrated" because they "aren't (yet) good enough" to be 1000 after 5 rated games. Meanwhile, GM Magnus Carlsen currently has a "live" FIDE rating of 2863.9 Elo (live, as opposed to published monthly rating). As a further comparison, popular Twitch chess streamers WFM Alexandra Botez and WFM Anna Cramling have FIDE ratings of 2020 and 2057 Elo respectively, and Chesscom blitz ratings of 2195 and 2084 glicko respectively. I think they would simply wipe the floor at most (but not all) chess clubs that exist in the world. There really aren't a lot of players who go through my chess club who are at or near this level. It really takes a lot of work to get anywhere close to that standard. The amount of work it takes for most to get that good would require chess to be their main hobby, if not something more. That, or they have to start it from a young age and also be quite obsessive about it. Exact numbers aside, this level strength in chess isn't to be sneezed at, and requires significant investments of time - often over a "lifetime" (where lifetime = 10 or more years of a high level of dedication, focus, and good guidance), which for many young players, it is most of their lifetime. Most chess players will never reach this standard of chess, despite following all the basic advice as per the post. So I just find top 5% / 95th percentile being bandied about to be applicable for any field as somewhat laughable or fabricated. Maybe the figure is real for Overwatch, but for things like chess, speedcubing, and instrumental/vocal music, it just doesn't quite work that way as much as many would wish it did. Yes, the overall message of practising and learning is good, but when extraordinary claims such as "95%-ile isn't that good" get thrown about with Overwatch being used as primary evidence, the substance isn't much more convincing than typical clickbait. Survivorship bias?
sheikheddy 5y ago
Wow, a lot of these comments seem to misunderstand that this is 95th percentile amongst people who participate, not people who practice actively. The article is mostly correct.
- coatmatter 5y ago
  Redefining percentile or who is considered an active participant as the author did is the greater self-serving misunderstanding. Following from my earlier comment here, this weekly graph already shows active players - those who aren't active aren't factored in: https://lichess.org/stat/rating/distribution/blitz https://lichess.org/stat/rating/distribution/blitz To be active on Lichess requires on to actually be active and have a Rating Deviation below a certain value. It's all easy to "reaching the top 1% is easy" if you're already at the top. I commented earlier explaining the difference between Overwatch and chess, pointing out how the author's description of what it took to get "good" at Overwatch simply doesn't work for chess. Without trying to slight Overwatch, chess is simply a very very tough game/sport to get very good at. As Grandmaster Ben Finegold would say, "I'm the best player in this chair!" Yep, so that also makes me in the 100th percentile when in my own chair, just with fancier words.
ummonk 5y ago
The caveat is that in any activity you’re disproportionately likely to encounter those who practice that activity the most. Therefore, you could be in the 90th percentile of participants, but below average against those you encounter.
- MonaroVXR 5y ago
  Also true (I guess) very difficult to make a good assumption about it.
themacguffinman 5y ago
> Despite all of the caveats above, my belief is that it's easier to become relatively good at real life activities relative to games or sports because there's so little delibrate practice put into most real life activities. > You're probably 99%-ile, but someone with no talent who's put in the time to practice the basics is going to have a serve that you can't return I have a soft spot for this kind of insight because it is supposed to be inspiring and I know I should take it in that spirit, but one thing that's always bothered me is the contradictory assertions that on the one hand, almost no one deliberately practices, but on the other hand, this makes it "easier". How can both assertions be true? Any sensible definition of difficulty has to consider how many people succeed at it. If very few people succeed at a task in practice, it is a difficult task no matter how "easy" it looks in theory. I've come to see passion & motivation as indistinguishable from "talent". Having the passion & motivation to put in the effort is part of one's "talent" and realistically is not that easy to change. Everyone knows that to write well, you need to practice writing a lot. Scott Alexander notes in his Parable of the Talents [1] that he never needed to really muster up the energy to practice writing a lot: > I know people who want to get good at writing, and make a mighty resolution to write two hundred words a day every day, and then after the first week they find it’s too annoying and give up. These people think I’m amazing, and why shouldn’t they? I’ve written a few hundred to a few thousand words pretty much every day for the past ten years. But as I’ve said before, this has taken exactly zero willpower. I've found this to be true in a lot of walks in life. While there are certainly people with genius intellect/ability, most of the "talented" people I meet aren't of that kind, they are just effortlessly more motivated or passionate about something than I am. For example, they often started programming when they were 10 or something and spent most of their childhood choosing to do little side projects instead of the normal variety of childhood time wasters. I think this essay's (95%-ile is not that good) thesis is roughly this: to be good at something, you just need to care a little, because most people don't care at all. Just need to care a little, that doesn't sound so hard, right? All I want to say is yes, in my experience caring even a little is so, so hard. There's a reason why most people don't care at all and that reason is probably not "you're better and more special than most people as a reader of this essay". [1] https://slatestarcodex.com/2015/01/31/the-parable-of-the-talents/ https://slatestarcodex.com/2015/01/31/the-parable-of-the-tal...
screye 5y ago
This article belies a key assumption that is also the biggest challenge when making such a statement. Distributional assumption - The assumption is that the mean and std dev of the participating class is in line with the general population. Overwatch and public speaking sneakily satisfy this assumption. Everyone is forced to speak in public at some point, and overwatch is the archetypal 'normie' game of the of the past few years. Dan then moves from these 2 scenarios to programming, where this assumption doesn't work. There are many formal and informal filters involved in making it an employable programmer. (Say complete a BSCS or equivalent experience). There are also supply-demand factors, where the top %ilers are presorting themselves into CS like no other time in history. Landing a FANGesque job straight out of college is already putting you in the 95th percentile, because of sheer number of filters that one person passes through. Other situations with strong filtering include inaccessible games like Dota2 or high barrier of entry sports like climbing. Now, I agree with Dan's title and the contents, but for a different reason. There is perpetual headroom in the pyramid of excellence. Eventually, those better than you at a particular thing go from being a number to a tangible list of names. But, whether that should mean anything is a purely personal decision. The argument that anyone can go beyond x-percetile through deliberate practice is always true until it isn't. That's because we tend to take our personal IQ, resilience, discipline and acumen for granted. So, I would rephrase that as: 'When the distributional assumption holds, 95th percentile would not be considered noteworthy by any institutions or people that the average FANG/HN reader would find worthy of respect.' Doesn't sound as useful with the qualifications. P.S: something something nuance for large scale communication is impossible. Criticizing danluu for one post lends credence to another post of his. Such is life.
lordnacho 5y ago
He's basically right. Here's another example, math contests. When I was in high school I was really into math. I was way ahead of my year group and I figured it was more interesting to do special problems of the "what is the last digit of 2^2022" variety, the kind of thing that didn't turn up on normal homework, but also not terribly complicated, the kind of thing you can figure out with a short burst of concentration. So I went to math contests around Europe with other schools. The thing was that most kids I met there were totally ordinary at maths. Sure, some were really quite good, but most of the bottom 75% were indistinguishable from your normal kid who doesn't know at 15 what an integral is. The friends on my team weren't even in the top set math class at my little school, but my observation was that most kids from the other schools weren't either. Most were along for the ride, much like myself at school sports events where you also got a trip across Europe for participating. I did okay at it, and in my final year we had a couple of really good kids on the team and won the contest. I thought that was pretty cool, but I also knew I wasn't close to the pinnacle of math contestery. Fact was I was like that kid who just had some natural ability but I didn't really practice a huge amount and nobody suggested there was a way to get much better. If you added up all the extra practice compared to just sitting in math class, maybe it was a couple of extra lessons a week, with more interesting homework. My math teacher was a great motivator whom I still keep in touch with, but we only had him and his experience. Fast forward to now and I recently discovered a friend of mine got a top 10 at the IMO. Basically top 10 among all high school kids on the planet. I can ask him the occasional interesting question that I see on FB, and he'll know how to do it. He did proper years of training in math, had a very famous coach among others, and went through the eye of the needle several times. There's a world of difference between that and your casual have-a-go person.
BasDirks 5y ago
After reaching that percentile the grind begins. And that's when learning begins. And that's when you develop character. The dopamine doesn't flow as freely any more as obstacles to improvement become the rule rather than the exception. Braggadocious anecdote: on December 25th, 2021, I started solving Kattis problems. I set myself a goal to reach a top 1000 rank in 2022. This took me two weeks instead of an entire year. I adjusted the goal to top 100. This is significantly harder, and after two more weeks I was tempted to dismiss the goal through excuses like: "these are just puzzles with limited real-world application". This is obvious bullshit, and I just wanted to avoid doing the work. I caught myself in the act and doubled down on the goal. I bought some books and learned a lot. Currently sitting on #229, and I _will_ reach top 100 this year.
birdyrooster 5y ago
Seems like their real life was Overwatch
aronhegedus 5y ago
article definitely made me think about what else I could be 95th percentile in. I agree that dedicated practice is very important compared with people that just play casually
dTal 5y ago
Sturgeon's law: 90% of everything is crap. Corollary: 95th percentile is only (approximately) 50th percentile of non-crap.
ajuc 5y ago
> Most people probably don't have the talent to play in a professional league regardless of their practice regimen, but when you can get to 95%-ile by fixing mistakes like "not realizing that you should stand on the objective", you don't really need a lot of talent to get to 95%-ile. In the only 2 competitive games I played enough to talk about this - sc2 and dota2 - the mistakes people make at my level (which at my peak was ~2000 mmr in dota and low plat in starcraft - probably top 60-40% of the active players in both) - were nothing like "not understanding game objectives". In starcraft it's easy to say what went wrong because there's much less variables. Usually it's going for a wrong build order blindly, not reacting to enemy unit composition in time, or just being too slow mechanically (usually in the economy part of the game, fights you can just attack-move into the enemy army, but you have to build buildings and units FAST or you don't have enough units to fight). I've been watching sc2 games on top level, I've even played vs people in top 90% in 8v8 free for all games, and there's just no way for me to win against them. I've played one 8v8 game where I was the only one in plat and the rest was in master/grand master. They left me alone for 20 minutes and fought it between themselves. I've build maxed 3/3/3 skytoss fleet with mothership and carriers, full wall of photon cannons and a lot of gates for instant remax. 20 minutes of reasonably fast base-building. After I started to fight I lost in like 5 minutes vs mass marines + medivacs :/ Guy just harassed me to death from 10 different angles while escaping my deathball of carriers. I had 10 times his resources but not the skill to defend from so many points at once. And he wasn't even grand master. Sc2 will show you exactly how much you suck. In dota there's much more chaos, sometimes it's lost in the picking phase, other times someone just feeds for 10 minutes, sometimes people don't understand who is more late-game and farm for 40 minutes vs later-game carry or constantly fight vs earlier-game lineup instead of farming. Other than these I usually don't know why we lose. It's never "I don't understand the game objective". I could probably get much higher if I wasn't spamming winter wyvern as pos4/5 every game. But it's the only hero I enjoy, so whatever.
flerchin 5y ago
The observations about 95% percentile being good or not are distracting from the point that focused, reviewed practice is necessary to level up.
- bo1024 5y ago
  True, and the point that people are often weirdly bad at deliberately improving, or weirdly unmotivated to practice deliberate improvement.
whoomp12342 5y ago
okay yeah but context matters here. Who cares about being in the 95th percentile of overwatch compared to say being in the 95th percentile of income earned?
- tandymodel100 5y ago
rapjr9 5y ago
In machine learning 95% accuracy is not very useful. It means the algorithm will be wrong 1 out of 20 times. So every 20th word in speech recognition will be incorrect, your medical diagnosis will be wrong for every 20th person, your face will not unlock your phone every 20th time, every 20th person will not be allowed to log into a web site. There are some machine learning applications where 95% is good enough or even great such as web search engines, but for many important applications it is nowhere near good enough and you need something more like 99.9% or better to be actually useful. Think about how often the algorithm can be allowed to make a mistake. If it's never, then don't use machine learning. If it's 1 in a million you need 99.9999% accuracy. 95% looks good in machine learning papers in computer science, but in the real world 95% often means dismal failure. Think of an autonomous killer robot, is mistakenly killing 1 out of 20 people ok? For a self driving car is harming the driver or car (or pedestrian/dog/child/other car) mistakenly even 1 out of 1000 (99.9%) times ok? I think not. One measure of what accuracy you need is how well people do the same task. If you can do better than people most of the time, maybe the algorithm is useful, though you also have to compare exactly how the algorithm fails to the way people fail. If people fail by choosing a fender bender and the algorithm fails by killing someone, the algorithm has to work better than people do.
- jvanderbot 5y ago
  95% accuracy has nothing to do with being in the 95pctile of active participants other than using the number 95. To achieve a 95th pctile ranking among the relevant machine learning frameworks means being more accurate than 95 % of other approaches, and that's probably quite good and useful.
- arbirk 5y ago
  Rank percentiles and statistical test accuracy / model accuracy percentiles can not be compared. The author of the article seems to be confused about statistics.
- deleted 5y ago
  [deleted]
whateveracct 5y ago
don't compare yourself to others
HDMI_Cable 5y ago
I think this only applies to some things, not others. If you're in the 95th percentile of high-school students, then you're probably getting into a good university. For university, the 95th percentile is enough to get into medicine, law, or a good business school. The 95th percentile for overall intelligence is 125. The 95th percentile for household income is $250 000. For things like video games, which have a broad net of casual players and a few who are very competitive, it isn't that good. All you need to do to get in the 95th percentile is to be competitive; but for applications where everyone is acting somewhat competitively, or for non-zero sum competitions like school or income, the 95th percentile can be very good.
D13Fd 5y ago
I think this is really "it's not hard to get into the 95th percentile of things that you are already fairly good at." Yes, if you have a history of playing video games, it's not hard to pick up another game and do well--and, with some focus, do very well. But what if he tried to pick up playing the flute? How about drawing comics? Competitive swimming? Investing? What about being in the 95th percentile of a complex field like medicine or law? Assuming he doesn't already have a background in those things, he's probably going to have a hard time reaching the "95th percentile" of skill.
- raydev 5y ago
  > But what if he tried to pick up playing the flute? How about drawing comics? Competitive swimming? Investing? Having seen exactly 2 of your examples in real life, particularly people who had no experience and no interest in painting or playing an instrument until their 40s, a few years of persistence and targeted practice made them better than me after a lifetime of “casual play”. I might have to put in less time to “get better than them” if I really sat down and tried for a year, but I’m not sure that matters in the way you’re thinking
mi100hael 5y ago
Golf is a real-world activity that can be used for comparison here. It's a good candidate because there is a standard handicap rating system issued by a central governing body, and there are a large number of amateur players who play regularly, compared to other sports. - Roughly 10% of golfers in the US maintain a handicap rating. - "Today, the 2,417,905 who have a handicap included 2,051,675 “active posters,” meaning they posted at least one score in 2020. However, the average number of posted scores was 38, almost double the average number of rounds played by golfers overall last year—so it’s pretty clear that those who get a handicap are among the game’s most engaged participants." [0] - "it has been found that on a given day the average golfer would be expected to post a score of approximately 100 strokes when following all the rules of golf" [1] which is 28 strokes over par. A 28 handicap is around the 5%-ile of golfers with a handicap [2] - As such, those 10% of golfers with a handicap should overlap pretty heavily with the top 10-20% of all golfers. The 50%-ile of golfers with a handicap is 13 [2], so by extrapolation that should be 90-95%-ile of all golfers. - 13 is still a solid handicap. That's someone who is breaking 90 (better than bogey golf) pretty much every round and likely has years of practice. Saying it "isn't very impressive because it's not that hard to do" would be an incorrect statement in my opinion. - (as a sanity check, the 90%-ile of golfers with a handicap is a handicap of 5. Most D1 golfers will have a handicap right around 0, and D3 will be maybe 2. Given the number of casual golfers, it seems likely that only about 1 or 2 out of 100 are nearing that level.) [0] https://www.linksmagazine.com/how-do-you-match-up-against-the-average-golf-handicap-in-2020/ https://www.linksmagazine.com/how-do-you-match-up-against-th... [1] https://golftips.golfweek.usatoday.com/average-golf-handicap-america-2455.html https://golftips.golfweek.usatoday.com/average-golf-handicap... [2] https://www.usga.org/content/usga/home-page/handicapping/handicapping-stats.html https://www.usga.org/content/usga/home-page/handicapping/han...
- ghaff 5y ago
  It also shows that the population you're comparing yourself to, i.e. the denominator, matters a lot. If you're comparing yourself mostly to people who do some activity pretty regularly, top 5-10% means you're pretty good compared to the average person on the street who may never have touched a golf club. But if you draw your population from anyone who has ever taken a swing at a golf ball, 5-10% is a lot less impressive.
westcort 5y ago
The biggest takeaway I got from this was that recording yourself doing something, then observing yourself completing it as an outsider can dramatically increase your performance. Relevant content: After watching a recording of myself writing code, I realized I was spending about a quarter of the total time implementing the feature tracking down which functions the bugs were in! This was completely non-obvious to me and I wouldn’t have found it out without recording myself Now that I’m aware that I spent so much time isolating which function a bugs are in, I now test each function as I write it to make sure they work This allows me to write code a lot faster as it dramatically reduces the amount of time it takes to debug my code

166 points by DantesKite 5y ago

A not quite obvious side effect of this - you should be aware of the vast majority of things you do that you are not in the top 5% - and don’t sweat it. Most things in life just need to be done, not done exceptionally well, so spend your effort on the things that matter. For example, I shop pretty badly at the store, no list, no plan - not worth it to me to develop those, I can spend time elsewhere on the things I can be good at.

loloquwowndueo 5y ago
Using a shopping list would actually save you time to spend elsewhere rather than at a crappy grocery store :)
- ryanbrunner 5y ago
  I find I do groceries at similar speeds with or without a list (if anything, I can often be faster if I just get what seems to make sense rather than hunting out something specific from a list). Where it makes the bigger difference is in reduced food waste (probably the biggest difference) and less money spent.
  - bombcar 5y ago
    Part of it is also “do I get basically the same stuff each time” or in my case the grocery store is a block away so it’s “what five items am I grabbing now”. And even if I did have a list I’d still be bottom 50% I’m sure.
- toolz 5y ago
  Time is rarely the limiting factor in achieving goals. The vast majority of us kill time with time sinks like playing on our phone or watching TV series - none of which are likely to be life goals. Instead optimizing for putting productive energy towards real goals and avoiding menial tasks that at best save some time seems like a more optimal strategy.
  - chii 5y ago
    > The vast majority of us kill time with time sinks like playing on our phone or watching TV series - none of which are likely to be life goals. People state a goal, but that's not a goal - that's an aspiration. The person doing the above "time wasting" is just setting the actual goal of "relax" and "hedonism", which tbh, is completely fine. What they say their goals are is irrelevant. Not very many people actually work towards the goal they say they want to achieve. Those who do are quite likely to achieve it.
- tasha0663 5y ago
  More time to spend bikeshedding the ultimate grocery list productivity process and integrating a smart refrigerator!
  - earleybird 5y ago
    I'll see your bikeshed and raise you a yak shave. Sometimes I just need to be "good enough" at something because it's not my ultimate goal (aka the thing I want to accomplish or be good at).
- coatmatter 5y ago
  I'll use a shopping list on the rare occasion where I really need to buy something, and very much am in tune with how to use both digital and paper lists / notes, but with the vast majority of the time, I know what I want to buy or what I want to browse, and that most of the time, having a list makes very little difference to my day to day life. Sometimes I feel that people try to over-optimise grocery shopping. Basically, my process is simple and works: 1. realise I need more food, 2. grab keys, phone (and maybe wallet) and reusable bags, 3. go to the supermarket, 4. browse the usual stuff I'm interested in and look for what's on sale, 5. grab it off the shelf if I want it. It's really not that complicated, nor at all stressful to shop without a list. Not everything in life needs a list.
rp3 5y ago

elil17 5y ago

A video game and public speaking seem like really weird/unusual examples. A ton of people play video games, very few people do it professionally. A ton of people give talks, but it’s usually a small part of their jobs. Being even 50%-ile among professionals for either would certainly be extremely impressive. When I think about normal jobs, which are less competitive fields than professional gaming or speaking, being 95%-ile is very good (although 50%-ile is not great). Certainly, the top 5% of people at my company (200 people in my office, so 10 people I work with) are on the “top of their game”, so to speak - they always drive towards the most important objectives, rarely make errors, come up with incredible ideas, etc.

indymike 5y ago
> A video game and public speaking seem like really weird/unusual examples. I thought they were great examples because most people are familiar with both. Dan's not wrong: most people don't improve because they don't invest in getting better, and often getting better is easy to do. Incidentally, I watched one of my kids do this with Overwatch. She started watching the replays and seeing what she did wrong, and then that turned into what is that player doing right. Two weeks later top 50%.
- ghaff 5y ago
  I agree. That probably do require spending a fair bit of time to build confidence and skills. But they also benefit from some deliberate practice/skills building as opposed to just repeating some of the same mistakes over and over again.
e4e78a06 5y ago
> Being even 50%-ile among professionals for either would certainly be extremely impressive. I think you'd be surprised at how low the bar is. And how people either 1) don't care about actively improving or 2) don't effectively learn from their mistakes. For a lot of people their career is just a way to get paid. As long as they don't get fired that's enough for them. And for other people they never developed a growth mindset and a habit of introspection, or they treat mistakes/failure as a painful memory to be avoided rather than something to be learned from.

dang 5y ago

Discussed at the time: 95th percentile isn't that hard to reach - https://news.ycombinator.com/item?id=22265197 https://news.ycombinator.com/item?id=22265197 - Feb 2020 (286 comments)

rp3 5y ago

More interesting is that reaching the 99.99th percentile is very difficult, and yet completely unremarkable. I love David Foster Wallace’s essay on this subject: “ Tennis Player Michael Joyce's Professional Artistry as a Paradigm of Certain Stuff about Choice, Freedom, Discipline, Joy, Grotesquerie, and Human Completeness.” He follows a top 200 and is impressed by how absolutely amazing he is at tennis, but ultimately knows the player will know neither fame nor fortune.

lostdog 5y ago
It's important to choose the right thing to be in the top 0.01 percent of. Being the 200th best bobsledder probably sucks. Being in the top 200 lawyers is probably pretty awesome.
- thetallstick 5y ago
  And if you’re in the top, say 1/3, of engineers you’ll make a good living. If you’re in the top 1% of say, chess players, you probably won’t even get tournament invites.
  - eru 5y ago
    Though you might be able to make a living with teaching chess. Similarly for musicians: teaching music is a reasonably stable income stream.
  - ghaff 5y ago
    Tournament jobs is even a term. Look for any field that has unpaid or very lowly paid internships (or would-be actors waiting tables) because so many people will work for nothing to get a foot in the door hoping for a big break.
- ipaddr 5y ago
  Pool sizes and opportunity vary. Once you are part of a bobsledder team you are in a limited group where bobsled track time is available. You could already be in the top 200. To be in a top 200 lawyer you would need to beat out a huge pool.

rr808 5y ago

Trivial to be top 5 percent of the general population at programming Possible to be top 5 percent of the programmers Very difficult to be in the top 5 percent of FANG programmers Nearly impossible to be in the top 5 percent of Google Principal engineers

thisisonthetest 5y ago
I would counter with the idea that seniority and programming ability are inversely proportional via The Gervais Principal (mentioned on hn again just recently)
- solveit 5y ago
  Principal Engineers at Google know their stuff, a viral blogpost notwithstanding.
  - eru 5y ago
    That blogpost doesn't even contradict this. In the terms of that article, Principal Engineers would perhaps be classified as 'losers'. > The Gervais principle differs from the Peter Principle, which it superficially resembles. The Peter Principle states that all people are promoted to the level of their incompetence. It is based on the assumption that future promotions are based on past performance. The Peter Principle is wrong for the simple reason that executives aren’t that stupid, and because there isn’t that much room in an upward-narrowing pyramid. They know what it takes for a promotion candidate to perform at the to level. So if they are promoting people beyond their competence anyway, under conditions of opportunity scarcity, there must be a good reason. > Scott Adams, seeing a different flaw in the Peter Principle, proposed the Dilbert Principle: that companies tend to systematically promote their least-competent employees to middle management to limit the damage they can do. This again is untrue. The Gervais principle predicts the exact opposite: that the most competent ones will be promoted to middle management. Michael Scott was a star salesman before he become a Clueless middle manager. The least competent employees (but not all of them — only certain enlightened incompetents) will be promoted not to middle management, but fast-tracked through to senior management. To the Sociopath level. From https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-or-the-office-according-to-the-office/ https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-... To be clear: I mean to imply no opinion on whether 'The Gervais Principle' is true nor applicable. I'm just saying that 'The Gervais Principle' is perfectly compatible with Principal Engineers at Google knowing their stuff.

twofornone 5y ago

>But I think it's just the opposite: most people can become (relatively) good at most things. I think this point of view is not only blatantly false but also much more dangerous than most people realize. It sets an unrealistic standard across all disciplines, with a wide range of detrimental effects: 1. People entering majors that they are not qualified for, taking on more debt and failing later than than they otherwise would 2. Incentives for schools and colleges to inflate grades, since people are less likely to question unrealistically high graduation rates 3. 1 and 2 combining in the worst case to uncrease the number of incompetent workers across technical industries We've had decades of this sort "you can do anything if you work hard" propaganda in the west now and I fear that in addition to reducing competence across the board, it reduces the average person's ability to recognize competence, since the default assumption is that we are all endowed with the same potential, something fundamentally incompatible with human nature.

lucasmullens 5y ago
The article was mostly about things like table tennis and card games, not really careers (since I would consider getting an education "practicing regularly" which the article said it wasn't referring to). Most people who practice can become relatively good at table tennis, drawing, juggling, etc.
eru 5y ago
Your observation is perfectly compatible with the notion you quoted from the article. It's just that the 95%-ile photography major or art major etc is not gonna make money from photography or art. Nobody buys the records of a 95%-ile musician either. (Though you can probably teach music at that skill level and earn a living.) In comparison, a 95%-ile engineer can expect to make a decent living from engineering. > I think this point of view is not only blatantly false but also much more dangerous than most people realize. I partially agree about the notion being dangerous, despite probably being true. The thing is that most people never put in the conscious effort required to get to 95%-ile proficiency. Despite any exhortations to the contrary.
- chii 5y ago

xigency 5y ago

Admittedly, I only skimmed this. But within the space of video games, I’m not sure Overwatch is a stunning example since it’s generally a casual game. I’ve been playing various iterations of Counter-Strike and StarCraft for decades. Sometimes with deliberate practice, including reading strategies, playing custom maps to practice certain skills, and watching series aimed at improving level of play. And despite the hundreds of hours invested I don’t believe I’ve ever cracked the top 50% in any leaderboard rankings. Anyone in the top 5% is definitely that good, executing complex strategies with extreme precision. There are lots of areas in life that are highly competitive — where being the best takes raw talent, a huge time investment, and a bit of luck. I wouldn’t try to diminish that.

temp8964 5y ago
Top 5% of leaderboard may equal to 99.99% of participants.
- xigency 5y ago
  Leaderboard may be the wrong term. In StarCraft 2 for instance, there are Bronze, Silver, Gold, Platinum, Diamond, Masters, and Grandmasters leagues. Diamond 1 to GM might include around 5% of the ranked players.
  - temp8964 5y ago
    Ranked players maybe top 10% of all players? Purely guessing.
    - BirdieNZ 5y ago
      No, although the median of the bell curve of ranked players is probably further to the right than the median of the bell curve of unranked players.
    - xigency 5y ago
      In SC2, unranked and ranked players are actually mixed in matchmaking. So if you are playing unranked you still have an MMR, just not one you can see. As an unranked player, you can tell what division you are in based on the portrait frame of your (ranked) opponents.

XorNot 5y ago

The classic "oh actually that's a 1 in 20 occurrence rate" issue.

sprobertson 5y ago
"95 percentile" vs "1 in 20" is a good example of how confusing statistics are. First sounds excellent, second sounds like anyone could (try to) do it.
- eru 5y ago
  Yes. Though I think the more interesting part of the article are when the author goes into the detailed mechanics of how that effort looks like for two examples.
- rightbyte 5y ago
  20 people is about a school class. Being the best at something everyone does in a class is rather hard. If you suck at skipping rope (I did) you could probably climb the implicit rank quite easily by practice, till you were up against the sporty girls with aptitude.

Buttons840 5y ago

On lichess, the 95th percentile rating of active participants is 2125. I wonder if this author would agree such a rating "isn't that good"? I think it's pretty darn good.

ummonk 5y ago
The whole point of the article is that the average active Lichess user would be 95th percentile of people who know how to play chess.
- Buttons840 5y ago
  So someone who only knows how to play chess would be considered a participant? Maybe this is what the author intended, I don't know. However, I find this definition of "participant" strange, because it allows us to truthfully say that someone is a participant, even though they don't play chess (they know the rules, but don't play).
  - tempestn 5y ago
    I've played chess plenty of times. Against my parents growing up, against friends, computers, etc. Probably played 50 games of chess in my life. Compared to none at all, that's a lot. But I've never actually studied the game. I don't know any theory, haven't memorized any openings, etc. I have no idea what my rating would be, but I'm sure it'd be low. I expect there are plenty of similar people out there, who "play" chess, meaning they have done so on occasion, but are not in any way students of the game.
- Invictus0 5y ago
  That still sounds completely wrong. I might give you 75th percentile but certainly not 95th.
coatmatter 5y ago
Yep, I'm forever 'amazed' (read: not surprised) that this type of content continually makes it to the top of the front page of this website. With the way the author described Overwatch, it sounds like the equivalent to 95th percentile for chess would be more 50th percentile for chess, because in order to reach around that level, one would have to put in that type of work as described to purportedly reach the 95th percentile in Overwatch. Based on my understanding with the Lichess's blitz rating system (which of course can't be directly mapped to any other rating system), I expect that players rated 2175 Glicko in Lichess blitz could roughly be getting close to 1800-2000 Elo with FIDE if they played FIDE classical tournaments and were rated accordingly. For comparison, FIDE's rating floor is 1000, but newbie players would likely remain "unrated" because they "aren't (yet) good enough" to be 1000 after 5 rated games. Meanwhile, GM Magnus Carlsen currently has a "live" FIDE rating of 2863.9 Elo (live, as opposed to published monthly rating). As a further comparison, popular Twitch chess streamers WFM Alexandra Botez and WFM Anna Cramling have FIDE ratings of 2020 and 2057 Elo respectively, and Chesscom blitz ratings of 2195 and 2084 glicko respectively. I think they would simply wipe the floor at most (but not all) chess clubs that exist in the world. There really aren't a lot of players who go through my chess club who are at or near this level. It really takes a lot of work to get anywhere close to that standard. The amount of work it takes for most to get that good would require chess to be their main hobby, if not something more. That, or they have to start it from a young age and also be quite obsessive about it. Exact numbers aside, this level strength in chess isn't to be sneezed at, and requires significant investments of time - often over a "lifetime" (where lifetime = 10 or more years of a high level of dedication, focus, and good guidance), which for many young players, it is most of their lifetime. Most chess players will never reach this standard of chess, despite following all the basic advice as per the post. So I just find top 5% / 95th percentile being bandied about to be applicable for any field as somewhat laughable or fabricated. Maybe the figure is real for Overwatch, but for things like chess, speedcubing, and instrumental/vocal music, it just doesn't quite work that way as much as many would wish it did. Yes, the overall message of practising and learning is good, but when extraordinary claims such as "95%-ile isn't that good" get thrown about with Overwatch being used as primary evidence, the substance isn't much more convincing than typical clickbait. Survivorship bias?

sheikheddy 5y ago

Wow, a lot of these comments seem to misunderstand that this is 95th percentile amongst people who participate, not people who practice actively. The article is mostly correct.

coatmatter 5y ago
Redefining percentile or who is considered an active participant as the author did is the greater self-serving misunderstanding. Following from my earlier comment here, this weekly graph already shows active players - those who aren't active aren't factored in: https://lichess.org/stat/rating/distribution/blitz https://lichess.org/stat/rating/distribution/blitz To be active on Lichess requires on to actually be active and have a Rating Deviation below a certain value. It's all easy to "reaching the top 1% is easy" if you're already at the top. I commented earlier explaining the difference between Overwatch and chess, pointing out how the author's description of what it took to get "good" at Overwatch simply doesn't work for chess. Without trying to slight Overwatch, chess is simply a very very tough game/sport to get very good at. As Grandmaster Ben Finegold would say, "I'm the best player in this chair!" Yep, so that also makes me in the 100th percentile when in my own chair, just with fancier words.

ummonk 5y ago

The caveat is that in any activity you’re disproportionately likely to encounter those who practice that activity the most. Therefore, you could be in the 90th percentile of participants, but below average against those you encounter.

MonaroVXR 5y ago
Also true (I guess) very difficult to make a good assumption about it.

themacguffinman 5y ago

> Despite all of the caveats above, my belief is that it's easier to become relatively good at real life activities relative to games or sports because there's so little delibrate practice put into most real life activities. > You're probably 99%-ile, but someone with no talent who's put in the time to practice the basics is going to have a serve that you can't return I have a soft spot for this kind of insight because it is supposed to be inspiring and I know I should take it in that spirit, but one thing that's always bothered me is the contradictory assertions that on the one hand, almost no one deliberately practices, but on the other hand, this makes it "easier". How can both assertions be true? Any sensible definition of difficulty has to consider how many people succeed at it. If very few people succeed at a task in practice, it is a difficult task no matter how "easy" it looks in theory. I've come to see passion & motivation as indistinguishable from "talent". Having the passion & motivation to put in the effort is part of one's "talent" and realistically is not that easy to change. Everyone knows that to write well, you need to practice writing a lot. Scott Alexander notes in his Parable of the Talents [1] that he never needed to really muster up the energy to practice writing a lot: > I know people who want to get good at writing, and make a mighty resolution to write two hundred words a day every day, and then after the first week they find it’s too annoying and give up. These people think I’m amazing, and why shouldn’t they? I’ve written a few hundred to a few thousand words pretty much every day for the past ten years. But as I’ve said before, this has taken exactly zero willpower. I've found this to be true in a lot of walks in life. While there are certainly people with genius intellect/ability, most of the "talented" people I meet aren't of that kind, they are just effortlessly more motivated or passionate about something than I am. For example, they often started programming when they were 10 or something and spent most of their childhood choosing to do little side projects instead of the normal variety of childhood time wasters. I think this essay's (95%-ile is not that good) thesis is roughly this: to be good at something, you just need to care a little, because most people don't care at all. Just need to care a little, that doesn't sound so hard, right? All I want to say is yes, in my experience caring even a little is so, so hard. There's a reason why most people don't care at all and that reason is probably not "you're better and more special than most people as a reader of this essay". [1] https://slatestarcodex.com/2015/01/31/the-parable-of-the-talents/ https://slatestarcodex.com/2015/01/31/the-parable-of-the-tal...

screye 5y ago

This article belies a key assumption that is also the biggest challenge when making such a statement. Distributional assumption - The assumption is that the mean and std dev of the participating class is in line with the general population. Overwatch and public speaking sneakily satisfy this assumption. Everyone is forced to speak in public at some point, and overwatch is the archetypal 'normie' game of the of the past few years. Dan then moves from these 2 scenarios to programming, where this assumption doesn't work. There are many formal and informal filters involved in making it an employable programmer. (Say complete a BSCS or equivalent experience). There are also supply-demand factors, where the top %ilers are presorting themselves into CS like no other time in history. Landing a FANGesque job straight out of college is already putting you in the 95th percentile, because of sheer number of filters that one person passes through. Other situations with strong filtering include inaccessible games like Dota2 or high barrier of entry sports like climbing. Now, I agree with Dan's title and the contents, but for a different reason. There is perpetual headroom in the pyramid of excellence. Eventually, those better than you at a particular thing go from being a number to a tangible list of names. But, whether that should mean anything is a purely personal decision. The argument that anyone can go beyond x-percetile through deliberate practice is always true until it isn't. That's because we tend to take our personal IQ, resilience, discipline and acumen for granted. So, I would rephrase that as: 'When the distributional assumption holds, 95th percentile would not be considered noteworthy by any institutions or people that the average FANG/HN reader would find worthy of respect.' Doesn't sound as useful with the qualifications. P.S: something something nuance for large scale communication is impossible. Criticizing danluu for one post lends credence to another post of his. Such is life.

lordnacho 5y ago

He's basically right. Here's another example, math contests. When I was in high school I was really into math. I was way ahead of my year group and I figured it was more interesting to do special problems of the "what is the last digit of 2^2022" variety, the kind of thing that didn't turn up on normal homework, but also not terribly complicated, the kind of thing you can figure out with a short burst of concentration. So I went to math contests around Europe with other schools. The thing was that most kids I met there were totally ordinary at maths. Sure, some were really quite good, but most of the bottom 75% were indistinguishable from your normal kid who doesn't know at 15 what an integral is. The friends on my team weren't even in the top set math class at my little school, but my observation was that most kids from the other schools weren't either. Most were along for the ride, much like myself at school sports events where you also got a trip across Europe for participating. I did okay at it, and in my final year we had a couple of really good kids on the team and won the contest. I thought that was pretty cool, but I also knew I wasn't close to the pinnacle of math contestery. Fact was I was like that kid who just had some natural ability but I didn't really practice a huge amount and nobody suggested there was a way to get much better. If you added up all the extra practice compared to just sitting in math class, maybe it was a couple of extra lessons a week, with more interesting homework. My math teacher was a great motivator whom I still keep in touch with, but we only had him and his experience. Fast forward to now and I recently discovered a friend of mine got a top 10 at the IMO. Basically top 10 among all high school kids on the planet. I can ask him the occasional interesting question that I see on FB, and he'll know how to do it. He did proper years of training in math, had a very famous coach among others, and went through the eye of the needle several times. There's a world of difference between that and your casual have-a-go person.

BasDirks 5y ago

After reaching that percentile the grind begins. And that's when learning begins. And that's when you develop character. The dopamine doesn't flow as freely any more as obstacles to improvement become the rule rather than the exception. Braggadocious anecdote: on December 25th, 2021, I started solving Kattis problems. I set myself a goal to reach a top 1000 rank in 2022. This took me two weeks instead of an entire year. I adjusted the goal to top 100. This is significantly harder, and after two more weeks I was tempted to dismiss the goal through excuses like: "these are just puzzles with limited real-world application". This is obvious bullshit, and I just wanted to avoid doing the work. I caught myself in the act and doubled down on the goal. I bought some books and learned a lot. Currently sitting on #229, and I _will_ reach top 100 this year.

birdyrooster 5y ago

Seems like their real life was Overwatch

aronhegedus 5y ago

article definitely made me think about what else I could be 95th percentile in. I agree that dedicated practice is very important compared with people that just play casually

dTal 5y ago

Sturgeon's law: 90% of everything is crap. Corollary: 95th percentile is only (approximately) 50th percentile of non-crap.

ajuc 5y ago

> Most people probably don't have the talent to play in a professional league regardless of their practice regimen, but when you can get to 95%-ile by fixing mistakes like "not realizing that you should stand on the objective", you don't really need a lot of talent to get to 95%-ile. In the only 2 competitive games I played enough to talk about this - sc2 and dota2 - the mistakes people make at my level (which at my peak was ~2000 mmr in dota and low plat in starcraft - probably top 60-40% of the active players in both) - were nothing like "not understanding game objectives". In starcraft it's easy to say what went wrong because there's much less variables. Usually it's going for a wrong build order blindly, not reacting to enemy unit composition in time, or just being too slow mechanically (usually in the economy part of the game, fights you can just attack-move into the enemy army, but you have to build buildings and units FAST or you don't have enough units to fight). I've been watching sc2 games on top level, I've even played vs people in top 90% in 8v8 free for all games, and there's just no way for me to win against them. I've played one 8v8 game where I was the only one in plat and the rest was in master/grand master. They left me alone for 20 minutes and fought it between themselves. I've build maxed 3/3/3 skytoss fleet with mothership and carriers, full wall of photon cannons and a lot of gates for instant remax. 20 minutes of reasonably fast base-building. After I started to fight I lost in like 5 minutes vs mass marines + medivacs :/ Guy just harassed me to death from 10 different angles while escaping my deathball of carriers. I had 10 times his resources but not the skill to defend from so many points at once. And he wasn't even grand master. Sc2 will show you exactly how much you suck. In dota there's much more chaos, sometimes it's lost in the picking phase, other times someone just feeds for 10 minutes, sometimes people don't understand who is more late-game and farm for 40 minutes vs later-game carry or constantly fight vs earlier-game lineup instead of farming. Other than these I usually don't know why we lose. It's never "I don't understand the game objective". I could probably get much higher if I wasn't spamming winter wyvern as pos4/5 every game. But it's the only hero I enjoy, so whatever.

flerchin 5y ago

The observations about 95% percentile being good or not are distracting from the point that focused, reviewed practice is necessary to level up.

bo1024 5y ago
True, and the point that people are often weirdly bad at deliberately improving, or weirdly unmotivated to practice deliberate improvement.

whoomp12342 5y ago

okay yeah but context matters here. Who cares about being in the 95th percentile of overwatch compared to say being in the 95th percentile of income earned?

tandymodel100 5y ago

rapjr9 5y ago

In machine learning 95% accuracy is not very useful. It means the algorithm will be wrong 1 out of 20 times. So every 20th word in speech recognition will be incorrect, your medical diagnosis will be wrong for every 20th person, your face will not unlock your phone every 20th time, every 20th person will not be allowed to log into a web site. There are some machine learning applications where 95% is good enough or even great such as web search engines, but for many important applications it is nowhere near good enough and you need something more like 99.9% or better to be actually useful. Think about how often the algorithm can be allowed to make a mistake. If it's never, then don't use machine learning. If it's 1 in a million you need 99.9999% accuracy. 95% looks good in machine learning papers in computer science, but in the real world 95% often means dismal failure. Think of an autonomous killer robot, is mistakenly killing 1 out of 20 people ok? For a self driving car is harming the driver or car (or pedestrian/dog/child/other car) mistakenly even 1 out of 1000 (99.9%) times ok? I think not. One measure of what accuracy you need is how well people do the same task. If you can do better than people most of the time, maybe the algorithm is useful, though you also have to compare exactly how the algorithm fails to the way people fail. If people fail by choosing a fender bender and the algorithm fails by killing someone, the algorithm has to work better than people do.

jvanderbot 5y ago
95% accuracy has nothing to do with being in the 95pctile of active participants other than using the number 95. To achieve a 95th pctile ranking among the relevant machine learning frameworks means being more accurate than 95 % of other approaches, and that's probably quite good and useful.
arbirk 5y ago
Rank percentiles and statistical test accuracy / model accuracy percentiles can not be compared. The author of the article seems to be confused about statistics.
deleted 5y ago
[deleted]

whateveracct 5y ago

don't compare yourself to others

HDMI_Cable 5y ago

I think this only applies to some things, not others. If you're in the 95th percentile of high-school students, then you're probably getting into a good university. For university, the 95th percentile is enough to get into medicine, law, or a good business school. The 95th percentile for overall intelligence is 125. The 95th percentile for household income is $250 000. For things like video games, which have a broad net of casual players and a few who are very competitive, it isn't that good. All you need to do to get in the 95th percentile is to be competitive; but for applications where everyone is acting somewhat competitively, or for non-zero sum competitions like school or income, the 95th percentile can be very good.

D13Fd 5y ago

I think this is really "it's not hard to get into the 95th percentile of things that you are already fairly good at." Yes, if you have a history of playing video games, it's not hard to pick up another game and do well--and, with some focus, do very well. But what if he tried to pick up playing the flute? How about drawing comics? Competitive swimming? Investing? What about being in the 95th percentile of a complex field like medicine or law? Assuming he doesn't already have a background in those things, he's probably going to have a hard time reaching the "95th percentile" of skill.

raydev 5y ago
> But what if he tried to pick up playing the flute? How about drawing comics? Competitive swimming? Investing? Having seen exactly 2 of your examples in real life, particularly people who had no experience and no interest in painting or playing an instrument until their 40s, a few years of persistence and targeted practice made them better than me after a lifetime of “casual play”. I might have to put in less time to “get better than them” if I really sat down and tried for a year, but I’m not sure that matters in the way you’re thinking

mi100hael 5y ago

Golf is a real-world activity that can be used for comparison here. It's a good candidate because there is a standard handicap rating system issued by a central governing body, and there are a large number of amateur players who play regularly, compared to other sports. - Roughly 10% of golfers in the US maintain a handicap rating. - "Today, the 2,417,905 who have a handicap included 2,051,675 “active posters,” meaning they posted at least one score in 2020. However, the average number of posted scores was 38, almost double the average number of rounds played by golfers overall last year—so it’s pretty clear that those who get a handicap are among the game’s most engaged participants." [0] - "it has been found that on a given day the average golfer would be expected to post a score of approximately 100 strokes when following all the rules of golf" [1] which is 28 strokes over par. A 28 handicap is around the 5%-ile of golfers with a handicap [2] - As such, those 10% of golfers with a handicap should overlap pretty heavily with the top 10-20% of all golfers. The 50%-ile of golfers with a handicap is 13 [2], so by extrapolation that should be 90-95%-ile of all golfers. - 13 is still a solid handicap. That's someone who is breaking 90 (better than bogey golf) pretty much every round and likely has years of practice. Saying it "isn't very impressive because it's not that hard to do" would be an incorrect statement in my opinion. - (as a sanity check, the 90%-ile of golfers with a handicap is a handicap of 5. Most D1 golfers will have a handicap right around 0, and D3 will be maybe 2. Given the number of casual golfers, it seems likely that only about 1 or 2 out of 100 are nearing that level.) [0] https://www.linksmagazine.com/how-do-you-match-up-against-the-average-golf-handicap-in-2020/ https://www.linksmagazine.com/how-do-you-match-up-against-th... [1] https://golftips.golfweek.usatoday.com/average-golf-handicap-america-2455.html https://golftips.golfweek.usatoday.com/average-golf-handicap... [2] https://www.usga.org/content/usga/home-page/handicapping/handicapping-stats.html https://www.usga.org/content/usga/home-page/handicapping/han...

ghaff 5y ago
It also shows that the population you're comparing yourself to, i.e. the denominator, matters a lot. If you're comparing yourself mostly to people who do some activity pretty regularly, top 5-10% means you're pretty good compared to the average person on the street who may never have touched a golf club. But if you draw your population from anyone who has ever taken a swing at a golf ball, 5-10% is a lot less impressive.

westcort 5y ago

The biggest takeaway I got from this was that recording yourself doing something, then observing yourself completing it as an outsider can dramatically increase your performance. Relevant content: After watching a recording of myself writing code, I realized I was spending about a quarter of the total time implementing the feature tracking down which functions the bugs were in! This was completely non-obvious to me and I wouldn’t have found it out without recording myself Now that I’m aware that I spent so much time isolating which function a bugs are in, I now test each function as I write it to make sure they work This allows me to write code a lot faster as it dramatically reduces the amount of time it takes to debug my code