26 ms·

The Lesson to Unlearn

• xrd 7y ago
  A corollary: software engineering interviews are tests. When you are young you can memorize more things. When you are older you've experienced many different ways of building software. But, interviews, most of them, generally just ask you to regurgitate concise facts, just like PG points out tests do. And, our brains are tuned to pick out mistakes in those facts and make decisions based on that. I'm not saying hiring for engineers is biased against older engineers, but it makes me think after reading this essay.
• Wowfunhappy 7y ago
  > If you merely read good books on medieval history, most of the stuff you learned wouldn't be on the test. It's not good books you want to read, but the lecture notes and assigned reading in this class. It is the professor's job to decide which parts of medieval history are the most important to learn. Therefore, those are what I need to study. Maybe not all professors do this effectively—but as long as we're talking about ideals, this is how it should work.
  • thepete2 7y ago
    > And even most of that you can ignore, because you only have to worry about the sort of thing that could turn up as a test question.
    • Wowfunhappy 7y ago
      That part just did not describe my experience in college. Maybe I was bad at knowing what "could turn up as a test question." My college also did not have any old tests floating around, at least as far as I was aware.
    • fhennig 7y ago
      I think we should just drop book work questions altogether and allow students to take notes into the exam. Just make the tests about analysis and synthesis, which is the stuff you should be able to do if you truly understood something and didn't just memorize some facts.
    • watwut 7y ago
      In classes I have seen, things you could ignore were minority of it and usually side stuff like funny anecdotes, minor players that are there for you to get feel for the context, famous nun dick tree and such. This stuff helps you learn important stuff and makes it all less dry, but it absolutely does not make sense to penalize students for not learning it actively.
  • watwut 7y ago
    Yeah, this came across to me as overly simplistic not really thought out argument too. I dont think that college medieval history not being the "students can pick any part of it and anything you learn about medieval history is good to go" is some kind of massive issue with medieval history course. It does not mean that you did not learned or that grade does not measure learning. It just means the topic is not whole medieval history (which would be ridiculously broad) but only selected parts of it. It is kind of like complaining that algorithms 101 are allowing students to pick any algorithms to learn and require everyone to learn the same set of algorithms.
  • bonoboTP 7y ago
    Right, I had a similar reaction. Autodidacts often end up reading a bunch of stuff from here and there with sub-par results. One of the main benefits of a (university) course is focus and guidance. That the prof or teacher selects specific topics they deem important for that stage of education. As a student it can be difficult to judge that yourself.
• 1k 7y ago
  Studying, getting hired, career promotions, raising funds... It’s all about beating the system. You’re evaluated based on certain criteria and you optimize for those.
• jsherwani 7y ago
  We (Screenhero, W13) may well have been one of the many YC companies PG’s referring to! But instead of hoping there was some test to hack, we were worried that there was some unknown hack out there that we didn’t know. It was quite a huge relief when PG told us something to the effect of “all the successful startups have found their initial grown through just one thing: delighting their users with a great product”. That realization / confirmation freed us four engineering cofounders from worrying about some “growth hack” we never learnt, and helped us focus on prioritizing our product development through talking to users. We then used Sean Ellis’ “very disappointed” survey methodology (which Rahul @ Superhman recently wrote about) and used it as our North Star to find the few features to focus on polishing. We got acquired by Slack in 2015, and built out Slack Calls. I left last year, and I’m now in the final stages of launching a new product around super-charged meetings for remote teams (picking up where Screenhero / Slack Calls / Zoom left off), and am using the same principles again. No hacking of tests, just building a product that people want, and using their feedback as the only valuable bit of data in determining priorities. Thanks PG and YC (and Sean Ellis!) for startup principles and methodologies that have stood the test of time.
• k__ 7y ago
  I never cared about grades, but I was never very good or very bad. Just slightly above average In academia, having good grades is sometimes helpful to climb the ladder. Not all people can study medicine. Not all people can get a master degree. Often these gigs are given to the best students only, but sometimes you can circumvent the grading issue by waiting longer, going to the military, be friends with a prof. or working for free.
  • robocat 7y ago
    > Not all people can study medicine I think you are falling for the fallacy that the essay is trying to expose? Surely those that pass medicine are those that are good at medicine exams. Plenty of people would make great doctors that don’t due to the exam system, not due to a lack of capability.
• shrubble 7y ago
  This makes me wonder if pg has ever read any of the writings of John Taylor Gatto? Various quotes: https://www.goodreads.com/author/quotes/41319.John_Taylor_Gatto https://www.goodreads.com/author/quotes/41319.John_Taylor_Ga...
  • avindroth 7y ago
    I feel like he definitely has, he seems like he reads a lot and Gatto is not unpopular for people with thoughts on school
• esotericn 7y ago
  Not sure I agree with this. The real world is very much about optimizing for and beating tests. It might be producing a specific CV or preparing for an interview whiteboard session to land a job. It might be socializing and networking in a specific way in order to land funding. The real world does not commonly reward just being really good at arbitrary things. It's almost always focused on meeting a need that someone else has defined, much like a test.
  • buboard 7y ago
    There is truth to this. It reflects the shift of mentality in tech entrepreneurs since the 2000s (experimentally entrepreneurial, introverted hackers, libertarian) to the 2010s (work for FANG or make a feature that FANG wants to buy, well-curated github and social media, social justice). I could dare say that this mentality flowed from academia to the greater economy. Academic funding has been generally formalistic and test-driven for decades.
    • fhennig 7y ago
      So in the greater economy, how was funding allocated before that?
      • deleted 7y ago
        [deleted]
    • BlueTemplar 7y ago
      How many decades? I've heard that things were very different in the 60's/70's...
  • chadcmulligan 7y ago
    > The real world It's not the real world - its the human world. The real world is very different, and getting more so imho. Skills to survive the human world are usually useless in the real physical world. It's a distinction I read recently from a philosopher that I lost track of, maybe someone here knows. Her point was (I think) that when someone says the real world, with relation to education, it's not really - it's the world of human systems that are very much open to degradation and corruption, and they can be changed.
• irjustin 7y ago
  So while I agree with what he is saying directly, to me, testing is a function of filtering at large scale. To a secondary degree, measuring as feedback for that system. Testing and grades in general are a society level filtering mechanism and is a measuring proxy for how the work force filters for candidates at the entry level. As most know, once you've got about 10 years, most of that is largely ignored at the bottom of your resume. Entrepreneurs, in the eyes of this system, are deviants of sorts. Hacks who couldn't/didn't want to make it in the system, but turns out that system is not the only way. A non-western example of this are a lot of the families that came up in money in China the past few decades. They didn't make it in the rigor of Chinese school system, but when they said fuck-it, I'll survive by starting something, they were wildly successful. Controversially there's evidence pointing to education doesn't help people at scale[1]. So that whole structure, debatably, has diminishing returns. So, I agree with Paul Graham 100%, schooling isn't the only way, and I mean that by - if my children decide they do not want to attend school anymore because they have a plan that, while unconventional, can do great things; Support all the way.
• buboard 7y ago
  I can't find whose quote it is that "the most difficult thing is to teach people to think simply". Unfortunately education cannot test that, because tests are by definition testing only a fragment of a system. Knowing and understanding one thing end-to-end is meaningful knowledge, and qualitatively better than knowing fragments of 100 things. Perhaps you can assess the former by grading a diploma thesis on a very specific subject. Tests may have an important role in motivating students to delve more deeply in subjects but they don't go beyond that. There's similarly more value to knowing some technology "end-to-end" rather than specializing in only one level and never understanding what's below. It's also interesting to read this: > There are now ways to get rich by doing good work, and that's part of the reason people are so much more excited about getting rich than they used to be. Which may reflect how things were 15 years ago. There were 2 threads in "Ask HN" last week which basically concluded that these days you're much better off (money wise) working for an uncreative role in FANG.
  • avindroth 7y ago
    https://www.wikiwand.com/en/Gordian_Knot https://www.wikiwand.com/en/Gordian_Knot
    • buboard 7y ago
      the gordian knot is more of a brutish non-solution to a hard problem. "thinking simply" is more about developing intuitive understanding to the point where you can also explain it simply
  • fhennig 7y ago
    > Tests may have an important role in motivating students to delve more deeply in subjects but they don't go beyond that. I feel like the university doesn't allow for time to stroll from the beaten path and explore topics that you're interested in. Most people I talked to have a very stressful studying experience. You know people care about your grade in the end, and you pay a lot of money for your degree. So it makes sense to also spent the majority of your time to learn the course material in and out.
• jakobmi 7y ago
  It's exactly the same with hiring at FANG/McKinsey/BCG/... :). You optimize for the (well known) interview process. You study 10 hours/day for 2 weeks. And you get the job with 99% accuracy.
  • avindroth 7y ago
    The artificiality (?) is what gets to me tbh
• hertavein 7y ago
  https://todaypkjobs.com/ https://todaypkjobs.com/
• crocal 7y ago
  I think the elephant in the room is the question of purpose. What is the purpose guiding your actions? Getting good grades is a mean that /can/ be necessary to achieve the purpose you have chosen in life. But it’s not sufficient for most purposes. On the other hand, once you have a purpose, you truly make the most of education.
• gcc_programmer 7y ago
  TL; DR; Einstein said sth along the lines of "Don't let your education get in the way of your knowledge." Finish uni with high grades then spend the rest of your life learning.
  • username90 7y ago
    > Finish uni with high grades then spend the rest of your life learning. I'd rather not waste 30% of my life on that nonsense. Better to use uni for what it is meant for: Learn whatever seems interesting and don't mind the grades.
• manmal 7y ago
  My wife and I are contemplating unschooling our kids. The longer our son is in primary school, the more we see the deterioration of his willingness to learn anything school-related. He hates homework (as do most kids), and this is even more painful to see given that it has been shown that homework is essentially useless for learning performance. He still likes to write and do math, but as long as he is in school, we have to take care that his interest doesn’t go south. During school holidays, he actually starts to do school exercises for fun; those are the very same exercises he would fight against doing for hours on a regular school day. Go figure. Btw, Gates, Page, Brin - all were not in a conventional school. The capacity to just build things without fear of being judged is invaluable.
  • jraby3 7y ago
    We thought the same way but it’s difficult to replace the social component. Still it’s hard to see how bad the school system is and force our kids to participate. It could be so much better.
    • c64112af69ea8a3 7y ago
      Look for an unschooling school, strange as that may sound. Free democratic and Sudbury Valley model are some of the things to search for. There aren't many around but the numbers are increasing. We send our son to one. It's a leap of faith; the kids essentially "do whatever". But it seems worth it compared to the damage done by middle and high school. And the kids that I've seen come out of this school seem to do well in society (and college) and generally seem much more mature than high school graduates.
      • jblwps 7y ago
        Montessori schools as well: "Follow the child"
        • 80386 7y ago
          Be careful with Montessori. I went to a "traditional Montessori" school for a few months when I was very young. Long story short, their educational method insisted that we needed instruction in how to play with toys before we were allowed to do so, and we needed to do so on mats, which needed to be rolled up tidily at the end of the prescribed playtime. The problem with that was that we already knew what Legos were. But we were supposed to pretend we didn't, in order to appease the teacher. I think this was the same school that insisted to my parents that I couldn't read because they (the school) hadn't taught me to read. My parents taught me to read. So they had me demonstrate, and the teacher decided my parents must have... trained me to pretend to read.
• astatine 7y ago
  I don't think students are the blameless victim of this "system". I know of a continuous evaluation system adopted in a college, where the final tests had a less than 50% weightage for the grade. Thr students rebelled. It was far easier for a majority to burn the midnight oil for a few weeks than be diligent through the course. The test model has stuck because it's convenient for everyone, regardless of its clear weaknesses.
  • matwood 7y ago
    I'm one of those weird outliers I guess, but even when there were only 1-2 exams for the whole course I stayed diligent throughout. The night before a test I just did a bit of review, and got a good nights sleep. I never understood those who tried to cram because it's not something that would ever work for how I learn and understand.
• mtreis86 7y ago
  I wonder if there will be any changes made to the YC application process now that Paul is thinking about these things.
• fhennig 7y ago
  This is a very interesting thing to think about, I thought about it a bunch of times already, and have a couple of thoughts about it too. First of all, once you're 55, it's easy to say stuff like that, because you won't be tested anymore. And I think there is a good chunk of survivorship bias: "Hey, I made it without worrying about tests, so you can, too!" Although that is probably not true for a big chunk of the population, especially once we look at non CS people. Then I think, he is fundamentally right. Grades shouldn't matter so much, it is about what you learn. And I like to approach things that way too, but in the end, I always have to study for the tests as well. The problem is exactly how deeply it is ingrained in everything. If I just stop caring, it doesn't really help. I will just end up in a worse position, since everyone around me still cares about test results. The people that need to change their mind are the people that use tests as measures of qualities that they are not a good measure for. I think a lot of people are falling thought the cracks because they don't fit the expectation of a HR person close enough. In the end, we will always have to have tests. A completely individualized assessment of peoples qualities or fit for certain roles just doesn't scale. It would be great though to come up with new ways of testing that are maybe more in line with actual learning. For example, it would be nice to have tests at university where I can google things, just like in the real world. I am lucky enough to be able to work in the booming tech sector where there are so many jobs that it is fine if I don't work towards a test.
  • volume 7y ago
    > The problem is exactly how deeply it is ingrained in everything You can you use "test" as a proxy for the impression you want to make on your boss and peers. Maybe even friends and family. It's up to you to redefine that test (or create a complementary one) based on your own goals and your own timeline.
  • vntx 7y ago
    > For example, it would be nice to have tests at university where I can google things, just like in the real world. One of the best professors I had at university for a Linux course always said: “If you don’t know something, google it!” He backed up his words by allowing googling on the actual coding part of his exams. You had to understand the underlying concepts but you never had to memorize syntax. One time, he had us write a small program in C that involved threading during a proctored timed exam. Everyone ran out of time for that part. His course and exams were _rigorous_. You could never cram for his class and expect to pass and indeed I know of one student repeatedly failing his course. This was one of the courses in university where I gained the bulk of my useful knowledge and practical skills I use to this day. I can count the courses that were of similar quality I took at university on one(1) hand. The rest of my degree involved hackable tests, apathetic professors and were massive time-wasters. I learned nothing from them, obviously. The good professors made me realize how bad the rest of my degree was and how much of a racket higher education could be. I look back at my undergrad days and wonder how much more I would have learned if I had not focused so much on grades. I’m still actively trying to unschool myself.
• bavcyc 7y ago
  Interesting essay, however it is written from a perspective of having basic needs met and having the resources to spend time 'hacking' a system. When you are struggling to pay the bills for the basics, you don't focus on the hacking just on the surviving. The question I have is how do you hack the system such that you help those who are struggling to survive?
• AlchemistCamp 7y ago
  This is great seeing so many PG essays! I've really missed his writing. > And at elite universities, that means nearly everyone, since someone who didn't care about getting good grades probably wouldn't be there in the first place. The result is that students compete to maximize the difference between learning and getting good grades. > When I started advising startup founders at Y Combinator, especially young ones, I was puzzled by the way they always seemed to make things overcomplicated. I suspect part of the problem (of early founders being focused on hacking tests) was related to how heavily YC funded people from elite universities—which select people who do exactly that.
  • anonytrary 7y ago
    > This is great seeing so many PG essays! I've really missed his writing. I love reading his tweets! They're all so down to earth and illuminating. That said, I'm not a fan of his blog writing as much. I find it full of overly-sparse nuggets of wisdom. It needs to be more concise, to the point, with fewer clever metaphors. This article could've been compressed 70% and still have been human readable.
    • pyb 7y ago
      pg seems to have a backlog of essay ideas that he's catching up on. This is all good, but also means that they get written a bit too quickly. It's the classic "had I had more time to spend on this text, it would have been shorter" syndrome.
      • anonytrary 7y ago
        Indeed, time is of the essence; playing tug-of-war with quality.
    • kashyapc 7y ago
      I've read the article in full. It raises an important point, but it's indeed a laborious read. As I read it, the haunting paragraph from The Elements of Style came to mind: "Omit needless words. Vigorous writing is concise. A sentence should contain no unnecessary words, a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts. This requires not that the writer make all sentences short, or avoid all detail and treat subjects only in outline, but that every word tell." - - - If you enjoy the essay form, I highly recommend the "Essays of E.B. White" (yes, the same White of The Elements of Style). Some of the essays stay with you for a long time. https://www.harpercollins.com/9780060932237/essays-of-e-b-white/ https://www.harpercollins.com/9780060932237/essays-of-e-b-wh...
• empath75 7y ago
  And yet I’ve interviewed with 2 or 3 different y combinator companies that hire based on coding tests that test nothing other than that you’ve memorized “cracking the coding interview”.
• anonytrary 7y ago
  > No one was pulling all-nighters two weeks into the semester. As a physics student, I can say we definitely pulled 2am-ers 2 weeks into the semester. Our Condensed Matter professor gave us some of the most random questions that no one could anticipate. To be honest, I was one of those "diligent" students who got good grades. I'll tell you right now his quizzes were not hackable. The following held true: |quiz subject matter| >> |lecture matter| He tested on the former, so we had no way of memorizing or hacking the quiz. You had to really understand the concepts to do well on his quizzes. Ditto for my grad Quantum Mechanics class. On the other hand, I've also had professors who give exams pulled from the internet and allow use of the internet while taking those exams. I hacked that immediately during the exam, then almost got in trouble for it. > ...if the professor tells you that there were three underlying causes of the Schism of 13... you'd better know them. A professor like my Condensed Matter Physics professor would go on and on about 3 fundamental causes and then quiz students about the 5 fundamental causes, giving a zero to anyone who couldn't think of all the base 3, then grading the remaining students who thought creatively of 2 additional causes against each other on a bell curve. :)
  • majos 7y ago
    > A professor like my Condensed Matter Physics professor would go on and on about 3 fundamental causes and then quiz students about the 5 fundamental causes, giving a zero to anyone who couldn't think of all the base 3, then grading the remaining students who thought creatively of 2 additional causes against each other on a bell curve. :) To me, this sounds like a plausible argument against such tests. The issue of creatively grading responses to an ill-defined question often pops up in discussion here about interview practices. In those discussions, typically someone will say “I don’t do a generic whiteboard interview. Instead I do [idiosyncratic thing x]. It really gives me amazing insight into the candidate.” Then someone else says “Yeah right, a weird test with unclear metrics just gives you a big empty space to fill in with all your biases and pick someone who answers the way you would”. Of course, both sides are exaggerated here. But it’s not clear to me that “creative” tests are necessarily any better.
• mattbee 7y ago
  This spends too many words justifying the stale "real hacker" tradition of dunking on education, and kindof ignores the problem of how to measure something so subjective as learning. So people who are good at tests get ahead. We get that you're not interested in grades. We all know people who play the game and win, not sincerely engage with their job or education or whatever. But I'd like to hear from pg / other people in YC how they think they are "hacked". What do their successful applicants actually optimise for when "delight" and even "growth" are just as subjective as "learning"? After all they fund lots and lots of companies and founders, many of whom make no returns. How do founders keep their funding "success" long past the point it was deserved?
  • dkyc 7y ago
    "Growth" in the YC definition ("revenue growth with positive unit economics", or the closest proxy to that) seems a pretty objective test to me.
    • sutterbomb 7y ago
      They’re often selecting applicants before growth is a factor, no? Surely there are hacks to this process
    • killerstorm 7y ago
      It can take a lot of resources to get to that point, no? So there's still a question what companies should do until they reach this growth possibility.
• njacobs5074 7y ago
  That is a great essay. Thank you for sharing. I've worked in tech for quite a while now. I used to buy into the meritocracy cant but as the years wore on, I realized how much bullshit it was. I think the biggest moment came when I started to think about how to explain to people in my new home, South Africa, that working in tech would free you economically. There's some truth to that. But that's not the whole truth. The truth is that all those years ago when I got my first coding job in financial services, I ticked 3 important boxes for the interview: white, male, and university-educated.
  • i_am_new_here 7y ago
    You are derailing the discussion and I hope you get downvoted into oblivion Besides that: The (classical) financial industry (banks and insurances) has a relatively trival IT (of copying data around and displaying it). Combined with external pressure, they are -today- leading on progressive indicators, such as diversity, inclusion, near-shoring and far-shoring.
• mekazu 7y ago
  I wonder why pg didn’t launch this article on a Tuesday.
• anonytrary 7y ago
  > The real problem is grades Bingo! The real problem is the using incorrect metrics. If you judge a fish by its ability to climb a tree, it will spend its entire life thinking it is stupid!
• deleted 7y ago
  [deleted]
• jpm_sd 7y ago
  > At this point I've told the founders something you'd think would be completely obvious: that they should make a good company by making a good product. Sure, ok. How do you make a good product? That definitely is not "completely obvious"
  • BlueTemplar 7y ago
    At least one user is going to love it.
• daly 7y ago
  I took a class from John White at UCONN. He gave us a couple dozen CS papers at the start of the year. Every class had 3 paper presentations, 20 minutes each. A random student was picked for each presentation. So you had to read and understand each of the upcoming 3 papers because there was a random chance you would have to present one of the three. The result is that you eventually read all of the papers well enough that you could give a 20 minute presentation. There were no grades. It was pass/fail. It was one of the best classes I ever took.
  • bonoboTP 7y ago
    But the person who presented the very first paper won't have to care about the rest of the semester right? Or can you get randomly picked multiple times?
    • daly 7y ago
      There were only 10 students allowed in the class. So you got picked about every 3rd class (sometimes more than once per class).
      • jventura 7y ago
        I wish I could do that to my students, but I have an average of 120 students on my classes each semester.. :/
  • jimbob45 7y ago
    I had a high school professor do a somewhat abridged version of that for our final. It was impossible to spend an adequate amount of time on all three so you really had to focus on two and hope you got lucky. Easily the most stressful final I've ever taken.
• ynniv 7y ago
  Artificial "hackable" tests are opportunities to see progress at scale. While I appreciate the recognition that bespoke, perhaps Montessori, education might better prepare people for the open ended ness of life, there aren't enough unicorn startup opportunities in the world to use that as a primary means of knowing how well you know things. Evaluation through essays is better than multiple choice, the Socratic method is better than essays, but we don't know how to scale those things yet. Perhaps some day we will have a Young Ladies Illustrated Primer, but until then we have hackable tests.
• daly 7y ago
  I got into "computer science" before there was a CS degree. Our CS classes were taught in the math dept (fortran), business dept (cobol), or engineering (assembler). The profs were literally 1 chapter ahead of the students. I, however, was in love with the subject. I was the "student advisor" in our "machine room". We had 5 teletypes connected to a remote mainframe at Rutgers. I lived and breathed CS. I wrote tests for the profs (even thought I was also in the class) and I answered questions for students about the tests when they came for computer help. Pick a subject you love so deeply that you're always at the leading edge (which essentially means a new subject the school wants to teach). For example, CMU introduced an AI curiculum. Learn how to do NNs, DNNs, GANs, etc. Read the latest papers. Write working code. Chat with the profs. Hang around the dept. Proof systems are a hot topic. Learn LEAN, COQ, AGDA. Learn to write proofs by machine. Read the papers. Get ahead of the curve. It is hard to "fail" a student who knows a lot more than you, especially when they help craft the tests. Intel stuck an FPGA inside its CPU. Learn Verilog and learn to hack new CPU instructions (e.g. read Gustufson's book "The End of Error" that introduces a new kind of floating point arithmetic). Write code that outperforms BLAS code. Learn BPF (Berkeley Packet Filter) so you can do impossible things about the kernel in User mode. Make your networking class look like it is stone-age. I could go on but you either "get it" or you don't.
  • daly 7y ago
    BPF link: http://www.brendangregg.com/blog/2019-12-02/bpf-a-new-type-of-software.html http://www.brendangregg.com/blog/2019-12-02/bpf-a-new-type-o...
  • deleted 7y ago
    [deleted]
  • kaymanb 7y ago
    I think I fall into the "don't get it" category. Could you explain what I should be taking away from this? I understand that being at the leading edge has benefits, but it's not feasible to understand the state-of-the-art across an entire CS curriculum.
    • graycat 7y ago
• dennis_jeeves 7y ago
  It gets worse: In my view schools/universities are institutions for mass indoctrination. Noam Chomsky has said something along similar lines. I borrowed the words from him. A bunch of my other observations: https://realminority.wordpress.com/observations-of-the-world-2/ https://realminority.wordpress.com/observations-of-the-world...
• wrinklytidbits 7y ago
  I'm a non degree student at an ivy league and I took a course this semester because I wanted to learn the basic fundamentals of electrical engineering. Because I'm learning because I want to, I put in effort into things I find interesting and difficult without the pressure of deadlines and grades. I love grades because it gives me the feedback I need (otherwise I would "learn" from youtube videos and pdfs) to assess how well I know the subject. I can take breaks from truly hard problems until I have enough clarity of mind to approach the problem with a new perspective. I can't do that with a deadline. I don't care if the answers "are out there" meaning the professor has returned graded tests and posted the answers; I do the work because I want to learn. That paradox of students wanting to review graded homeworks and students who want to learn by doing the homework is resolved practically by those that are more vocal about their wants. I'm happy to take my time, doing my best (with the understanding that my best takes time) and observing my grade afterwards. Sometimes my learning takes me down paths that harder and more rewarding with the consequence that I take longer than others: I am a slow learner.
• brador 7y ago
  Good grades = Passing well paying jobs resume filter = startup initial capital for 90% of the population without access to the bank of mom and dad or a large inheritance. PG starting to lose touch with regular joe.
  • 0x445442 7y ago
    Yeah, I had a similar take. A lot of what he writes seems to come back to keeping the YC pipeline full. Half way through the essay I was thinking, surely this can't be the first time PG has realized the idea of "playing the game", which is what I've known it as all my life. But then at the end of the essay he uses that exact same phrase to hit home the point that "playing the game" is antiquated now if one desires to become rich. While I'd agree there are more opportunities for individuals or small groups to get rich now than in 1960, for the vast majority of us, the best bet to become rich is to "play the game".
    • dennis_jeeves 7y ago
      Had to chime in with a nod of agreement to both brador and 0x445442 . I'm not being normally someone to comment on such things ( why bother with the effort?). I am making concrete efforts to get noticed by identifying groups/people who might align with my own thoughts/beliefs for my own vested interests.
• ph2082 7y ago
  And Lesson to Learn is when real life problems come, people will ask can you solve the problem not what grade you got.
• spion 7y ago
  To use a fun analogy, the teachers write the unit tests and you pass them by writing a mock or fake implementation of knowledge. edit: and now I got to a point in the article where that's pretty much already said. Well, too late to delete this comment now.
• smiley1437 7y ago
  I love Paul Graham's articles, but isn't this Goodhart's Law but with more words? (When a measure becomes a target, it ceases to be a good measure)
  • acidburnNSA 7y ago
    Searched the page for Goodhart and found this. There's added value here as this is a discussion of why Goodhart's Law is bad and how it's so prevalent in our education and business systems. I knew about the law but still got a lot out of this essay.
    • dccoolgai 7y ago
      I did a Ctrl+f "goodhart", too. Glad to see that there is growing awareness of this principle... I think it underlies a lot of basic problems with society/tech/government in general. There is a related real-life parable I like about the ww2 plane designer who put more armor where returning bombers _weren't_ shot up, but I forget his name.
• bkohlmann 7y ago
  An important corollary is that school and tests teach you not to be wrong. They teach you that incorrect answers will be punished. Yet, most interesting questions in life don’t yet have defined answers. Thus you need to have and test a hypothesis, which will very often be “incorrect” the first time around. But that doesn’t actually matter - the mere act of thinking about and defining what an answer could be sets you up to iterate and test. Eventually, you may find product market fit (a - not the - right answer). You have to be willing to be wrong at first to learn...and win.
  • robocat 7y ago
    Even worse, most tests have a well defined question and a correct answer. In real life you don’t know the question, there isn’t a single canonical answer, and there is no score-keeper to tell you if you are passing.
• rsp1984 7y ago
  I completely agree with the the theory, however what 90% or more of us do when they finish university is to go work for an established (i.e. large) company, where winning means hacking bad tests (promotion cycles). So in some perverse sense school does prepare most of us well for working life, as most of the time doing well in BigCo isn't to actually perform well or to actually make a difference, it's to leave the impression of it at the right time with the people that matter for your next promotion.
  • throwawaytemp1 7y ago
    This isn't fair to tests. They are actually far more fair and objective than BigCo promo processes. You can't fake doing a variety of hard math questions the way you can move up the ladder due things like nepotism.
• throwawwy 7y ago
  Several schools of economic thought claims "Money" also operates as a test for "value for society". It's also as porous as a colander. > I suspect many people implicitly assume that doing working in a field with bad tests is the price of making lots of money. > Now you can make lots of money by making cool things. An analogous set of incentive structures, which compels people in elite universities to play the tests applies to intra & inter-company interactions as well: companies who play for the tests will enjoy structural advantages over companies which do not; and this lever allows winner-takes-most mechanisms via buyouts, consolidations, private equity. While founders, individually, may make out of this wealthy, companies, and society as a whole suffers, and playing for the tests, as a behavior, saturates.
• Jgrubb 7y ago
  There's a passage in Zen and the Art of Motorcycle Maintenance about exactly this. --- Phaedrus' argument for the abolition of the degree and grading system produced a nonplussed or negative reaction in all but a few students at first, since it seemed, on first judgment, to destroy the whole University system. One student laid it wide open when she said with complete candor, "Of course you can't eliminate the degree and grading system. After all, that's what we're here for." She spoke the complete truth. The idea that the majority of students attend a university for an education independent of the degree and grades is a little hypocrisy everyone is happier not to expose. Occasionally some students do arrive for an education but rote and the mechanical nature of the institution soon converts them to a less idealistic attitude. The demonstrator was an argument that elimination of grades and degrees would destroy this hypocrisy. Rather than deal with generalities it dealt with the specific career of an imaginary student who more or less typified what was found in the classroom, a student completely conditioned to work for a grade rather than the knowledge the grade was supposed to represent. Such a student, the demonstrator hypothesized, would go to his first class, get his assignment and probably do it out of habit. He might go to his second and third as well. But eventually the novelty of the course would wear off and, because his academic life was not his only life, the pressure of other obligations or desires would create circumstances in where he just would not be able to get an assignment in. Since there was no degree or grading system he would incur no penalty for this. Subsequent lectures which presumed he'd completed the assignment might be a little more difficult to understand, however, and this difficulty, in turn, might weaken his interest to a point where the next assignment, which he would find quite hard, would also be dropped. Again no penalty. In time his weaker and weaker understanding of what the lectures were about would make it more and more difficult for him to pay attention in class. Eventually he would see that he wasn't learning much; and facing the continual pressure of outside obligations, he would stop studying, feel guilty about this and stop attending class. Again, no penalty would be attached. But what had happened? The student, with no hard feelings on anybody's part, would have flunked himself out. Good! This is what should have happened. He wasn't there for a real education in the first place and he had no real business there at all. A large amount of money and effort had been saved and there would be no stigma of failure and ruin to haunt him the rest of his life. No bridges had been burned. The student's biggest problem was a slave mentality which had been built into him by years of carrot-and-whip grading, a mule mentality which said, "If you won't whip me, I won't work." He didn't get whipped. He didn't work. And the cart of civilization, which he supposedly was being trained to pull, was just going to have to creak along a little slower without him. This is a tragedy, however, only if you presume that the cart of civilization, "the system," is pulled by mules. This is a common, vocational, "location" point of view, but it's not the Church attitude. The Church attitude is that civilization, or "the system" or "society" or whatever you want to call it, is best served not by mules but by free men. The purpose of abolishing grades and degrees is not to punish mules or get rid of them but to provide an environment in which that mule can turn into a free man. The hypothetical student, still a mule, would drift around for a while. He would get another kind of education quite as valuable as the one he'd abandoned, in what used to be called the "school of hard knocks." Instead of wasting money and time as a high-status mule, he would now have to get a job as a low-status mule, maybe as a mechanic. Actually his real status would go up. He would be making a contribution for a change. Maybe that's what he would do for the rest of his life. Maybe he'd found his level. But don't count on it. In time - six months; five years, perhaps - a change could easily begin to take place. He would become less and less satisfied with a kind of dumb, day-to-day shop-work. His creative intelligence, stifled by too much theory and too many grades in college, would now become reawakened by the boredom of the shop. Thousands of hours of frustrating mechanical problems would have made him more interested in machine design. He would like to design machinery himself. He'd think he could do a better job. He would try modifying a few engines, meet with success, look for more success, but feel blocked because he didn't have the theoretical information. He would discover that when before he felt stupid because of his lack of interest in theoretical information, he'd now find a brand of theoretical information which he'd have a lot of respect for, namely, mechanical engineering. So he would come back to our degreeless and gradeless school, but with a difference. He'd no longer be a grade-motivated person. He'd be a knowledge motivated person. He would need no external pushing to learn. His push would come from inside. He'd be a free man. He wouldn't need a lot of discipline to shape him up. In fact, if the instructors assigned him were slacking on the job he would be likely to shape them up by asking rude questions. He'd be there to learn something, would be paying to learn something and they'd better come up with it. Motivation of this sort, once it catches hold, is a ferocious force, and in the gradeless, degreeless institution where our student would find himself, he wouldn't stop with rote engineering information. Physics and mathematics would come within his sphere of interest because he'd see he needed them. Metallurgy and electrical engineering would come up for attention. And, in the process of intellectual maturing that these abstract studies gave him, he would be likely to branch out into other theoretical areas that weren't directly related to machines but had become part of a larger goal. This larger goal wouldn't be the imitation of an education in Universities today, glossed over and concealed by grades and degrees that gave the appearance of something happening when, in fact, almost nothing is going on. It would be the real thing.
• ZeljkoS 7y ago
  Yes, most tests are bad, but there is a quick advice that improves them immensely: never ask questions that are not at least Level 3 in Bloom's taxonomy of knowledge: https://blog.testdome.com/blooms-taxonomy/ https://blog.testdome.com/blooms-taxonomy/ I cofounded a technical screening startup, and despite our efforts to educate our customers that enter their questions on our platform, 95% of their custom questions are bad questions. They tend to ask trivia questions, that can quickly be googled, instead of asking work-sample questions (which we at TestDome.com prefer). I think they learned that from years of paper tests in school, it is very hard to unlearn. Just to give you an idea, for testing web programmers we suggest questions where you need to find bugs in HTML code (https://www.testdome.com/questions/html-css/inspector/17629 https://www.testdome.com/questions/html-css/inspector/17629), while our customers would ask questions like "What does CSS stand for?"
  • appleflaxen 7y ago
    can anybody comment on the difference between "understanding" and "applying" in that taxonomy. The examples provided don't seem very good to me. is it the difference between understanding a language and not speaking it, perhaps? following a conversation, but not being able to participate in it?
    • abdullahkhalids 7y ago
      I have learned Bloom's taxonomy with comprehension instead of understanding. So if in an algorithm's class, you were taught that algorithm X was exponential time, then the exam question at the comprehension level can be, "describe why does it take exponential time to run algorithm X". An applying level question could be, "Find the running time of algorithm Y (which was never discussed in class)."
    • namibj 7y ago
      Understanding is being able to read python code with no reference book/manual. Applying is writing python code with no reference book/manual. In both cases doing it properly means you are reasonably fast while being confident about the correctness (either that you correctly predicted the program's behavior or that you wrote bug-free software).
• ken 7y ago
  I'm glad he made the leap from school tests to funding tests, as it seemed to be a thinly veiled analogy from the start. I'm disappointed, though, that he didn't take the final step and admit that money itself is also a poor, hackable test. Funding, growth, usage -- these are all still one level removed from something worthwhile or beneficial. Cigarettes have amazing usage numbers even in 2019. I'm waiting for the entrepreneur who will say he doesn't care how much money he makes or how many customers he has, only that it's worthwhile and needed doing.
  • zamfi 7y ago
    > I'm waiting for the entrepreneur who will say he doesn't care how much money he makes or how many customers he has, only that it's worthwhile and needed doing. Closest to this today might be Elon Musk, maybe...but there are other complexities there?
    • tr33house 7y ago
      This guy is literally overworking all his employees. I find it hard to look up to his ways
      • zamfi 7y ago
        A controversial figure to be sure, but at least no one is really forced to work for him -- they know what they're getting into, no?
  • jtbayly 7y ago
    Every free (as in speech) program is written by somebody that said that. But they are generally not entrepreneurs. The entrepreneur is by definition somebody that cares about money.
    • friendlybus 7y ago
      Like Wozniak? Twitter launched without a business plan, Jack was on the Colbert Report in the 2000s saying he'll figure out how to monetize it eventually. I really question how much Uber cares about money recently...
• deleted 7y ago
  [deleted]
• keithyjohnson 7y ago
  As an engineer, this makes me wish there were a better path to FANG employment than hacking their whiteboarding interviews by leetcoding for weeks. They probably do this because it's more objective and simpler than something like the "long conversation with a professor" test that pg suggests.
• daly 7y ago
  Are you in EECS? Most big data centers use FPGAs "on the wire" to do things like encryption on the fly or compression so that they get "wire speedups" without burning CPU cycles. So FPGA solutions that "live on the wire" and can be "plugged into the LAN port" are a really useful area you can hack in your dorm room. I worked in the security area. One of the key problems is "exfiltrations" where someone tries to copy valuable files off a server. I created a startup to attack this "on the wire" with hardware so it can't be hacked. (It failed because nobody knew what an FPGA was, nor how TCP packets worked. Sigh). The idea is to create crypto-hashes of the valuable files. Hand-install the hashes into an FPGA using a read-only micro card. The FPGA sits on the wire, hashing files being sent. If the hash matches, scream. THe same idea of "unhackable" malware virus scans can be done on an FPGA. Only someone with physical access to the FPGA card can modify it. It operates "at wire speeds". The hard part is finding people who can spell FPGA and know what malware means. Be that person. You could do this in your dorm room. The side effects are that you learn a LOT very quickly, it is grounded in real code, and you could actually convince YC that your "homework" is worth a passing grade. Besides, FPGAs are just plain fun. Make your own hardware Neural Nets. Hack your bike with a camera and an NN to warn of approaching cars.
  • rbinv 7y ago
    Wrong thread?
    • klik99 7y ago
      or a very overtrained neural net machine commenter
  • strictfp 7y ago
    If you're one of those persons, where would you look for a job?
• zopine 7y ago
  TLDR: The way to succeed in school is by gaming the system, but the way to succeed in business is by making a great product. But PG is wrong. Startups can, and do, game the system. By showing unsustainable, artificial growth with no real value behind it, they can fool investors long enough to make a lucrative exit by IPO or acquisition.
  • buboard 7y ago
    an IPO or acquisition without proving their market value should count as startup failure.
• thecopyeditor 7y ago
  Part of the challenge is society has organized the distribution of certain rewards and necessary services for playing along with the system. For example - healthcare insurance. Generally a nightmare to secure unless you're with a large company or in a demographic group which statistically probably needs little beyond accident coverage. Want to be a maverick? Better be a high earning one, since you're going to pay a fortune for a bunch of little things and get locked away from the rewards for being a top level corporate servant....
• vezycash 7y ago
  >The real problem is that most tests don't come close to measuring what they're supposed to This is what happens when the teacher's hidden goal is to make marking/grading easier.
  • TeMPOraL 7y ago
    Couple that with students' hidden goal being "not ruining my career prospects and/or angering my parents by getting bad grades", and you arrive at modern school.
    • vezycash 7y ago
      >angering my parents by getting bad grades At secondary school level, it's no longer an issue. Parents just change schools. A few months ago, an uber unserious secondary girl altered her end of term results. The father brought the slip to school to complain about the unprofessional look of the slip due to multiple alterations. Found out what his daughter did and change schools.
• Rerarom 7y ago
  Yeah, but what you learned can be changed in the future (by learning more when you're in a more comfortable position), grades can't.
• hooande 7y ago
  I know that learning how to hack systems is valuable because that's how I got into YCombinator. I set out specifically to hack the application process and succeeded. It wasn't even that difficult to do. If pg believes what he's saying in this essay, I have two questions: 1. why did he design a testing system that was easily hacked? 2. why does YCombinator itself focus so much on hacking the venture capital system, as opposed to making great products? [1] I think the answer to both is that the ideas presented in this essay don't scale. If you can't create a 200 person per year startup incubator that focuses on true learning instead of process hacking, then how could someone do it for a 20,000 person university? Things get boiled down to metrics like grades and valuations because they are the only way to measure/manage the productive output of a large number of people. The insight that I've had is that as long as large civilizations and organizations exist, there will be easily hackable systems. There is a direct relationship between the number of humans involved in a thing, and the number of opportunities to hack that thing. Understanding and exploiting this seems like a valuable ability. [1] I was personally struck by how much of YCombinator was (in 2008) oriented around learning what magic words to say to investors and exactly when to time techcrunch launch articles, etc. There seemed to be no time given to "how to make a great product", likely because this is something that can't be taught
  • BoiledCabbage 7y ago
    That's also my largest concern with this essay. PG describes the world they way he wants it to be, but not the world the way it is. And I haven't been inside of YC, but everything I've seen is that YC is essentially program that combines some level of "support" with essentially learning to hack VCs. If it were simply build the best product then there wouldn't be a need for YC. If it were simply build the best product there wouldn't be a need for marketing or sales or business degrees. The hacks involved in marking and sales became so important to any companies success that they've over time become entire fields of their own. Same thing with an MBA - essentially a degree to hack business. Regardless of what you think of it, it's clear based on the financial returns of it that it works. One of the biggest lessons constantly mentioned of startups is how naive engineers are to the business side of things and there is more than building a great product. In this essay PG takes all of that and throws it out the window. Is that suddenly no longer true? My read of it is that he (rightfully) likes YC and as a result doesn't see how it has the same flaws as the systems he describes. Essentially like every parent who only sees their kid as beautiful.
• krosaen 7y ago
  I actually have found that, at least for math and science classes, focusing on learning was a less stressful way to do well on tests, even if cramming was potentially a workable way to get the grade too. The remaining problem is that often times the course load dictated that some classes had to be undertaken in hack mode.
• avindroth 7y ago
  I am surprised this book is not mentioned. Talks about how optimization doesn't lead to optimal goals in complex environments (e.g. life). https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237 https://www.amazon.com/Why-Greatness-Cannot-Planned-Objectiv...
• daly 7y ago
  Forget the grade. Impress your profs. Let them learn from you. For example, you could work on analog computers (look it up). The early AI systems were all analog, using motors on "volume knobs" to adjust the weights. Analog is MUCH faster than digital. You could probably train an analog neural net "at movie speeds" because it all happens in parallel. Analog weights are trivial (capacitors), backprop is just differentiation which analog does trivially. Function inversion just involves putting the function in the feedback loop of an op-amp. A board with 100x100 mini-caps would be dirt cheap and would have 10,000 "weights". Besides being "up to speed on AI NNs" you would also be breaking new ground (well, really old ground but nobody remembers anything before last year). Imagine talking the department into an "analog AI lab" in cooperation with the EE dept and you get to be "the person". Heck, now you've nearly "joined the department" rather than being a student. Think of college as a big company that lets you do ANYTHING. Profs have to prove themselves to get tensure. Students have to "get good grades" and "learn to hack the FAANG interviews so they can be a 'good cultural fit (aka grade hackers)". But YOU are at college as a "free agent" and can use all the resources to excel at what you love. Imagine having 4 years at a company with the resources of a college that lets you do ANYTHING. You will never have it this good again. Forget the grade.
• Hendrikto 7y ago
  > In theory you shouldn't have to prepare for a test in a class any more than you have to prepare for a blood test. The key here is the “in theory” part. It is very hard to design exams that actually test your understanding of some topic as opposed to your memorization skills. As somebody who recently got his bachelor’s CS degree (currently working on my master’s degree), and who also puts more emphasis on actually learning rather that getting good grades, I can tell you that understanding a subject might be enough to pass, but is seldom enough to get good grades.
  • asdfasgasdgasdg 7y ago
    This is especially true in a domain that has no practical purpose. In computer science, the "exam" could be to go solve some hard problem, or write a program with a particular effect. In engineering, you can have students build a bridge and then stress test it. In athletics, the exam is competition. In business, it's making money. In dating, it's finding a mate (or whatever your personal goals are in that space). In farming, it's making food come out of the ground. For writers, it's a compelling or profitable story or book. For painters, it's a painting. Woodworkers can build a chair. All those domains are testable. But what is the practical work product of a deep knowledge of medieval history? It's extremely difficult to test the past, so it's certainly not predictions about what already happened. Nobody needs knowledge of medieval history for any practical purpose in the modern day. There is no possible way to test for this knowledge in a practical scenario because there is no practical outlet for the knowledge. Literary criticism is the same, along with much that is called "liberal arts" today. So for subjects like this, exams and essays are the only possible work product, and it's true that at that point you're going to have to use contrived tests, since real tests don't exist.
    • spats1990 7y ago
      >But what is the practical work product of a deep knowledge of medieval history? It's extremely difficult to test the past, so it's certainly not predictions about what already happened. This strikes me as a really sad way to see the world. Would you then agree that say, archaeology or palaeontology are things that nobody needs knowledge of for any practical purpose in the modern day? If you disagree, why?
• mettamage 7y ago
  You need a metric to measure performance. It's hard to get such a metric right. You need this metric because the government institutionalized schooling and need to have aggregate data on performance. However, for the student, it'd be better if they'd focus on the intersection of what they're interested in and what is societally relevant and not to be graded at all. I know a friend of mine who went to a school for 1 year where he didn't get graded, all he learned was soft business skills. I can clearly see that he grew a lot from it. But I can also see that it is tough for anyone else to see that he learned anything at all. Partially this is because he isn't graded and the behaviors he picked up are now natural to him and it just looks like it was always there.
• QueensGambit 7y ago
  This is a great essay. But, I find it odd that YC's famous "cohorts" are invented by PG. Cohorts work because startups in that batch compare against each other, which in itself is a grading system. Getting into YC, topping the batch and raising money are inevitable symptoms of "cohort" thinking. We could argue YC uses cohorts to make startups compete for meaningful metrics instead of vanity metrics. But still, YC is based on this innate human nature to compete and get social acceptance.
• arh68 7y ago
  I think "hackability" is a spectrum [0]. The idea is that VC funding is less hackable than college admissions, and that market growth/acceptance is very far on the un-hackable end, along with fairly-referee'd sporting events. He doesn't admit any ways to actually hack market growth (as if there were none), which is weird. Seems like Use Dark Patterns, Pad Your Numbers, Fake It 'Til You Make It, Regulatory Capture, &c would all make the list. I agree with the general idea, though: I think the least hackable tests are either Wimbledon/tennis or GSL/StarCraft tournaments; you simply cannot schmooze your way to even the lower brackets. [0] like most things people put in 2 categories
• remotecool 7y ago
  I think this is the difference between education in many countries In Asia and the US: they care only about the grades, not what you actually learn.
• adwn 7y ago
  > The way you get lots of users is to make the product really great. That is wrong – or, at the very least, incomplete. "Build it and they will come" is the dream and misconception of every programmer who's talented at software development but not at sales. Hell, this is the number one advice of every business how-to book ever: You can't rely on people finding out about your great product by chance, you need to put in the work and sell it! > Then people will not only use it but recommend it to their friends [...] That's not how it works for B2B software, and only rarely for B2C. Usually, if a product is successful, then as a result of good marketing (a good product ist somewhat necessary, but not sufficient).
  • going_to_800 7y ago
    If the product is great(not only the software, but support, vision, market fit, etc), you need a minimum base of users until it takes off, doesn't matter is B2C or B2B, people will talk.
  • deleted 7y ago
    [deleted]
  • TeMPOraL 7y ago
    100% right, and the corollary of this is: you don't actually need to make a great product, if your sales&marketing game is good enough. Or, in other words, a lot of successful business - including startups - is exactly the "test hacking" PG urges founders to unlearn. I'm honestly surprised by this essay arguing that "hacking the test" is the wrong approach in startups/business. "Hacking the test" is essentially what your marketing is supposed to do. These "non-authoritarian" tests like selling things are just as hackable as school tests; you just have to discover how the system really operates. The whole authoritarian/non-authoritarian split doesn't carve reality at the joints, IMO. Football match is really an authoritarian test - it tests who wins under the game rules, which are given from the top. It may be hard to hack, but that's because passing that test is usually synonymous with the terminal goals of the test takers - i.e. they want to win the game. You'll note however that, once money gets involved, people are sometimes made to deliberately lose games; in these scenarios, there are usually other people involved who are absolutely hacking the test. The best way I've found to carve the reality at the joints is to talk about terminal and instrumental goals. Learning useful things isn't the terminal goal for most students, getting a good career is (and/or not pissing off parents by getting bad grades). So I'm ultimately surprised that PG argues that building good things, not hacking the test, is how you win the startup game - I would think most of the startups have products as instrumental goals; the exit is the terminal goal. And building a great product isn't the best way to achieve that goal.
• songzme 7y ago
  > I would explain that what makes a startup promising, not just in the eyes of investors but in fact, is growth. Growth is like cancer. Eventually a company will get big enough to bully others for the sake of growth. Last night I had dinner with a friend who is building a (yet another) language learning app. Simple fact about language learning, or learning in general: You must practice almost every single day. Even if it is only 5-10 minutes. PG himself said, if you don't die (aka give up), you will eventually succeed: http://www.paulgraham.com/die.html http://www.paulgraham.com/die.html To me, the best, most sensical way for learning applications to measure success is to delete user accounts with > 3 days of inactivity. This way you can measure the absolute effectiveness of your program. Students who stick through with your application should 100% become fluent. Target the users who actually want to learn, and measure your success by how many users you made successful. "NO no no", he says. You'll have no users if you do that.
  • mattmaroon 7y ago
    There's a lot of evidence that taking breaks helps with language learning. And there are several other ways to learn languages. I'll sometimes stop using duo lingo but listen to Spanish language podcasts instead for a couple weeks
• BenoitEssiambre 7y ago
  This is one of the reasons I quit academia and am skeptical of much of the research coming out of universities. As a grad student, at some point, it became pretty clear that I would have to choose between doing good science or doing career advancing work (by ignoring biases in my research, p-hacking etc.). Universities have self selected for people who do bad but publishable science.
• louma12 7y ago
  I think it is characterized by using the ideal solution
• ViralBShah 7y ago
  Thus the famous quote - Don't let your schooling get in the way of your education.
• vegiraghav 7y ago
  So i have been looking at architecture and security lately. Is it a thing that you automatically start noticing what is not right in a sight? For instances first thing i saw in that link was that the site runs on http not https.
• graycat 7y ago
  When I was teaching math in college, it was easy enough to give tests that (i) really tested knowledge of the material and (ii) were not hackable. Here's how I did that: Just announce at the beginning of the class that (i) the homework assignments were just to work enough of the exercises in the book to be good at working the exercises and (ii) the tests would be just some of the exercises in the book with just minor or no modifications. Worked fine. In D. Knuth's The TeXBook is: "The traditional way is to put off all creative aspects until the last part of graduate school. For seventeen or more years, a student is taught examsmanship, then suddenly after passing enough exams in graduate school he's told to do something original." In particular, there can be some comeuppance to too much course hacking and not enough learning: Without enough actual learning, may have too much trouble doing "something original". On the course hacking Paul describes: (1) Dad had his Master's in education and to me emphasized just learning and never grades. So, I emphasized learning. He was a bit extreme in ignoring my grades: A few times my neglect of grades cost me, never much but a little. And for Paul's remarks on studying for tests, that's what I did: I never studied for tests. Essentially as Paul explained, all along in the course I'd been trying to learn the material so no way would just cramming and losing sleep help me. Broadly Dad was correct: Most of what I got out of education that actually helped me was just the learning. And one of the best things I did learn was how to learn, i.e., pick good subjects, topics, and sources and learn independently from the sources. E.g., the college I went to as a freshman was selected because it was nearly for free and I could walk to it. But the college was so much like just a grade 13 that they didn't let me start with calculus. But I'd been a math major at a relatively good college prep high school, e.g., that the year before me had sent three students to Princeton and my year sent one to MIT. And in my year, of #1, #2, #3 on the Math SAT, the MIT guy was #3 and I was #2. Similarly on the CEEB Math knowledge test. Sooooo, I SHOULD have been in calculus, and the course they had me in was 90% lower than what I'd already learned in high school and for the other 10% I just taught that to myself in a few days. But I wanted to be a college math major and not fall behind so got a good calculus book and dug in and did okay. The next year I went to a college with a quite good math department, continued with their sophomore calculus from the same text Harvard was using, pretty good text, made As, and did fine. So, I never took freshman calculus; my ability to teach myself saved my tail feathers. On Paul's remarks on grades, there was a cute story: When the SAT scores came back, one of the teachers, I'd had in the sixth grade, was appointed the counselor for giving the scores to the students; so, she knew me well and knew my reputation among the teachers. She read my Verbal SAT score and had a nice smile, said it was "Very good". It wasn't so good, but she was happy because the score was MUCH better than my reputation! Then she read my Math SAT score and became afraid, upset, confused, speechless. When she recovered she said "There must be some mistake." Yup, there had been, hers and the other teachers for 12 painful years. E.g., there was one really good day for learning: In plane geometry, my favorite subject, the teacher was competent but really nasty. No way did I want her to have any credit for my learning; so no way would I ever admit to doing any homework. Each day she assigned three not very difficult exercises. I outlined solutions in the margin of the book for the most difficult exercises, maybe some of hers, and then turned to the more difficult exercises in the back and did them ALL, never once missing one. One of them took me from Friday afternoon to Sunday evening. In class on Monday, she worked at the board an easy exercise with the same figure I'd worked on all weekend. So, I woke up from having my head down resting as usual and for the first and last time spoke in class: "There's an exercise in the back with the same figure." She took the bait! About 15 minutes later, with frustration, she was exhorting the class "Think, class, think. Think about the given ....". Heck, I didn't know that I was the only one doing ALL the homework and that even the teacher had not done all the homework! I didn't want to be accused of ruining the class so blurted out "Why don't we ..." and in a few more words, with great anger, she interrupted me. NO WAY did she want my solution! With anger she said "You knew how to do it all along!". Of COURSE I did: No way would I ask HER for help in math! This story might make a good scene in a movie! There was one more with her: On the state test, the last problem was how to inscribe a square in a semi-circle. After the test I kept thinking about a good solution so asked to see her after school. She didn't like that. I mentioned the problem, and she asked "Why are you working on that?". Well, it was curious, and I'd already solved everything else in sight! So, I went off on the side and constructed a square, circumscribed the semi-circle, constructed a fourth proportional, so had the crucial length in the original figure and did that construction. She said it was not legal to construct the other figure. In college, a student in a geometry for teachers class showed me, given triangle ABC, construct D on AB and E on BC so that the lengths of AD = DE = EC. So, off on the side I started with the angle BAC, picked any length for the AD for that figure, finished constructing the figure on the side similar to the given figure, found the crucial fourth proportional, and completed the construction in the original figure. She said, "How'd you know to do that? That is smilimitude, an advanced technique we are studying." I said "I rediscovered it in high school, and the teacher said can't do that.". (2) I started my career in applied math, computing, and physics for mostly problems in US national security near DC. There no one cared what course grades I'd made but cared a LOT what I could DO that was important and based on what I'd learned or could learn quickly. And I was learning like drinking from a fire hose. Net, what I knew and learned got me a good career, at one time an annual salary six times what a new high end Camaro cost. (3) I did well in freshman physics, led the class. One of the guys I'd beaten got some respect for me so contacted me when FedEx was struggling with scheduling the fleet. I'd been teaching computer science at Georgetown University (had never taken a course in computing but had been learning from that fire hose) so joined FedEx, typed software furiously, got a schedule for the fleet, thrilled the Founder Smith, pleased the BoD, enabled some crucial funding, and saved the company, all based on what I'd learned, in physics class and in computing. Later did a little calculus for some revenue projections, pleased two crucial BoD Members, and saved the company again. (4) In grad school, what I'd learned before entering helped a LOT: On the five Ph.D. qualifying exams, I did the best in the class on four of the exams, and for three of them what I did was all from what I'd learned in class or taught myself before entering. Later my interest in learning and neglect of formalities torqued off my department Chair. So, in a course in optimization, I saw a question with no answer in the course. I found no answer in the library. I got some rough ideas for how to get an answer so signed up for a reading course to address the problem, not necessarily solve it. In two weeks I had a nice solution with a nice new general theorem and also a nice solution to a related problem stated but not solved in the famous paper of Arrow, Hurwicz, and Uzawa in mathematical economics. My work looked publishable (it was; I did publish it later in JOTA). Word of my solution spread in the department. Then I had a halo: No one cared about formalities and let me just LEARN and, then, do my Ph.D. research, with no faculty direction. Again, what I had LEARNED saved my tail feathers. And for Knuth's remark about something "original", the learning let me do that. I did find that the thinking techniques I'd used to do the more difficult exercises in Rudin, Royden, Neveu, Loeve, Halmos, Fleming, etc. were sufficient to do publishable original research. So, net, just learn and do that by doing the more difficult exercises! E.g., back to high school a challenging test question got me to reinvent similitude!
• analog31 7y ago
  I've got one kid in college, the other is finishing high school. Here's what I'm telling them, although not in these words: If you get something of real lasting value out of your education, then you're the one who is hacking the system. Figuring out the tricks to pass tests is not hacking, it's going with the flow. Or at best, it's hacking in the "script kiddie" fashion.
• aex 7y ago
  I've found about how to hack the tests quite early on. I've had straight A's until I no longer cared about the grades. I've also realized that entering and pursuing a career at a tech corporations is a game as well. What has been driving me to startups is the fact that creating a profitable business has no place for hacks. Fundraising is a game that could be easily hacked though. I thought it was only possible to hack the fundraising game for a couple of rounds but Adam Neumann et al. changed my mind that hacks could carry a business even to post-IPO.
  • geofft 7y ago
    > What has been driving me to startups is the fact that creating a profitable business has no place for hacks. Not sure that's true. Marketing is a hack, for one. If you have the best product but nobody knows about it, it's no use. More generally, there's a reason that the tech career ladder is hackable: performance reviews are both important and also too expensive to do right. If you want a real evaluation of what I've done over the past year, what you really need is someone following me 40 hours a week, noting how I contributed (positively or negatively) at small meetings, keeping track of whether my projects are late because I helped someone with something truly important or I spent my time on HN, etc. But you can't assign one reviewer per employee, so you make an approximate process where employees self-report what they did and managers report the fraction they've seen. You also can't get rid of the process, because a simple profit motive demands you incentivize employees for actually delivering business value. So you have a process that's vulnerable to exploits like flashy launches that will wither in a year. If you as a business owner aren't evaluating your employees, you won't be profitable. If you're watching each moment of your employees, you're wasting time. If you spend your time developing a fairer review process, you're not working on your actual business. If you don't hire employees and just put yourself out there in the market, your customers certainly aren't evaluating you fairly. (And of course your employees and potential hires are evaluating you on partial data, and it's in your interest to hire and retain good employees.) So whatever you do, it's hackable, and if you don't play the game you'll forfeit it.
  • dunkelheit 7y ago
    > What has been driving me to startups is the fact that creating a profitable business has no place for hacks. I feel that this hack/not hack distinction comes from confusion of goals. If your end goal really is creating a profitable business then hack/not hack is not a useful distinction. Everything you do either gets you closer to your goal (then it is good, hack or not) or not (then it is bad, even if it "feels right"). But suppose your real goal is "doing most good for society" or whatever and you think that you can achieve this goal by building a profitable business. Then if you e.g. create some bullshit product and market the hell of it (thus achieving your proxy goal), it will feel like a hack that doesn't get you closer to your real goal. TLDR: hack/not hack distinction arises when there are two goals: a proxy and the real one. A hack is something that gets you closer to your proxy goal but not the real one.
• flint 7y ago
  "A"s are for cowards.
• deleted 7y ago
  [deleted]
• wglb 7y ago
  It took me several years to learn how to get good grades. I was not that great a student in the beginning, and was motivated by things that were interesting to me. The two eventually converged, and I ended up with good grades by the end. One test I had was a counterexample to the hackable tests that Paul talked about. And it was the only one of that kind that was of that nature. It was in the EE Static Fields course. The final exam was one question, and you could hear the groans as the test was passed out: Derive Maxwell's Equations This test was a great indicator of how well you got the basic facts of the course.
• raldi 7y ago
  Hacking tests is a lesson much older than modern schools. It's a fundamental part of being human, or indeed, an animal. When a peacock is being judged for his feathers or song or mating dance, these are all being used as proxies for the actually important thing, the quality of his genes. And it's not just a male thing; women do all kinds of work to appeal to arbitrary male preferences. On the other hand, survival of the fittest is an honest judge, so maybe these peacock tests work better than we think?
• gerbilly 7y ago
  >For me, as for most students, the measurement of what I was learning completely dominated actual learning in college. Well I never had this problem in school. I remember a high school calculus exam where I got the answer to the derivatives of the fundamental functions wrong (simple memorization - boring!) but I got the delta epsilon part of the test perfect. I got a C- on the exam, but the teacher wrote a comment that always stuck with me: "You understood the most complicated part , as usual!" Same thing in University, I never did homework in groups. I wasn't looking for the answer to the questions on my problem sets, I wanted a unique answer, or a more elegant answer. I would outdo the professors sometimes. One time I saved a ton of busywork on a problem by setting up a homomorphism with another vector space, proof done in 5 steps!
• mentos 7y ago
  "When a measure becomes a target, it ceases to be a good measure" -Charles Goodhart I've read most of PG's letters but I think this is now my favorite because while I've noticed each of these issues separately I've never connected them together through this common flaw in our education system. I find it really profound.
• viburnum 7y ago
  The hoop-jumping that most shapes society is the need to maximize profit, at the expensive of all other goals.
• tmsh 7y ago
  Fwiw I responded vi Twitter thread https://twitter.com/p10q/status/1203348614338113536 https://twitter.com/p10q/status/1203348614338113536
• hodder 7y ago
  I actually had to drop out of Spanish courses in university as they were going to crush my GPA if I didn’t drop them. I was learning a ton and enjoying them, but for my targeted profession my resume would be tossed if my GPA didn’t meet an arbitrary cutoff.
• jmull 7y ago
  I think the worst thing school teaches you -- especially STEM -- is that 99%+ questions have a certain right answer. IRL most interesting and important questions are not anywhere close to that.
• souterrain 7y ago
  The free market itself is the ultimate hackable test, no? Our society has examples where superb work goes unrewarded financially. Does pg argue that they simply should ignore this test grade?
  • erikerikson 7y ago
    He doesn't offer help there but... Yes and the job interview.
• whack 7y ago
  > How does one get lots of users? They had all kinds of ideas about that. They needed to do a big launch that would get them "exposure." They needed influential people to talk about them. They even knew they needed to launch on a tuesday, because that's when one gets the most attention. > No, I would explain, that is not how to get lots of users. The way you get lots of users is to make the product really great. Then people will not only use it but recommend it to their friends, so your growth will be exponential once you get it started. Ironically enough, if there is one thing I've heard from HN, it is to not believe in the "if you build it, they will come" myth. That no matter how good a product you build as an engineer, it is all pointless if you aren't willing to hit the ground and start aggressively marketing and selling it. That a mediocre product with great marketing/sales, will win over a better product with poor marketing/sales. I've heard this directly from YC partners themselves, when they keep telling early-stage founders to market themselves via "do things that don't scale". At Reddit, one of YC's early successes, the founders were literally spending their time spamming their own site with fake posts and comments using sock-puppets, in order to create the illusion of activity. As someone who loves building products, I hate the idea of spending time and energy on tactics like that. I'd love to spend all that time and energy on making my product better. But I've learnt grudgingly from YC that product development is pointless unless I'm using guerilla tactics to hawk my product in front of users. Maybe I'm in the minority, but I've actually had the opposite experience as what PG describes in his essay. Back in school and University, I did study very hard to ace tests. But the way I studied was to genuinely learn and understand the material as well as possible. Not to "hack" it in some way. Whereas once I graduated, and especially once I tried out entrepreneurship, I realized that just building a great product was insufficient. I now had to "hustle" and use "street smarts" and "growth hacking" in order to get people to notice what I'm doing. The lesson I had to unlearn from school was that the quality of your work will speak for itself, and will win the day. I never needed marketing and sales in school, but it seems indispensable in the real world.
  • BlueTemplar 7y ago
    > That no matter how good a product you build as an engineer, it is all pointless if you aren't willing to hit the ground and start aggressively marketing and selling it. You have pretty much summarized another one of his talks : http://paulgraham.com/ds.html http://paulgraham.com/ds.html (Or even two : http://paulgraham.com/schlep.html http://paulgraham.com/schlep.html )
• burntoutfire 7y ago
  The solution seems to be - avoid people with degrees from top universities? Or, in more extreme form, all college graduates. Less extreme variant of it would be - OK, you went to college - but show me what you did on your own (personal projects, small attempts at startups etc.) during that time. That's what the game dev industry is doing. If the answer to this question is "nothing", then you're probably facing a person who's happy to take directions and land himself a cushy job in some large organization, where he will use his intelligence to navigate towards maximum pay and minimum workload.
• deleted 7y ago
  [deleted]
• flyGuyOnTheSly 7y ago
  Funny to read and enjoy this article this morning... Open up HN again this afternoon for a quick work break... And find my eyes IMMEDIATELY glancing at my karma score in the top right before anything else. Perhaps PG should practice what he preaches and rip that out of the UI? I would encourage it... damnit... I just looked again.
  • darkkindness 7y ago
    Just inject: td:nth-child(3) > span.pagetop { color: transparent; } Hides karma, leaves user and logout intact (as they're links)
    • deleted 7y ago
      [deleted]
• z3t4 7y ago
  Once in a while there will be a unicorn that will spread from mouth to mouth simply because its better then everything out there.
• tcj 7y ago
  For anyone who enjoyed this, a lot of these ideas were really well discussed a few years back in Neil Postman's "Teaching as a Subversive Activity" (1969) and John Dewey's "Democracy & Education" (1916). As you can see these books are over 50 years old. These ideas aren't new, which goes to say how long we've known this and what little have we done. The real question Graham poses here is: what is the point of schooling? At the moment, it teaches and conditions people to "hack tests" as Graham puts it. So people continue to do that throughout their lives. It also conditions people to think there is a "correct" answer to problems. Which in the open-ended nature of the real questions we have to grapple with in life is a complete lie. A teacher will rarely say: "It depends." What passes for "education" today is really just "training" and very far removed from real learning. People only learn about things that interest them. And you learn by asking questions, which most of the time will lead to further questions to which you will only ever find tentative, temporary answers. Not that different from how scientists treat their "solutions" to big questions as tentative. What can we do to change that? Change schools. It's the one place where we have kids kept for 15 years by mandate! It's the biggest lever we have to change perceptions and attitudes. Remove tests and grading. Learning will free up when that happens. It's remarkable what happens when at the beginning of the year the teacher gives everyone an A. Let go of the idea that permeates American society that everything needs to be measured. Some things do, but things like intelligence can't and shouldn't. I know the immediate response is that this is unpractical. And to that I ask why? Because we need a way to control? Control what? What district is "performing" better? When was a score on a math test a true measure of human potentiality? We need to go back to core questions as to what is the point of schools and what is worth teaching and learning in today's world. The world has changed immensely, but our "curriculums" are the same. The only difference is we use tablets and Microsoft Word to hand in assignments. I agree with Graham's conclusion. When we liberate ourselves from these useless idealogical metaphors we will free people and society up to stop hacking tests and start living.
• paulsutter 7y ago
  Getting tested accelerates learning. That’s why we learn more from actual work, every action is tested by reality. Or why we learn language by using it, every interaction is a test. > I was genuinely interested in most of the classes I took, and I worked hard. And yet I worked by far the hardest when I was studying for a test.
• artemisyna 7y ago
  I'm not sure if the correct takeaway here is "unlearn how to hack systems" but something more akin to "prioritize the right hacks you need to do". Making a good product (with whatever shortcuts/hacks come with it) is necessary for doing well. Getting exposure and launching on a Tuesday are neither necessary nor sufficient, but they will give a marginal edge. Both the former and the latter are 'hacks', the only difference is degree of relevance. pg throws the baby out with the bathwater a bit by saying don't do both types of hacks.
• jimduk 7y ago
  I think at various points in life, you worry about the whole system and its effects on people, not just maximizing your local niche. This seems good, both morally and from a systems perspective. For me when younger this was about Ivan Illich (deschooling society), Feynman (the Brazilian light polarization story) etc. Now I'm older it's reading the anger in 'Excellent Sheep', watching the pain in low-trust workplaces, and telling my kids I value real learning over test-passing (they listen, but huge pressure to conform). It seems true that a new field is more intellectually stimulating and liberating than a codified one (I've talked to older female analyst programmers from the 70s/80s who had a blast with computers and left when it got rigid and boring, I worked in Gurgaon early 00s when those interested in IT were self-educating geeks doing it for love, who later got moved aside + down when the money arrived). Maybe this is why IT people chase the new shiny, as a defensive strategy against codification, a red queen game to keep the game interesting ( though inefficient). Anyway, glad Paul is publishing these viewpoints, even if the actions to be taken are not clear. Also, society is over-geeked currently. It's not obvious that hackers ( or intellectuals or logistics experts or technocrats) should have too much power instead of History students, or politicians or sports-stars or whoever. Life is both a game and not a game, short term gaming has long-term consequences. History and Biology and Literature teach us this clearly.
• StacyRoberts 7y ago
  >In the mid-twentieth century, when the economy was composed of oligopolies, the only way to the top was by playing their game. But it's not true now. It is still true. A FAANG job is the easiest path for a engineer to millionaire status in about 5 - 10 years. The interview process is eminently hackable. Or perhaps you want to do a startup, get funding. You build a this for that .com. Do some social media, find a few users who love it. Read pg's advice, pitch deck advice, go to some demo days. Hack the test. Get funding. The incubators, the vc fiends are themselves, oligopolic. A handful of firms with common expectations. Mmr. Hockey stick user growth. It's the team! Here's your A good chap. Those are the tests here. But there are other ways to get an A+. You don't have to start a business the way software people start a business. You can do it like the cabinet maker, or the electrician, or the store owner. You can buy one increasingly easier today because the baby boomers who started than decades ago want to retire. Imagine how you can take that business into the future with the software you can use to automate it. Business brokers abound. Go talk to one. Stop thinking go big or go home. Go talk to a business broker. Go ask about owner absent business options. Residual income and you didn't even have to be there. Use automation to make the employees lives better and more prodictive. And your customers' lives. Package that software you write into a SaaS and sell it to you competitors. Profit more. You are your own case study.
• ronilan 7y ago
  Q: >You're investing time in the same company you're asking them to invest money in. If it's not a good investment, why are you even doing it? A: Because you and them don’t have the same circumstances, and thus don’t have the same utility function. You have different values. The whole (long) thing is a mess of obvious misses. Too bad.
• NPMaxwell 7y ago
  "how to raise money ... read as the test. It came at the end of YC. It had numbers attached to it, and higher numbers seemed to be better. It must be the test." This is a great sharing of internal mental processes. The same process evolves around investment reports. "Oh look! I got a good grade!"
• japhyr 7y ago
  My son is 8, and yesterday he came home from school and said, "I'm not above grade level in math." He sounded a little disappointed, but not too much; he also seemed to be asking if that really means anything. We had a really interesting conversation. I've been a middle and high school math teacher most of my life, so this is quite familiar ground for me. Over dinner I asked him how he would go about measuring whether a kid is "above grade level". He got really animated and started to tell me what he thought second graders should know and be able to do, and how third graders should expand on that, and what he thought he'd start to learn in fourth and fifth grade. I told him I could start to teach him some of the operations and skills he'll learn in the coming years, but that it probably wouldn't mean a whole lot. Instead I asked him what ideas he's already familiar with that other kids his age don't know about, because we talk about math almost every day. "I know what phi is, and pi!" Yes, and we've also talked about limits, calculus, negative numbers, fractions, decimals, and he's looked at pages of math that's well beyond my understanding as well. I told him that when he gets to middle and high school classes, he's going to see higher level math and it's going to sound familiar because we've already been talking about the ideas for years. He's going to watch other kids fail and drop out of those classes because the ideas and the skills will be too new and too difficult for them to take in. I said we're probably much better off just continuing to talk about fun and interesting areas of math for the time being instead of worrying about "grade level".
• asveikau 7y ago
  > If getting into college were merely a matter of having the quality of one's mind measured by admissions officers the way scientists measure the mass of an object, we could tell teenage kids "learn a lot" and leave it at that. It is equally naive to think some idea of "quality of mind" or what someone has already learned in their first ~17 years is what they should go by. This is just shifting from one bias to another. Evaluating a human being, reducing them to a number or a boolean, is hard, and people tasked with doing this, no matter how confident or how much thought and intelligence they put into it, will get it wrong, often with bad consequences. So perhaps we should take such systems lightly, stay humble, and not get overconfident in our judgements.
• pavlov 7y ago
  The tech industry has unfortunately adopted the methodology of centralized hackable tests as the canonical gatekeeping method in the form of programming interviews. Most big tech companies don't care about how good you have been at delivering some value through creating software: they want to see you deliver a very specific type of performance at a whiteboard. Interviewers are given specific math puzzle questions to ask. Interviewees are explicitly told by the same companies' hiring departments that they should aim to hack the system by studying books like "Cracking the Code Interview". This is an industry that prides itself on supposedly making data-driven decisions through A/B testing. When it comes to hiring people to make those decisions, everybody just plays along to a decades-old script.
  • burlesona 7y ago
    Being a hiring manager at a big company, I can tell you this is just as frustrating for me as it is for candidates. I hate “leet code” and frankly find algorithmic interviews to be very low signal compared to more practical, open-ended, domain-specific problems. I will say though, the problem is one of “standardization” across an organization where it’s too big for everyone to fit in a room. Suppose you give each team high autonomy to hire whoever they like using whatever “good” process they come up with. 90% of the time this results in good hires. But as you grow, that ten percent of underperforming people becomes large in absolute numbers, and is very painful to deal with. It becomes a real problem when relatively lower performing people end up concentrated on a team, and then start being the hiring gatekeepers for that team, thus multiplying the number of lower performing hires. Later you start having institutional problems when everyone starts to perceive that the engineers in Department A are generally better than the engineers in Department B. Engineers in Department A are more likely to leave if they perceive the company is getting worse at engineering - it becomes a self-fulfilling prophecy. Then you get enormous pressure to come up with standardized testing - aka algorithms on the whiteboard, or some other academic inspired exercise - imposed by higher level leadership that wants to address a genuine problem (skill disparity across the org) but does not know any better way to do it. I think, as PG points out, there may be a real opportunity to innovate here, and probably a big financial opportunity if anyone can figure out how to productize and scale a solution. I struggle to see an easy answer, though. In a utopian universe (for a hiring manager) I’d do something like pay candidates to come on site and work for a week, then make a hire/no-hire decision based on that. But I think that is far too onerous for candidates (and a big company) to have legs.
• kresten 7y ago
  Nothing said here about how entrepreneurs think who have not been to university. It would have been an interesting contrast. I wonder is that because almost all people who do YC are university educated. Perhaps Paul graham doesn’t have much experience of entrepreneurs who are not university people.
• halfway 7y ago
  It seems that hackability of tests, along with their purpose can be generalized. Tests, like big company politics, are signalling mechanisms. The totality of all tests, culminating in a degree from an institution of a certain prestige, is a signal of potential to contribute. Companies and other big institutions need very strong assurance that supplicants will indeed "work out" by comporting themselves to the culture and making valuable contributions to their goals. And the reason such absurdly strong signals of assurance are needed is the ridiculous level of friction inherent in admission to and ejection from these institutions and their hierarchy of stations. So, to get into Big Company X, you need a prestigious degree, and to advance once you're there, you must identify and play their internal signalling games. Such is the attachment to signalling, that you'd often do much better to lead the holiday decoration committee than to create actual value by solving some important problem. The relative lack of friction in open source is why, given the ability and inclination, even a dog can contribute at a high level.
• projektfu 7y ago
  I suspect that some of the things people do to get good grades, like proofread their essays and edit them down to reduce redundancy, are good things to teach in school, and are broadly applicable outside of the university.
• andreyk 7y ago
  This post glosses over something important, which is that many college classes don't assign most of your grade based on tests. In Stanford's CS 229 Machine Learning class (which I am TAing right now), 20% of the grade is the midterm, 40% is HW, and 40% is a big project that students do throughout the quarter (and for which they get to decide the problem, approach, and most anything with just feedback and guidance from TAs). In my experience, "In practice, the phrase "studying for a test" was almost redundant, because that was when one really studied." is absolutely false, because studying also happened when doing assignments and projects. Of course one could cheat if they just cared about the grade, but I did not because I don't think most smart people really only care about grades, it feels bad to cheat. So TLDR is, I dont think the situation is so bad, because grades are not only a function of tests but also assignments and projects. I think the deeper issue is that people learn to be told how to learn (which assignments or projects to do), and not taught how to learn by themselves. There are exceptions (at Stanford a ton of CS classes have projects where the students map out the problem and solution themselves), but I think this is pretty important too.
• knzhou 7y ago
  As somebody who has designed some of the tests Paul Graham is complaining about, it really is hard from the other side! For example, it's long been known in the physics education research community that students come away from introductory courses with very little physical understanding, even if they can do the plug and chug problems on typical tests just fine. Students can all recite Newton's third law, but immediately afterward claim that when a truck hits a car, the truck exerts a bigger force. They know the law for the gravitational force, but can't explain what kept astronauts from falling off the moon, since "there's no gravity in space". Another common claim is that a table exerts no force on something sitting on it -- instead of "exerting a force" it's just "getting in the way". For research purposes, we measure physical understanding using a battery of tests, such as the Force Concept Inventory, containing only simple conceptual questions with unambiguous answers. So then everybody asks: if ordinary tests are so hackable, why not just switch to these conceptual ones? But that wouldn't work. There are less than ~100 distinct FCI-style questions. If these conceptual tests were the norm, students would just memorize the answers and parrot them back, with a flimsy understanding that crumples the second any follow-up question is asked. It would be just the same problem as before, except they would be worse computationally, too. The FCI only works as long as it doesn't count for a grade. The problem isn't tests, it's scale. If the people aren't motivated, any standardized measure will miss the mark -- even the entrepreneurship Paul Graham advocates for. God knows I've seen a lot of bullshit in that direction.
  • rglullis 7y ago
    > If the people aren't motivated, any standardized measure will miss the mark Shouldn't that be a sign that trying to come up with standardized measures is a fool's errand? Short of blatant social engineering, I don't see how we can have a system that could improve people's motivation and drive for education. And if we don't have a way to actually change human nature, why do we keep trying to design this system that would magically work for so many different people?
    • TeMPOraL 7y ago
      It's a trust issue. It comes from grades being used to evaluate people outside the particular schools they attend. We could give more autonomy to individual schools or teachers in terms of designing tests, but then we'd be facing two problems. One, the grades given by different schools/teachers would not be directly comparable. An A in one school could be awarded only for exceptional circumstances, and in another just for above-local-average. Two, once you start using those grades to judge people in a competitive situation like the job market, you'll have schools selling grades, teachers inflating grades to help their pupils compete against those from schools that sell grades, etc. and the whole system will collapse to uselessness. Standardized testing mitigates these problems by forcing everyone to grade by the same criteria, but in order to do that, the grading must be done on things that are extremely easy to measure - which removes both individual judgement of the teachers and any relation between testing and understanding of the material. The more I think about it, the more I feel it all boils down to scale. The grading system needs to be scaled up with the size of the market people compete on, otherwise it'll be either trivially gamed, or deeply unfair to unlucky people.
• wwarner 7y ago
  Makes me hopeful, and why I am so excited about the Recurse Center. https://www.recurse.com https://www.recurse.com
• darkkindness 7y ago
  The following paragraph is a good summary, I think: > Anyone who cares about getting good grades has to play this game, or they'll be surpassed by those who do. And at elite universities, that means nearly everyone, since someone who didn't care about getting good grades probably wouldn't be there in the first place. The result is that students compete to maximize the difference between learning and getting good grades. It's a race to the bottom.
• ve55 7y ago
  This post reminded me of Eliezer Yudkowsky's Guessing the Teacher's Password https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-the-teacher-s-password https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-t..., which focuses more on the need to unlearn answering questions in the style that school expects you to answer them in, since that style has been optimized to make testing easy, rather than demonstrate in-depth understanding
• lixtra 7y ago
  Paul should talk to Seth about education. He seems to have had the insights before him[1]. [1] https://seths.blog/2018/09/education-needs-to-be-inconvenient/ https://seths.blog/2018/09/education-needs-to-be-inconvenien...
• deleted 7y ago
  [deleted]
• lettergram 7y ago
  Interesting enough, I have the same philosophy - I never cared about grades, not even a degree really. I took what interested me and what I felt would help me. For me I always took 16-20 credit hours Per semester, including summers and averaged a GPA if 2.8-3.4 per semester or something. I also took enough classes in 5 years to get two bachelors. Unfortunately, during job interviews many companies ask for GPA on graduation. I always had to explain “I learned a lot! I just pushed myself to the limit and when you do that you get B’s” It’s always an interesting discussion and occasionally will lose my potential interviews. The truth is that school is really about certifying what you know, not learning. Personally, I think that should change, because building the skill of curiosity and The ability to research and understand a topic is far more valuable. But to learn to be curious you also have to be able to explore - can’t do that craming for test.
• pvg 7y ago
  No, no, no, experienced students are saying to themselves. Oh, no no no, it was too cold always (Still the dead one lay moaning) I was much too far out all my life And not waving but drowning. Somehow captures the mood of reading pg's later oeuvre.
• DataWorker 7y ago
  Unit tests are unit tests. Sometimes the unit is the worker.
• d--b 7y ago
  How is the YC interview not hackable? It seems to me that it consists of ticking boxes mostly...
• edanm 7y ago
  Is anyone else not seeing the footnotes inline? I'm only seeing the notes section at the end.
• Ericson2314 7y ago
  The managerial class will keep on embarrassing themselves sweating bullets over facile things like tests, college admissions, and school districts until they stop feeling precarious and downwardly mobile. There's no point looking at one of their idiosyncrasies in isolation. But then Paul Graham plays himself because all the startups are looking to get or softbanked. Absorbed into the monopolies or put in permanent luxery lift support. What a joke. Playing the VCs is more important than playing the increasingly poorer consumer hoards of the decreasing yield suburban consumer farms. We're years deep into the startup fad, and blinded test takers appealing to their blinded test taker money holders is all that's left.
• hardwaregeek 7y ago
  Hackathons are a great example of this. Ostensibly they should just be about the project that is the coolest or most interesting. But participants started to realize that it wasn't really about the most fascinating project. It was about the project that got the usually non-technical, naive judges to go "ooooh". People started hacking hackathons by building projects that were totally infeasible and completely fake in terms of technical implementation, but had some sort of wow factor through buzzwords (AI/ML/CV/VR/etc) or "making the world a better place". It's no coincidence that some of the largest hackathons are run and attended by students from the top universities pg mentions in the essay. Which is why YC's hackathon was so satisfying. It was great seeing people pitch the normal hackathon-y ideas, then getting simple but direct questions such as "Why would someone use this app?" or "Why wouldn't I use <alternative> instead?".
• correlator 7y ago
  It sounds a lot like raising venture funding. Make sure there's a slide that says "$10 billion" because that's the new billion. Talk about your "AI" which was formerly "blockchain" or "big data" and make sure you fit the mold of whatever the funding is looking for. Leave creativity at the door please.
• bmahmood 7y ago
  Likely unpopular opinion here, but it seems a little unfair to fault YC founders for believing "that the way to win was to hack the test", when the YC application itself would seemingly select for founders that exhibit this behavior with questions like "When have you most successfully hacked a non-computer system to your advantage?" I'd posit the YC application in and of itself shares some of the facets that the article is critiquing in a "test". The fact there are paid services popping up to review YC applications reminds me of SAT Prep services.
• codingdave 7y ago
  I went to a small liberal arts school, and tests were rare. Papers were common, presentations as well. Projects, and experiments in the science courses. But not tests. Most of the time, our grades were based on the work we produced, and our participation in discussions. Even in Art History, when the job was to study and memorize slides and identify them, the test was not to just say what they were, but to write a paragraph about how they fit into the cultural context from which they came.
• listle 7y ago
  Listen to this essay here: https://listle.app.link/v0haf9qve2 https://listle.app.link/v0haf9qve2
• guptaneil 7y ago
  The problem is not that fundraising feels like a test, it's that fundraising is a test posed by authority figures, and by pg's own definition, is probably hackable. How hackable is proportional to the quality of the VC. Luckily startups themselves are indeed not a test. So you can avoid any test by making enough revenue to not need external funding, but the second best strategy is take the VC's test without trying to hack it as your own test of the investor. Assuming you're a good investment, you'll beat those trying to hack the test unless the investor's test is susceptible to hacking. If you're not a good investment (which is by definition not obvious in the early days), maybe you should just try to learn to hack the test? Ironically, one of YC's biggest value adds is that it helps you hack those tests by lending you their name.
  • ignoramous 7y ago
    Paul, though he doesn't reference his previous essay [0] (which has a section on gaming the system) in the current one, did write then that fooling the investors and treating it like a test is only going to delay the inevitable. [0] http://paulgraham.com/before.html http://paulgraham.com/before.html
• caro_douglos 7y ago
  Recently I have been charged with helping someone setup paperwork for their startup. I am not a lawyer so I simply have the paperwork, standard agreements, and general company structure that any lawyer would (I assume) give away for free. Avodocs has pretty much all that paperwork covered, carta has the captable, and pilot has the books under wrap. After reading this article I'm somewhat torn about mentioning things like c corps and filing in Delaware to my friend since in the end it really is only the product that matters. Corporate structure steeped in law/history is a hackable test which everyone must take...resulting in either getting bitten or by running to attorneys and tax advisors (because they were bitten in the past). TLDR opinion. The hackable c corp structure Y pushes has payed off well for Ycombinator yet it is the very thing that can distract from building a good product. Then again I'm sure someone might make the claim that c corps are also a way to easily share equity with your employees.
• eloff 7y ago
  What if we moved to pass/fail grading? Every assignment and test is just marked pass or fail. Realistically tempered by what percentage you're allowed to fail for the class, e.g. 40% for university calculus - no different than grading on a curve today. Fail any N assignments/tests and fail the course. Maybe some are weighted to count extra, e.g. Final project, midterm, final exam. This is really not much different than the current system, but if you stop caring about the difference between C and A, students can either slack off or focus more on learning (if the subject is interesting). Nobody cares about your grades after your first job anyway. I had nearly a perfect GPA in college, and it was a terrible waste of time in hindsight. If I'd spent more time socializing maybe I would have made some lifelong contacts that would have benefited my career or become co-founders with me. I've always thought to tell my hypothetical kids to not sweat the grades and remember that the benefit of university is really the friends (and sometimes life partner) who you meet there. It's total shit when it comes to learning stuff. A self motivated person can learn more in a year than a 4 year degree. Somewhat depends on the subject though - I wouldn't want a self taught doctor who didn't do the practical parts of the education.
• dnprock 7y ago
  I feel like Paul Graham is unlearning his own experience. Maybe, he's at the top of the game and tries to figure out why it's messed up. Life is full of hacks. People hack stuff: school, test, career. The site we're on now called itself Hacker News. I suppose Paul Graham gave it the name. When I was in school, I did all kinds of hacks. I didn't graduate high school in the US. I took the GED to go to college. I did challenge tests to skip classes and save money. I got to know people who collected old tests. They trade old tests with me for homework help. Everyone graduated, got jobs, and continued their life hacks. The focus on learning and doing is important. As long as we're delivering, we can hack some stuff. The philosophy debate between Kant and Locke will never end. There's a balance for each person. We need to discover it ourselves.
  • GavinMcG 7y ago
    It seems like you're responding to the idea that "hacking is bad" and that's not at all what the essay got across to me. In fact, it outright agrees that "the focus on learning and doing is important." Instead of "hacking is bad", the essay was saying "incentivizing young people with a single easily-hacked methodology is bad".
• choppaface 7y ago
  “But you can't blame teachers if their tests are hackable. Their job is to teach, not to create unhackable tests. The real problem is grades, or more precisely, that grades have been overloaded.” Two important things here: * Grade inflation makes for happier students and counterbalances the value of test hacking. Today there are very few schools that don’t do grade inflation. Without considering grade inflation, much of this essay is simply Graham entertaining his own nostalgia. (I’m not necessarily a proponent of inflation but it’s a key phenomena missing from the essay). * Good teachers hold lots of office hours and give good feedback through those offerings. Email support is also a lot more popular today. You can also play games like test corrections, rough drafts, etc. Students don’t learn well without feedback, and most teachers who don’t invest in it won’t succeed today. This focus towards feedback in modern teaching is at odds with the education community profiled in this essay. If there’s one lesson to un-learn, it’s to forget about giving attention to non-constructive papers that lack evidence. While this essay gives a thorough criticism of tests, it offers no concrete alternative. And it offers no grounding for the claimed ‘good’ student who ‘focuses on valuable learning.’ Graham, like any other VC, seeks to upwell sentiment in the interest of controlling it (e.g. creating an investment asset out of it). Reading this essay doesn’t teach you anything about hacking unless you recognize that the author is trying to hack you himself.
  • pmichaud 7y ago
    The suggested alternative I got from the essay was "try to cause a real, specific outcome in the world. If the action you took caused the outcome, you passed the test."
    • choppaface 7y ago
      Did Sam Altman pass the test with Loopt? Just because you made waves somehow or filled your pockets with money doesn’t mean your time was ‘better spent’ than trying to make A’s in organized education. School can very well teach you how to make better outcomes. But, yes, school steers youthful energy away from making change... err making money for VCs.
• meterplech 7y ago
  As a new parent I think about this a lot in terms of how to educate my son. I find it especially relevant as my first job was starting my own SAT tutoring company. Anecdotally I found it interesting that immigrant parents & most of the wealthiest US-born parents shared a recognition that this was a game. They were direct and explicit to their kids that this was a system that they must participate in, so they might as well win at it. I think that is the key: make sure your kids can recognize when something is a bs system they need to hack (like selling enterprise software) and when something is actually about craft and the best will win (like building great products that end-users want to use).
• r3trohack3r 7y ago
  I personally don’t think this is _the_ problem with education. Education has a lot of shortcomings but, as some siblings point out, having a standardized measure of learning is a _hard_ problem. The thing that had always left me a bit uneasy after my rocky road through education was just how damned certain everyone is about everything. Our education system, at least up through undergraduate unless you dip your toes into undergraduate research (and even sometimes still then!), teaches you things we “know” about the world we live in. Humans have discovered an _incredible_ amount about the world around us, but we have a _tonne_ left to figure out. We don’t lean into the unknown with our education system until late stage. The masses miss out on the beauty of the unknown.
• qwerty456127 7y ago
  Good point, but there have been countless amazing products that failed to raise money and/or to attract enough users. Many have even succeeded just to be bought and terminated. Doing the actual job is not enough, in real the real world you are doomed to hack users, investors, laws and many other things anyway.
  • BlueTemplar 7y ago
    Did he ever pretend otherwise?
• Terretta 7y ago
  > No, I would explain, that is not how to get lots of users. The way you get lots of users is to make the product really great. Then people will not only use it but recommend it to their friends, so your growth will be exponential once you get it started. > At this point I've told the founders something you'd think would be completely obvious: that they should make a good company by making a good product. And yet their reaction would be something like the reaction many physicists must have had when they first heard about the theory of relativity: a mixture of astonishment at its apparent genius, combined with a suspicion that anything so weird couldn't possibly be right. The essay context is students and startup founders, but it turns out most multi-billion dollar enterprises have forgotten this as well. This notion does not come embedded in the heads of most “senior management” from either world.
  • BlueTemplar 7y ago
    The point is that big companies have other, easier roads to success.
• stornetn 7y ago
  The point is summarized well by one of my favorite aphorisms (often attributed incorrectly to Mark Twain): "I have never let my schooling interfere with my education."
• yosefzeev 7y ago
  I actually did what this article suggests in college. It didn't go well grade wise. However, I had money and time to study and learn about things I wanted to learn. I found that my path often was interwoven with the path of the class, but that the perspective was different and sometimes professors are only grading on perspective--specifically their own. The biggest dividing line is, I suspect, that college does not, at this point teach one how to think. How to think and good grades are not necessarily the same thing at all. I was busy learning how to think, and deciding what could be relegated to "It's in a book, I can find it if I need it" vs "I should memorize this because it will be useful in life for me later".
• marccarre 7y ago
  The problem might be a lot bigger. Possibly most of one's life is spent (over?) fitting to some "test" in a system, or "expectation" someone else has, instead of doing what they truly want: trying to pass a test at school, trying to land that job (you think) you want, trying to be socially acceptable in the schoolyard, at a party, or in the office, trying to please a parent, a friend, a partner, trying to conform to some unconsciously shared vision of "success", trying to conform to your own vision of "success", etc. The list is long. Fairer, unhackable systems would definitely help in reducing the amount of such "tests" we artificially created as a society. We should strive for this. However is it possible to solve human nature? To solve "affinity bias", "confirmation bias", "psychological needs", etc.? Is it possible to create a society in which everyone is truly free? Or at least as free as possible from all this? Is it possible to solve freedom & wisdom? :-)
• hirundo 7y ago
  > When I was a kid, you could either become an engineer and make cool things, or make lots of money by becoming an "executive." Now you can make lots of money by making cool things. Software is outrageously accessible. The threshold cost for make-lots-of-money-by-making-cool-things has fallen to a used laptop and a wifi password. The prerequisite talent is mostly the ability to focus relentlessly while illegitimi non carborundum. But since those are rare talents the laptop can only elevate rare people. It's a path to relative wealth for only a pareto distribution of people, like pretty much anything else. But it's a path to absolute wealth for most of the rest of us. Cool things that make money tend to make our lives better. Make-lots-of-money-by-making-cool-things is a recipe for both more absolute and unequal wealth.
• parr0t 7y ago
  As someone who got very high grades in my CS degree I agree, transitioning to the real world with my first job was 12 months of pain.
• slipmagic 7y ago
  Great and inspirational article that’s pretty practical. That said, the website is a pain to use on mobile safari and difficult to properly capture when sharing it to Notion or Bear. Notion doesn’t capture it at all, and Bear can’t capture the hyphens or line/paragraph breaks. It’s probably because of the read more button. It also has some jaggy scrolling issue where if you switch between apps it scrolls up and sometimes collapses the article or jumps to the bottom of the page.
• scscsc 7y ago
  As a teacher, the way to fix this is to give "aligned tests". That is, where the test tests for exactly what you want the student to learn. See https://en.m.wikipedia.org/wiki/Constructive_alignment https://en.m.wikipedia.org/wiki/Constructive_alignment.
• hogFeast 7y ago
  What is funny about this is that he doesn't realise most medieval historians have actually recognised this and solved it...answer: no exams. Just get students to write essays. Most history classes do exams but, where it is taught well (i.e. not the US), they give you the questions that will be on the exams...these aren't exams like in maths. This tests for understanding, and actual ability to conduct research yourself (you still get bias from professors, I had a Chinese politics professor who had some fairly inflexible views about the Chinese economy...but it does average out). I still have no idea why this isn't used more generally in life. Want to see if someone can do this job...hm, rather than actually seeing them do the job...have you ever thought about how many windows there are in New York? All you are testing is whether someone has either been asked that kind of question before or, more likely, whether they went online and looked at what you would ask them. It is truly stunning how stupid people are about this. And, tbh, I think it largely reflects what most students about the world at university where certain professors seem to revel in the utter banality of pointless tests.
• yotamoron 7y ago
  Ivan Illichlready said it 50 years ago. Read 'Deschooling Society', its all there. We are schooled to confuse process with essence, and the bad results are apparent.
• dunkelheit 7y ago
  In my opinion the most insidious feature of the education system is not tests. After all a proper hard STEM program is not that easily hackable - the easiest way to complete it may be to actually study! What is worse (and although it is related to grades it is not the same), you only do what you are told to do. Your goals are set for you - read this chapter, solve this problem, answer this question. Thus your inherent creativity is slowly but surely gets sapped away until you become a "professional" - someone who solves problems of others without questioning them much. Not sure what to do with this though - every way to overcome this problem seems inherently not scalable. Also not sure if this is really a bad thing - a society where everybody suddenly becomes more creative can suddenly collapse because of all the excess creativity!
• gnicholas 7y ago
  I usually find PG’s writing to be very illuminating, but I largely disagree with this piece. I think the reason founders want introductions to influential people is not because they think that this is what you need to be successful. It’s because they have seen many cases where influential people talked up startups that seemed to be pretty mediocre, and it helped them succeed. And they think “well my startup is at least as good as those other ones, so publicity by famous people will accelerate our path to success.” They’ve seen press coverage of lousy startups propel them into fundraising successes and growing revenues, even if they never became profitable. And they think “I’d succeed faster if my legit startup had that kind of exposure.” It’s not that they think these things are a substitute for having a good product. They think (correctly) that having these things will accelerate their growth and somewhat lower the bar to success (particularly if there are network effects involved).
  • ajju 7y ago
    > They think (correctly) that having these things will accelerate their growth and somewhat lower the bar to success Incorrectly, unless you define success as a local maxima. You usually can’t influence your way to broad user adoption
    • gnicholas 7y ago
      What makes you say that? For early stage startups, it seems especially likely that increased awareness can (though usually will not) create a material amount of additional success. I know that a few fortunate tweets my startup received drove tons of traffic our way, and likely resulted in the press coverage we received soon after. And that press coverage brought more users, and also the proof points of having been covered by major media. That makes fundraising and partnerships easier, etc.
• softwaredoug 7y ago
  One solution might be to silo the people who evaluate the students from those that teach the materials. Treat the test more about evaluating the effectiveness of instructors and less about the students.
• DFXLuna 7y ago
  This article reminds me of my interview for workstations at HPi. I came away from it knowing that I had to work there. The interview was 5 hours: 4 one hour sessions and lunch with the team. I didn't write a single line of code the whole day. Every question came with the disclaimer that the answer was less important than the thought process. Disclaimer: I work there now and my opinions don't necessarily represent my employer's etc.
• adebelov 7y ago
  I never let my schooling get in a way of my education :)
• noobermin 7y ago
  As someone who's not entirely convinced VC funding start-ups isn't the best model for how society can allocate resources, I'm always impressed with how pg thinks, but I find he misses the boundaries of the world he lives in. He's right about artificial tests, but he fails when he says the current situation really allows "people doing good things" to get ahead. I mean, just look WeWork. It's hard to claim WeWork won at an artificial test, unless you count convincing others with your confidence as your test. I'm not sure that's what he thinks it is though. You can claim non-artificial tests are "better," but humans (VC's) still hold the purse-strings so to speak. If you can convince them you're winning the non-artificial tests say of the market, then you can get money and do coke and party for a while, whilst sure your company will crash and burn but you escape unscathed with millions of sweet free money. My point is the problem is deeper than just "artificial" vs. "non-artificial", although it is an important aspect.
• randomsearch 7y ago
  A related observation: what is the point of a “B” grade? It means “this person didn’t learn everything.” If you’re going to say “oh ok they got a B so we need to teach them some more”, that’s really useful. But if you just given someone a “B” and send them up a year at school/university, then presumably they find it harder and harder to do subsequent courses. And over time they learn they’re a B student. Which is demoralising, and possibly also insulting as it ascribes failure of the learning process to the student and not the teacher.
• renlo 7y ago
  School grades are more a measure of conformity and ability to adapt a wage-work environment. Performing well in both wage-work and school requires an individual to jump through arbitrary hoops (performance reviews, office politics requiring certain procedures, etc). Teachers are just as capricious as bosses and upper management with their directives. Perhaps if we move away from wage work then school will change too.
• auggierose 7y ago
  Love it. It has been a while since I read a P.G essay, although this was what first attracted me to HN. It also means that it is OK to work on your idea without entering YC. After all, being accepted into YC is just another test by authority at this point.
• grappler 7y ago
  I think there are some good insights to be gleaned from this essay, as with a lot of pg's essays. The perspective of the guy who founded and ran Y Combinator is always an interesting one to hear from. However for this essay in particular, had I been among the set of draft readers, I would have suggested toning down the “I hadn't realized X until now, so now I think other people haven't realized X either” message. Back in college I remember noticing choices of “better learning vs better grade” and deliberately choosing the former with a proudly “screw the authorities” attitude. I'm sure plenty of other Tara Ploughmans could say the same.
• gliese1337 7y ago
  In theory, tests are merely what their name implies: tests of what you've learned in the class. In theory you shouldn't have to prepare for a test in a class any more than you have to prepare for a blood test. In theory you learn from taking the class, from going to the lectures and doing the reading and/or assignments, and the test that comes afterward merely measures how well you learned. That was always how I approached tests in college. While everyone around me was stressing out, not once did I actively study for finals. Finals week is for relaxing, sleeping, watching movies, reading books, and having parties in between tests. Because, y'know, learning the stuff on the test was what the whole frickin' semester was for, and if you weren't paying attention up to now, what makes you think you can learn it all over a weekend? I was constantly amazed at how many of my peers, who were paying good money to learn stuff at school, were utterly shocked by that philosophy. I really wish more students understood it.
• JesseAldridge 7y ago
  > Why did founders tie themselves in knots doing the wrong things when the answer was right in front of them? Because that was what they'd been trained to do. Maybe I'm overly cynical, but I see another possibility: They want to create the appearance of running a hot startup without actually having to do the work. Why make something people want when you can just dazzle some investors and get them to dump a million dollars in your lap, no strings attached? I wonder if that also might explain why such a tiny fraction of startups generate essentially all the wealth -- most of them aren't really trying.
• aardvarks 7y ago
  There is a parallel with one of pg's other essays, 'Why nerds are unpopular'[1]. He says (I paraphrase): 'nerds care more about being smart than being popular'. But that's another way of saying nerds think the tests you have to pass to be popular are (on balance) a waste of time, ie bad tests. [1] http://paulgraham.com/nerds.html http://paulgraham.com/nerds.html
• eBombzor 7y ago
  Wonder what this guy thinks of Asian schools, where you literally have to dedicate yourself to studying 9 hours a day, 6 days a week, through middle and high school (where you have to dedicate 11+ hours regularly) for one test that literally determines your entire career path, unless you want to move to a different country that is.
• sinameraji 7y ago
  Love it. I wanna put this in the context of innovation in education: IMHO, a radical innovation in education is one that can address at least two of the following flaws and failures in the current education system, at any growable scale: –Duration and cost –Content –Delivery If I were to satisfy a VC, I could also add a 4th bullet point called "Supply/Demand", but since I'm not writing for a VC, then I'll write what actually matters. In the case of education, that is, it's okay to ignore the existing supply and demand, and focus on the demand that must exist that doesn't yet. Wrote a memo on these and elaborated more https://www.linkedin.com/pulse/q4-2019-what-i-see-current-state-innovation-education-sina-meraji/ https://www.linkedin.com/pulse/q4-2019-what-i-see-current-st... P.S: I think I got lucky that my mom was a teacher, so I grew up having a definition of right and wrong different from what was expected of me, and that stayed with me throughout my education. I didn't do what was asked, but what I believe was right for my growth.
  • BlueTemplar 7y ago
    Considering that "education" is being used for daycare and conformity first, it might not be "fixable" when these goals are still requirements...
• jhatemyjob 7y ago
  Paul Graham is one of the most powerful people on the on the planet. Of course he sees "hacking bad tests" (aka politics) as something that will become obsolete. He has the privilege of ejecting from any political situation he finds unpleasant. Meanwhile, the other 7B population with substantially less power, we have to deal with each other. Politics are, and always will be, a thing. As long as people exist.
• ZguideZ 7y ago
  Tsunebaro Makiguchi, an early 20th century Japanese educator, was striving towards deconstruction of rote education but his work was disrupted by authorities who worried that his educational revolution would disrupt orderly Japanese society. Careful, PG, you might be rocking the boat too much with this. ;)
• ryacko 7y ago
  I disagree with this essay, I have no knowledge in the field that would allow me to argue against this essay, I think this essay lacks brevity, and the moral of the story seems weak.
• thallukrish 7y ago
  The way we filter candidates into any stream be it college or companies need to change. People are inherently pressurized by capitalistic mindsets and competition that makes them to not question things but just follow the herd mindset of setting up papers and interviews mechanically. We always are chasing the next than solving the now. Some related thoughts I wrote a while ago on the state of Education https://medium.com/@thallukrish/why-the-why-matters-d7b8170b5a2c https://medium.com/@thallukrish/why-the-why-matters-d7b8170b...
• CivBase 7y ago
  One of my favorite college courses was a computer science course that taught C++. It only had 4 graded components, each worth 25% of your final grade. There were two tests (programming tests with paper and pencil only) and two projects. It wasn't hard, but it was the fifth most failed course in the university because the tests were extremely difficult to "hack" and the projects were big enough that a student could not rush them at the last minute. There was no homework or attendance to pad grades. It was one of the few times in my college career that I felt like a professor actually cared whether or not students actually learned about the subject.
• thatthatis 7y ago
  I always shot for a “B+” in college. My logic was that in 5 years I’d remember the content at a B+ level, and any incremental effort was better spent socializing or building student organizations.
• rajacombinator 7y ago
  This is actually wrong and surprising to hear from PG. For the most part, life is hackable. Sure, being the “real deal,” whatever that is, is great. But most of “success” in life (again, whatever that is) comes from hacking. (And luck.) Look at WeWork, for instance, one of the greatest and most successful hacks of our time. YC itself became a hack as soon as it acquired useful signal. If you’re optimizing for success, you should teach students to hack more not less.
• woodandsteel 7y ago
  I think one solution to the problem PG describes is self-directed study.
• donmatito 7y ago
  The most interesting class I took in my entire studies (Soft Matter physics) was also the one with the most interesting exams. Our year's 2-hour exam was the following. The professor gave us a small cup of water, a piece of paper, and a piece of toilet paper. There was only 2 questions - place one end of the paper in the water. Describe what happens. Explain - place one end of the toilet paper in the water. Describe what happens. Explain The year before us was with a small piece of glass and a small piece of potatoe - breathe on the glass. Describe. Explain - rub the potatoe on the glass. Breathe on it. Describe. Explain You REALLY needed to understand the class material to succeed in these seemingly simple tests
• Bost 7y ago
  Is there any tl;dr ? Thank you in advance.
• adyer07 7y ago
  I’m returning to school for a BFA in art, and I’m constantly surprised by exactly how un-hackable the assignments are, compared to my experience studying engineering. It’s finals season, and many people (myself included) take it as an excuse to push and make work that is harder and more complex than they could during the rest of the semester. Most classes are pass/fail, so the only reward is having better work (and a better portfolio). You could duct tape a banana to the wall, but that’s on you to live with. Maybe there aren’t a lot of fine artists starting YC companies, but how about designers who went to art school? Do they treat these problems differently? Do they have less to unlearn, or different bad habits?
• mncolinlee 7y ago
  Perhaps one of the most noticeably hackable tests I've participated in is democratic elections. I ran for office a few times and learned a lot about how the system actually works. I believe this is why politics has such a negative connotation. Ultimately, voters want politicians to do what's right and politicians learn they have to gain the genuine love of one group and appease a large enough cross-section of groups without offending an important one. If there's one thing I learned in politics, the biggest hack is in having a great narrative. Ultimately, brands, companies, and politicians are characters in our consciousness that only achieve meaning through the stories they tell. Essentially, the story must be believable and ring true to the audience being targeted according to their past experience. For example, evangelicals love a good redemption and hard work story. Each group has their own highest morals. I'd recommend "Don't Think of An Elephant" by George Lakoff, which explains how politicians hack the human mind by crafting an entire language that plays into their hopes and fears. I'd love to have every test in life be genuine and unhackable. But when tests are clearly so hackable, it doesn't help to not call out how the hack works. Call it responsible cultural exploit disclosure.
• sirspacey 7y ago
  This is the difference between project-based learning and standardized testing. When you learn through projects your ability to apply concepts is tested by the results you generate, connecting to our intrinsic motivations. Standardized testing is, functionally a disease. Check out “DIY University” to see the effect it has on society long-term. If you think projects can’t be the path to teaching highly abstract disciplines, such as physics, check out “Surely You’re Joking Mr. Feynman.” “Most Likely to Succeed” is probably the best book written on this topic. I spent 10 years as an edtech entrepreneur. It is hard to overstate the degree to which we have built a Testing system at the expense of an educational one.
• ALBERT300 7y ago
  HIRE A PROFESSIONAL HACKER FOR ANY HACKING RELATED ISSUES: OUR SERVICES INCLUDE: CELLPHONE MONITORING SOCIAL MEDIA HACKING DATABASE ALTERATION SPYING ON ALL SORT OF DEVICES CLONING ( WEBSITES, DEVICES, CELLULAR etc. ) HACKING SPOUSE PHONE UNIVERSITIES GRADE CHANGE RECOVER LOST CRYPTO CURRENCIES GET IN TOUCH WITH THE BEST HACKER VIA: EMAIL:THEREDHACKERGROUP AT GMAIL DOTCOM PHONE/WHATSAPP: 5 7 1 3 1 8 9 4 9 8
• grappler 7y ago
  Coming back to this discussion a few days later as I've just started an online course on Kubernetes from the Linux Foundation and I came across this section in the introduction materials that seemed very relevant here. Quoting from it: Training/Certification Firewall The Linux Foundation has two separate training divisions: Course Delivery and Certification. These two divisions are separated by a firewall. The curriculum development and maintenance division of the Linux Foundation Training department has no direct role in developing, administering, or grading certification exams. Enforcing this self-imposed firewall ensures that independent organizations and companies can develop third party training material, geared towards helping test takers pass their certification exams. Furthermore, it ensures that there are no secret "tips" (or secrets in general) that one needs to be familiar with in order to succeed. It also permits the Linux Foundation to develop a very robust set of courses that do far more than teach the test, but rather equip attendees with a broad knowledge of the many areas they may be required to master to have a successful career in open source system administration. Running with this idea a bit, suppose schools were organized this way, with a similar "teaching/testing firewall" across everything they do. This doesn't get us all the way to solving problems around "teaching to the test" but it does put in place separate entities with the explicit mandate of making tests less 'hackable'.
• classified 7y ago
  So many words and nothing said. Let me just say something to raise the informational content of that article to 1 Bit: Water is wet.
• Xelaz 7y ago
  Is the latest essays the in-text links to notes are missing. (My favorite from this one: "Learning is the naive algorithm for getting good grades.")
• Mauricio_ 7y ago
  I'm too late to comment, but my 2 cents: As long as there's something important at stake people will try to hack the test. There's no way around that. The best that can be done is to try to keep the correlation between the "test" and what it's trying to measure add high as possible and to teach everyone how to hack it, that way everyone is on a level play field. If everyone hacks it then the difference in scores will be mostly about what is being measured and not about who hacked it and who didn't. That's why big companies actually advice interview candidates to practice on leetcode and give them example interview questions. But I agree with the main point that being good only at "hacking tests" in life instead of being good at your profession is very bad.