19 ms·

General and Surprising

417 points by maguay 9y ago

jacalata 9y ago
Well, that was certainly general.
- enriquto 9y ago
  ...and not very surprising. Thus, platitude! :)
  - deleted 9y ago
    [deleted]
  - jacalata 9y ago
    Indeed. The number of votes this has received is an amusing insight into how many people must upvote based on origin and not content. If this were published on some anonymous medium blog, it would be lucky to be +2.
    - hasenj 9y ago
      It sounds like a clip from some kind of a speech. There's some missing context. or, there's hardly any content. Like, I'm not sure what he's trying to get at.
    - tinco 9y ago
      Not everything that is a platitude is necessarily bad. Reinforcing, clarifying or rediscovering knowledge certainly has value. If Paul Graham writes an essay then that is relevant to Hacker News so it should be upvoted regardless of whether it is shit.
      - jacalata 9y ago
        So...you are agreeing with me that this has been upvoted based on origin and not content, and you're just repeating what I said to reinforce the point?
        zaptheimpaler 9y ago
        Thou shalt not take PG's name in vain.
        tinco 9y ago
        Origin and context give extra meaning and weight to content. That seems trivially true to me. If Paul Graham writes this then that means the topic is on the mind of someone influential and important in our community, thus it is worthy of scrutiny and thought, thus worthy of an upvote. Hacker News is not some game where people compete to create content with maximum merit. It is a source of new content relevant to this community.
    - WA 9y ago
      Nothing new here. The messenger always plays an important role.
quuquuquu 9y ago
It's funny, a lot of my imposter syndrome comes from an anxiety about not having massive "deltas of novelty", as PG would call it. I guess all I can do now is hope that whoever is listening can see the value in incremental change :)
- ismail 9y ago
  Same here, you constantly wonder if what your thinking is truly unique. I believe the key is figuring out when your view is unique. What I have been doing is asking people questions on their view of a problem or something I am thinking about and looking at the deltas there. I have not thought about it with the term delta. Maybe looking for deltas would be another addition to the toolkit when searching for insights. I prefer to use the term contradictions. Sometimes that leads you to further insights.
  - heavenlyblue 9y ago
    And you spend most of your life trying to prove how special you are, instead of productively producing things and playing with the world that is also shaping them? People tend to forget that ideas are worth nothing. It's not that hard to build a model of the world which would be parametrisable for any possible outcome. Then those parameters would be defined by your experience, which requires you to get out of your mind and earn on the outside.
    - ismail 9y ago
      It seems my comment was not precise enough. The result is the following assumptions being made: 1. The thinking/insight/ideas is in pursuit of proving my own special nature 2. The thinking/insights/ideas are purely theories and have no practical application Notice that the word "problem" was used. What is meant here an actual problem that I am trying to solve with actual practical implications. Let me clarify what I mean, using concrete examples: In product development/startups/business space: "My customers are telling me their problem is X, but what they are doing or how they are doing it is Contradictory. Why is this?" "Two customers have completely different and contradictory views on what feature/problem is important to them, why is this?" The actual purpose is to understand your customer better so you can cater for what they actually need. This part is confusing, can you clarify what you mean: "Then those parameters would be defined by your experience, which requires you to get out of your mind and earn on the outside"
yeukhon 9y ago
[T]he more general the ideas you're talking about, the less you should worry about repeating yourself. Organisms are used to train over and over to get better at things. Certain abilities are innate, like a calf knows to stand up as soon as out of the womb. But to stand firmly the calf has to keep trying. When it comes to an idea, I believe that the broader I present the idea, the easier it is to start a conversation, because others are more likely to show interest, and eventually reach the point you find a focus point. However, I still repeat myself, because I wouldn't invent a whole new dialogue next time I bring up the same broad idea with another person. On the other hand, I feel developing a product (startup or not), the more general the idea is, the harder for me to explain my ideas concretely, much like films have leading roles. The ideas present by the film can be broad, but you don't want camera rolling on 200 actors in 1 hour 30 minutes, would you? Your message can be interesting and novel, but I am going to be lost. But yeah, on HN topics like "I quite FB", "security is hard", "Google is evil", etc are common here, but I still comment on those topics when I feel like to. My responses are usually similar, but I might add new thoughts, or rephrase what I had written before. I called this introspection. I believe we are not comfortable repeating ourselves, because we want to be interesting and original, but we still have the urge to try to perfect our speech the next time someone asks.
fiokoden 9y ago
A very academic / theoretical post. Not entirely sure the point. It might have been easier to understand if bound to real world examples.
harscoat 9y ago
"surprising" in patent law https://en.wikipedia.org/wiki/Inventive_step_and_non-obviousness https://en.wikipedia.org/wiki/Inventive_step_and_non-obvious...
deleted 9y ago
[deleted]
stonesixone 9y ago
There's also a class of valuable insights that are general and not surprising, but rather obvious. For these, the challenge is that no one previously had the insight to observe and state what is obvious. Maybe the surprise for these isn't in the truth but rather that no one had stated them before.
paulajohnson 9y ago
The essay describes a 2x2 grid with axes of generality and surprise. Things that are suprising and general are valuable. Things that are general but unsurprising are platitudes, and things that are specific but surprising are gossip. The essay itself is a platitude.
- zaptheimpaler 9y ago
  Brutal but accurate summary! I get the feeling there is something he is not (or cannot) saying that prompted the essay.
  - Kluny 9y ago
    Yeah, it has a bit of a vaguebooking feel!
    - mattacular 9y ago
      I can't see the exact published date but it strikes me as a rebuff to critics of the iPhone X unveil
- bambax 9y ago
  It misses an important element that may seem obvious but isn't: truth. Some insights are indeed, general and surprising, but when examined closely, turn out to be false. That's what happened to many of the ideas presented by Daniel Kahneman in Thinking, fast and slow. He obviously aimed for "general and surprising" but accepted as solid results the outcome of lone, irreproducible experiments. We (our current culture -- it wasn't always like that) love originality and "surprises", but as a rule, the more surprising the result, the more scrutiny it should withstand.
  - tlb 9y ago
    Circulating ideas where people disagree about whether or not they're true is an excellent way to discover things. Not just whether the particular ideas true, but why or why not, and where the uncertainty comes from. In your example, perhaps the most lasting value of priming studies is a much greater understanding of how p-hacking actually misleads us. Everyone knew that it could in theory, but it was general and surprising that entire fields could have an apparent scientific consensus entirely based on p-hacking and file drawer effects.
QuantumRoar 9y ago
This is not true as often you find surprising things that are not general which ARE NOT GOSSIP. To stay with the physics theme, the Schwarzschild solution of general relativity is a very special one and I don't think anybody thinks that black holes are gossip. And it is certainly not surprising that amateurs in general forget that F = ma is only valid if the mass does not change. The more general expression is F = dp/dt, where p is the momentum. But this, of course, is also only valid in inertial systems. It's not really important to the article but it does kind of annoy me that he uses the most special case of an expression in an argument about it being general. He could have actually made the point about generality by comparing this expression to the most general one for the force (in a frame of reference that accelerates). That would have also shown why generality can quickly become infeasible in practice. If he knew how many approximations people make in the real world, not because they want to, but because they HAVE TO, his worldview might be a different one. I feel like he's trying to make a point about a very specific scenario but doesn't mention it explicitly. Instead he tries to be general and therefore fails to understand that his view doesn't actually apply in general.
- Danihan 9y ago
  You should have read the draft of this, instead of the people who did.
- oldandtired 9y ago
  Theoretical entities called "black holes" cannot exist in finite universe (no matter the perspective changes that are applied). The result (if I understand it correctly) applies to a universe in which there is a single massive body and universe is eternal. Hence, if time dilation exists (under increasing levels of gravity) "black hole" cannot form in any finite time. Since there are many massive bodies in our universe, solution to problem is not applicable. So, in that context, "black holes" ARE GOSSIP!!!! Let the "GOSSIP" wars commence. Other than that, I think you make some very salient points, especially about the approximations that we make and the validity of those approximations to the specific applications/situations being studied.
  - QuantumRoar 9y ago
econ1212 9y ago
F = ma is interesting because there are universal forces like electricity, gravitation and electromagnetism that we use for our daily life. Moreover there is a marriage between maths and physics since acceleration is the second derivative of space with respect of time. If you are a new Newton you should create the new concepts and develop the new maths to create a surprising step forward. Looking for a word that goes from gossip to F=ma is like using a telescope to watch a worm. Concepts are about frameworks and using a right scale to focus concepts.
w_t_payne 9y ago
There are also the platitudes that everybody knows but that nobody does anything about.... Oftentimes there is virtue in simply being unusually conscientious.
matt4077 9y ago
This genre of writing is also neatly categorised into one of three categories: Obvious, unactionable, or wrong. PG sometimes meanders into actual meaning. But the likes of Tim Ferris or Seth Godin absolutely excel at this drivel.
roceasta 9y ago
>that territory tends to be picked clean, precisely because those insights are so valuable People often think that a field has been 'picked clean' and conclude that either no further progress is possible or that further discoveries are going to be rare and incremental. e.g. "the future truths of physical science are to be looked for in the sixth place of decimals" -- Michelson, 1894 But there are always deep problems and this implies that unlimited progress is possible. The really big discoveries can happen at any time and are of an unpredictable nature, not least because they affect multiple fields. They leave behind plenty of smaller problems for everyone to pick up.
- andrewflnr 9y ago
  That's especially funny from the same Michelson who did the Michelson-Morley experiment that basically kicked off relativity.
perpetualcrayon 9y ago
"the more general the ideas you're talking about, the less you should worry about repeating yourself" A rule-of-thumb I've generally followed when brainstorming is "never write down the ideas". And if at all possible, try to forget them, good or bad. It inevitably forces me to re-think the ideas "from scratch". I've never been worried that an idea would escape me or that I would forget it, as I think the exercise of re-thinking ideas from scratch has helped me build a relatively solid mental model of the world. I feel that, with practice, this method of brainstorming has started bringing me to the solid / useful (general / surprising) ideas faster. It has likely, over the years, added weeks or possibly months worth of time that I've spent brainstorming the same (general) ideas over and over.
- vanderZwan 9y ago
  As an aside, I can't say for sure how you use the term "brainstorming" here, but there is plenty of evidence that classic-style brainstorming in groups leads to less ideas, and less creative ideas. When I was forced by my boss to teach designers brainstorming, against my better judgement, I instead made this slideshow summarising my frustrations and what I would suggest people do instead: https://docs.google.com/presentation/d/1EYyS4AtRhNa6i299R_RB0a54ePU2PEQsILQnwWgY9KU/edit?usp=sharing https://docs.google.com/presentation/d/1EYyS4AtRhNa6i299R_RB...
  - afro88 9y ago
    The ideas in this presentation are great. Thanks for posting
    - xrd 9y ago
      Agreed, really stimulating and well put. Thanks for sharing.
      - ronjouch 9y ago
        Indeed, great presentation, thanks for sharing :)
JepZ 9y ago
My personal image about this topic is the fact that the general relativity theory is more advanced than the special relativty theory. It reminds me that creating something more general is harder than creating something specialized for a specific problem. For programmers this seems to be obvious, but for others this is counter intuitive.
- marcosdumay 9y ago
  For anybody that ever used some kind of math, it is obvious. But "counter intuitive" could be Mathematics middle name.
thriftwy 9y ago
For me, general and surprising things are coming in the form of "X doesn't seem to work". The reasoning goes like this: X is described like a good thing and a lot of resources poured in it. But there are documented failures of X - are they spurious or causes by X not actually doing what is advertised? More often than not the working answer "it probably doesn't work" which is often "a thing you can not say" due to general consensus, good feelings and above mentioned amount of resources poured in X.
- jakubp 9y ago
  Can you give examples of such things that probably don't work, but you can't say it if you don't want to be ostracized?
  - thriftwy 9y ago
    For example, Yugoslavia had what was described as the most progressive ethnic minorities policy in the world (regarding official languages use, representation of minorities, etc, etc), and then it violently fell apart in a series of ethnic conflicts.
  - deleted 9y ago
    [deleted]
jackblack8989 9y ago
This guy pretends to be an intellectual but is the biggest wannabe I know. He tried to block the latest Steve Jobs film because it showed how much of an asshole Jobs was
- 123lalala123 9y ago
  The guy, ironically enough, is the living proof that capitalism doesn't reward talent.
mack1001 9y ago
PG makes it all look so neat and tight, however he fails to mention the number of serendipitous instances that drive insights and new ideas.
- jamesrcole 9y ago
  I think the essay is intended to describe one approach, not to provide a complete account of how you can get good ideas.
ismail 9y ago
Only slightly related but there is a great book for your children: "What to do you do with an idea" https://www.amazon.com/What-Do-You-Idea/dp/1938298071 https://www.amazon.com/What-Do-You-Idea/dp/1938298071
- almostarockstar 9y ago
  What should one do?
csomar 9y ago
Paul Graham is running out of ideas these days. (Partly joking). This goes on to my same discovery: Life itself is a constant process of incremental trial and error builds-up. Over a few billions years, you get something out of it (human being playing snapchat) If you keep counting from 0 to an infinitely large number, at some numbers you'll "discover": Windows 7, Windows 7 Korean Language, Microsoft Office, OSX, and every other game that human designed and developed. Plus a bunch of other software from alternate realities that compile on our machines. Cool. The process the human are doing is not much different (albeit one can argue that it is much more efficient than stupidly counting) "Everything" is already there. The new Audi A4? It was already "possible". If there is one thing that will change the future, it is our discovery of the link between "abstract" and "real" concepts. They share some important points that I'd not be surprised that "real" and "abstract" are the same thing/continuum. Like space and time.
- edanm 9y ago
  That's an interesting thought. Just want to point out that complexity theory from Computer Science fits nicely into this, because it asks, not what is "possible" (e.g. finding all programs by counting all the integers), but what is possible within limited resources, like e.g. time. A nice addition to the thought that, theoretically, even if you can find the Windows 10 source code by counting all integers, you can't actually do it within the lifetime of the universe without a smarter algorithm.
  - csomar 9y ago
    More like infinite resources. But there is a lifetime for the universe?
- hammock 9y ago
  Before you lay one thousand blocks of marble. Inside every one is a world famous sculpture. Eventually you will find it...
- andrewflnr 9y ago
  > keep counting You can always find the successor of a given integer, but there's not really an analogous process for human discovery, where it's easy to go in circles. The process really is that different. Are there theoretical situations where random searches are guaranteed to be complete on infinite spaces given "infinite time" (probability of finding solution approaches 1)?
coldtea 9y ago
>Thanks to Sam Altman, Patrick Collison, and Jessica Livingston for reading drafts of this. This (a very basic 400 word essay/post) needed "drafts" and 3 people going through them? I like some of PG's essays, but this one just has a couple of trivial insights.
- HiroshiSan 9y ago
  Yes it's called writing.
  - coldtea 9y ago
    Wooosh. That would be a relevant answer if I had asked about the practice of writing drafts and having friends go through them in general. My point was different: several drafts and editors were needed for this tiny and banal result? It's like seeing someone ordering a $10,000 AGA cooker, buying all kinds of spices, herbs and fresh ingredients, amassing several Kitchenaid appliances, getting into an apron and a chef's hat, and then proceed to make ...a grilled cheese sandwich.
    - zimpenfish 9y ago
      This isn't even a grilled cheese sandwich. It's not even toast. It's just a slice of slightly stale Wonderbread.
    - corpMaverick 9y ago
      May be if the sandwich was going to be consumed by one person. But this "sandwich" will be read by thousands. Well worth the effort of three reviewers.
      - coldtea 9y ago
        If you expect thousands, and have 3 reviewers, at least come up with a decent sandwich. Or, failing that, just don't show anything at all until you got something good.
    - 9y ago
amelius 9y ago
Too abstract to be useful.
lr4444lr 9y ago
I'd say this post was "general, not surprising".
kadenshep 9y ago
Paul Graham doing what he does best. Pseudo-philosophical musings with no basis or any kind of rigorous pass through. Why is this gaining visibility? If anyone else had posted this it'd never see the light of day on here.
- dullgiulio 9y ago
  From gospel of Hacker News, blog post. May the Founder be with you!
- alexashka 9y ago
  I found it to be a nice little reminder of 'don't worry about being original, keep digging and maybe there'll be a gem there someday'. The simple things are good to be reminded of, on a regular basis, that's why :)
ballenf 9y ago
> It's not true that there's nothing new under the sun. There are some domains where there's almost nothing new. But there's a big difference between nothing and almost nothing, when it's multiplied by the area under the sun. I found that statement the most insightful of the essay. Helped me resolve some cognitive dissonance. Whether there really are totally new things and not just existing things in new packaging depends on which level of abstraction you stop at with when analyzing. For example, at a very high level you could say that the Internet as a whole is really just a very advanced/efficient printing press. (And automation like the printing press or assembly line, just efficiently organized production.) That level of abstraction may be less than helpful, unless there are insights to be gleaned from where the internet is going by looking at printing press technology. Their impacts on empowering the masses are similar, but they each encouraged/enabled network effects to consolidate much power (newspapers and goog/fb) and thereby effectively countering much of the effect.
- wcarss 9y ago
  I agree, this statement seems like the most interesting one in the essay. It reminds me of expected value calculations: extremely low probability events of tremendously high impact could actually out-rate moderately probable events of moderate impact in terms of how concerned you should be with them. Assigning meaningful probabilities and impact values can be tricky though, which adds a whole layer of complexity and hand-waving to the problem. This is where discussions of nuclear war, strong AI, nazi-oriented political resurgences, geothermal storms, epidemics, etc. can really go off the rails in my opinion: two people can separately evaluate "worst-case scenario" as very different realities, assigning very different intuited impact-values, and also assign very different probability values. Even if they think about these things from an expected-value perspective, a discussion between them becomes at least a 2-variable linear equation, and the odds of even understanding each other are stretched, let alone finding the same x and y and agreeing upon what that means for the actions we should take in response. Graham's point here on the positive side though is a refreshing step outside of that domain. An idea recycled, if broadly applicable enough, only needs a hint of novelty. It gives me some hope for being able to come up with useful things.
  - deleted 9y ago
happy-go-lucky 9y ago
The essay itself is general and surprising, and the combination never loses its effectiveness. Ideas keep evolving. There was nothing new about the idea of a search engine when Google came into existence in 1998. In 1945, Dr. Vannevar Bush wrote an article titled As We May Think: https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/ https://www.theatlantic.com/magazine/archive/1945/07/as-we-m... > he urges that men of science should then turn to the massive task of making more accessible our bewildering store of knowledge. In the 1960’s, Gerard Salton created and developed Salton’s Magic Automatic Retriever of Text (SMART). He also authored a book called A Theory of Indexing detailing his initial tests that search is largely based off of relevancy algorithms. Here’s a very interesting blog post titled Search Down Memory Lane by Tom Evslin who worked with Salton during this project: http://blog.tomevslin.com/2006/01/search_down_mem.html http://blog.tomevslin.com/2006/01/search_down_mem.html Fast forward to 1990, a man named Alan Emtage created the first search engine known as Archie which retrieved a database files by matching user queries using regexes. Following the growth of Archie’s popularity, two similar search engines Veronica and Jughead were created and they started indexing plain text files. In 1993, the first bot called World Wide Web Wanderer was created and then it was upgraded to capture active URLs and store them in a database. Then came ALIWEB (Archie-Like Indexing of the Web), which crawled meta information of pages. There’re others including WebCrawler (1994), Lycos(1994), AltaVista(1995), Excite(1995), Yahoo(1995), Dogpile(1996), Ask Jeeves(1996). This is just my superficial understanding of how the idea of search engines was born and has evolved over time. They were very different from Google, but they’ve similarities to how data is processed and analyzed today.
- roymurdock 9y ago
  What general ideas did we sow in the past 20-30 years that, through minor iterations, are coming to (surprising) fruition today?
  - santaclaus 9y ago
    Machine learning
    - 9y ago
brlewis 9y ago
At the beginning: The most valuable insights... Toward the end: So it's doubly important not to let yourself be discouraged by people who say there's not much new about something you've discovered. "Not much new" is a real achievement when you're talking about the most general ideas. Maybe if you keep going, you'll discover more. I really feel like Paul is conflating valuable and novel here. When you create something that solves someone's problem, who cares whether the ideas behind it are novel or not?
- robertlagrant 9y ago
  This is a good hackers and painters juxtaposition: in STEM, it doesn't have to be novel as long as it is better. In art it doesn't have to be better as long as it's novel :-)
golemotron 9y ago
I think Nassim Nicholas Taleb's work epitomizes the "general yet surprising" territory.
tw1010 9y ago
When I read stuff like this and reflect on the fact that this is about the peak quality of the philosophical discussion that is going on in our little niche of the world, I get slightly depressed. I get that it's hard to find people at the intersection of astounding writing and technical chops, but still, it makes me want to distance myself and just go back to reading the classics.
- deleted 9y ago
  [deleted]
- jessriedel 9y ago
  What community centered around professional skills do you think has better philosophical discussion? Sure, there exists better philosophical discussion elsewhere, and communities centered around academic disciplines can be more sophisticated because that is their raison d'etre. But when I think of professional communities (e.g., accounting, medicine, law, etc.), I can't think of any better except insofar as they bleed into academia (e.g., blogs by law professors, not practicing lawyers).
  - m_fayer 9y ago
    The medical field is an embarrassment of riches when it comes to talented essayists, journalists, and biographers. I've also read a few wonderful biographies written by restauranteurs. Our field unfortunately lags behind, I think partially because it's rather dry compared to other fields, and partially because SV ideology is pervasive, stifling, and indifferent to many swathes of the human experience.
    - jessriedel 9y ago
      I don't necessarily disagree, but little of that falls under the heading of "philosophical discussion". Medicine is romantic and salient to laymen, so of course journalists are going to write more biographies and whatnot for them on that topic. I almost never find such pieces philosophically insightful, and they aren't competing with Paul Graham essays.
    - 9y ago
clavalle 9y ago
An unfortunate corollary is that slightly different general ideas are very difficult to get people to listen to because they feel like they already know it and it is not much new or the difference is negligible. This might be true of any particular case someone might think of but when the entire landscape is taken into account the multiplier pg mentions applies and can change things dramatically. But people don't seem to be well wired to think of the general. And for any general multiplier to work, people generally need to be on board. For example: there is a difference between economic inequality and desperate economic inequality -- the first is a mere difference in ownership and control. The second is /also/ a difference in ownership and control but that leaves one party or group bargaining against their physical well-being. There is much said about mere economic inequality but, by leaving out the desperation factor, the public conversation is tied up in knots (e.g. "Well why don't we give everyone a million dollars?!" nonsense) and the real problem, desperation and the conditions that lead to that state, persist despite having the tools to potentially tackle it. But any single example -- a single mother that has to take two minimum wage jobs to feed her kids -- can be reduced away. She could just do x, y, or z and her particular problem would be solved and she'd still be economically disadvantaged but not desperate to the point of worrying about her kids starving. And the conversation ends. We are back to the inconvenience of a weak bargaining position which is also described by the phrase 'economic inequality' and it all feels like the same, well trod and religiously guarded ground.
rehevkor5 9y ago
F=ma is surprising? We took two values and combined them to define a new one. You can define all sorts of new things this way, how is that surprising?
- alexashka 9y ago
  You're making a valid point in a not-so-nice way. What he meant was F=ma combined with other formulas that make use of F, make up a system that's surprisingly useful while remaining simple to use. I imagine you already knew that. If you're acting as an editor, you can phrase your point better. if you're merely nitpicking for things to 'gotcha' someone - I assure you that's not a smart longterm strategy :)
nate 9y ago
On the "surprising" part of this - Murray Davis had some related research and writing about being interesting: https://proseminarcrossnationalstudies.files.wordpress.com/2009/11/thatsinteresting_1971.pdf https://proseminarcrossnationalstudies.files.wordpress.com/2... "An audience finds a proposition ‘interesting’ not because it tells them some truth they thought they already knew, but instead because it tells them some truth they thought they already knew was wrong."
peterwwillis 9y ago
tl;dr if you have an idea that isn't very "new" then keep working on it until it is "more new" There should be a twitter account that just summarizes rambling think pieces.
raldi 9y ago
What are some examples of past insights someone has made by discovering something slightly surprising about the general world?
hnaccy 9y ago
This post is lacking in the latter.
kantian 9y ago
Might a parallell be draw to Kant here? General and bland would be as akin to "analytic a priori". All bachelors are unmarried, that sorta thing. Specific and surprising would be like "synthetic posteriori". Such and such plant is blue and grows in the Himalayas. The real gold would be in the "synthetic a priori" findings. Blending the generalness of logic with the newness of empiricism. Like figuring out geometrical laws from observing nature. New information AND far-reaching implications. Am I on to something here? https://en.wikipedia.org/wiki/Analytic%E2%80%93synthetic_distinction https://en.wikipedia.org/wiki/Analytic%E2%80%93synthetic_dis...
- kantian 9y ago
  https://plato.stanford.edu/entries/analytic-synthetic/ https://plato.stanford.edu/entries/analytic-synthetic/ "While some trivial a priori claims might be analytic in this sense, for Kant the seriously interesting ones were synthetic."
- kantian 9y ago
  Synthetic a priori = getting epistemological alpha (to speak financese) :)
d_burfoot 9y ago
A couple of years ago I wrote a book about a new philosophy of science based on the following methodology: - Obtain a large dataset of observations related to a phenomenon of interest (eg English text, bioimaging data, economic reports, car-mounted camera recordings, etc) - Develop a theory of the phenomenon, or revise a previous theory - Build the theory into a lossless data compression program - Score the theory by invoking the compressor on the dataset, taking into account the size of the compressor itself; lower codelength is better. - Adopt or discard the theory as appropriate and return to step #2. I believe that this is a valid variant of the traditional scientific method, in which data compression takes the place of experimental prediction. In Graham's terms, this idea is a small delta of novelty, but since it refers to a predecessor idea of tremendous significance and generality, it could be very important. The book is available here: https://arxiv.org/abs/1104.5466 https://arxiv.org/abs/1104.5466 I've spent a few years pursuing this methodology in the domain of English text. The result is a sentence parser, which works quite well, but was built completely without the use of hand-labelled training data (eg the Penn Treebank). You can check out the parser here: http://ozoraresearch.com/crm/public/parseview/UserParseView.jsp http://ozoraresearch.com/crm/public/parseview/UserParseView.... I'm always happy to talk to people about these ideas, ping me at daniel dot burfoot at gmail if you're interested.
- jsweojtj 9y ago
  This is interesting. I don't have an intuition on how well this could work on different problems. Do you think this approach would work for something, let's start simple, like ballistics neglecting rotation of the Earth and drag? If you fed in thousands of starting velocities and angles, and the resulting distance traveled, will this process end up with equations of motion that are as compact as can be found in first year physics?
  - indiv0 9y ago
    I think the problem here is that you are still the one that needs to develop the equations of motion that will be tested for compression. What this describes is a method for testing theories, not generating them. You still need to study the data to try and properly analyze it so you can understand the rules yourself, or at least be able to see the edges of them. Now if you could find a way to generate moderately accurate equations automatically by analyzing the dataset, I think that would be a real leap forward. Is there a related mechanism to the one presented by the parent which could be used to generate theories?
dasil003 9y ago
I think pg is falling prey to the whole post-hoc founder story phenomenon here. He sees so many startups, he hears so many stories about them, and he works hard to distill it into some kind of actionable and scalable investment strategy. The problem is that the facts which led to the success of the business are not the same thing as the best insights a founder can come up with about why it succeeded. The latter is subject to an incredible amount of confirmation bias and the simple desire to tell a good story—in fact you have to do that to get investment. In reality, novelty of ideas is meaningless, because the massively overwhelming majority of ideas in the world never get executed on. Even within contexts where people and companies have the means, most ideas—even good ones—don't get executed on because they conflict with other ideas and don't win the battle for resources. What matters for an idea is execution and context. If you bring the right resources: skills, connections, go-to-market strategy, smaller supporting ideas, and it gels with the current zeitgeist of whatever market you are in, then it can get some traction. From there you learn and gain more insights which can be leveraged to scale up. You do this over and over again, and if you are lucky you can make an 8, 9, 10-figure company. By the time all this happens you'll have some tremendous insights, but they will be specific to the context in which they were gleaned. The "generality" we observe of these insights is just a result of our pattern-matching and story-telling brains. The "surprisingness" comes after the fact to varying degrees based on existing preconceptions and perhaps how the world has changed over time. In other words, I don't think there is such a thing as a hugely valuable insight per se, rather this: we have insights, we believe in them to varying degrees, and based on how well it maps to our actions and their results, we later declare the "value" of the insight.
burger_moon 9y ago
I'm glad Paul is still writing essays. It seems it has dropped off in frequency, this essay kind of hints at one such reason for that. I hope he continues to write even if people disagree with his ideas or if they are repetitive.
- tw1010 9y ago
  I commented a post with a negative sentiment below, but I agree with you. A lot of his ideas and insights are valuable and thought provoking, and I wouldn't want the tone of these comments to make him stop.
- saycheese 9y ago
  On the same note, PG's "official" HN account hasn't been active for years: https://news.ycombinator.com/threads?id=pg https://news.ycombinator.com/threads?id=pg
  - ComodoHacker 9y ago
    One year ten months, to be precise.
    - dsacco 9y ago
      His account now makes comments once every year or year and a half, but it hasn't been generally active for nearly 3.5 years (the last bunch of comments in a row in a short time period was 1264 days ago, as of this writing).
deleted 9y ago
[deleted]
robertlagrant 9y ago
Is this a stealth "I'm done with the small delta of novelness Arc already has and am stopping on it. If you can't appreciate that then you need to look harder" article? :-)
trapperkeeper74 9y ago
Diff eq-like behavior is everywhere: species population, popularity, technology, market saturation, economics, spring rates, etc. (Malthus was partially ;) mistaken.)
graphememes 9y ago
> It's not true that there's nothing new under the sun. Philosophically this statement is false. Mental health wise, it's a lot nicer to keep in mind than the truth of reality that you reside within.
- msla 9y ago
  Hm. Can something be philosophically false but factually correct?
  - graphememes 9y ago
    Love is an example I can think of.
outworlder 9y ago
> It's not true that there's nothing new under the sun. There are some domains where there's almost nothing new. But there's a big difference between nothing and almost nothing, when it's multiplied by the area under the sun. PG, I see what you did there. The rest of the essay is just a setup for this single paragraph. This is your own small delta of novelty. Well done.
grappler 9y ago
Clearly Paul Graham is defending the Bodega vending machine here ;)
lotusko 9y ago
It's very resonant,ths for sharing.

417 points by maguay 9y ago

jacalata 9y ago

Well, that was certainly general.

enriquto 9y ago
...and not very surprising. Thus, platitude! :)
- deleted 9y ago
  [deleted]
- jacalata 9y ago
  Indeed. The number of votes this has received is an amusing insight into how many people must upvote based on origin and not content. If this were published on some anonymous medium blog, it would be lucky to be +2.
  - hasenj 9y ago
    It sounds like a clip from some kind of a speech. There's some missing context. or, there's hardly any content. Like, I'm not sure what he's trying to get at.
  - tinco 9y ago
    Not everything that is a platitude is necessarily bad. Reinforcing, clarifying or rediscovering knowledge certainly has value. If Paul Graham writes an essay then that is relevant to Hacker News so it should be upvoted regardless of whether it is shit.
    - jacalata 9y ago
      So...you are agreeing with me that this has been upvoted based on origin and not content, and you're just repeating what I said to reinforce the point?
      - zaptheimpaler 9y ago
        Thou shalt not take PG's name in vain.
      - tinco 9y ago
        Origin and context give extra meaning and weight to content. That seems trivially true to me. If Paul Graham writes this then that means the topic is on the mind of someone influential and important in our community, thus it is worthy of scrutiny and thought, thus worthy of an upvote. Hacker News is not some game where people compete to create content with maximum merit. It is a source of new content relevant to this community.
  - WA 9y ago
    Nothing new here. The messenger always plays an important role.

quuquuquu 9y ago

It's funny, a lot of my imposter syndrome comes from an anxiety about not having massive "deltas of novelty", as PG would call it. I guess all I can do now is hope that whoever is listening can see the value in incremental change :)

ismail 9y ago
Same here, you constantly wonder if what your thinking is truly unique. I believe the key is figuring out when your view is unique. What I have been doing is asking people questions on their view of a problem or something I am thinking about and looking at the deltas there. I have not thought about it with the term delta. Maybe looking for deltas would be another addition to the toolkit when searching for insights. I prefer to use the term contradictions. Sometimes that leads you to further insights.
- heavenlyblue 9y ago
  And you spend most of your life trying to prove how special you are, instead of productively producing things and playing with the world that is also shaping them? People tend to forget that ideas are worth nothing. It's not that hard to build a model of the world which would be parametrisable for any possible outcome. Then those parameters would be defined by your experience, which requires you to get out of your mind and earn on the outside.
  - ismail 9y ago
    It seems my comment was not precise enough. The result is the following assumptions being made: 1. The thinking/insight/ideas is in pursuit of proving my own special nature 2. The thinking/insights/ideas are purely theories and have no practical application Notice that the word "problem" was used. What is meant here an actual problem that I am trying to solve with actual practical implications. Let me clarify what I mean, using concrete examples: In product development/startups/business space: "My customers are telling me their problem is X, but what they are doing or how they are doing it is Contradictory. Why is this?" "Two customers have completely different and contradictory views on what feature/problem is important to them, why is this?" The actual purpose is to understand your customer better so you can cater for what they actually need. This part is confusing, can you clarify what you mean: "Then those parameters would be defined by your experience, which requires you to get out of your mind and earn on the outside"

yeukhon 9y ago

[T]he more general the ideas you're talking about, the less you should worry about repeating yourself. Organisms are used to train over and over to get better at things. Certain abilities are innate, like a calf knows to stand up as soon as out of the womb. But to stand firmly the calf has to keep trying. When it comes to an idea, I believe that the broader I present the idea, the easier it is to start a conversation, because others are more likely to show interest, and eventually reach the point you find a focus point. However, I still repeat myself, because I wouldn't invent a whole new dialogue next time I bring up the same broad idea with another person. On the other hand, I feel developing a product (startup or not), the more general the idea is, the harder for me to explain my ideas concretely, much like films have leading roles. The ideas present by the film can be broad, but you don't want camera rolling on 200 actors in 1 hour 30 minutes, would you? Your message can be interesting and novel, but I am going to be lost. But yeah, on HN topics like "I quite FB", "security is hard", "Google is evil", etc are common here, but I still comment on those topics when I feel like to. My responses are usually similar, but I might add new thoughts, or rephrase what I had written before. I called this introspection. I believe we are not comfortable repeating ourselves, because we want to be interesting and original, but we still have the urge to try to perfect our speech the next time someone asks.

fiokoden 9y ago

A very academic / theoretical post. Not entirely sure the point. It might have been easier to understand if bound to real world examples.

harscoat 9y ago

"surprising" in patent law https://en.wikipedia.org/wiki/Inventive_step_and_non-obviousness https://en.wikipedia.org/wiki/Inventive_step_and_non-obvious...

deleted 9y ago

[deleted]

stonesixone 9y ago

There's also a class of valuable insights that are general and not surprising, but rather obvious. For these, the challenge is that no one previously had the insight to observe and state what is obvious. Maybe the surprise for these isn't in the truth but rather that no one had stated them before.

paulajohnson 9y ago

The essay describes a 2x2 grid with axes of generality and surprise. Things that are suprising and general are valuable. Things that are general but unsurprising are platitudes, and things that are specific but surprising are gossip. The essay itself is a platitude.

zaptheimpaler 9y ago
Brutal but accurate summary! I get the feeling there is something he is not (or cannot) saying that prompted the essay.
- Kluny 9y ago
  Yeah, it has a bit of a vaguebooking feel!
  - mattacular 9y ago
    I can't see the exact published date but it strikes me as a rebuff to critics of the iPhone X unveil
bambax 9y ago
It misses an important element that may seem obvious but isn't: truth. Some insights are indeed, general and surprising, but when examined closely, turn out to be false. That's what happened to many of the ideas presented by Daniel Kahneman in Thinking, fast and slow. He obviously aimed for "general and surprising" but accepted as solid results the outcome of lone, irreproducible experiments. We (our current culture -- it wasn't always like that) love originality and "surprises", but as a rule, the more surprising the result, the more scrutiny it should withstand.
- tlb 9y ago
  Circulating ideas where people disagree about whether or not they're true is an excellent way to discover things. Not just whether the particular ideas true, but why or why not, and where the uncertainty comes from. In your example, perhaps the most lasting value of priming studies is a much greater understanding of how p-hacking actually misleads us. Everyone knew that it could in theory, but it was general and surprising that entire fields could have an apparent scientific consensus entirely based on p-hacking and file drawer effects.

QuantumRoar 9y ago

This is not true as often you find surprising things that are not general which ARE NOT GOSSIP. To stay with the physics theme, the Schwarzschild solution of general relativity is a very special one and I don't think anybody thinks that black holes are gossip. And it is certainly not surprising that amateurs in general forget that F = ma is only valid if the mass does not change. The more general expression is F = dp/dt, where p is the momentum. But this, of course, is also only valid in inertial systems. It's not really important to the article but it does kind of annoy me that he uses the most special case of an expression in an argument about it being general. He could have actually made the point about generality by comparing this expression to the most general one for the force (in a frame of reference that accelerates). That would have also shown why generality can quickly become infeasible in practice. If he knew how many approximations people make in the real world, not because they want to, but because they HAVE TO, his worldview might be a different one. I feel like he's trying to make a point about a very specific scenario but doesn't mention it explicitly. Instead he tries to be general and therefore fails to understand that his view doesn't actually apply in general.

Danihan 9y ago
You should have read the draft of this, instead of the people who did.
oldandtired 9y ago
Theoretical entities called "black holes" cannot exist in finite universe (no matter the perspective changes that are applied). The result (if I understand it correctly) applies to a universe in which there is a single massive body and universe is eternal. Hence, if time dilation exists (under increasing levels of gravity) "black hole" cannot form in any finite time. Since there are many massive bodies in our universe, solution to problem is not applicable. So, in that context, "black holes" ARE GOSSIP!!!! Let the "GOSSIP" wars commence. Other than that, I think you make some very salient points, especially about the approximations that we make and the validity of those approximations to the specific applications/situations being studied.
- QuantumRoar 9y ago

econ1212 9y ago

F = ma is interesting because there are universal forces like electricity, gravitation and electromagnetism that we use for our daily life. Moreover there is a marriage between maths and physics since acceleration is the second derivative of space with respect of time. If you are a new Newton you should create the new concepts and develop the new maths to create a surprising step forward. Looking for a word that goes from gossip to F=ma is like using a telescope to watch a worm. Concepts are about frameworks and using a right scale to focus concepts.

w_t_payne 9y ago

There are also the platitudes that everybody knows but that nobody does anything about.... Oftentimes there is virtue in simply being unusually conscientious.

matt4077 9y ago

This genre of writing is also neatly categorised into one of three categories: Obvious, unactionable, or wrong. PG sometimes meanders into actual meaning. But the likes of Tim Ferris or Seth Godin absolutely excel at this drivel.

roceasta 9y ago

>that territory tends to be picked clean, precisely because those insights are so valuable People often think that a field has been 'picked clean' and conclude that either no further progress is possible or that further discoveries are going to be rare and incremental. e.g. "the future truths of physical science are to be looked for in the sixth place of decimals" -- Michelson, 1894 But there are always deep problems and this implies that unlimited progress is possible. The really big discoveries can happen at any time and are of an unpredictable nature, not least because they affect multiple fields. They leave behind plenty of smaller problems for everyone to pick up.

andrewflnr 9y ago
That's especially funny from the same Michelson who did the Michelson-Morley experiment that basically kicked off relativity.

perpetualcrayon 9y ago

"the more general the ideas you're talking about, the less you should worry about repeating yourself" A rule-of-thumb I've generally followed when brainstorming is "never write down the ideas". And if at all possible, try to forget them, good or bad. It inevitably forces me to re-think the ideas "from scratch". I've never been worried that an idea would escape me or that I would forget it, as I think the exercise of re-thinking ideas from scratch has helped me build a relatively solid mental model of the world. I feel that, with practice, this method of brainstorming has started bringing me to the solid / useful (general / surprising) ideas faster. It has likely, over the years, added weeks or possibly months worth of time that I've spent brainstorming the same (general) ideas over and over.

vanderZwan 9y ago
As an aside, I can't say for sure how you use the term "brainstorming" here, but there is plenty of evidence that classic-style brainstorming in groups leads to less ideas, and less creative ideas. When I was forced by my boss to teach designers brainstorming, against my better judgement, I instead made this slideshow summarising my frustrations and what I would suggest people do instead: https://docs.google.com/presentation/d/1EYyS4AtRhNa6i299R_RB0a54ePU2PEQsILQnwWgY9KU/edit?usp=sharing https://docs.google.com/presentation/d/1EYyS4AtRhNa6i299R_RB...
- afro88 9y ago
  The ideas in this presentation are great. Thanks for posting
  - xrd 9y ago
    Agreed, really stimulating and well put. Thanks for sharing.
    - ronjouch 9y ago
      Indeed, great presentation, thanks for sharing :)

JepZ 9y ago

My personal image about this topic is the fact that the general relativity theory is more advanced than the special relativty theory. It reminds me that creating something more general is harder than creating something specialized for a specific problem. For programmers this seems to be obvious, but for others this is counter intuitive.

marcosdumay 9y ago
For anybody that ever used some kind of math, it is obvious. But "counter intuitive" could be Mathematics middle name.

thriftwy 9y ago

For me, general and surprising things are coming in the form of "X doesn't seem to work". The reasoning goes like this: X is described like a good thing and a lot of resources poured in it. But there are documented failures of X - are they spurious or causes by X not actually doing what is advertised? More often than not the working answer "it probably doesn't work" which is often "a thing you can not say" due to general consensus, good feelings and above mentioned amount of resources poured in X.

jakubp 9y ago
Can you give examples of such things that probably don't work, but you can't say it if you don't want to be ostracized?
- thriftwy 9y ago
  For example, Yugoslavia had what was described as the most progressive ethnic minorities policy in the world (regarding official languages use, representation of minorities, etc, etc), and then it violently fell apart in a series of ethnic conflicts.
- deleted 9y ago
  [deleted]

jackblack8989 9y ago

This guy pretends to be an intellectual but is the biggest wannabe I know. He tried to block the latest Steve Jobs film because it showed how much of an asshole Jobs was

123lalala123 9y ago
The guy, ironically enough, is the living proof that capitalism doesn't reward talent.

mack1001 9y ago

PG makes it all look so neat and tight, however he fails to mention the number of serendipitous instances that drive insights and new ideas.

jamesrcole 9y ago
I think the essay is intended to describe one approach, not to provide a complete account of how you can get good ideas.

ismail 9y ago

Only slightly related but there is a great book for your children: "What to do you do with an idea" https://www.amazon.com/What-Do-You-Idea/dp/1938298071 https://www.amazon.com/What-Do-You-Idea/dp/1938298071

almostarockstar 9y ago
What should one do?

csomar 9y ago

Paul Graham is running out of ideas these days. (Partly joking). This goes on to my same discovery: Life itself is a constant process of incremental trial and error builds-up. Over a few billions years, you get something out of it (human being playing snapchat) If you keep counting from 0 to an infinitely large number, at some numbers you'll "discover": Windows 7, Windows 7 Korean Language, Microsoft Office, OSX, and every other game that human designed and developed. Plus a bunch of other software from alternate realities that compile on our machines. Cool. The process the human are doing is not much different (albeit one can argue that it is much more efficient than stupidly counting) "Everything" is already there. The new Audi A4? It was already "possible". If there is one thing that will change the future, it is our discovery of the link between "abstract" and "real" concepts. They share some important points that I'd not be surprised that "real" and "abstract" are the same thing/continuum. Like space and time.

edanm 9y ago
That's an interesting thought. Just want to point out that complexity theory from Computer Science fits nicely into this, because it asks, not what is "possible" (e.g. finding all programs by counting all the integers), but what is possible within limited resources, like e.g. time. A nice addition to the thought that, theoretically, even if you can find the Windows 10 source code by counting all integers, you can't actually do it within the lifetime of the universe without a smarter algorithm.
- csomar 9y ago
  More like infinite resources. But there is a lifetime for the universe?
hammock 9y ago
Before you lay one thousand blocks of marble. Inside every one is a world famous sculpture. Eventually you will find it...
andrewflnr 9y ago
> keep counting You can always find the successor of a given integer, but there's not really an analogous process for human discovery, where it's easy to go in circles. The process really is that different. Are there theoretical situations where random searches are guaranteed to be complete on infinite spaces given "infinite time" (probability of finding solution approaches 1)?

coldtea 9y ago

>Thanks to Sam Altman, Patrick Collison, and Jessica Livingston for reading drafts of this. This (a very basic 400 word essay/post) needed "drafts" and 3 people going through them? I like some of PG's essays, but this one just has a couple of trivial insights.

HiroshiSan 9y ago
Yes it's called writing.
- coldtea 9y ago
  Wooosh. That would be a relevant answer if I had asked about the practice of writing drafts and having friends go through them in general. My point was different: several drafts and editors were needed for this tiny and banal result? It's like seeing someone ordering a $10,000 AGA cooker, buying all kinds of spices, herbs and fresh ingredients, amassing several Kitchenaid appliances, getting into an apron and a chef's hat, and then proceed to make ...a grilled cheese sandwich.
  - zimpenfish 9y ago
    This isn't even a grilled cheese sandwich. It's not even toast. It's just a slice of slightly stale Wonderbread.
  - corpMaverick 9y ago
    May be if the sandwich was going to be consumed by one person. But this "sandwich" will be read by thousands. Well worth the effort of three reviewers.
    - coldtea 9y ago
      If you expect thousands, and have 3 reviewers, at least come up with a decent sandwich. Or, failing that, just don't show anything at all until you got something good.
  - 9y ago

amelius 9y ago

Too abstract to be useful.

lr4444lr 9y ago

I'd say this post was "general, not surprising".

kadenshep 9y ago

Paul Graham doing what he does best. Pseudo-philosophical musings with no basis or any kind of rigorous pass through. Why is this gaining visibility? If anyone else had posted this it'd never see the light of day on here.

dullgiulio 9y ago
From gospel of Hacker News, blog post. May the Founder be with you!
alexashka 9y ago
I found it to be a nice little reminder of 'don't worry about being original, keep digging and maybe there'll be a gem there someday'. The simple things are good to be reminded of, on a regular basis, that's why :)

ballenf 9y ago

> It's not true that there's nothing new under the sun. There are some domains where there's almost nothing new. But there's a big difference between nothing and almost nothing, when it's multiplied by the area under the sun. I found that statement the most insightful of the essay. Helped me resolve some cognitive dissonance. Whether there really are totally new things and not just existing things in new packaging depends on which level of abstraction you stop at with when analyzing. For example, at a very high level you could say that the Internet as a whole is really just a very advanced/efficient printing press. (And automation like the printing press or assembly line, just efficiently organized production.) That level of abstraction may be less than helpful, unless there are insights to be gleaned from where the internet is going by looking at printing press technology. Their impacts on empowering the masses are similar, but they each encouraged/enabled network effects to consolidate much power (newspapers and goog/fb) and thereby effectively countering much of the effect.

wcarss 9y ago
I agree, this statement seems like the most interesting one in the essay. It reminds me of expected value calculations: extremely low probability events of tremendously high impact could actually out-rate moderately probable events of moderate impact in terms of how concerned you should be with them. Assigning meaningful probabilities and impact values can be tricky though, which adds a whole layer of complexity and hand-waving to the problem. This is where discussions of nuclear war, strong AI, nazi-oriented political resurgences, geothermal storms, epidemics, etc. can really go off the rails in my opinion: two people can separately evaluate "worst-case scenario" as very different realities, assigning very different intuited impact-values, and also assign very different probability values. Even if they think about these things from an expected-value perspective, a discussion between them becomes at least a 2-variable linear equation, and the odds of even understanding each other are stretched, let alone finding the same x and y and agreeing upon what that means for the actions we should take in response. Graham's point here on the positive side though is a refreshing step outside of that domain. An idea recycled, if broadly applicable enough, only needs a hint of novelty. It gives me some hope for being able to come up with useful things.
- deleted 9y ago

happy-go-lucky 9y ago

The essay itself is general and surprising, and the combination never loses its effectiveness. Ideas keep evolving. There was nothing new about the idea of a search engine when Google came into existence in 1998. In 1945, Dr. Vannevar Bush wrote an article titled As We May Think: https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/ https://www.theatlantic.com/magazine/archive/1945/07/as-we-m... > he urges that men of science should then turn to the massive task of making more accessible our bewildering store of knowledge. In the 1960’s, Gerard Salton created and developed Salton’s Magic Automatic Retriever of Text (SMART). He also authored a book called A Theory of Indexing detailing his initial tests that search is largely based off of relevancy algorithms. Here’s a very interesting blog post titled Search Down Memory Lane by Tom Evslin who worked with Salton during this project: http://blog.tomevslin.com/2006/01/search_down_mem.html http://blog.tomevslin.com/2006/01/search_down_mem.html Fast forward to 1990, a man named Alan Emtage created the first search engine known as Archie which retrieved a database files by matching user queries using regexes. Following the growth of Archie’s popularity, two similar search engines Veronica and Jughead were created and they started indexing plain text files. In 1993, the first bot called World Wide Web Wanderer was created and then it was upgraded to capture active URLs and store them in a database. Then came ALIWEB (Archie-Like Indexing of the Web), which crawled meta information of pages. There’re others including WebCrawler (1994), Lycos(1994), AltaVista(1995), Excite(1995), Yahoo(1995), Dogpile(1996), Ask Jeeves(1996). This is just my superficial understanding of how the idea of search engines was born and has evolved over time. They were very different from Google, but they’ve similarities to how data is processed and analyzed today.

roymurdock 9y ago
What general ideas did we sow in the past 20-30 years that, through minor iterations, are coming to (surprising) fruition today?
- santaclaus 9y ago
  Machine learning
  - 9y ago

brlewis 9y ago

At the beginning: The most valuable insights... Toward the end: So it's doubly important not to let yourself be discouraged by people who say there's not much new about something you've discovered. "Not much new" is a real achievement when you're talking about the most general ideas. Maybe if you keep going, you'll discover more. I really feel like Paul is conflating valuable and novel here. When you create something that solves someone's problem, who cares whether the ideas behind it are novel or not?

robertlagrant 9y ago
This is a good hackers and painters juxtaposition: in STEM, it doesn't have to be novel as long as it is better. In art it doesn't have to be better as long as it's novel :-)

golemotron 9y ago

I think Nassim Nicholas Taleb's work epitomizes the "general yet surprising" territory.

tw1010 9y ago

When I read stuff like this and reflect on the fact that this is about the peak quality of the philosophical discussion that is going on in our little niche of the world, I get slightly depressed. I get that it's hard to find people at the intersection of astounding writing and technical chops, but still, it makes me want to distance myself and just go back to reading the classics.

deleted 9y ago
[deleted]
jessriedel 9y ago
What community centered around professional skills do you think has better philosophical discussion? Sure, there exists better philosophical discussion elsewhere, and communities centered around academic disciplines can be more sophisticated because that is their raison d'etre. But when I think of professional communities (e.g., accounting, medicine, law, etc.), I can't think of any better except insofar as they bleed into academia (e.g., blogs by law professors, not practicing lawyers).
- m_fayer 9y ago
  The medical field is an embarrassment of riches when it comes to talented essayists, journalists, and biographers. I've also read a few wonderful biographies written by restauranteurs. Our field unfortunately lags behind, I think partially because it's rather dry compared to other fields, and partially because SV ideology is pervasive, stifling, and indifferent to many swathes of the human experience.
  - jessriedel 9y ago
    I don't necessarily disagree, but little of that falls under the heading of "philosophical discussion". Medicine is romantic and salient to laymen, so of course journalists are going to write more biographies and whatnot for them on that topic. I almost never find such pieces philosophically insightful, and they aren't competing with Paul Graham essays.
  - 9y ago

clavalle 9y ago

An unfortunate corollary is that slightly different general ideas are very difficult to get people to listen to because they feel like they already know it and it is not much new or the difference is negligible. This might be true of any particular case someone might think of but when the entire landscape is taken into account the multiplier pg mentions applies and can change things dramatically. But people don't seem to be well wired to think of the general. And for any general multiplier to work, people generally need to be on board. For example: there is a difference between economic inequality and desperate economic inequality -- the first is a mere difference in ownership and control. The second is /also/ a difference in ownership and control but that leaves one party or group bargaining against their physical well-being. There is much said about mere economic inequality but, by leaving out the desperation factor, the public conversation is tied up in knots (e.g. "Well why don't we give everyone a million dollars?!" nonsense) and the real problem, desperation and the conditions that lead to that state, persist despite having the tools to potentially tackle it. But any single example -- a single mother that has to take two minimum wage jobs to feed her kids -- can be reduced away. She could just do x, y, or z and her particular problem would be solved and she'd still be economically disadvantaged but not desperate to the point of worrying about her kids starving. And the conversation ends. We are back to the inconvenience of a weak bargaining position which is also described by the phrase 'economic inequality' and it all feels like the same, well trod and religiously guarded ground.

rehevkor5 9y ago

F=ma is surprising? We took two values and combined them to define a new one. You can define all sorts of new things this way, how is that surprising?

alexashka 9y ago
You're making a valid point in a not-so-nice way. What he meant was F=ma combined with other formulas that make use of F, make up a system that's surprisingly useful while remaining simple to use. I imagine you already knew that. If you're acting as an editor, you can phrase your point better. if you're merely nitpicking for things to 'gotcha' someone - I assure you that's not a smart longterm strategy :)

nate 9y ago

On the "surprising" part of this - Murray Davis had some related research and writing about being interesting: https://proseminarcrossnationalstudies.files.wordpress.com/2009/11/thatsinteresting_1971.pdf https://proseminarcrossnationalstudies.files.wordpress.com/2... "An audience finds a proposition ‘interesting’ not because it tells them some truth they thought they already knew, but instead because it tells them some truth they thought they already knew was wrong."

peterwwillis 9y ago

tl;dr if you have an idea that isn't very "new" then keep working on it until it is "more new" There should be a twitter account that just summarizes rambling think pieces.

raldi 9y ago

What are some examples of past insights someone has made by discovering something slightly surprising about the general world?

hnaccy 9y ago

This post is lacking in the latter.

kantian 9y ago

Might a parallell be draw to Kant here? General and bland would be as akin to "analytic a priori". All bachelors are unmarried, that sorta thing. Specific and surprising would be like "synthetic posteriori". Such and such plant is blue and grows in the Himalayas. The real gold would be in the "synthetic a priori" findings. Blending the generalness of logic with the newness of empiricism. Like figuring out geometrical laws from observing nature. New information AND far-reaching implications. Am I on to something here? https://en.wikipedia.org/wiki/Analytic%E2%80%93synthetic_distinction https://en.wikipedia.org/wiki/Analytic%E2%80%93synthetic_dis...

kantian 9y ago
https://plato.stanford.edu/entries/analytic-synthetic/ https://plato.stanford.edu/entries/analytic-synthetic/ "While some trivial a priori claims might be analytic in this sense, for Kant the seriously interesting ones were synthetic."
kantian 9y ago
Synthetic a priori = getting epistemological alpha (to speak financese) :)

d_burfoot 9y ago

A couple of years ago I wrote a book about a new philosophy of science based on the following methodology: - Obtain a large dataset of observations related to a phenomenon of interest (eg English text, bioimaging data, economic reports, car-mounted camera recordings, etc) - Develop a theory of the phenomenon, or revise a previous theory - Build the theory into a lossless data compression program - Score the theory by invoking the compressor on the dataset, taking into account the size of the compressor itself; lower codelength is better. - Adopt or discard the theory as appropriate and return to step #2. I believe that this is a valid variant of the traditional scientific method, in which data compression takes the place of experimental prediction. In Graham's terms, this idea is a small delta of novelty, but since it refers to a predecessor idea of tremendous significance and generality, it could be very important. The book is available here: https://arxiv.org/abs/1104.5466 https://arxiv.org/abs/1104.5466 I've spent a few years pursuing this methodology in the domain of English text. The result is a sentence parser, which works quite well, but was built completely without the use of hand-labelled training data (eg the Penn Treebank). You can check out the parser here: http://ozoraresearch.com/crm/public/parseview/UserParseView.jsp http://ozoraresearch.com/crm/public/parseview/UserParseView.... I'm always happy to talk to people about these ideas, ping me at daniel dot burfoot at gmail if you're interested.

jsweojtj 9y ago
This is interesting. I don't have an intuition on how well this could work on different problems. Do you think this approach would work for something, let's start simple, like ballistics neglecting rotation of the Earth and drag? If you fed in thousands of starting velocities and angles, and the resulting distance traveled, will this process end up with equations of motion that are as compact as can be found in first year physics?
- indiv0 9y ago
  I think the problem here is that you are still the one that needs to develop the equations of motion that will be tested for compression. What this describes is a method for testing theories, not generating them. You still need to study the data to try and properly analyze it so you can understand the rules yourself, or at least be able to see the edges of them. Now if you could find a way to generate moderately accurate equations automatically by analyzing the dataset, I think that would be a real leap forward. Is there a related mechanism to the one presented by the parent which could be used to generate theories?

dasil003 9y ago

I think pg is falling prey to the whole post-hoc founder story phenomenon here. He sees so many startups, he hears so many stories about them, and he works hard to distill it into some kind of actionable and scalable investment strategy. The problem is that the facts which led to the success of the business are not the same thing as the best insights a founder can come up with about why it succeeded. The latter is subject to an incredible amount of confirmation bias and the simple desire to tell a good story—in fact you have to do that to get investment. In reality, novelty of ideas is meaningless, because the massively overwhelming majority of ideas in the world never get executed on. Even within contexts where people and companies have the means, most ideas—even good ones—don't get executed on because they conflict with other ideas and don't win the battle for resources. What matters for an idea is execution and context. If you bring the right resources: skills, connections, go-to-market strategy, smaller supporting ideas, and it gels with the current zeitgeist of whatever market you are in, then it can get some traction. From there you learn and gain more insights which can be leveraged to scale up. You do this over and over again, and if you are lucky you can make an 8, 9, 10-figure company. By the time all this happens you'll have some tremendous insights, but they will be specific to the context in which they were gleaned. The "generality" we observe of these insights is just a result of our pattern-matching and story-telling brains. The "surprisingness" comes after the fact to varying degrees based on existing preconceptions and perhaps how the world has changed over time. In other words, I don't think there is such a thing as a hugely valuable insight per se, rather this: we have insights, we believe in them to varying degrees, and based on how well it maps to our actions and their results, we later declare the "value" of the insight.

burger_moon 9y ago

I'm glad Paul is still writing essays. It seems it has dropped off in frequency, this essay kind of hints at one such reason for that. I hope he continues to write even if people disagree with his ideas or if they are repetitive.

tw1010 9y ago
I commented a post with a negative sentiment below, but I agree with you. A lot of his ideas and insights are valuable and thought provoking, and I wouldn't want the tone of these comments to make him stop.
saycheese 9y ago
On the same note, PG's "official" HN account hasn't been active for years: https://news.ycombinator.com/threads?id=pg https://news.ycombinator.com/threads?id=pg
- ComodoHacker 9y ago
  One year ten months, to be precise.
  - dsacco 9y ago
    His account now makes comments once every year or year and a half, but it hasn't been generally active for nearly 3.5 years (the last bunch of comments in a row in a short time period was 1264 days ago, as of this writing).

deleted 9y ago

[deleted]

robertlagrant 9y ago

Is this a stealth "I'm done with the small delta of novelness Arc already has and am stopping on it. If you can't appreciate that then you need to look harder" article? :-)

trapperkeeper74 9y ago

Diff eq-like behavior is everywhere: species population, popularity, technology, market saturation, economics, spring rates, etc. (Malthus was partially ;) mistaken.)

graphememes 9y ago

> It's not true that there's nothing new under the sun. Philosophically this statement is false. Mental health wise, it's a lot nicer to keep in mind than the truth of reality that you reside within.

msla 9y ago
Hm. Can something be philosophically false but factually correct?
- graphememes 9y ago
  Love is an example I can think of.

outworlder 9y ago

> It's not true that there's nothing new under the sun. There are some domains where there's almost nothing new. But there's a big difference between nothing and almost nothing, when it's multiplied by the area under the sun. PG, I see what you did there. The rest of the essay is just a setup for this single paragraph. This is your own small delta of novelty. Well done.

grappler 9y ago

Clearly Paul Graham is defending the Bodega vending machine here ;)

lotusko 9y ago

It's very resonant,ths for sharing.