23 ms·

Marginally related (software?) rant: every time I want to learn how something specific works or I want to do something oddly specific I keep running in the same

Marginally related (software?) rant: every time I want to learn how something specific works or I want to do something oddly specific I keep running in the same phenomenon. Google mostly returns vague abstract fluff and Stack Overflow tells me it really, really shouldn't be done because [reason]. It's like most of human written content caps out at about the level of description ChatGPT could deliver. Like there's a "knowledge event horizon". After that it always ends in hard to parse research papers, specifications, jumping around the source code of multiple libraries, debugging or reverse engineering. If I get lucky I might land on a 15 year old blog post that no man has seen since inception and it describes exactly what I wanted to know. And that's just software. Humans know exactly what parts went into it and how it works. Why must wisdom always be squeezed out of a rock? Why is only the fluff copied thousands of times making research harder each day? Why are there never indicators which way to dig for more details? T__T

vinyl7 4y ago
See also the Preventing the Collapse of Civilization talk by Jonathan Blow https://youtube.com/watch?v=ZSRHeXYDLko https://youtube.com/watch?v=ZSRHeXYDLko
kovac 4y ago
This is true and very unfortunate. My response to this is that when I want to learn anything in-depth, I use textbooks written by experts (e.g. professors, industry experts often from the past). It's hard to verify the expertise of blog authors and SO contributors unlike the authors, and blog posts can often be more about self-promotion than sharing deep insights. Sometimes I even write to these authors asking questions and they actually respond. To some extent, old internet is still alive and beautiful. It's just buried under a lot of noise.
- jvans 4y ago
  Agree, textbooks are very underrated. The three blue one brown guy had one of the best tips for textbook selection that's been extremely helpful for me. He suggests that textbooks by single authors are the easiest to understand and in retrospect that matches with my experience. Multiple authors tend to refine the work to the point where it's 100% correct but explained in a way that only someone who is already an expert would understand.
Shinmon 4y ago
I feel you. I think this is due to the fact that the overall concepts are usually quite easy to understand and therefore easy to talk about. As soon as you have to dig deeper you need to invest significantly more time to talk about it and your audience becomes really small. This is amplified by people creating newsletters, content (marketing) in general, but the actual knowledge/wisdom is not given away. It's incredibly frustrating but happens in almost all technological field, especially when they are fast moving. In mechanical engineering for example a lot of wisdom is also put into norms, standards and so on. Books are written about it. But 10 year old books are still completly valid while in software every couple of years a new framewokr is the new hot shit.
- Swizec 4y ago
  > But 10 year old books are still completly valid while in software every couple of years a new framewokr is the new hot shit The more I look at it, the more it feels like software hasn’t had a fundamentally new idea in decades. Frameworks improve, networks get bigger, compute gets cheaper, and we spend most of our time dealing with domain modeling, cache, and statistical inference. We’ve made great strides in the ease of building and encoded many lessons in our frameworks, scale is bigger too, but the underlying guts of software engineering have been very stable for decades.
  - Shinmon 4y ago
    I agree actually...kind of. And there are good books about software architecture and so on. Unfortunately, for each good one there are at least 50 that are really really bad. There are a lot of technology-related books as well, that teach you about responsive web design with react or stuff like this. Encoding lessons in frameworks isn't always helpful either in my opinion. Your mental model of the software needs to be correct, otherwise it's just a really painful experience. What works for large scale software is often just pain for beginners.
AlexB138 4y ago
Google has been SEOed to death essentially. Little to no valuable content can be found there anymore, beyond exactly what you laid out. The only place I've found to find good, solid content to fill the gap between blog-spam and the sort of deep dive you laid out is technical books. Granted, there's a lot of chaff in books as well, but their quality to junk ratio is wildly better than Google, and they generally go much deeper.
- solardev 4y ago
  I can't wait until you can give ChatGPT an ISBN and ask it to answer any question by citing that book. The SNR of a book may be high, but when the cost of finding that signal is having to go through 400 pages of dense material, line by line, word by word... well, Google wins.
  - volkk 4y ago
    wow. i love this idea
  - russellendicott 4y ago
    So Google Books? [1] Only problem is that pesky copyright that makes you buy them [1] https://en.m.wikipedia.org/wiki/Google_Books https://en.m.wikipedia.org/wiki/Google_Books
    - solardev 4y ago
      Google Books is just a search engine. It doesn't yet have the natural language parsing and information synthesis power that chat gpt has
- titzer 4y ago
  > Google has been SEOed to death essentially. It's the nature of the incentive system that they set up. Their only grand vision for the internet was one where Google hit its growth targets every quarter. If you do the math on that, then Google has to be enormous in 2022. Guess what, Google is enormous in 2022, and the only way they could figure out how to do that was to keep growing their ad business. So here we are.
jstanley 4y ago
> It's like most of human written content caps out at about the level of description ChatGPT could deliver. Don't worry, it won't be long until we're lamenting that most of ChatGPT's output caps out at about the level a human could deliver.
plastiquebeech 4y ago
When I was young, we didn't have StackOverflow or Google. We had to hike through snow to access the internet at all, uphill both ways! In those days, we would usually read the manual when we needed to dig past a surface-level understanding of how things work. The manuals are dry and dense, but they're much easier to read with modern niceties like "Ctrl+F". IMO, this is one of the reasons that people recommend using software with a long track record. If you have a question about some parameter in a systemd service script, and the internet doesn't have a ready made answer, the details are all written down in the manual. https://www.man7.org/linux/man-pages/man5/systemd.service.5.html https://www.man7.org/linux/man-pages/man5/systemd.service.5.... Plenty of modern tools have comprehensive detailed docs like this. Python, Go, even most widely-used JS frameworks. The primary sources are often downranked in search engines because they don't sell ads, but it's a good idea to find and bookmark them when you start working with a new technology. RTFM!
- mhuffman 4y ago
  >In those days, we would usually read the manual when we needed to dig past a surface-level understanding of how things work. The manuals are dry and dense, but they're much easier to read with modern niceties like "Ctrl+F". Then, after that didn't work, we had to go to find the weird bearded guy that didn't like to talk to people, but knew everything. He would usually tell you the answer, but in a condescending way that made you understand that you only came to him in "emergencies" and that you were kind of stupid for asking an "obvious" question. Actually, now that I think of it, that is stackoverflow now! Thank god message boards started taking off! ... but sadly, then the "real" Internet came and killed it. :-(
  - clarge1120 4y ago
    Before the real internet came and killed the forums, the weird bearded guy would lurk on the forums. He's the guy who makes you answer general questions like, "Why would you want to do that?" "What's this for?" "Did you try searching before asking such a stupid question?"
    - 4y ago
winReInstall 4y ago
The problem is one of the precision of the question and the scope of the answer. Usually some questions are just indicative of a complete lack of understanding of the field, basically making a whole comp science course necessary to answer in depth. https://www.youtube.com/watch?v=36GT2zI8lVA https://www.youtube.com/watch?v=36GT2zI8lVA
interroboink 4y ago
I feel your pain, and have had similar experiences. I do feel like wisdom is inherently an "internalize for your own mind" process, and on some level, nobody can do that for you. So of course you will be scrabbling around trying to piece things together until it clicks in your own mind. I've found having a discussion with another human, who understands the topic, is very helpful. And it is one case where an "academic" setting can be useful, assuming you can actually talk to the professor rather than being forced to parse some unfathomable textbook. Even that example — a textbook — is similar to what you said. It is literally designed to help you understand a topic. And yet, my experience with them mimics yours. A lot of the material is uselessly general, and a lot of it is uselessly specific and technical. And it almost never answers the specific question you actually want to know at a given moment. So, you have to internalize the information (slowly) and eventually get there. Maybe there is an "event horizon" of me being able to absorb new material, rather than something external.
crazygringo 4y ago
> Humans know exactly what parts went into it and how it works. Why must wisdom always be squeezed out of a rock? Partly, because so many developers famously hate writing documentation and hate commenting their code. If developers followed Donald Knuth's Literate Programming [1] then it would be a big improvement. People rely on a lot of "tribal knowledge" without ever bothering to write it down, and out of all the fields, software development seems to be particularly notorious for its anti-documentation bias. [1] https://en.wikipedia.org/wiki/Literate_programming https://en.wikipedia.org/wiki/Literate_programming
yamtaddle 4y ago
The degree to which most of the Internet's actual intellectual/informational value is still, in 2022, simply providing more-efficient replacement for inter-library loan (that is, book piracy websites), for many fields, has me wondering if it'll ever fully deliver on the potential that many of us used to think it had.
mattgreenrocks 4y ago
Entry-level content is easier to produce and consume. It is also guaranteed a larger percentage of traffic due to the high influx of newer developers. Example: it's 2022 and we still somehow upvote blog posts detailing one's stack as if that has any impact on anyone else.
p0nce 4y ago
Try books.
marcosdumay 4y ago
Well, there's a reason ChatGPT delivers this kind of content. It's what is was trained into, and it's just rewiring the logic in the facts it was given. (It's very impressive how it rewires the logic, but it can't really know anything, by design.) And yes, that horizon is very real. It's what the internet rewards, so it's what almost everybody delivers.
worldsayshi 4y ago
This knowledge event horizon is bound to stay around the current position for as long as our primary medium for information is text/video/sound (i.e. linear). When you google for some programming story you typically look for a story that describes how to combine concept A, B, C and maybe F. The more concepts you add the likelyhood of somebody having described a good implementation story for the combination of those concepts goes down. But you don't need to add many such concepts before the number of potential stories become very large. So for any non trivial number of concepts a blog article is not likely to exist. We could only really change the game by going from text representation to intelligent representations. Rules or general AI. However, my main surprise in all this: why do we need this many combinations of concepts in the first place? Why isn't the total domain of things that are interesting to do for utilitarian reasons more limited in size?
shanebellone 4y ago
I cannot relate to your experience. Python docs are almost always sufficient for my needs. I rarely use Stack Overflow but find it more useful for examples or semantic discussions. I view documentation as a set facts and stack overflow as additional color. What type of projects require this workflow?
simpsond 4y ago
It seems like SEO and content marketing have long term side effects. I have experienced what you describe, and it’s frustrating.
swayvil 4y ago
"Vague and abstract" and "a warning" are the minimum-effort responses. Therefore they will be found in the greatest number. Most responses are there just to get your attention. This whole thing runs on attention.
eternalban 4y ago
> Why must wisdom always be squeezed out of a rock? Dictionary says there are 3 components to wisdom: knowledge, experience, and good judgment. Machines can help with the first requirement, knowledge, to an extent but leave you frustrated (as you note). They can even help you dig out wise chestnut buried somewhere (as you did) but then you hit the other two. This is how it is supposed to work: armed with knowledge, you apply this and gain experience, and after a few close calls the fortunate also develop good judgment, informed by experience of application of knowledge. You want all that squeezed out how?
johnfn 4y ago
This is correct. It's because, basically definitionally, the amount of people who have deeply engaged with any topic is very low compared to the number of people who have shallowly engaged with it. This means that the amount of reference material written by people with deep understanding will be much much less than the amount of reference material written by people with shallow understanding. "Back in the day", this problem was addressed by having significant hurdles in order to publish material on a topic. E.g., books used to be the primary way to learn about a topic, and in order to publish a book you must 1) deeply engage with the material for quite a while simply to write the book and 2) become fairly credentialed / convince a publisher you're worth their time, etc. Same thing with academic papers. The internet has completely removed all barriers to publishing, meaning it's up to us to sort the wheat from the chaff.
- Jensson 4y ago
  Yeah, like here on HN the blogs are usually written by people who just learned about the topic and then you go to the comments to see what the people who know more says. People who know seems to not think it is worth it to write a blog about it, but they can comment on posts others write and correct them when they are wrong.
  - volkk 4y ago
    misaligned incentives. people get rewarded for simply publishing SOMETHING. look at companies, bootcamps, whatever the hell. most promotions are given bc you've written some low effort blog post that really add nothing to the broader intelligence of software building. hell, even bootcamps are like "go write/publish something!!" and most of these people have barely a clue as to how software operates. GL sifting through all of that useless content i wouldn't be surprised if a very large percentage of the last decade of tech blog posts are just remnants of someone's promotion, and is now hyperlinked on some resume, or linkedin page
  - dsfyu404ed 4y ago
    But how do you know the people in the comments actually know more and aren't just an equivalently low but slightly different common denominator? You don't. The same "shallow knowledge vs deep knowledge" problem still applies.
the-printer 4y ago
> After that it always ends in hard to parse research papers, specifications, jumping around the source code of multiple libraries, debugging or reverse engineering. If I get lucky I might land on a 15 year old blog post that no man has seen since inception and it describes exactly what I wanted to know. What you’re describing takes a lot of time and nerve. Those are two of the foremost requisites of the pursuit of knowledge. The Web is working. It’s enabling you to even access those old manuals via the Internet Archive, papers via Sci-hub, source code on GitHub and of course that old blog post on page 7 of a DuckDuckGo query. The Web works y’all!
deleted 4y ago
[deleted]
hosh 4y ago
Wisdom can't be squeezed out of a rock. The experience of wisdom is individual, because wisdom is transformative. What's being transformed is not the knowledge but you, your consciousness. It's local in that sense. It's not something that can be packaged or productized. Wisdom comes from an awareness of the greater whole. The insights do not come from analysis, but rather, synthesis. It engages with the intuition rather than the intellect. Although it can seem mystical, I think there are authors who have been able to express ways to engage in wisdom even if they are not directly talking about it. For example, Christopher Alexander has some interesting things to say about wholes, centers, and unfolding: Seeing Wholes - https://iamronen.com/blog/2018/01/14/christopher-alexander-seeing-wholeness/ https://iamronen.com/blog/2018/01/14/christopher-alexander-s... Centers - https://iamronen.com/blog/2018/01/08/christopher-alexander-centers/ https://iamronen.com/blog/2018/01/08/christopher-alexander-c... Unfolding - https://www.livingneighborhoods.org/ht-0/whatisanunfolding.htm https://www.livingneighborhoods.org/ht-0/whatisanunfolding.h...
- cloutchaser 4y ago
  I think what you are talking about are what a few hundred years ago you called prophets. Some people can pass on wisdom but it’s a very rare skill, and followers can often exhibit the cult like behaviour that’s so repulsive to others. Probably because they can’t pass on that wisdom to others. Also, many so called prophets can easily exploit this skill, which they might well do if that’s their personality type. Just typing this out has made me think it’s almost like passing on wisdom and the religious experience are probably inseparable.
  - hosh 4y ago
    Yet, in the Tantric and Classical view, art was a way that ordinary people can connect to wisdom. No mystical experiencies or psychedelic substances necessary. What's amazing to me about Christopher Alexander and his work is that he's able to describe the process of generating such art (in the form of architecture) to ordinary people, using plain, relatable language. The links I posted in the earlier comment are examples. In other words, you don't need to be someone with a rare skill.
  - 4y ago
elorant 4y ago
Because it’s easy to monetize crap. So no one takes the time to reproduce valuable content.
dylan604 4y ago
Best description I've heard for this is "institutional knowledge". It is not uncommon for a workflow/process to be "well documented", but over the course of time, the people that actually do the process have found small little tweaks that are not part of the original documentation nor do they ever get added to the documentation. Due to this, following the directions will never result in what the current workflow does. You see this a lot in recipes, and I've heard tales of military being susceptible to this, large manufacturing processes, etc where the tweaks are susceptible to being lost if large cutbacks/layoffs were to affect the people with that tribal knowledge.
pojzon 4y ago
Its worse than that. Someone posts a good solution on github as a comment -> 50 ppl create a paper about it on medium.com -> 500 copy-paste websites replicate it to their database of shitty ads ridden sites. Finding something more complex than few cm deep is becoming a hellish task. I blame Google for lack of moderation and getting their SOE gamed like a little pipsqueek. Juniors are dying happy coz of how fast they can find trivial answers. Seniors get shafted. Its so bad that I started to build my own github repo woth various answers I needed over the years.
BizarroLand 4y ago
That is kind of exciting though! I'm in the same boat and feeling the frustration of not being able to easily branch swing to the specific information that I want because the path is no longer clear. That doesn't happen for trivial topics or problems though! It's a sort of new frontier where we have to re-solve potentially solved problems because the solution isn't pervasive enough to be trivial. We have to push ourselves to overcome a knowledge limit, which throws a nice fat monkey wrench into the idea of having "All human knowledge in the palm of your hand". There's undiscovered territory out there and it's hard to see unless you get above the trees.
nathias 4y ago
it's because you're confusing knowledge wrappers for knowledge, the payload is always difficult to parse
unity1001 4y ago
Google screwed its search during Eric Schmidt's term with the mentality of "Brands will sort out the mess" to 'combat spam'. The result is a few big sites dominating the search in everything instead of the original results that used to be before those updates. The first infamous one being 'Panda' I believe. It succeeded in eliminating spam from a lot of small sites and instead resulting in an even bigger spam from the same big sites for every category. Great for people who were doing shoe-shopping at, well, Amazon, probably, but bad for everyone else. Then they increasingly personalized search results via AI. And that changes your search results so fast that you may not get the same seach results for the same exact keyword in a week's time. Im increasingly of the opinion that corporate, 'business mentality' should not run any major tech corp. It sees everything in numbers and extreme abstractions, juggling them to make a desirable false reality happen in paper and in metrics, screwing up the real world for every user.
- fleddr 4y ago
  Panda has taught me a lot. Just before it hit, I had a small site that was on the rise in terms of traffic. A hobbyist site, no ads, all original content, and very much not "thin". An enthusiast site, much like the original internet. Incoming links were organic and spontaneous, from serious sites. Panda crushed it. Massive pagerank downgrade and traffic decimated to about 5%. In mere hours. A false positive, I suppose. I spent 3 months trying to figure out the reason but nobody, including Google or SEO specialists were able to provide any tangible answer. That day etched a few important lessons into my long term memory: 1) The size of Google's power. It effectively has the power to decide whether you digitally exist or not. The ultimate traffic controller. 2) Even a good faith move by Google may have 1% of false positives. Which seems totally rational until you realize that at Google's scale this could mean millions of people being totally fucked out of the blue. 3) Worse, when that happens, there's no accountability. You can't go anywhere for help, undo being a false positive, you're just randomly fucked and that's it. You cannot depend on anything ran by Google. Although I'm fine and it wasn't exactly a pivotal moment in my life, it radically changed my view on Google. Early on, I was a fanboy. Google kicking lame Microsoft's butt with miraculous products like search, maps, gmail. I was cheering them on. As of my "lesson", I've grown increasingly cynical. That was 10 years ago. They've become infinitely worse.
quickthrower2 4y ago
The problem here is Google and SEO incentives. Or even perceived SEO gains by writing shallow articles. Notice how many of the articles have a “Pricing” and a bold “We’re Hiring” link ;-)
User23 4y ago
At this rate we're going to have to go back to getting information on programming computers from books. On the plus side, that means if you're willing to dig through the research papers and other hard but useful sources, you have an opportunity to make a difference by writing that book.
numpad0 4y ago
Easy, it’s because a lot of(I’m hopeful it’s not all of) so-called values of modern world come from exploitation, negative externalities, and little creative choices. Working on a honest straight reality is like drinking water, has no calories in it. Coke is better. Wines are more better, and everyone other than than I these days seems to be always on a magic fissile cocktails, and while I don’t get that, they seem well off doing it.
dataflow 4y ago
Stack Overflow is indeed quite painful in this regard, and it really, really doesn't need to be like this. And in fact, it isn't all the time, or in every Stack Exchange. Sometimes, finding the right Stack Exchange site helps a lot (like with retrocomputing), where the premise (say, "I want to get this to run on Windows 95") won't turn into arguments over security patches (or whatever) derailing the question. Or sometimes you just have to find the right people in the community, and nudge them to answer somehow. Certain users tend to be quite generous and go out of their way to answer everything you asked with more to guide you, while others love to chide you and slap you with a question closure or deletion for even thinking about walking off the beaten path.
- dsfyu404ed 4y ago
  >Or sometimes you just have to find the right people in the community, and nudge them to answer somehow. You see this on Reddit all the time. There will be a Q&A sub for a topic dominated by people with scant if any experience who think they're helping people by googling stuff for them and there will be a comedy subreddit for the same topic. Smart people ask questions in the joke sub because they know that having sufficient understanding to be amused by highly contextual stuff weeds out a lot of the casuals.
f0e4c2f7 4y ago
When I hit that point I've found that books, textbooks, and YouTube videos can be a good resource.
yrgulation 4y ago
__MatrixMan__ 4y ago
Because there's no market for ads targeting one guy fifteen years from now. We've built a web that actively discourages the kind of depth you're after.
ruste 4y ago
I love how quickly after ChatGPT's release we're conflating the chicken and the egg. It's not that the content online is barely better than ChatGPT. ChatGPT is what it is because it was trained on that content.
namaria 4y ago
There are a lot of drivers for fragmentation and opacity online. Venture capital throwing money at start ups to see what sticks, search engine optimization, gut-feeling opinionated blogging to build personal brands (self selecting for unexperienced being loud and misleading a lot of the new arrivals) etc. The cure for me has been two fold: 1) Focusing on books from reputable publishing houses (curated long form content) - but even that takes filtering. I noticed O'Riley has been putting out a lot of noise lately. 2) Focusing on open-standards, sound engineering. I'd rather focus on understanding hardware, operating systems, networking etc then myriad software as a service offerings full of vendor lock-in dark patterns.
antman 4y ago
Have you tried ChatGPT? It gives me better results for a lot of things. Comparison of ChatGPT and Google on 500 coding queries: https://news.ycombinator.com/item?id=34089423 https://news.ycombinator.com/item?id=34089423

vinyl7 4y ago

See also the Preventing the Collapse of Civilization talk by Jonathan Blow https://youtube.com/watch?v=ZSRHeXYDLko https://youtube.com/watch?v=ZSRHeXYDLko

kovac 4y ago

This is true and very unfortunate. My response to this is that when I want to learn anything in-depth, I use textbooks written by experts (e.g. professors, industry experts often from the past). It's hard to verify the expertise of blog authors and SO contributors unlike the authors, and blog posts can often be more about self-promotion than sharing deep insights. Sometimes I even write to these authors asking questions and they actually respond. To some extent, old internet is still alive and beautiful. It's just buried under a lot of noise.

jvans 4y ago
Agree, textbooks are very underrated. The three blue one brown guy had one of the best tips for textbook selection that's been extremely helpful for me. He suggests that textbooks by single authors are the easiest to understand and in retrospect that matches with my experience. Multiple authors tend to refine the work to the point where it's 100% correct but explained in a way that only someone who is already an expert would understand.

Shinmon 4y ago

I feel you. I think this is due to the fact that the overall concepts are usually quite easy to understand and therefore easy to talk about. As soon as you have to dig deeper you need to invest significantly more time to talk about it and your audience becomes really small. This is amplified by people creating newsletters, content (marketing) in general, but the actual knowledge/wisdom is not given away. It's incredibly frustrating but happens in almost all technological field, especially when they are fast moving. In mechanical engineering for example a lot of wisdom is also put into norms, standards and so on. Books are written about it. But 10 year old books are still completly valid while in software every couple of years a new framewokr is the new hot shit.

Swizec 4y ago
> But 10 year old books are still completly valid while in software every couple of years a new framewokr is the new hot shit The more I look at it, the more it feels like software hasn’t had a fundamentally new idea in decades. Frameworks improve, networks get bigger, compute gets cheaper, and we spend most of our time dealing with domain modeling, cache, and statistical inference. We’ve made great strides in the ease of building and encoded many lessons in our frameworks, scale is bigger too, but the underlying guts of software engineering have been very stable for decades.
- Shinmon 4y ago
  I agree actually...kind of. And there are good books about software architecture and so on. Unfortunately, for each good one there are at least 50 that are really really bad. There are a lot of technology-related books as well, that teach you about responsive web design with react or stuff like this. Encoding lessons in frameworks isn't always helpful either in my opinion. Your mental model of the software needs to be correct, otherwise it's just a really painful experience. What works for large scale software is often just pain for beginners.

AlexB138 4y ago

Google has been SEOed to death essentially. Little to no valuable content can be found there anymore, beyond exactly what you laid out. The only place I've found to find good, solid content to fill the gap between blog-spam and the sort of deep dive you laid out is technical books. Granted, there's a lot of chaff in books as well, but their quality to junk ratio is wildly better than Google, and they generally go much deeper.

solardev 4y ago
I can't wait until you can give ChatGPT an ISBN and ask it to answer any question by citing that book. The SNR of a book may be high, but when the cost of finding that signal is having to go through 400 pages of dense material, line by line, word by word... well, Google wins.
- volkk 4y ago
  wow. i love this idea
- russellendicott 4y ago
  So Google Books? [1] Only problem is that pesky copyright that makes you buy them [1] https://en.m.wikipedia.org/wiki/Google_Books https://en.m.wikipedia.org/wiki/Google_Books
  - solardev 4y ago
    Google Books is just a search engine. It doesn't yet have the natural language parsing and information synthesis power that chat gpt has
titzer 4y ago
> Google has been SEOed to death essentially. It's the nature of the incentive system that they set up. Their only grand vision for the internet was one where Google hit its growth targets every quarter. If you do the math on that, then Google has to be enormous in 2022. Guess what, Google is enormous in 2022, and the only way they could figure out how to do that was to keep growing their ad business. So here we are.

jstanley 4y ago

> It's like most of human written content caps out at about the level of description ChatGPT could deliver. Don't worry, it won't be long until we're lamenting that most of ChatGPT's output caps out at about the level a human could deliver.

plastiquebeech 4y ago

When I was young, we didn't have StackOverflow or Google. We had to hike through snow to access the internet at all, uphill both ways! In those days, we would usually read the manual when we needed to dig past a surface-level understanding of how things work. The manuals are dry and dense, but they're much easier to read with modern niceties like "Ctrl+F". IMO, this is one of the reasons that people recommend using software with a long track record. If you have a question about some parameter in a systemd service script, and the internet doesn't have a ready made answer, the details are all written down in the manual. https://www.man7.org/linux/man-pages/man5/systemd.service.5.html https://www.man7.org/linux/man-pages/man5/systemd.service.5.... Plenty of modern tools have comprehensive detailed docs like this. Python, Go, even most widely-used JS frameworks. The primary sources are often downranked in search engines because they don't sell ads, but it's a good idea to find and bookmark them when you start working with a new technology. RTFM!

mhuffman 4y ago
>In those days, we would usually read the manual when we needed to dig past a surface-level understanding of how things work. The manuals are dry and dense, but they're much easier to read with modern niceties like "Ctrl+F". Then, after that didn't work, we had to go to find the weird bearded guy that didn't like to talk to people, but knew everything. He would usually tell you the answer, but in a condescending way that made you understand that you only came to him in "emergencies" and that you were kind of stupid for asking an "obvious" question. Actually, now that I think of it, that is stackoverflow now! Thank god message boards started taking off! ... but sadly, then the "real" Internet came and killed it. :-(
- clarge1120 4y ago
  Before the real internet came and killed the forums, the weird bearded guy would lurk on the forums. He's the guy who makes you answer general questions like, "Why would you want to do that?" "What's this for?" "Did you try searching before asking such a stupid question?"
  - 4y ago

winReInstall 4y ago

The problem is one of the precision of the question and the scope of the answer. Usually some questions are just indicative of a complete lack of understanding of the field, basically making a whole comp science course necessary to answer in depth. https://www.youtube.com/watch?v=36GT2zI8lVA https://www.youtube.com/watch?v=36GT2zI8lVA

interroboink 4y ago

I feel your pain, and have had similar experiences. I do feel like wisdom is inherently an "internalize for your own mind" process, and on some level, nobody can do that for you. So of course you will be scrabbling around trying to piece things together until it clicks in your own mind. I've found having a discussion with another human, who understands the topic, is very helpful. And it is one case where an "academic" setting can be useful, assuming you can actually talk to the professor rather than being forced to parse some unfathomable textbook. Even that example — a textbook — is similar to what you said. It is literally designed to help you understand a topic. And yet, my experience with them mimics yours. A lot of the material is uselessly general, and a lot of it is uselessly specific and technical. And it almost never answers the specific question you actually want to know at a given moment. So, you have to internalize the information (slowly) and eventually get there. Maybe there is an "event horizon" of me being able to absorb new material, rather than something external.

crazygringo 4y ago

> Humans know exactly what parts went into it and how it works. Why must wisdom always be squeezed out of a rock? Partly, because so many developers famously hate writing documentation and hate commenting their code. If developers followed Donald Knuth's Literate Programming [1] then it would be a big improvement. People rely on a lot of "tribal knowledge" without ever bothering to write it down, and out of all the fields, software development seems to be particularly notorious for its anti-documentation bias. [1] https://en.wikipedia.org/wiki/Literate_programming https://en.wikipedia.org/wiki/Literate_programming

yamtaddle 4y ago

The degree to which most of the Internet's actual intellectual/informational value is still, in 2022, simply providing more-efficient replacement for inter-library loan (that is, book piracy websites), for many fields, has me wondering if it'll ever fully deliver on the potential that many of us used to think it had.

mattgreenrocks 4y ago

Entry-level content is easier to produce and consume. It is also guaranteed a larger percentage of traffic due to the high influx of newer developers. Example: it's 2022 and we still somehow upvote blog posts detailing one's stack as if that has any impact on anyone else.

p0nce 4y ago

Try books.

marcosdumay 4y ago

Well, there's a reason ChatGPT delivers this kind of content. It's what is was trained into, and it's just rewiring the logic in the facts it was given. (It's very impressive how it rewires the logic, but it can't really know anything, by design.) And yes, that horizon is very real. It's what the internet rewards, so it's what almost everybody delivers.

worldsayshi 4y ago

This knowledge event horizon is bound to stay around the current position for as long as our primary medium for information is text/video/sound (i.e. linear). When you google for some programming story you typically look for a story that describes how to combine concept A, B, C and maybe F. The more concepts you add the likelyhood of somebody having described a good implementation story for the combination of those concepts goes down. But you don't need to add many such concepts before the number of potential stories become very large. So for any non trivial number of concepts a blog article is not likely to exist. We could only really change the game by going from text representation to intelligent representations. Rules or general AI. However, my main surprise in all this: why do we need this many combinations of concepts in the first place? Why isn't the total domain of things that are interesting to do for utilitarian reasons more limited in size?

shanebellone 4y ago

I cannot relate to your experience. Python docs are almost always sufficient for my needs. I rarely use Stack Overflow but find it more useful for examples or semantic discussions. I view documentation as a set facts and stack overflow as additional color. What type of projects require this workflow?

simpsond 4y ago

It seems like SEO and content marketing have long term side effects. I have experienced what you describe, and it’s frustrating.

swayvil 4y ago

"Vague and abstract" and "a warning" are the minimum-effort responses. Therefore they will be found in the greatest number. Most responses are there just to get your attention. This whole thing runs on attention.

eternalban 4y ago

> Why must wisdom always be squeezed out of a rock? Dictionary says there are 3 components to wisdom: knowledge, experience, and good judgment. Machines can help with the first requirement, knowledge, to an extent but leave you frustrated (as you note). They can even help you dig out wise chestnut buried somewhere (as you did) but then you hit the other two. This is how it is supposed to work: armed with knowledge, you apply this and gain experience, and after a few close calls the fortunate also develop good judgment, informed by experience of application of knowledge. You want all that squeezed out how?

johnfn 4y ago

This is correct. It's because, basically definitionally, the amount of people who have deeply engaged with any topic is very low compared to the number of people who have shallowly engaged with it. This means that the amount of reference material written by people with deep understanding will be much much less than the amount of reference material written by people with shallow understanding. "Back in the day", this problem was addressed by having significant hurdles in order to publish material on a topic. E.g., books used to be the primary way to learn about a topic, and in order to publish a book you must 1) deeply engage with the material for quite a while simply to write the book and 2) become fairly credentialed / convince a publisher you're worth their time, etc. Same thing with academic papers. The internet has completely removed all barriers to publishing, meaning it's up to us to sort the wheat from the chaff.

Jensson 4y ago
Yeah, like here on HN the blogs are usually written by people who just learned about the topic and then you go to the comments to see what the people who know more says. People who know seems to not think it is worth it to write a blog about it, but they can comment on posts others write and correct them when they are wrong.
- volkk 4y ago
  misaligned incentives. people get rewarded for simply publishing SOMETHING. look at companies, bootcamps, whatever the hell. most promotions are given bc you've written some low effort blog post that really add nothing to the broader intelligence of software building. hell, even bootcamps are like "go write/publish something!!" and most of these people have barely a clue as to how software operates. GL sifting through all of that useless content i wouldn't be surprised if a very large percentage of the last decade of tech blog posts are just remnants of someone's promotion, and is now hyperlinked on some resume, or linkedin page
- dsfyu404ed 4y ago
  But how do you know the people in the comments actually know more and aren't just an equivalently low but slightly different common denominator? You don't. The same "shallow knowledge vs deep knowledge" problem still applies.

the-printer 4y ago

> After that it always ends in hard to parse research papers, specifications, jumping around the source code of multiple libraries, debugging or reverse engineering. If I get lucky I might land on a 15 year old blog post that no man has seen since inception and it describes exactly what I wanted to know. What you’re describing takes a lot of time and nerve. Those are two of the foremost requisites of the pursuit of knowledge. The Web is working. It’s enabling you to even access those old manuals via the Internet Archive, papers via Sci-hub, source code on GitHub and of course that old blog post on page 7 of a DuckDuckGo query. The Web works y’all!

deleted 4y ago

[deleted]

hosh 4y ago

Wisdom can't be squeezed out of a rock. The experience of wisdom is individual, because wisdom is transformative. What's being transformed is not the knowledge but you, your consciousness. It's local in that sense. It's not something that can be packaged or productized. Wisdom comes from an awareness of the greater whole. The insights do not come from analysis, but rather, synthesis. It engages with the intuition rather than the intellect. Although it can seem mystical, I think there are authors who have been able to express ways to engage in wisdom even if they are not directly talking about it. For example, Christopher Alexander has some interesting things to say about wholes, centers, and unfolding: Seeing Wholes - https://iamronen.com/blog/2018/01/14/christopher-alexander-seeing-wholeness/ https://iamronen.com/blog/2018/01/14/christopher-alexander-s... Centers - https://iamronen.com/blog/2018/01/08/christopher-alexander-centers/ https://iamronen.com/blog/2018/01/08/christopher-alexander-c... Unfolding - https://www.livingneighborhoods.org/ht-0/whatisanunfolding.htm https://www.livingneighborhoods.org/ht-0/whatisanunfolding.h...

cloutchaser 4y ago
I think what you are talking about are what a few hundred years ago you called prophets. Some people can pass on wisdom but it’s a very rare skill, and followers can often exhibit the cult like behaviour that’s so repulsive to others. Probably because they can’t pass on that wisdom to others. Also, many so called prophets can easily exploit this skill, which they might well do if that’s their personality type. Just typing this out has made me think it’s almost like passing on wisdom and the religious experience are probably inseparable.
- hosh 4y ago
  Yet, in the Tantric and Classical view, art was a way that ordinary people can connect to wisdom. No mystical experiencies or psychedelic substances necessary. What's amazing to me about Christopher Alexander and his work is that he's able to describe the process of generating such art (in the form of architecture) to ordinary people, using plain, relatable language. The links I posted in the earlier comment are examples. In other words, you don't need to be someone with a rare skill.
- 4y ago

elorant 4y ago

Because it’s easy to monetize crap. So no one takes the time to reproduce valuable content.

dylan604 4y ago

Best description I've heard for this is "institutional knowledge". It is not uncommon for a workflow/process to be "well documented", but over the course of time, the people that actually do the process have found small little tweaks that are not part of the original documentation nor do they ever get added to the documentation. Due to this, following the directions will never result in what the current workflow does. You see this a lot in recipes, and I've heard tales of military being susceptible to this, large manufacturing processes, etc where the tweaks are susceptible to being lost if large cutbacks/layoffs were to affect the people with that tribal knowledge.

pojzon 4y ago

Its worse than that. Someone posts a good solution on github as a comment -> 50 ppl create a paper about it on medium.com -> 500 copy-paste websites replicate it to their database of shitty ads ridden sites. Finding something more complex than few cm deep is becoming a hellish task. I blame Google for lack of moderation and getting their SOE gamed like a little pipsqueek. Juniors are dying happy coz of how fast they can find trivial answers. Seniors get shafted. Its so bad that I started to build my own github repo woth various answers I needed over the years.

BizarroLand 4y ago

That is kind of exciting though! I'm in the same boat and feeling the frustration of not being able to easily branch swing to the specific information that I want because the path is no longer clear. That doesn't happen for trivial topics or problems though! It's a sort of new frontier where we have to re-solve potentially solved problems because the solution isn't pervasive enough to be trivial. We have to push ourselves to overcome a knowledge limit, which throws a nice fat monkey wrench into the idea of having "All human knowledge in the palm of your hand". There's undiscovered territory out there and it's hard to see unless you get above the trees.

nathias 4y ago

it's because you're confusing knowledge wrappers for knowledge, the payload is always difficult to parse

unity1001 4y ago

Google screwed its search during Eric Schmidt's term with the mentality of "Brands will sort out the mess" to 'combat spam'. The result is a few big sites dominating the search in everything instead of the original results that used to be before those updates. The first infamous one being 'Panda' I believe. It succeeded in eliminating spam from a lot of small sites and instead resulting in an even bigger spam from the same big sites for every category. Great for people who were doing shoe-shopping at, well, Amazon, probably, but bad for everyone else. Then they increasingly personalized search results via AI. And that changes your search results so fast that you may not get the same seach results for the same exact keyword in a week's time. Im increasingly of the opinion that corporate, 'business mentality' should not run any major tech corp. It sees everything in numbers and extreme abstractions, juggling them to make a desirable false reality happen in paper and in metrics, screwing up the real world for every user.

fleddr 4y ago
Panda has taught me a lot. Just before it hit, I had a small site that was on the rise in terms of traffic. A hobbyist site, no ads, all original content, and very much not "thin". An enthusiast site, much like the original internet. Incoming links were organic and spontaneous, from serious sites. Panda crushed it. Massive pagerank downgrade and traffic decimated to about 5%. In mere hours. A false positive, I suppose. I spent 3 months trying to figure out the reason but nobody, including Google or SEO specialists were able to provide any tangible answer. That day etched a few important lessons into my long term memory: 1) The size of Google's power. It effectively has the power to decide whether you digitally exist or not. The ultimate traffic controller. 2) Even a good faith move by Google may have 1% of false positives. Which seems totally rational until you realize that at Google's scale this could mean millions of people being totally fucked out of the blue. 3) Worse, when that happens, there's no accountability. You can't go anywhere for help, undo being a false positive, you're just randomly fucked and that's it. You cannot depend on anything ran by Google. Although I'm fine and it wasn't exactly a pivotal moment in my life, it radically changed my view on Google. Early on, I was a fanboy. Google kicking lame Microsoft's butt with miraculous products like search, maps, gmail. I was cheering them on. As of my "lesson", I've grown increasingly cynical. That was 10 years ago. They've become infinitely worse.

quickthrower2 4y ago

The problem here is Google and SEO incentives. Or even perceived SEO gains by writing shallow articles. Notice how many of the articles have a “Pricing” and a bold “We’re Hiring” link ;-)

User23 4y ago

At this rate we're going to have to go back to getting information on programming computers from books. On the plus side, that means if you're willing to dig through the research papers and other hard but useful sources, you have an opportunity to make a difference by writing that book.

numpad0 4y ago

Easy, it’s because a lot of(I’m hopeful it’s not all of) so-called values of modern world come from exploitation, negative externalities, and little creative choices. Working on a honest straight reality is like drinking water, has no calories in it. Coke is better. Wines are more better, and everyone other than than I these days seems to be always on a magic fissile cocktails, and while I don’t get that, they seem well off doing it.

dataflow 4y ago

Stack Overflow is indeed quite painful in this regard, and it really, really doesn't need to be like this. And in fact, it isn't all the time, or in every Stack Exchange. Sometimes, finding the right Stack Exchange site helps a lot (like with retrocomputing), where the premise (say, "I want to get this to run on Windows 95") won't turn into arguments over security patches (or whatever) derailing the question. Or sometimes you just have to find the right people in the community, and nudge them to answer somehow. Certain users tend to be quite generous and go out of their way to answer everything you asked with more to guide you, while others love to chide you and slap you with a question closure or deletion for even thinking about walking off the beaten path.

dsfyu404ed 4y ago
>Or sometimes you just have to find the right people in the community, and nudge them to answer somehow. You see this on Reddit all the time. There will be a Q&A sub for a topic dominated by people with scant if any experience who think they're helping people by googling stuff for them and there will be a comedy subreddit for the same topic. Smart people ask questions in the joke sub because they know that having sufficient understanding to be amused by highly contextual stuff weeds out a lot of the casuals.

f0e4c2f7 4y ago

When I hit that point I've found that books, textbooks, and YouTube videos can be a good resource.

yrgulation 4y ago

__MatrixMan__ 4y ago

Because there's no market for ads targeting one guy fifteen years from now. We've built a web that actively discourages the kind of depth you're after.

ruste 4y ago

I love how quickly after ChatGPT's release we're conflating the chicken and the egg. It's not that the content online is barely better than ChatGPT. ChatGPT is what it is because it was trained on that content.

namaria 4y ago

There are a lot of drivers for fragmentation and opacity online. Venture capital throwing money at start ups to see what sticks, search engine optimization, gut-feeling opinionated blogging to build personal brands (self selecting for unexperienced being loud and misleading a lot of the new arrivals) etc. The cure for me has been two fold: 1) Focusing on books from reputable publishing houses (curated long form content) - but even that takes filtering. I noticed O'Riley has been putting out a lot of noise lately. 2) Focusing on open-standards, sound engineering. I'd rather focus on understanding hardware, operating systems, networking etc then myriad software as a service offerings full of vendor lock-in dark patterns.

antman 4y ago

Have you tried ChatGPT? It gives me better results for a lot of things. Comparison of ChatGPT and Google on 500 coding queries: https://news.ycombinator.com/item?id=34089423 https://news.ycombinator.com/item?id=34089423