16 ms·

Why Is the Web So Monotonous? Google

303 points by amf12 4y ago

deleted 4y ago
[deleted]
norwalkbear 4y ago
I've personally thought it's a side effect of bringing the masses to the internet.
- leobg 4y ago
  Probably true. The only Internet had a great filter. You needed to have money, technical know-how, and a great deal of openness and vision to do anything on the Internet, especially publishing anything there. Nowadays, there is no more filter, neither for access to the Internet, nor for publishing on it. Also, getting people to pay you money is now a proven concept on the Internet. Whereas, in the early days, not only where there few options for taking payments or for monetizing content with ads, but it also wouldn’t have been worth the effort because that just wasn’t a big enough target audience.
HeyItsMatt 4y ago
- Kye 4y ago
  It could be a situation where Google doesn't mean to do it, but also isn't incentivized to fix it. The people who make the most money off Google have the most time and motivation to find ways to manipulate it to protect and increase their profit. Anyone at Google who wants to fix it is incentivized to come up with new features for promotion rather than bugfix.
  - HeyItsMatt 4y ago
- stonemetal12 4y ago
  > if your goal is to maximize the next few quarters results and don't care if you crash the company. Also makes sense if you are optimizing a metric and only considering the data, not where it is leading you.
- stevesearer 4y ago
  I had a scenario today where I looked something up and the search result page yielded zero results. However, there were 3-4 ads above and below the text that said there were zero results.
drivebycomment 4y ago
This is a clear demonstration of the user's expectation of search having exceeded what's possible today. From the article: > Lets look at some examples. One of my favorite places in the world is Koh Lanta, Thailand. When traveling, I’m always on the lookout for places that give off the Koh Lanta vibe. What does that mean? Hard to say, exactly, but having tourist amenities without being touristy. Charming, slow, cheap. I don’t know exactly; if I did, it’d be easier to find. Anyway, forgetting that Google is bad at long tails, I search for what is the koh lanta of croatia? and get: This is a near impossible query for human beings, let alone for computer given the state of the AI at this point in time.
- svnpenn 4y ago
  Vectoring Words (Word Embeddings) - Computerphile https://youtu.be/gQddtTdmG_8?t=835 https://youtu.be/gQddtTdmG_8?t=835
- cfeduke 4y ago
  Maybe. If this were a couple of decades ago and this search were performed, the searcher may simply see zero results. This used to be an acceptable response from a search engine - unless there's some website that specifically contains the search term being sought, simply return zero results. Not thousands or tens of thousands results. What we get instead are incorrect results which are being presented as reasonable answers to the search query in the name of advertising revenue.
  - marcodiego 4y ago
    I remember, in the 90's, I typed "hexdit" in altavista and spent the time visiting exactly 100 websites to find exactly the hex editor I was searching for. I remember the number exactly because it was in the last position (10) of the last page and there were 10 pages.
    - macintux 4y ago
      Ugh. I distinctly remember a dinner party in 1993-94 where we talked about how amazing the web was and how impossible it was to find anything. We discussed registered keywords a la AOL, and I’m sure someone suggested search engines would eventually solve it, but I certainly wouldn’t have wagered significant money on that based on Altavista. When I discovered Google (I think it was running at beta.stanford.edu at the time) it was just short of a miracle.
jaimex2 4y ago
Is there a search engine out there that implements the good old page rank algorithm? The patent expired a while ago now.
- bluGill 4y ago
  It wouldn't work if you did. Back in 2000 everyone created their own web site with a list of links to others they found intersting. Now nobody is creating that input and so you don't get and data to rank by.
  - jaimex2 4y ago
    That's true but we now also have things like Reddit providing those links.
swayvil 4y ago
>websites are penalized for even linking to low-ranking pages! So the top hundred sites link to each other and everything else gets ignored. This the equivalent of social media's echo chambers.
- quickthrower2 4y ago
  I would like to see a source for that claim though. Seems odd to me. Also most pages are low ranking. Even on say HN which is probably high ranking, the page for this comment is likely to be low ranking, because it is new and nothing is linking to it (other than the one link from another leaf-like page).
- codefreeordie 4y ago
  This gets even more effective once you layer in content-based ranking adjustments -- where defined types of content or viewpoints get automatically downranked. You downrank the content directly, and then the rest of the web is highly penalized from even linking to the "disapproved" content, and so that content effectively disappears from the internet, except in tiny dark corners that aren't found via search.
bluGill 4y ago
I miss yahoo of 1996 when they were a table of contents. Practically unmaintainable which is why they gave up, but it was nice to follow a few links and find something interesting on weird subjects. No search at all in those days, but it was better for finding things.
- quickthrower2 4y ago
  Like this: https://dmoz-odp.org/ https://dmoz-odp.org/
  - HeyItsMatt 4y ago
    - bluGill 4y ago
      Maybe i'm just lucky, but every link I tried was not only live but updated reciently. Though i'm still disappointed they are not updating.
  - bluGill 4y ago
    Wow, it actually lets me discover interesting things.
  - beej71 4y ago
    Is this still active? Seems like a great idea, but the page says it's an archive and the search doesn't work. :(
    - wilburTheDog 4y ago
      They've apparently moved to https://curlie.org https://curlie.org Their discussion forum is https://www.resource-zone.com/forum/ https://www.resource-zone.com/forum/
warning26 4y ago
There's a lot of Google hate on HN, and much of it is deserved, but I think this particular trend of blaming Google for the prevalence of useless clickbait articles is a bit much. The real answer to this question is that walled-garden social media took over everything. The much-pined-for "Old Google" worked because people used to actually create content on their own sites, and not just post it on walled garden monolithic social sites like Facebook or Twitter.
- marginalia_nu 4y ago
  I disagree. The fact that Google has such a power over the shape of web traffic, along with a few other websites, absolutely re-shapes the Internet to fit whatever its standards make a website rank well. It's in many ways almost exactly the same process as natural selection. Whatever works gets traffic and gets copied by other websites, whatever doesn't fades away.
  - alexalx666 4y ago
    yes but we can build our own query tools, for ex. Im using `$ so graphql vs sql` and such terminal commands more and more these days
    - marginalia_nu 4y ago
      Oh yeah, this is primarily a problem for Google. They're sort of stuck being chased by the shadow they cast. I've had very little problems cutting through the SEO spam running an independent search engine, but then I don't shape traffic like they do.
    - hn_throwaway_99 4y ago
      Sorry, unrelated, but your comment hit a pet peeve of mine. I love GraphQL, and I think it has a number of features that make it better and more usable for most APIs than a RESTful interface. The thing I despise about GraphQL is they put "QL" in the name, so legions of software developers think that it is somehow comparable to SQL, or somehow is a generic query language for data. GraphQL has absolutely nothing to do with SQL. Comparing "GraphQL vs SQL" is like comparing "HTML vs Java".
abruzzi 4y ago
this page makes a comment that I wish more site developers understood–no results to my search is a much more meaningful result than changing my search to provide results I didn't ask for.
quickthrower2 4y ago
> As it happens, Google rewards websites which use keywords in their url, title, headings, and first 100 words. Just by eyeballing, we can see that this particular website is targeting the keywords “water”, “system”, “irrigation”, and “garden”. Pages like these hyper-optimized to come up for particular searches. The stupid expository stuff exists only to pack “important keywords” into the first 100 words. Another explanation is that the type of company that buys thousands of links is also the type that pays rock bottom for their writers. I don't think you need to necessarily write trite crap in order to do well in SEO.
a9h74j 4y ago
> Mass-appeal queries are, almost by definition, not particularly hard to find. If I need a bus schedule, I know to talk to my local transit authority. If I’m looking to keep up with the Kardashians, I’m not going to have any problems (at least, no search problems.) In identifying a 95/5 reward for supporting "mass-appeal queries", is much of the answer right there? And will the average general-purpose alternative search engine escape similar incentives?
marginalia_nu 4y ago
> What I dream of is Google circa 2006. A time where a search engine searched what you asked for. A time before aggressive SEO. A time before social media, when the only people on the internet had a reason to be there. A time before sticky headers and full-screen modal pop-ups asking you to subscribe to a newsletter before reading the article. A time before click-bait and subscription-only websites which tease you with a paragraph before blurring out the rest of the content. Well, I built sort of exactly this: https://search.marginalia.nu/ https://search.marginalia.nu/ It's not great, but it sure has its moments. Should be noted, regarding Koh Lanta, that travel is one of the most aggressively SEO-spammed topics, along with pharma and online casinos. It's extremely difficult to cut through the noise and reach any sort of signal.
- taftster 4y ago
  [Quoting your quote] - A time before aggressive SEO? No, just that Google was able to defeat it with Page Rank (one of their possibly few innovative ideas to the search arena). Mathematically, Page Rank did very well to weed out the SEO'd websites of the time. Point is, I don't think SEO at the time was any more or less "aggressive", per se. Just different. And clearly, the game has since escalated altogether.
- onlyrealcuzzo 4y ago
  > Well, I built sort of exactly this: https://search.marginalia.nu/ https://search.marginalia.nu/ What's the elevator pitch? What do you actually do that is special? From the about section: > This is an independent DIY search engine that focuses on non-commercial content, and attempts to show you sites you perhaps weren't aware of in favor of the sort of sites you probably already knew existed. > The software for this search engine is all custom-built, and all crawling and indexing is done in-house. The project is open source. Feel free to poke about in the source code or contribute to the development! But that doesn't really give me much info.
  - tigerlily 4y ago
    I know it’s of low value to ask, but who really gives a shit?
ArtofIndirect 4y ago
kmprod 4y ago
> My last point: websites are penalized for even linking to low-ranking pages! No they aren't.
- gweinberg 4y ago
  That was my impression also. My understanding is that your page "quality" is 1 + the sum of the quality all pages that link to your page, divided by the total number of outgoing links. So the quality of pages with incoming links matters, but the quality of pages linked to doesn't, just the number.
RajT88 4y ago
Google is not the front page of my internet. I recently decided to create my own landing page, stored locally on disk, with a funny gif and favicon.ico, and a shortlist of my most used work and personal links (including this site!). No ads, no tracking. Shockingly - I had to install browser extensions to override the new tab page in both Edge and Chrome. This used to be a built in setting! Shady guys. Very shady. Although... Not as shady as the fact that Chrome by default sends your entire browsing history to Google for analysis.
- guntars 4y ago
  > Not as shady as the fact that Chrome by default sends your entire browsing history to Google for analysis. Citation needed. Are you saying that, as I click around the site here, Google knows what comments threads I'm interested in? Edit: I think they're talking about the History sync, which does indeed by default send Google every link you visit, but only if you explicitly enable it. We should get access to this data and leak porn preferences of every senator. That ought to get the laws changed quickly.
  - RajT88 4y ago
    Yes. And I can prove it to you. Settings > Sync and Google Services > Other Google Services > Make Searches and Browsing Better. Underneath the setting it explains, "Send URLs of pages you visit to Google".
    - cma 4y ago
      we need to expand wiretapping laws to this stuff, the disclosure should be upfront and default off
      - RajT88 4y ago
        Indeed. When I discovered this setting existed, and was enabled on all my devices, it made the hairs on my neck stand on end. I think Today is the day I try moving back to Firefox again. I hope it's gotten snappier.
i_have_an_idea 4y ago
The author seems to only have a superfluous understanding of SEO. Yes, keywords matter, as well as inbound and outbound links, but, these days Google is also quite sophisticated about surfacing results that meet the criteria for Good Page Experience and pass the Core Web Vitals. That means responsive web sites, with highly optimized images, font delivery, minimal on-load jitter that load super fast, even on old mobile devices. All of that is good for the end user.
- billyhoffman 4y ago
  True but by Google own posts about initiatives like Core Web Vital and mobile first these are used as tie-breaking factors not as ranking factors. If you have 2 pages that are judged to have equally relevant content per all of Google’s other content ranking, the one with faster core web vitals will appear first. Pages with amazing Core Web Vitals, but poor content ranking, are not ranking more highly than relevant pages with poor web vitals. Example.com has amazing Core Web Vitals.
  - i_have_an_idea 4y ago
    I do work in this area and my observation is that they're used as a lot more than tie-breaking factors. Yes, you can't have an empty page / filled with spam, I never said that you could.
devmunchies 4y ago
I disagree that its google's fault. I think it went from a diverse mainstreet to a walmart. Likely an inevitable progression of any economy if not deliberately prevented. Was likely quickened by Google though. "They paved paradise to put up a parking lot"
redavni 4y ago
I have resorted to using Russian (Yandex) and Chinese (Baidu) search engines in the last few years. It is interesting that non commercial sources have more of a voice on these search engines than on Google today. Google is good at many things, but their leadership is questionable now, unlike in the past. I would pay a subscription fee for a service that provided partity with Google and customer service that answers the phone.
- akudha 4y ago
  Have you tried this one? https://neeva.com/ https://neeva.com/ No idea how good it is, I remember it was posted here a while ago
  - redavni 4y ago
    Have not. I will try it now though.
- outworlder 4y ago
  Aren't those heavily censored?
  - mojuba 4y ago
    Yandex censors the results mostly for Russian IPs as far as I can tell, but they stay neutral for the outside users.
    - trasz 4y ago
      Nope, just yandexed for “bucha massacre”, 2nd place is already Rashist propaganda.
- the_duke 4y ago
  > It is interesting that non commercial sources have more of a voice on these search engines than on Google today. I sort of agree, I also sometimes use Yandex and am usually positively surprised. I can immediately see two reasons: A) SEO efforts are targeted Google/Bing, not other search engines that don't focus on the target market/language/region. Of course that argument can be twisted around: these search engines don't focus on Western languages, so could in theory be more susceptible to SEO trickery. It's interesting that they aren't. B) Ad money. Google is strongly incentivized to prefer search results that show Google ads. For foreign search engines there is no money to be made, so they can disregard that signal.
yazzku 4y ago
I would also ask, why is HN so monotonous? If you compare 'new' vs what goes on the wall, you'll see a non-trivial amount of Big Tech bias and very good posts (often criticisms) effectively getting censored. Is the paycheck that good?
- sebastianconcpt 4y ago
  Cancel culture fear => Peer pressure hypersensibility => Monotonicity
- krapp 4y ago
  It shouldn't be a surprise that most of the content posted to a forum owned by a Silicon Valley startup incubator is related to "Big Tech."
- talkingtab 4y ago
  Exactly! The top "comment" right now is: "this is a clear demonstration of the user's expectation of search having exceeded what's possible today".
- pixl97 4y ago
  Because to avoid trolling and arguments HN filters out controversy and most politics. Yea, it's more monotonous, but it's far less hate filled than most forums and keeps on target far more often.
- Barrin92 4y ago
  > and very good posts (often criticisms) effectively getting censored I browse new occasionally and I encounter a lot of lowbrow substack spam to be honest. Rants against big tech make you a contrarian, problem is you have to be contrarian and right to be interesting. I don't think HN has a big tech bias, most criticism of big tech is just plain awful. If anything in today's discourse there is a stupendous anti big-anything bias (for context, I'm not paid by big tech, I work for a small German company on the other side of the pond and find much of SV culture annoying personally)
- joshka 4y ago
  This comment seems pretty off-topic from the article linked to, can you provide a bit more context on how it's related other than the concept of monotony?
haskell_melody 4y ago
The About page of Read Something Interesting (https://readsomethinginteresting.com/about https://readsomethinginteresting.com/about) conveys this well.
sebastianconcpt 4y ago
...and purple? Huh? Why everything is purple? Seriously
sebastianconcpt 4y ago
It's not technology. It's about culture (which is linked to the social imaginary, a kind of group consciousness operative system, hence a kind of immaterial technology itself). Psychologically, at the core of the problem, this is what happens: Cancel culture fear => Peer pressure hypersensibility => Monotonicity
throwaway539324 4y ago
I was thinking about this recently, because at my business we're working on improving our SEO. The big problem is that Google is the major way people discover goods and services. If you have a site that sells garden hoses, the best way to get eyeballs is to write blog posts that a potential user of garden hoses might search for. "How to install a garden hose." "What is the best garden hose." This kind of content marketing can drive huge traffic and thus huge sales. But only results on the first page of Google matter, and the farther down the first page you are, the less traffic you get. Now, the 15 companies that produce garden hoses are fighting against each other to get the 10 slots in Google's first page for any given keyword. With AI-generated articles, it's become a race to the bottom. Google rewards AI-generated, keyword-optimized blog spam. If you want to be one of the 15 companies that makes it into those 10 slots, you better believe you need to write AI-generated blogspam too. And when everyone has to do that in order to compete, all that's left is the AI-generated blogspam that has infested the modern web. You almost have no choice as a business. Either write AI-generated blogspam that Google loves, or your competition will, and bury you in the search results.
- stevage 4y ago
  This is a good summary of the situation.
will_walker 4y ago
Be careful what you wish for. AI capable of writing weird garbage nonsense content that looks truthful is coming down the pipeline, with a reward function built in for user engagement. In a world of effortless surreal content, authoritatively truthful answer will become more valuable. We're going to need encyclopedias again!
whateveracct 4y ago
Sounds like a use-case for search but - should it ever re-materialize [1] A bazaar of boutique search indexes provided by People seems like it would be nice to have. Better than Google constantly trying to sell me shit. [1] https://paste.sr.ht/~sircmpwn/048293268d4ed4254659c3cd6abe671cb87ecbf9 https://paste.sr.ht/~sircmpwn/048293268d4ed4254659c3cd6abe67...
alexalx666 4y ago
i dig your layout
shadowgovt 4y ago
FTA: > Why don’t other search engines compete on search results? It can’t be hard to do better than Google for the long tail. I think that's the "citation needed" that the author's missing. It's not like Google is unaware of this problem, and a huge piece of their research spend goes into improving search. It's entirely possible nobody's doing better than Google because nobody knows how. The entire paradigm Google's leading in might be a saddle-point with significant activation energy needed to escape it while still having something as usually-useful as Google.
- trasz 4y ago
  Except that “improving search” here means maximizing Google profits, from ads in particular, as opposed to what users would consider an improvement to search.
  - sidibe 4y ago
    There are probably thousands or tens of thousands of people working on search. Some of them are interested in and interface with the business side but I'm sure most of them have objectives just related to search. It's possible they've found and not implemented massive improvements because it's getting sat on by higher ups who are more aware of the business and objectives of the ads teams. If that were the case though you'd probably hear about it and see people leaving in droves to start competitors
    - trasz 4y ago
      You wouldn’t hear about it at all because of NDAs. And people wouldn’t be “leaving in droves”, because they are already used to it.
carabiner 4y ago
Isn't this just getting older? You could just as well ask why is life so monotonous. There's a finite number of experiences, practically speaking, and as you go through each, subsequent versions lack novelty. This is also why time seems to speed up as we age.
wnissen 4y ago
Does this mean that pages with excessively complicated cookie acceptance/rejection also get ranked higher because they have a longer dwell time? Talk about a dark pattern.
lotw_dot_site 4y ago
Among other things, Linux on the Web can be used to usher in a new paradigm for accessing online resources. The current document-centric approach has an intensely graphical (attention grabbing) aspect which very much encourages the least common denominator types of results that the author abhors. Those who want to be more thoughtful about these things need better toolsets to allow them to focus the way that they go about their online lives (including general search). Google is in indeed more than search. They are also the machine learning framework, Tensorflow. Linux on the Web is in as good a position as anything else to start putting the JavaScript implementation (Tensorflow.js) to very good use. We don't need new search engines as much as we need new search engine interfaces.
RootKitBeerCat 4y ago
Hey yeah! Imagine like two small guys in a garage could like make a indexer for the web and call it a search engine and just make it work the best, and gain market dominance and become a little bit slower to innovate than authors “back in 2006” nostalgia…
seydor 4y ago
> like it consists of ten big sites Optimistic. Which are those?
keyle 4y ago
It's not just google. In the past, Mr Joe Blogs was into a passion, he would write about it on his own shoddy blog with gif links to other blogs part of a blog ring of nerdy stuff he cares about. Nowadays, Mr Joe Blogs writes on Medium, shares on Facebook and potentially has an outdated instagram account with 4 pictures of his passion. All buried down by algorithm to never be found, because his passion "isn't sexy". The closed web and their algorithms are just as much to blame.
- krapp 4y ago
  >Nowadays, Mr Joe Blogs writes on Medium, shares on Facebook and potentially has an outdated instagram account with 4 pictures of his passion. All buried down by algorithm to never be found, because his passion "isn't sexy". From what I've observed, the opposite tends to be true more often than not. Plenty of people have been able to create communities around niche interests because platforms bring something independent blogs never really did - an audience. Joe Blogs probably started a subreddit and facebook group for his passion and gets more people reading his content in a week than likely happened to stumble across his blog in a year.
  - qzx_pierri 4y ago
    Yeah but you get no independence that way. Slip up and post the wrong "badspeak" phrase of the month? Shadow banned. As long as content is sitting underneath a huge lingering banhammer, the content will never be as raw and authentic as content was back in 2006. This reinforces the echo chamber issue and also makes the internet feel contrived and soulless.
    - krapp 4y ago
      Getting banned isn't an issue for most people, at least not under the politically driven terms you're describing (just look at the kind of extremist content than flies on Twitter), but even thought banning exists (and it did on the "old" web as well) there's still far more authentic content out there than ever. It just happens that the web is more complex than it was in 2006, and far more culturally culturally diverse, and most people can be authentic on the internet now without also being edgelords or pushing the bounds of TOS.
fudged71 4y ago
This SEO blog ecosystem isn't going to die anytime soon but alternatives will exist. In the next 5 years, "Knowledge Networks" are going to become curated high-density graphs of domain-specific knowledge. Some will be paywalled, some will have ads, but you'll know how to find the information you're looking for.
labrador 4y ago
We all have to get better at using search engines to get past the SEO Google "reminds me of Koh Lanta" turns up Boracay, Philippines in the top three links https://www.google.com/search?q=%22reminds+me+of+Koh-Lanta%22 https://www.google.com/search?q=%22reminds+me+of+Koh-Lanta%2... No you ever used this phrase about anything in Croatia https://www.google.com/search?q=%22reminds+me+of+Koh-Lanta%22%20croatia https://www.google.com/search?q=%22reminds+me+of+Koh-Lanta%2... Translating "reminds me of Koh Lanta" to Croatian doesn't help https://www.google.com/search?q=%22podsje%C4%87a+me+na+Koh-Lantu%22 https://www.google.com/search?q=%22podsje%C4%87a+me+na+Koh-L...
- TimTheTinker 4y ago
  Google's "It looks like there aren't many great matches for your search" response probably means you're onto something interesting :)
  - labrador 4y ago
    I get that a lot. They have a really interesting little character fishing that is animated. It appears to be SVG animated by CSS. I need to learn how to do that.
Karellen 4y ago
> Google search results are significantly less helpful than they used to be. > Anyway, forgetting that Google is bad at long tails, I search for `what is the koh lanta of croatia?` Well, in at least one significant case Google searches are more helpful than they used to be, in that they now take into account natural language queries like "what is..." and try pretty hard to DTRT. Contrast that to "the internet of yore" that the author yearns for, where search engines - including Google - treated all words as keywords, and tended to weight them in the order they were given, so that the query would mostly look for pages containing the word "what", and "is" and "the", and then hope to find a subset containing "koh" and "lanta", and finally, almost incidentally, rank those by whether they might seem relevant to "croatia". Incidentally, those old-style keyword searches may still provide better results in some cases. Try searching `croatia place charming slow cheap`, and see how that compares.
- svnpenn 4y ago
  > and try pretty hard to DTRT TRTD works fine though.
jorgenbuilder 4y ago
Projecting.
awinter-py 4y ago
> Google rewards websites which use keywords in their url, title, headings, and first 100 words can't say whether this is true for sites but they definitely encourage this behavior in ads -- the ads dashboard will shame you ('poor quality ad! we may not show this' kind of prompts) if you don't obey silly rules about 'how many keywords' appear in your title.
smm11 4y ago
I saved all kinds of sites on ZIP drives in the mid-90s. It's amazing how much stuff/information there used to be, instead of whatever it is we have today on the WWW.
llaolleh 4y ago
It's not all Google. The entire medium has shifted for the average user.Overfitted even. The prior barrier if entry for quality content was that you had to be tech savvy. Now anyone and their mom can publish shit content, play the SEO game, and spam dunk the Internet. Maybe the solution to this is really some kind fo web3 variant which Wil put up some hurdles again.
deleted 4y ago
[deleted]
claudiojulio 4y ago
RSS or similar is the solution.
xivzgrev 4y ago
“ For example, have you ever noticed that the main content of most websites is something like 70% down the page? Every recipe site I’ve ever seen is like this — nobody cares about how this recipe was originally your great-grandmother’” If anyone is curious, this is at least part due to copyright law. A recipe is a set of facts, which is no copyrighted. But recipes with substantial literary expression (making it unique) can be copyrighted. Content creators don’t want their stuff copied so viola…that’s how you get a long winded story before the goods. https://copyrightalliance.org/are-recipes-cookbooks-protected-by-copyright/#:~:text=Can%20You%20Copyright%20a%20Recipe%3F,be%20protected%20by%20copyright%20law https://copyrightalliance.org/are-recipes-cookbooks-protecte....
- lethologica 4y ago
  I always thought it was more for the SEO than the copyright.
- GavinMcG 4y ago
  I see this claim repeated but is there any evidence for it? It’s not like writing a long story adds any protection for the recipe. Seems the author’s explanation is simpler (though even simpler is that the audience of those long-winded sites appreciates the content, or the authors enjoy creating it).
- np_tedious 4y ago
  Surely they could put that crap after and achieve the same status?
megablast 4y ago
It’s why recipe sites have a dumb story. In the old days, you might search something simple and it would have the answer in the listing. So you wouldn’t have to click. They are now gone too.
deleted 4y ago
[deleted]
alphabetting 4y ago
> Why don’t other search engines compete on search results? It can’t be hard to do better than Google for the long tail. Search is ridiculously hard. Everyone underestimates the forces you are up against with SEO/Spam industry. Microsoft has invested into Bing for 13 years and has no market share and the #1 search term is "google."
- powerhour 4y ago
  It seems like we've all individually been able to find spam, and I'm guessing Google engineers that use Google for searching have the same experience. Is it just a matter of them allowing us to help train their algorithms? I'd be more than willing to report sites if I had the option.
mullikine 4y ago
We're moving towards the imaginary internet. https://semiosis.github.io/about/ https://semiosis.github.io/about/ https://semiosis.github.io/posts/imaginary-internet-survival-guide/ https://semiosis.github.io/posts/imaginary-internet-survival... Better start encoding your personal mythology into code right now.
Li7h 4y ago
I think eventually more people will realize that the modern web is terrible and they will get fed up with it. People are already building alternatives. There are several situations that could cause Google to lose a foothold as the search behemoth and drive consumers to find alternate ways to find products. Possible catalysts include: regulation of search or advertising negatively affecting google, an unknown search competitor entering the market, another of the behemoths competing (amazon with search, bing getting their act together, etc.), antitrust suits, insert black swan event here. I think the pendulum is also swinging towards smaller, local communities. Maybe that's just my bubble. Eventually everyone will get sick of it though, and something will change. I don't think we can predict what will replace it, but I hope it's not another big, centralized source. We've seen the negatives of that. I don't know how to build it, but to me the ideal Google search killer would be: 1) Decentralized somehow. Hosting search indexes collectively in order to reduce the need for a single entity to host the data. Crypto has this possibility maybe. But I don't know how well it could be implemented, and whether you could have search still be "free" like Google is. How do you solve for that problem? 2) Filters out the SEO spam. Yes, this is a huge problem. My idea to fix it is manual curation. Not scalable. How do you solve that? I think something like StumbleUpon. A curated list of sites that is searchable in its own index. Perhaps there could be a trusted network of curated indexes. Members only to host the curated indexes to keep out the black hats, but free to search for the public at large. I also think we need to educate everyday users to host their own sites on their own hardware. Making it super simple to spin up a web host in a VM that is running on a laptop in the bedroom. Or off a raspberry pi 0. Or whatever is cheap and available. Turn it off when you need to and the website goes offline. Yes, but that is okay in the interim to get Joe Regular hosting a website independently. Sure it's not okay for a business, but for your hobby site about bonsai trees, why not? Of course this is against most ISP's ToS so how do you solve for that? How do you give Joe Regular the ability to deploy a simple, secure website easily? How do you incentivize them to get out of the walled gardens? Give them a garden of their own. That was what made the early web (as I knew it in the 90s and 00s) awesome. You were figuring it out on your own. Google/Bing/DuckDuckGo can stick around and be the digital yellow pages. They are good at that. We need to build a new web for ourselves.
- richardsocher 4y ago
muststopmyths 4y ago
As an aside, 15 years ago I spent a week in koh lanta. It was a lovely time and having returned to briefly visit other parts of Thailand in the years since, I am pleasantly surprised to learn that koh lanta still has some of the charm from that long ago. I too look for koh lanta in every beach town I have visited since then
rrwright 4y ago
“Google is awful for the long tail.” Exactly right. Big opportunity.
jsemrau 4y ago
90% of content writers that I have talked during product validation are using Google to search for writing financial content. They mirrored the sentiment of the author that search results have gotten significantly worse and do not provide much auxiliary insights. The noise/hit ratio is really high. Interestingly, Google Search works really well for Stack Overflow.
- unsafecast 4y ago
  > Interestingly, Google Search works really well for Stack Overflow. Maybe it's because people who use these kinds of websites are generally less tolerant of BS websites than the average person, so Google tends to derank the crappy ones just because we always click away instantly.
  - hnfong 4y ago
    Maybe it's because Google Search is maintained by software engineers and they actually dog food their stuff in this context?
Sunspark 4y ago
When the web was younger than it is now, everything on it was a novelty. It was interesting to see because pages would appear that had never existed before. Now it's all corporate influencer recycling with ads slapped on it.
thayne 4y ago
> None of this is hard to do. It just requires someone to get started. I'm not an expert in this area but that seems very optimistic. Creating an algorithm that gives better quality results than google for certain kinds of queries probably isn't too difficult. But scaling it up to indexing the entire web, continuously, would require a LOT of resources.
SergeAx 4y ago
I glad we have GPT-3 and DALLE-E 2 and all the improvements in generative ML. It means the internet will switch from wide discovery mode to narrow subscription and recommendation. This is the only way to deal with generated search spam.
seu 4y ago
You don't need such a complex analysis, it's actually much simpler: you are mostly shown websites that try to make money from you, because those are the ones optimizing as hell to be at the front of the search. Any other website, where people just share information freely, where you might get different opinions, experiences, hell, _anything that is different from just trying to sell you something_ (even if it is ads), will be very far in the back.
yarg 4y ago
Google, facebook, walled gardens in general, and the fact that ~100% of users neither know nor care how much better things could be.
wyuenho 4y ago
> thus changing the problem of trustworthiness to one of authority I wish Google would just penalize Quora. I can't ask any question on Google without seeing a bunch of Quora fake experts on the first page providing vacant answers to any question.
- intelVISA 4y ago
  Even worse when it's CS related and you get fed sheer nonsense from a "Dev Guru 25YOE" like damn idk what the target demographic is for the SWE side of Quora but I sure hope to never be part of it.
hooby 4y ago
For decades now, SEOptimizers have constantly found new tricks to get their websites ranked higher than they should, while search engines have constantly updated their algorithms to make them harder to game like that. These two things have co-evolved in response to each other, and what this article laments is the result of exactly that process. But the interaction between SEO and Google search is way more complex than the article implies. In particular, the influence goes both ways, and it's not a clear one-way cause and effect relationship. Also playing into this are a lot of other factors, like the increasing commercialization of the web, how the online advertising business works, Googles dominant position in both web search and web advertising, etc... I really don't think the solution to this can be as simple as "change how Google rewards keywords". It's not that simple.
edpichler 4y ago
For those who agree that the internet is getting worse, I recommend joining the http://www.indieweb.org http://www.indieweb.org movement.
nsmog767 4y ago
SEO person here. If you look at any topic or vertical with a lot of competition, there is bound to be monotony. That's because SEO people are trained to study "search intent", meaning, what do people think about when they search? What questions/desires can we, as publishers, help them fulfill? It's a good theory, but becomes circular when you consider that one of the primary ways to determine search intent is . . . seeing what does well on Google. So if you search for, say, "customer feedback", you see a lot of general guides about strategy, offering general definitions. That's likely because the first couple pages that ranked did it, and then more players saw that and said "oh, search intent for 'customer feedback' is [whatever they are doing]." When you see all the listicles for certain queries, that's what you're seeing. Yes, it probably reflects some desire from people to see a list of options. But it's also just SEO people saying "look, the search intent is X." It leads to everyone copying each other and reduces the incentive to try and be innovative/think outside the box with content.
tensionheadx 4y ago
"If I’m looking to keep up with the Kardashians, I’m not going to have any problems (at least, no search problems.)" Touché :]
mpcannabrava 4y ago
What are your best links to the dark, beautiful corners of the internet?

303 points by amf12 4y ago

deleted 4y ago

[deleted]

norwalkbear 4y ago

I've personally thought it's a side effect of bringing the masses to the internet.

leobg 4y ago
Probably true. The only Internet had a great filter. You needed to have money, technical know-how, and a great deal of openness and vision to do anything on the Internet, especially publishing anything there. Nowadays, there is no more filter, neither for access to the Internet, nor for publishing on it. Also, getting people to pay you money is now a proven concept on the Internet. Whereas, in the early days, not only where there few options for taking payments or for monetizing content with ads, but it also wouldn’t have been worth the effort because that just wasn’t a big enough target audience.

HeyItsMatt 4y ago

Kye 4y ago
It could be a situation where Google doesn't mean to do it, but also isn't incentivized to fix it. The people who make the most money off Google have the most time and motivation to find ways to manipulate it to protect and increase their profit. Anyone at Google who wants to fix it is incentivized to come up with new features for promotion rather than bugfix.
- HeyItsMatt 4y ago
stonemetal12 4y ago
> if your goal is to maximize the next few quarters results and don't care if you crash the company. Also makes sense if you are optimizing a metric and only considering the data, not where it is leading you.
stevesearer 4y ago
I had a scenario today where I looked something up and the search result page yielded zero results. However, there were 3-4 ads above and below the text that said there were zero results.

drivebycomment 4y ago

This is a clear demonstration of the user's expectation of search having exceeded what's possible today. From the article: > Lets look at some examples. One of my favorite places in the world is Koh Lanta, Thailand. When traveling, I’m always on the lookout for places that give off the Koh Lanta vibe. What does that mean? Hard to say, exactly, but having tourist amenities without being touristy. Charming, slow, cheap. I don’t know exactly; if I did, it’d be easier to find. Anyway, forgetting that Google is bad at long tails, I search for what is the koh lanta of croatia? and get: This is a near impossible query for human beings, let alone for computer given the state of the AI at this point in time.

svnpenn 4y ago
Vectoring Words (Word Embeddings) - Computerphile https://youtu.be/gQddtTdmG_8?t=835 https://youtu.be/gQddtTdmG_8?t=835
cfeduke 4y ago
Maybe. If this were a couple of decades ago and this search were performed, the searcher may simply see zero results. This used to be an acceptable response from a search engine - unless there's some website that specifically contains the search term being sought, simply return zero results. Not thousands or tens of thousands results. What we get instead are incorrect results which are being presented as reasonable answers to the search query in the name of advertising revenue.
- marcodiego 4y ago
  I remember, in the 90's, I typed "hexdit" in altavista and spent the time visiting exactly 100 websites to find exactly the hex editor I was searching for. I remember the number exactly because it was in the last position (10) of the last page and there were 10 pages.
  - macintux 4y ago
    Ugh. I distinctly remember a dinner party in 1993-94 where we talked about how amazing the web was and how impossible it was to find anything. We discussed registered keywords a la AOL, and I’m sure someone suggested search engines would eventually solve it, but I certainly wouldn’t have wagered significant money on that based on Altavista. When I discovered Google (I think it was running at beta.stanford.edu at the time) it was just short of a miracle.

jaimex2 4y ago

Is there a search engine out there that implements the good old page rank algorithm? The patent expired a while ago now.

bluGill 4y ago
It wouldn't work if you did. Back in 2000 everyone created their own web site with a list of links to others they found intersting. Now nobody is creating that input and so you don't get and data to rank by.
- jaimex2 4y ago
  That's true but we now also have things like Reddit providing those links.

swayvil 4y ago

>websites are penalized for even linking to low-ranking pages! So the top hundred sites link to each other and everything else gets ignored. This the equivalent of social media's echo chambers.

quickthrower2 4y ago
I would like to see a source for that claim though. Seems odd to me. Also most pages are low ranking. Even on say HN which is probably high ranking, the page for this comment is likely to be low ranking, because it is new and nothing is linking to it (other than the one link from another leaf-like page).
codefreeordie 4y ago
This gets even more effective once you layer in content-based ranking adjustments -- where defined types of content or viewpoints get automatically downranked. You downrank the content directly, and then the rest of the web is highly penalized from even linking to the "disapproved" content, and so that content effectively disappears from the internet, except in tiny dark corners that aren't found via search.

bluGill 4y ago

I miss yahoo of 1996 when they were a table of contents. Practically unmaintainable which is why they gave up, but it was nice to follow a few links and find something interesting on weird subjects. No search at all in those days, but it was better for finding things.

quickthrower2 4y ago
Like this: https://dmoz-odp.org/ https://dmoz-odp.org/
- HeyItsMatt 4y ago
  - bluGill 4y ago
    Maybe i'm just lucky, but every link I tried was not only live but updated reciently. Though i'm still disappointed they are not updating.
- bluGill 4y ago
  Wow, it actually lets me discover interesting things.
- beej71 4y ago
  Is this still active? Seems like a great idea, but the page says it's an archive and the search doesn't work. :(
  - wilburTheDog 4y ago
    They've apparently moved to https://curlie.org https://curlie.org Their discussion forum is https://www.resource-zone.com/forum/ https://www.resource-zone.com/forum/

warning26 4y ago

There's a lot of Google hate on HN, and much of it is deserved, but I think this particular trend of blaming Google for the prevalence of useless clickbait articles is a bit much. The real answer to this question is that walled-garden social media took over everything. The much-pined-for "Old Google" worked because people used to actually create content on their own sites, and not just post it on walled garden monolithic social sites like Facebook or Twitter.

marginalia_nu 4y ago
I disagree. The fact that Google has such a power over the shape of web traffic, along with a few other websites, absolutely re-shapes the Internet to fit whatever its standards make a website rank well. It's in many ways almost exactly the same process as natural selection. Whatever works gets traffic and gets copied by other websites, whatever doesn't fades away.
- alexalx666 4y ago
  yes but we can build our own query tools, for ex. Im using `$ so graphql vs sql` and such terminal commands more and more these days
  - marginalia_nu 4y ago
    Oh yeah, this is primarily a problem for Google. They're sort of stuck being chased by the shadow they cast. I've had very little problems cutting through the SEO spam running an independent search engine, but then I don't shape traffic like they do.
  - hn_throwaway_99 4y ago
    Sorry, unrelated, but your comment hit a pet peeve of mine. I love GraphQL, and I think it has a number of features that make it better and more usable for most APIs than a RESTful interface. The thing I despise about GraphQL is they put "QL" in the name, so legions of software developers think that it is somehow comparable to SQL, or somehow is a generic query language for data. GraphQL has absolutely nothing to do with SQL. Comparing "GraphQL vs SQL" is like comparing "HTML vs Java".

abruzzi 4y ago

this page makes a comment that I wish more site developers understood–no results to my search is a much more meaningful result than changing my search to provide results I didn't ask for.

quickthrower2 4y ago

> As it happens, Google rewards websites which use keywords in their url, title, headings, and first 100 words. Just by eyeballing, we can see that this particular website is targeting the keywords “water”, “system”, “irrigation”, and “garden”. Pages like these hyper-optimized to come up for particular searches. The stupid expository stuff exists only to pack “important keywords” into the first 100 words. Another explanation is that the type of company that buys thousands of links is also the type that pays rock bottom for their writers. I don't think you need to necessarily write trite crap in order to do well in SEO.

a9h74j 4y ago

> Mass-appeal queries are, almost by definition, not particularly hard to find. If I need a bus schedule, I know to talk to my local transit authority. If I’m looking to keep up with the Kardashians, I’m not going to have any problems (at least, no search problems.) In identifying a 95/5 reward for supporting "mass-appeal queries", is much of the answer right there? And will the average general-purpose alternative search engine escape similar incentives?

marginalia_nu 4y ago

> What I dream of is Google circa 2006. A time where a search engine searched what you asked for. A time before aggressive SEO. A time before social media, when the only people on the internet had a reason to be there. A time before sticky headers and full-screen modal pop-ups asking you to subscribe to a newsletter before reading the article. A time before click-bait and subscription-only websites which tease you with a paragraph before blurring out the rest of the content. Well, I built sort of exactly this: https://search.marginalia.nu/ https://search.marginalia.nu/ It's not great, but it sure has its moments. Should be noted, regarding Koh Lanta, that travel is one of the most aggressively SEO-spammed topics, along with pharma and online casinos. It's extremely difficult to cut through the noise and reach any sort of signal.

taftster 4y ago
[Quoting your quote] - A time before aggressive SEO? No, just that Google was able to defeat it with Page Rank (one of their possibly few innovative ideas to the search arena). Mathematically, Page Rank did very well to weed out the SEO'd websites of the time. Point is, I don't think SEO at the time was any more or less "aggressive", per se. Just different. And clearly, the game has since escalated altogether.
onlyrealcuzzo 4y ago
> Well, I built sort of exactly this: https://search.marginalia.nu/ https://search.marginalia.nu/ What's the elevator pitch? What do you actually do that is special? From the about section: > This is an independent DIY search engine that focuses on non-commercial content, and attempts to show you sites you perhaps weren't aware of in favor of the sort of sites you probably already knew existed. > The software for this search engine is all custom-built, and all crawling and indexing is done in-house. The project is open source. Feel free to poke about in the source code or contribute to the development! But that doesn't really give me much info.
- tigerlily 4y ago
  I know it’s of low value to ask, but who really gives a shit?

ArtofIndirect 4y ago

kmprod 4y ago

> My last point: websites are penalized for even linking to low-ranking pages! No they aren't.

gweinberg 4y ago
That was my impression also. My understanding is that your page "quality" is 1 + the sum of the quality all pages that link to your page, divided by the total number of outgoing links. So the quality of pages with incoming links matters, but the quality of pages linked to doesn't, just the number.

RajT88 4y ago

Google is not the front page of my internet. I recently decided to create my own landing page, stored locally on disk, with a funny gif and favicon.ico, and a shortlist of my most used work and personal links (including this site!). No ads, no tracking. Shockingly - I had to install browser extensions to override the new tab page in both Edge and Chrome. This used to be a built in setting! Shady guys. Very shady. Although... Not as shady as the fact that Chrome by default sends your entire browsing history to Google for analysis.

guntars 4y ago
> Not as shady as the fact that Chrome by default sends your entire browsing history to Google for analysis. Citation needed. Are you saying that, as I click around the site here, Google knows what comments threads I'm interested in? Edit: I think they're talking about the History sync, which does indeed by default send Google every link you visit, but only if you explicitly enable it. We should get access to this data and leak porn preferences of every senator. That ought to get the laws changed quickly.
- RajT88 4y ago
  Yes. And I can prove it to you. Settings > Sync and Google Services > Other Google Services > Make Searches and Browsing Better. Underneath the setting it explains, "Send URLs of pages you visit to Google".
  - cma 4y ago
    we need to expand wiretapping laws to this stuff, the disclosure should be upfront and default off
    - RajT88 4y ago
      Indeed. When I discovered this setting existed, and was enabled on all my devices, it made the hairs on my neck stand on end. I think Today is the day I try moving back to Firefox again. I hope it's gotten snappier.

i_have_an_idea 4y ago

The author seems to only have a superfluous understanding of SEO. Yes, keywords matter, as well as inbound and outbound links, but, these days Google is also quite sophisticated about surfacing results that meet the criteria for Good Page Experience and pass the Core Web Vitals. That means responsive web sites, with highly optimized images, font delivery, minimal on-load jitter that load super fast, even on old mobile devices. All of that is good for the end user.

billyhoffman 4y ago
True but by Google own posts about initiatives like Core Web Vital and mobile first these are used as tie-breaking factors not as ranking factors. If you have 2 pages that are judged to have equally relevant content per all of Google’s other content ranking, the one with faster core web vitals will appear first. Pages with amazing Core Web Vitals, but poor content ranking, are not ranking more highly than relevant pages with poor web vitals. Example.com has amazing Core Web Vitals.
- i_have_an_idea 4y ago
  I do work in this area and my observation is that they're used as a lot more than tie-breaking factors. Yes, you can't have an empty page / filled with spam, I never said that you could.

devmunchies 4y ago

I disagree that its google's fault. I think it went from a diverse mainstreet to a walmart. Likely an inevitable progression of any economy if not deliberately prevented. Was likely quickened by Google though. "They paved paradise to put up a parking lot"

redavni 4y ago

I have resorted to using Russian (Yandex) and Chinese (Baidu) search engines in the last few years. It is interesting that non commercial sources have more of a voice on these search engines than on Google today. Google is good at many things, but their leadership is questionable now, unlike in the past. I would pay a subscription fee for a service that provided partity with Google and customer service that answers the phone.

akudha 4y ago
Have you tried this one? https://neeva.com/ https://neeva.com/ No idea how good it is, I remember it was posted here a while ago
- redavni 4y ago
  Have not. I will try it now though.
outworlder 4y ago
Aren't those heavily censored?
- mojuba 4y ago
  Yandex censors the results mostly for Russian IPs as far as I can tell, but they stay neutral for the outside users.
  - trasz 4y ago
    Nope, just yandexed for “bucha massacre”, 2nd place is already Rashist propaganda.
the_duke 4y ago
> It is interesting that non commercial sources have more of a voice on these search engines than on Google today. I sort of agree, I also sometimes use Yandex and am usually positively surprised. I can immediately see two reasons: A) SEO efforts are targeted Google/Bing, not other search engines that don't focus on the target market/language/region. Of course that argument can be twisted around: these search engines don't focus on Western languages, so could in theory be more susceptible to SEO trickery. It's interesting that they aren't. B) Ad money. Google is strongly incentivized to prefer search results that show Google ads. For foreign search engines there is no money to be made, so they can disregard that signal.

yazzku 4y ago

I would also ask, why is HN so monotonous? If you compare 'new' vs what goes on the wall, you'll see a non-trivial amount of Big Tech bias and very good posts (often criticisms) effectively getting censored. Is the paycheck that good?

sebastianconcpt 4y ago
Cancel culture fear => Peer pressure hypersensibility => Monotonicity
krapp 4y ago
It shouldn't be a surprise that most of the content posted to a forum owned by a Silicon Valley startup incubator is related to "Big Tech."
talkingtab 4y ago
Exactly! The top "comment" right now is: "this is a clear demonstration of the user's expectation of search having exceeded what's possible today".
pixl97 4y ago
Because to avoid trolling and arguments HN filters out controversy and most politics. Yea, it's more monotonous, but it's far less hate filled than most forums and keeps on target far more often.
Barrin92 4y ago
> and very good posts (often criticisms) effectively getting censored I browse new occasionally and I encounter a lot of lowbrow substack spam to be honest. Rants against big tech make you a contrarian, problem is you have to be contrarian and right to be interesting. I don't think HN has a big tech bias, most criticism of big tech is just plain awful. If anything in today's discourse there is a stupendous anti big-anything bias (for context, I'm not paid by big tech, I work for a small German company on the other side of the pond and find much of SV culture annoying personally)
joshka 4y ago
This comment seems pretty off-topic from the article linked to, can you provide a bit more context on how it's related other than the concept of monotony?

haskell_melody 4y ago

The About page of Read Something Interesting (https://readsomethinginteresting.com/about https://readsomethinginteresting.com/about) conveys this well.

sebastianconcpt 4y ago

...and purple? Huh? Why everything is purple? Seriously

sebastianconcpt 4y ago

It's not technology. It's about culture (which is linked to the social imaginary, a kind of group consciousness operative system, hence a kind of immaterial technology itself). Psychologically, at the core of the problem, this is what happens: Cancel culture fear => Peer pressure hypersensibility => Monotonicity

throwaway539324 4y ago

I was thinking about this recently, because at my business we're working on improving our SEO. The big problem is that Google is the major way people discover goods and services. If you have a site that sells garden hoses, the best way to get eyeballs is to write blog posts that a potential user of garden hoses might search for. "How to install a garden hose." "What is the best garden hose." This kind of content marketing can drive huge traffic and thus huge sales. But only results on the first page of Google matter, and the farther down the first page you are, the less traffic you get. Now, the 15 companies that produce garden hoses are fighting against each other to get the 10 slots in Google's first page for any given keyword. With AI-generated articles, it's become a race to the bottom. Google rewards AI-generated, keyword-optimized blog spam. If you want to be one of the 15 companies that makes it into those 10 slots, you better believe you need to write AI-generated blogspam too. And when everyone has to do that in order to compete, all that's left is the AI-generated blogspam that has infested the modern web. You almost have no choice as a business. Either write AI-generated blogspam that Google loves, or your competition will, and bury you in the search results.

stevage 4y ago
This is a good summary of the situation.

will_walker 4y ago

Be careful what you wish for. AI capable of writing weird garbage nonsense content that looks truthful is coming down the pipeline, with a reward function built in for user engagement. In a world of effortless surreal content, authoritatively truthful answer will become more valuable. We're going to need encyclopedias again!

whateveracct 4y ago

Sounds like a use-case for search but - should it ever re-materialize [1] A bazaar of boutique search indexes provided by People seems like it would be nice to have. Better than Google constantly trying to sell me shit. [1] https://paste.sr.ht/~sircmpwn/048293268d4ed4254659c3cd6abe671cb87ecbf9 https://paste.sr.ht/~sircmpwn/048293268d4ed4254659c3cd6abe67...

alexalx666 4y ago

i dig your layout

shadowgovt 4y ago

FTA: > Why don’t other search engines compete on search results? It can’t be hard to do better than Google for the long tail. I think that's the "citation needed" that the author's missing. It's not like Google is unaware of this problem, and a huge piece of their research spend goes into improving search. It's entirely possible nobody's doing better than Google because nobody knows how. The entire paradigm Google's leading in might be a saddle-point with significant activation energy needed to escape it while still having something as usually-useful as Google.

trasz 4y ago
Except that “improving search” here means maximizing Google profits, from ads in particular, as opposed to what users would consider an improvement to search.
- sidibe 4y ago
  There are probably thousands or tens of thousands of people working on search. Some of them are interested in and interface with the business side but I'm sure most of them have objectives just related to search. It's possible they've found and not implemented massive improvements because it's getting sat on by higher ups who are more aware of the business and objectives of the ads teams. If that were the case though you'd probably hear about it and see people leaving in droves to start competitors
  - trasz 4y ago
    You wouldn’t hear about it at all because of NDAs. And people wouldn’t be “leaving in droves”, because they are already used to it.

carabiner 4y ago

Isn't this just getting older? You could just as well ask why is life so monotonous. There's a finite number of experiences, practically speaking, and as you go through each, subsequent versions lack novelty. This is also why time seems to speed up as we age.

wnissen 4y ago

Does this mean that pages with excessively complicated cookie acceptance/rejection also get ranked higher because they have a longer dwell time? Talk about a dark pattern.

lotw_dot_site 4y ago

Among other things, Linux on the Web can be used to usher in a new paradigm for accessing online resources. The current document-centric approach has an intensely graphical (attention grabbing) aspect which very much encourages the least common denominator types of results that the author abhors. Those who want to be more thoughtful about these things need better toolsets to allow them to focus the way that they go about their online lives (including general search). Google is in indeed more than search. They are also the machine learning framework, Tensorflow. Linux on the Web is in as good a position as anything else to start putting the JavaScript implementation (Tensorflow.js) to very good use. We don't need new search engines as much as we need new search engine interfaces.

RootKitBeerCat 4y ago

Hey yeah! Imagine like two small guys in a garage could like make a indexer for the web and call it a search engine and just make it work the best, and gain market dominance and become a little bit slower to innovate than authors “back in 2006” nostalgia…

seydor 4y ago

> like it consists of ten big sites Optimistic. Which are those?

keyle 4y ago

It's not just google. In the past, Mr Joe Blogs was into a passion, he would write about it on his own shoddy blog with gif links to other blogs part of a blog ring of nerdy stuff he cares about. Nowadays, Mr Joe Blogs writes on Medium, shares on Facebook and potentially has an outdated instagram account with 4 pictures of his passion. All buried down by algorithm to never be found, because his passion "isn't sexy". The closed web and their algorithms are just as much to blame.

krapp 4y ago
>Nowadays, Mr Joe Blogs writes on Medium, shares on Facebook and potentially has an outdated instagram account with 4 pictures of his passion. All buried down by algorithm to never be found, because his passion "isn't sexy". From what I've observed, the opposite tends to be true more often than not. Plenty of people have been able to create communities around niche interests because platforms bring something independent blogs never really did - an audience. Joe Blogs probably started a subreddit and facebook group for his passion and gets more people reading his content in a week than likely happened to stumble across his blog in a year.
- qzx_pierri 4y ago
  Yeah but you get no independence that way. Slip up and post the wrong "badspeak" phrase of the month? Shadow banned. As long as content is sitting underneath a huge lingering banhammer, the content will never be as raw and authentic as content was back in 2006. This reinforces the echo chamber issue and also makes the internet feel contrived and soulless.
  - krapp 4y ago
    Getting banned isn't an issue for most people, at least not under the politically driven terms you're describing (just look at the kind of extremist content than flies on Twitter), but even thought banning exists (and it did on the "old" web as well) there's still far more authentic content out there than ever. It just happens that the web is more complex than it was in 2006, and far more culturally culturally diverse, and most people can be authentic on the internet now without also being edgelords or pushing the bounds of TOS.

fudged71 4y ago

This SEO blog ecosystem isn't going to die anytime soon but alternatives will exist. In the next 5 years, "Knowledge Networks" are going to become curated high-density graphs of domain-specific knowledge. Some will be paywalled, some will have ads, but you'll know how to find the information you're looking for.

labrador 4y ago

We all have to get better at using search engines to get past the SEO Google "reminds me of Koh Lanta" turns up Boracay, Philippines in the top three links https://www.google.com/search?q=%22reminds+me+of+Koh-Lanta%22 https://www.google.com/search?q=%22reminds+me+of+Koh-Lanta%2... No you ever used this phrase about anything in Croatia https://www.google.com/search?q=%22reminds+me+of+Koh-Lanta%22%20croatia https://www.google.com/search?q=%22reminds+me+of+Koh-Lanta%2... Translating "reminds me of Koh Lanta" to Croatian doesn't help https://www.google.com/search?q=%22podsje%C4%87a+me+na+Koh-Lantu%22 https://www.google.com/search?q=%22podsje%C4%87a+me+na+Koh-L...

TimTheTinker 4y ago
Google's "It looks like there aren't many great matches for your search" response probably means you're onto something interesting :)
- labrador 4y ago
  I get that a lot. They have a really interesting little character fishing that is animated. It appears to be SVG animated by CSS. I need to learn how to do that.

Karellen 4y ago

> Google search results are significantly less helpful than they used to be. > Anyway, forgetting that Google is bad at long tails, I search for `what is the koh lanta of croatia?` Well, in at least one significant case Google searches are more helpful than they used to be, in that they now take into account natural language queries like "what is..." and try pretty hard to DTRT. Contrast that to "the internet of yore" that the author yearns for, where search engines - including Google - treated all words as keywords, and tended to weight them in the order they were given, so that the query would mostly look for pages containing the word "what", and "is" and "the", and then hope to find a subset containing "koh" and "lanta", and finally, almost incidentally, rank those by whether they might seem relevant to "croatia". Incidentally, those old-style keyword searches may still provide better results in some cases. Try searching `croatia place charming slow cheap`, and see how that compares.

svnpenn 4y ago
> and try pretty hard to DTRT TRTD works fine though.

jorgenbuilder 4y ago

Projecting.

awinter-py 4y ago

> Google rewards websites which use keywords in their url, title, headings, and first 100 words can't say whether this is true for sites but they definitely encourage this behavior in ads -- the ads dashboard will shame you ('poor quality ad! we may not show this' kind of prompts) if you don't obey silly rules about 'how many keywords' appear in your title.

smm11 4y ago

I saved all kinds of sites on ZIP drives in the mid-90s. It's amazing how much stuff/information there used to be, instead of whatever it is we have today on the WWW.

llaolleh 4y ago

It's not all Google. The entire medium has shifted for the average user.Overfitted even. The prior barrier if entry for quality content was that you had to be tech savvy. Now anyone and their mom can publish shit content, play the SEO game, and spam dunk the Internet. Maybe the solution to this is really some kind fo web3 variant which Wil put up some hurdles again.

deleted 4y ago

[deleted]

claudiojulio 4y ago

RSS or similar is the solution.

xivzgrev 4y ago

“ For example, have you ever noticed that the main content of most websites is something like 70% down the page? Every recipe site I’ve ever seen is like this — nobody cares about how this recipe was originally your great-grandmother’” If anyone is curious, this is at least part due to copyright law. A recipe is a set of facts, which is no copyrighted. But recipes with substantial literary expression (making it unique) can be copyrighted. Content creators don’t want their stuff copied so viola…that’s how you get a long winded story before the goods. https://copyrightalliance.org/are-recipes-cookbooks-protected-by-copyright/#:~:text=Can%20You%20Copyright%20a%20Recipe%3F,be%20protected%20by%20copyright%20law https://copyrightalliance.org/are-recipes-cookbooks-protecte....

lethologica 4y ago
I always thought it was more for the SEO than the copyright.
GavinMcG 4y ago
I see this claim repeated but is there any evidence for it? It’s not like writing a long story adds any protection for the recipe. Seems the author’s explanation is simpler (though even simpler is that the audience of those long-winded sites appreciates the content, or the authors enjoy creating it).
np_tedious 4y ago
Surely they could put that crap after and achieve the same status?

megablast 4y ago

It’s why recipe sites have a dumb story. In the old days, you might search something simple and it would have the answer in the listing. So you wouldn’t have to click. They are now gone too.

deleted 4y ago

[deleted]

alphabetting 4y ago

> Why don’t other search engines compete on search results? It can’t be hard to do better than Google for the long tail. Search is ridiculously hard. Everyone underestimates the forces you are up against with SEO/Spam industry. Microsoft has invested into Bing for 13 years and has no market share and the #1 search term is "google."

powerhour 4y ago
It seems like we've all individually been able to find spam, and I'm guessing Google engineers that use Google for searching have the same experience. Is it just a matter of them allowing us to help train their algorithms? I'd be more than willing to report sites if I had the option.

mullikine 4y ago

We're moving towards the imaginary internet. https://semiosis.github.io/about/ https://semiosis.github.io/about/ https://semiosis.github.io/posts/imaginary-internet-survival-guide/ https://semiosis.github.io/posts/imaginary-internet-survival... Better start encoding your personal mythology into code right now.

Li7h 4y ago

I think eventually more people will realize that the modern web is terrible and they will get fed up with it. People are already building alternatives. There are several situations that could cause Google to lose a foothold as the search behemoth and drive consumers to find alternate ways to find products. Possible catalysts include: regulation of search or advertising negatively affecting google, an unknown search competitor entering the market, another of the behemoths competing (amazon with search, bing getting their act together, etc.), antitrust suits, insert black swan event here. I think the pendulum is also swinging towards smaller, local communities. Maybe that's just my bubble. Eventually everyone will get sick of it though, and something will change. I don't think we can predict what will replace it, but I hope it's not another big, centralized source. We've seen the negatives of that. I don't know how to build it, but to me the ideal Google search killer would be: 1) Decentralized somehow. Hosting search indexes collectively in order to reduce the need for a single entity to host the data. Crypto has this possibility maybe. But I don't know how well it could be implemented, and whether you could have search still be "free" like Google is. How do you solve for that problem? 2) Filters out the SEO spam. Yes, this is a huge problem. My idea to fix it is manual curation. Not scalable. How do you solve that? I think something like StumbleUpon. A curated list of sites that is searchable in its own index. Perhaps there could be a trusted network of curated indexes. Members only to host the curated indexes to keep out the black hats, but free to search for the public at large. I also think we need to educate everyday users to host their own sites on their own hardware. Making it super simple to spin up a web host in a VM that is running on a laptop in the bedroom. Or off a raspberry pi 0. Or whatever is cheap and available. Turn it off when you need to and the website goes offline. Yes, but that is okay in the interim to get Joe Regular hosting a website independently. Sure it's not okay for a business, but for your hobby site about bonsai trees, why not? Of course this is against most ISP's ToS so how do you solve for that? How do you give Joe Regular the ability to deploy a simple, secure website easily? How do you incentivize them to get out of the walled gardens? Give them a garden of their own. That was what made the early web (as I knew it in the 90s and 00s) awesome. You were figuring it out on your own. Google/Bing/DuckDuckGo can stick around and be the digital yellow pages. They are good at that. We need to build a new web for ourselves.

richardsocher 4y ago

muststopmyths 4y ago

As an aside, 15 years ago I spent a week in koh lanta. It was a lovely time and having returned to briefly visit other parts of Thailand in the years since, I am pleasantly surprised to learn that koh lanta still has some of the charm from that long ago. I too look for koh lanta in every beach town I have visited since then

rrwright 4y ago

“Google is awful for the long tail.” Exactly right. Big opportunity.

jsemrau 4y ago

90% of content writers that I have talked during product validation are using Google to search for writing financial content. They mirrored the sentiment of the author that search results have gotten significantly worse and do not provide much auxiliary insights. The noise/hit ratio is really high. Interestingly, Google Search works really well for Stack Overflow.

unsafecast 4y ago
> Interestingly, Google Search works really well for Stack Overflow. Maybe it's because people who use these kinds of websites are generally less tolerant of BS websites than the average person, so Google tends to derank the crappy ones just because we always click away instantly.
- hnfong 4y ago
  Maybe it's because Google Search is maintained by software engineers and they actually dog food their stuff in this context?

Sunspark 4y ago

When the web was younger than it is now, everything on it was a novelty. It was interesting to see because pages would appear that had never existed before. Now it's all corporate influencer recycling with ads slapped on it.

thayne 4y ago

> None of this is hard to do. It just requires someone to get started. I'm not an expert in this area but that seems very optimistic. Creating an algorithm that gives better quality results than google for certain kinds of queries probably isn't too difficult. But scaling it up to indexing the entire web, continuously, would require a LOT of resources.

SergeAx 4y ago

I glad we have GPT-3 and DALLE-E 2 and all the improvements in generative ML. It means the internet will switch from wide discovery mode to narrow subscription and recommendation. This is the only way to deal with generated search spam.

seu 4y ago

You don't need such a complex analysis, it's actually much simpler: you are mostly shown websites that try to make money from you, because those are the ones optimizing as hell to be at the front of the search. Any other website, where people just share information freely, where you might get different opinions, experiences, hell, _anything that is different from just trying to sell you something_ (even if it is ads), will be very far in the back.

yarg 4y ago

Google, facebook, walled gardens in general, and the fact that ~100% of users neither know nor care how much better things could be.

wyuenho 4y ago

> thus changing the problem of trustworthiness to one of authority I wish Google would just penalize Quora. I can't ask any question on Google without seeing a bunch of Quora fake experts on the first page providing vacant answers to any question.

intelVISA 4y ago
Even worse when it's CS related and you get fed sheer nonsense from a "Dev Guru 25YOE" like damn idk what the target demographic is for the SWE side of Quora but I sure hope to never be part of it.

hooby 4y ago

For decades now, SEOptimizers have constantly found new tricks to get their websites ranked higher than they should, while search engines have constantly updated their algorithms to make them harder to game like that. These two things have co-evolved in response to each other, and what this article laments is the result of exactly that process. But the interaction between SEO and Google search is way more complex than the article implies. In particular, the influence goes both ways, and it's not a clear one-way cause and effect relationship. Also playing into this are a lot of other factors, like the increasing commercialization of the web, how the online advertising business works, Googles dominant position in both web search and web advertising, etc... I really don't think the solution to this can be as simple as "change how Google rewards keywords". It's not that simple.

edpichler 4y ago

For those who agree that the internet is getting worse, I recommend joining the http://www.indieweb.org http://www.indieweb.org movement.

nsmog767 4y ago

SEO person here. If you look at any topic or vertical with a lot of competition, there is bound to be monotony. That's because SEO people are trained to study "search intent", meaning, what do people think about when they search? What questions/desires can we, as publishers, help them fulfill? It's a good theory, but becomes circular when you consider that one of the primary ways to determine search intent is . . . seeing what does well on Google. So if you search for, say, "customer feedback", you see a lot of general guides about strategy, offering general definitions. That's likely because the first couple pages that ranked did it, and then more players saw that and said "oh, search intent for 'customer feedback' is [whatever they are doing]." When you see all the listicles for certain queries, that's what you're seeing. Yes, it probably reflects some desire from people to see a list of options. But it's also just SEO people saying "look, the search intent is X." It leads to everyone copying each other and reduces the incentive to try and be innovative/think outside the box with content.

tensionheadx 4y ago

"If I’m looking to keep up with the Kardashians, I’m not going to have any problems (at least, no search problems.)" Touché :]

mpcannabrava 4y ago

What are your best links to the dark, beautiful corners of the internet?