4 ms·
I've been known to start ranting when I'm searching on some topic that attracts SEO garbage (e.g., Python), and realize I'm in the nth different SEO garbage art
by neilv 4y ago
I've been known to start ranting when I'm searching on some topic that attracts SEO garbage (e.g., Python), and realize I'm in the nth different SEO garbage article search hit.
I wonder whether a winning solution to the coming massively-parallel "AI" megaspew will emerge, how it will work, and whether Google will be the ones to do it.
Or will the bulk of consumers overwhelmingly submit, and accept it as normal, much like we already do with huge regressions in the quality of interactions and thinking?
- pmontra 4y agoI almost decided to limit my searches to the search engines of the authoritative sites. Examples in the case of Python: the Python language, the Django framework, Stackoverflow (until it will surrender to AI posts then we see what happens.) I'll end up creating a home page with search boxes to those sites. Very '90s
- ethbr0 4y agoIronically, the 90s solution (human-curated link collections) to Internet-wide searches seems the easiest way out. It's relatively easy for a human to tell the difference between AI copy (as pollutes the web) and actual content. It's also strange that Google et al. haven't built human-in-the-loop flagging of spam content into their search UX ("Is this site low quality?"). They could get around flag spam via reputation, given that Google knows identity for most of its users (because they're logged in or via IP across their properties). Side note: In cursory research on current state, I came across the Chinese phrase "human flesh search engine" [0], which I wonder if it sounds as cyberpunk in the original Chinese. [0] https://en.m.wikipedia.org/wiki/Human_flesh_search_engine https://en.m.wikipedia.org/wiki/Human_flesh_search_engine
- fmajid 4y ago> It's relatively easy for a human to tell the difference between AI copy (as pollutes the web) and actual content. No: https://en.wikipedia.org/wiki/Sokal_affair https://en.wikipedia.org/wiki/Sokal_affair
- ethbr0 4y agoThe Sokal affair says more about the dynamics of academic publishing than it does about humans' ability to key to nuances. And "as pollutes the web" -> we're not talking about state of the art.
- concordDance 4y ago> It's relatively easy for a human to tell the difference between AI copy (as pollutes the web) and actual content. It is now. It definitely won't be when gpt gets popular.
- wizzwizz4 4y agoIt will be: GPT output is usually either garbage, repetitive, very wrong, subtly wrong, or inconsistent in writing style in a way humans aren't. (It sort of starts trying to say something different, in the "same style" as it started with, then switches style later.) When humans really can't tell the difference, well… https://xkcd.com/810/ https://xkcd.com/810/ seems appropriate.
- bitL 4y agoWithout revealing the origin, the latest models tested in private betas solve many of the points you mentioned.
- wizzwizz4 4y agoThis is why I'm not leaking all my discriminating heuristics. https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law strikes again.
- ethbr0 4y agoAt some point you also run up against the virus vs immune system endgame: it's evolutionarily inefficient to optimally outwit an adversarial system. Usually there exists a local maximum where you obtain most of the value with less-than-full effort. Sadly, the centralization of search directly cuts against that. I.e. outwitting Google search = $$$$ vs outwitting 1-or-4 equally-used search engines = $. :(
- concordDance 4y agoThe GPT output I've been making using openAI's chatpot is indistinguishable from what the bottom 70% of commentators put out. But an environment glutted with that content is also worthless. I read for the top 10% that has genuinely novel thoughts.
- wjholden 4y agoYour post led me to an interesting little quick experiment. So in 2020 I self-taught Python during Advent of Code. I remember remarking that official python.org results seldom made it to the top of Google searches. I was used to Java and Julia searches taking me right to the official sites. Anyways, so an experiment just search for "[language] [feature]". I ended up with W3Schools as my top result for "Java ArrayList", but Oracle JDK8 docs for "Java Socket". "Python Socket" takes me to language reference but "Golang Socket" does not. Maybe there's a trend to be discovered.
- welterde 4y agoYou could also put something like searx [1] in front to search all at once (or at least provide unified interface for all). [1] https://searx.github.io/searx/ https://searx.github.io/searx/
- CuriouslyC 4y agoThe solution is to create datasets that contain data and aesthetic judgment metadata from reputable sources together, and train models to perform aesthetic judgment. This will both filter out spam/low quality content, and provide a conditioning tool for generative models to improve their output. Even better, the arms race this will kick off will create a virtuous cycle that will push progress in AI. The only problem with aesthetic judgment models is that (much like human aesthetic judges) it will sometimes give poor scores to things that are dissimilar to the data it was trained on, even though many people might find those things aesthetically appealing.
- pixl97 4y ago>from reputable sources What does this even mean. Reputable sources tends to fail the 'chinese room' experiment because you can never tell if your source is reputable or just faking being reputable (or later becomes corrupted).
- CuriouslyC 4y agoWell, if we were talking about food, I'd say Gordon Ramsay, David Chang, Anthony Bourdain or The New York Times could be considered reputable, while a Yelp reviewer with a handful of reviews could be considered disreputable. Ultimately you can boil it down to: Trust sources when they make statements that are later observed by me to be true, or are trusted by other sources that I trust. The negative feedback loop boils down to: If a source made a statement that I later found to be untrue, or extended trust to an untrustworthy source, reduce that source's trust.
- pixl97 4y agoThe issue here is you have pre-AI reputation sources, the problem is for post-AI content generation how are you supposed to find this content in a nearly infinite ocean of 'semi-garbage'. Anthony is not making new content, and one day the other people will expire. In the meantime a million 'semi-trustable' AI sources of content with varying reach depending on how they've been promoted will take over those markets. There are any number of particular problems here that we already know do not mesh well with how humans think. You'd start your AI source 'true', build up a following, and then slowly them into the Q-anon pit of insanity. Most people will follow your truth and fight anything that questions it.
- nradov 4y agoSearch engines like Google are probably a dying technology. The next disruptive innovation in that space will be "answer engines" that use GPT like technology to dynamically generate specific answers to questions rather than linking users to third-party sites. This will allow for controlling the entire user experience and serving ads embedded in the generated content.
- fnordpiglet 4y agoHere’s a solution: Go ask ChatGPT your question about python. It’s spew is probably better than the SEO spew. I feel like species traitor taking the side of the AI, but I think this is probably a decent way of answering a lot of questions based on a factual and rational basis that have been answered multiple times. A lot of hoopla is made over the fact you can trick it with a false premise into giving a false answer (I’d note the false premise makes all answers true), but if it can simulate a Linux box reasonably well, it can answer your questions in python module imports better than sifting through page rank collections of SEO crap. Edit: I also think this doesn’t undermine the value of stack overflow. Bots answering there probably does. But stack overflow has always struggled with simple redundant questions and trying to curate them away in favor of new and interesting questions, and promoting novel and well explained answers. ChatGPT probably won’t do that for you. But it can load shed the redundant stuff highly effectively. And, over time, it’ll get more nuanced as more nuanced questions and answers are incorporated. But the “I need human help” button is still there and thank goodness. Maybe the load shedding will enhance the experience on SO for everyone?
- visarga 4y agoThe solution will be to chain Google with chatGPT. A system that allows chatGPT to perform Google searches and use the retrieved documents as reference when formulating the response. This means we don't need to actually see Google or the website, but still have grounded responses with references. Instead of Google it could also be a big dump of text, like 1TB of well curated web text, so with local search you get more privacy and speed. And instead of chatGPT we will use "chatStability", a LM running on the local machine, in privacy. This could be the end of ads and the creation of a new private creative space for people. <rant>I think Google is in big trouble and that explains why they are second when it comes to LLMs - this line of development will make it easier to escape their ads and tracking. There's no profit in it for them. Even though the Transformer was invented at Google 5 years ago, from all the authors only one still works there, most of them have left to implement their own visions, a bunch of startups. Why have they left Google if they wanted to innovate? Because they could not do it inside. Google got to protect the current revenue streams first.</> https://analyticsindiamag.com/how-the-authors-behind-the-transformers-research-paper-are-transforming-the-ai-startup-landscape/#:~:text=Niki%20Parmar%20and%20Ashish%20Vaswani,AI%20research%20and%20product%20startup https://analyticsindiamag.com/how-the-authors-behind-the-tra....