7 ms·

I have a set of test questions I use to gauge how badly a LLM model has been lobotomized whenever a new one is released. This post made me finally realize that

by Zuiii 3y ago

I have a set of test questions I use to gauge how badly a LLM model has been lobotomized whenever a new one is released. This post made me finally realize that google search is really going away (compromise core mission due to invalid DMCA request? really??) and that I will have to start looking for new search engines. This of course means that I need a way to gauge prospective search engines. My first attempt: - Search for software like newpipe and dolphine emulator - Search for content that very strong people fought hard to bury - Search for public library sites like libgen, zstd, and scihub. - Search for popular torrent sites - Search for far-right content if search engine is US-based (suggestions please) - Search for far-left content if search engine is US-based (suggestions please) What else have I missed? Sidenote: It's been clear for a while now that unbiased google-grade search engines are going away. Each search engine has at least one topic where it would deliberately return garbage results. We need a meta search engine that automatically routes a search query to the least damaged search engine.

d33 3y ago
Is "zstd" a typo? Couldn't find any sources on it (unless we mean the compression standard), including Wikipedia.
- Zuiii 3y ago
  Sorry, I always mix the two together. I meant zlibrary: https://en.wikipedia.org/wiki/Z-Library https://en.wikipedia.org/wiki/Z-Library Most search results on google give fake clones. The current urls in wikipedia seem to be accurate. Also, use the Tor version because it has far more books.
- deleted 3y ago
  [deleted]
- pallas_athena 3y ago
  z-library? wild guess.
Zuiii 3y ago
Thought up of a few more search engine tests: - Search for specific git hashes, model numbers, and other forms of UIDs - Search for known phone numbers - Search for Tiananmen Square and Winnie the Pooh - Search for the Armenian genocide - Search for Mein kampf, Der Judenstaat, and other symbols used by extremists Unfortunately, I have a fairly western-centric view of the world. I need the perspectives of others with different views to cover my blind spots. I don't care what values you hold, I just want a reliable search engine tool that doesn't hide information from me.
- TremendousJudge 3y ago
  >I just want a reliable search engine tool that doesn't hide information from me I don't think this is something that a ranked search algorithm can do while keeping everybody happy. As an example, let's search for "vaccines cause autism". If you put "vaccines cause autism" content on top, some people are going to get very angry and think you're "hiding information" -- you haven't shown all the content debunking the claim. But, if you put "vaccines don't cause autism" first, some people are going to get very angry and think you're "hiding information", because you're not listing the original sources of the claim. There are a million such examples with varying degree of controversy; you've listed some of them already, but others could be "penis enlargement pills", "best truck to buy 2023", "dakota access pipeline", "thai king opression". You can't make an algorithm to distinguish fact from fiction, what counts as "information" and what doesn't. You can, at most, rank by consensus or popularity, but what's "popular" (or "allowed by the government") isn't necessarily true (or false). And you must rank your results somehow, there's just too much content.
Gareth321 3y ago
> Search for far-right content if search engine is US-based (suggestions please) I use a similar litmus test. I search for the website for the Proud Boys. Google doesn't just censor it. They place obviously hand-curated results critical of the movement on the first page. Bing is the same. DuckDuckGo also fails the test. Kagi and Yandex both pass this test. Google fails almost every one of these tests.
- FireInsight 3y ago
  Tried this with ISIS. Same results, Yandex doesn't shy away from gore materials and finding the supposed ISIS website is easy.
- varispeed 3y ago
  Just because Yandex passes the test, doesn't mean the can be relied upon. Remember, because they are Russian, it is in their interest to show you content that US corporations censor, but they may be censoring the content that Russia wants to be censored or manipulate it for the benefit of Russian propaganda. What I am trying to say is that it is probably better to get information from many sources as every can be biased one way or another. edit: Just query yandex about WWII, you'll see links to conspiracy sites and sources whitewashing Soviet Union involvement in starting it.
  - ChoGGi 3y ago
    > What I am trying to say is that it is probably better to get information from many sources as every can be biased one way or another. I guess it's back to dogpile for me... (if they still did the multiple search engine thing)
    - heap_perms 3y ago
      > Powered by Metasearch technology, Dogpile returns all the best results from leading search engines including Google and Yahoo!, so you find what you’re looking for faster. According to their 'About' Page, they still do it right?
guerrilla 3y ago
Use Brave Search. It's way better.
talldatethrow 3y ago
As a test for far right, try to find the Daily Stormer site. It moves every few months or atleast a year. Currently at dailystormer.in Comes up on Yandex. Hidden by google

by Zuiii 3y ago

d33 3y ago

Is "zstd" a typo? Couldn't find any sources on it (unless we mean the compression standard), including Wikipedia.

Zuiii 3y ago
Sorry, I always mix the two together. I meant zlibrary: https://en.wikipedia.org/wiki/Z-Library https://en.wikipedia.org/wiki/Z-Library Most search results on google give fake clones. The current urls in wikipedia seem to be accurate. Also, use the Tor version because it has far more books.
deleted 3y ago
[deleted]
pallas_athena 3y ago
z-library? wild guess.

Zuiii 3y ago

Thought up of a few more search engine tests: - Search for specific git hashes, model numbers, and other forms of UIDs - Search for known phone numbers - Search for Tiananmen Square and Winnie the Pooh - Search for the Armenian genocide - Search for Mein kampf, Der Judenstaat, and other symbols used by extremists Unfortunately, I have a fairly western-centric view of the world. I need the perspectives of others with different views to cover my blind spots. I don't care what values you hold, I just want a reliable search engine tool that doesn't hide information from me.

TremendousJudge 3y ago
>I just want a reliable search engine tool that doesn't hide information from me I don't think this is something that a ranked search algorithm can do while keeping everybody happy. As an example, let's search for "vaccines cause autism". If you put "vaccines cause autism" content on top, some people are going to get very angry and think you're "hiding information" -- you haven't shown all the content debunking the claim. But, if you put "vaccines don't cause autism" first, some people are going to get very angry and think you're "hiding information", because you're not listing the original sources of the claim. There are a million such examples with varying degree of controversy; you've listed some of them already, but others could be "penis enlargement pills", "best truck to buy 2023", "dakota access pipeline", "thai king opression". You can't make an algorithm to distinguish fact from fiction, what counts as "information" and what doesn't. You can, at most, rank by consensus or popularity, but what's "popular" (or "allowed by the government") isn't necessarily true (or false). And you must rank your results somehow, there's just too much content.

Gareth321 3y ago

> Search for far-right content if search engine is US-based (suggestions please) I use a similar litmus test. I search for the website for the Proud Boys. Google doesn't just censor it. They place obviously hand-curated results critical of the movement on the first page. Bing is the same. DuckDuckGo also fails the test. Kagi and Yandex both pass this test. Google fails almost every one of these tests.

FireInsight 3y ago
Tried this with ISIS. Same results, Yandex doesn't shy away from gore materials and finding the supposed ISIS website is easy.
varispeed 3y ago
Just because Yandex passes the test, doesn't mean the can be relied upon. Remember, because they are Russian, it is in their interest to show you content that US corporations censor, but they may be censoring the content that Russia wants to be censored or manipulate it for the benefit of Russian propaganda. What I am trying to say is that it is probably better to get information from many sources as every can be biased one way or another. edit: Just query yandex about WWII, you'll see links to conspiracy sites and sources whitewashing Soviet Union involvement in starting it.
- ChoGGi 3y ago
  > What I am trying to say is that it is probably better to get information from many sources as every can be biased one way or another. I guess it's back to dogpile for me... (if they still did the multiple search engine thing)
  - heap_perms 3y ago
    > Powered by Metasearch technology, Dogpile returns all the best results from leading search engines including Google and Yahoo!, so you find what you’re looking for faster. According to their 'About' Page, they still do it right?

guerrilla 3y ago

Use Brave Search. It's way better.

talldatethrow 3y ago

As a test for far right, try to find the Daily Stormer site. It moves every few months or atleast a year. Currently at dailystormer.in Comes up on Yandex. Hidden by google