6 ms·

How do you surface organic human content? I happen to linger around the fediverse/tildeverse sphere where I see organic content from people I personally have a

How do you surface organic human content? I happen to linger around the fediverse/tildeverse sphere where I see organic content from people I personally have a direct (digital) connection to (and I started self-hosting my music after Epic bought Bandcamp), but I'm not clear on how I'd go about digging that kind of stuff up in the more general case.

pixl97 4y ago
Welcome to the billion dollar question. Any place that is authentic will face the zombie horde attempting to fake authenticity in order to capture attention.
- tomxor 4y ago
  I think your almost right, but it's not necessarily authenticity... I think it's just money. Large "authentic" search engines can exist to serve the rest of the web, those personal blogs and other small communities. Those sites have a natural tendency to not be trying to turn everything into a revenue stream, so if that was the prerequisite for an engine, it would be a perfect match and naturally dissuade marketing types.
  - pixl97 4y ago
    Authenticity is worth money. When you have a 'real' community you're talking about real people with real salaries and desires, add in that you tend to develop a real trust between members. Think of this as fertilized soil. You can grow crops in it, but weed seeds will eventually land and try to take over it. HackerNews is a good example of this, it takes a healthy amount of moderation to keep things on topic where things like politics get peared pretty ruthlessly. If for a minute Dang gave in found ways to additionally monetize the forums, something that would be profitable for a while at least, things would start down a bad path.
marginalia_nu 4y ago
I do a traditional web crawl and exclude anything that looks too much like it wants a high google ranking. Nothing to it.
- ratww 4y ago
  This might be controversial, but I wish Google would exclude those websites too. Google started punishing keyword spam, then it started punishing black-hat comment spam. Even Youtube backtracked on the "videos have to be 10 minutes to rank". I wish they would do the same for carefully manicured SEO content farms too, as those sites are causing a harm worse than keyword-spammer sites did.
  - marginalia_nu 4y ago
    They're probably doing all they can. The problem is their dominance, both means they have effectively an entire industry looking for loopholes in everything they do, as well as legal considerations (arbitrarily punishing individual smaller actors might skirt on the territory of anti-competitive behavior)
    - sdoering 4y ago
      I fear that Google also has a conflict of interest here. A lot of these non optimized sites are not interested in making money via ads. So Google wouldn't profit additionally from leading people there. And a lot of people (myself often times included) are looking for a quick answer. A good enough answer. So good enough, SEO optimized is being surfaced. The result of an optimization war on both sides combined with the inevitable monetary interests. I don't habe a solution. Sadly.
      - ratww 4y ago
        I think there's two kinds of SEO spam going on. The black-hat kind is definitely made to extract money from ads. But those are easy to avoid for web veterans IMO. And I also feel that Google is doing its part, even though it's costing them money from those sweet ads! But the white-hat kind, also known as content marketing, is made to let legit companies save money. Instead of paying for Google Advertisement, they get traffic by means of organic content. Think "Michelin Guide" or "Red Bull". Which is a jolly fine idea and responsible for a lot of good stuff, but the problem is that this has been taken to extremes, and now the web is littered with low-effort content made by freelancer writers getting peanuts. I would personally prefer if those freelancer writers were doing 10 interesting Red Bull articles per month rather than 500 rehashes of contents from other websites. But who am I to judge. In the news industry things are also very similar.
ysavir 4y ago
It's not about surfacing organic human content, it's about only indexing organic human content. The problem is automated indexing. So long as indexing works according to defined rules, the advantage will be to those able to shape their content to those rules, and the spammers and scammers will win. An idea I've had for a few years is making a social-network based index engine. The only pages that get indexed are pages that users themselves mark as worth indexing, and the only pages returned in your results are pages that were marked for indexing by people you added to your circles, or the people in their circles, or the people in those circles, etc (probably up to 5 or 6 degrees of separation).
- kmeisthax 4y ago
  ...so, blogrolls?
  - ysavir 4y ago
    Not familiar with blogrolls, but not quite. The idea is more to have standard search engine user experience, but with the requirement that each result is vetted by someone the user trusts, or trusts by proxy.
    - kthejoker2 4y ago
      > Not familiar with blogrolls Not directed at you specifically but this is the actual problem. We already had a good system for these things. Delicious, blogrolls, RSS, the folksonomy ..
- nyokodo 4y ago
  > up to 5 or 6 degrees of separation So basically everyone on earth?
  - ysavir 4y ago
    Alright, 2 or 3!
- 4y ago

pixl97 4y ago

Welcome to the billion dollar question. Any place that is authentic will face the zombie horde attempting to fake authenticity in order to capture attention.

tomxor 4y ago
I think your almost right, but it's not necessarily authenticity... I think it's just money. Large "authentic" search engines can exist to serve the rest of the web, those personal blogs and other small communities. Those sites have a natural tendency to not be trying to turn everything into a revenue stream, so if that was the prerequisite for an engine, it would be a perfect match and naturally dissuade marketing types.
- pixl97 4y ago
  Authenticity is worth money. When you have a 'real' community you're talking about real people with real salaries and desires, add in that you tend to develop a real trust between members. Think of this as fertilized soil. You can grow crops in it, but weed seeds will eventually land and try to take over it. HackerNews is a good example of this, it takes a healthy amount of moderation to keep things on topic where things like politics get peared pretty ruthlessly. If for a minute Dang gave in found ways to additionally monetize the forums, something that would be profitable for a while at least, things would start down a bad path.

marginalia_nu 4y ago

I do a traditional web crawl and exclude anything that looks too much like it wants a high google ranking. Nothing to it.

ratww 4y ago
This might be controversial, but I wish Google would exclude those websites too. Google started punishing keyword spam, then it started punishing black-hat comment spam. Even Youtube backtracked on the "videos have to be 10 minutes to rank". I wish they would do the same for carefully manicured SEO content farms too, as those sites are causing a harm worse than keyword-spammer sites did.
- marginalia_nu 4y ago
  They're probably doing all they can. The problem is their dominance, both means they have effectively an entire industry looking for loopholes in everything they do, as well as legal considerations (arbitrarily punishing individual smaller actors might skirt on the territory of anti-competitive behavior)
  - sdoering 4y ago
    I fear that Google also has a conflict of interest here. A lot of these non optimized sites are not interested in making money via ads. So Google wouldn't profit additionally from leading people there. And a lot of people (myself often times included) are looking for a quick answer. A good enough answer. So good enough, SEO optimized is being surfaced. The result of an optimization war on both sides combined with the inevitable monetary interests. I don't habe a solution. Sadly.
    - ratww 4y ago
      I think there's two kinds of SEO spam going on. The black-hat kind is definitely made to extract money from ads. But those are easy to avoid for web veterans IMO. And I also feel that Google is doing its part, even though it's costing them money from those sweet ads! But the white-hat kind, also known as content marketing, is made to let legit companies save money. Instead of paying for Google Advertisement, they get traffic by means of organic content. Think "Michelin Guide" or "Red Bull". Which is a jolly fine idea and responsible for a lot of good stuff, but the problem is that this has been taken to extremes, and now the web is littered with low-effort content made by freelancer writers getting peanuts. I would personally prefer if those freelancer writers were doing 10 interesting Red Bull articles per month rather than 500 rehashes of contents from other websites. But who am I to judge. In the news industry things are also very similar.

ysavir 4y ago

It's not about surfacing organic human content, it's about only indexing organic human content. The problem is automated indexing. So long as indexing works according to defined rules, the advantage will be to those able to shape their content to those rules, and the spammers and scammers will win. An idea I've had for a few years is making a social-network based index engine. The only pages that get indexed are pages that users themselves mark as worth indexing, and the only pages returned in your results are pages that were marked for indexing by people you added to your circles, or the people in their circles, or the people in those circles, etc (probably up to 5 or 6 degrees of separation).

kmeisthax 4y ago
...so, blogrolls?
- ysavir 4y ago
  Not familiar with blogrolls, but not quite. The idea is more to have standard search engine user experience, but with the requirement that each result is vetted by someone the user trusts, or trusts by proxy.
  - kthejoker2 4y ago
    > Not familiar with blogrolls Not directed at you specifically but this is the actual problem. We already had a good system for these things. Delicious, blogrolls, RSS, the folksonomy ..
nyokodo 4y ago
> up to 5 or 6 degrees of separation So basically everyone on earth?
- ysavir 4y ago
  Alright, 2 or 3!
4y ago