4 ms·

Yeah. One of the interesting things about running a search engine (Blekko) was when the crawler would discover web sites that are were "SEO[1] boosters." Under

Yeah. One of the interesting things about running a search engine (Blekko) was when the crawler would discover web sites that are were "SEO[1] boosters." Under the .tv (Tuvalu the country, not television) domain our crawler found hundreds of "discussion boards" using off the shelf open source phPBB software and, at the time, a text generator based on markov chains, where the entire "discussions" were presumably supposed to look like users discussing the merits of products and recommending one over the competitors. ChatGPT makes this even easier and harder to spot algorithmically. Another of the 1000 cuts in "Internet advertising" that are whittling away at "search advertising" and reducing the margins on those ads. At some point we'll look back on the "age of Google" and trace out its genesis as it killed Alta Vista, to its amazing growth, to its self destruction by relying on network scale to do what it does without protecting itself from adversaries with equivalent scale. [1] SEO, aka Search Engine Optimization, aka the somewhat shady practice of employing tactics in order to get a search engine to rank your content high enough to appear in the organic results.

MichaelZuo 4y ago
It's not game over yet for search engines. There are clever tricks using the scarcity of IPv4 addresses to filter out spam. As well as the possibility of some kind of 'RealID' system.
- ChuckMcM 4y ago
  Actually, I was involved in a project to create LLMs that would give the crawler a better basis for discriminating against spammy adversarial pages. There is a lot that could be done here that is being overlooked. I don't doubt that, had IBM followed up on the research Doug Smith and I started, they would have been a player in the current "AI revolution." That they didn't was an experience similar to one at Google where organizational dynamics rendered the collective group incapable of arriving at an optimal solution.
  - MichaelZuo 4y ago
    I take it there was no skunkworks team either?
    - ChuckMcM 4y ago
      Small team, just getting traction (which may have contributed to its demise) I have observed at least three forms of organizational dysfunction; One the "wrong" group has the winning idea. In companies where a particular group is associated with a technology, a group outside that organization that comes up with a good idea gets killed rather than nurtured. The idea is too far outside the organizations historical core competencies for leadership to appreciate the innovation it represents. The success of a team is counter to the organizational plan of the senior leadership of the organization, resulting in killing the team/idea. The really interesting thing is that peer teams see "ohh, that was a good idea and yet it got killed, better not work on that!" rather than "it was a mistake to kill that idea, we need to keep it alive." These things don't happen below a certain size threshold because its still possible for the entire organization to both be aware of something and its potential. I attribute this to senior leadership at the "division" level is really focused on what is best for them and not the company but that is just a guess, of all the levels of management that I've participated in I have never been that senior in a company with that sort of scale.

MichaelZuo 4y ago

It's not game over yet for search engines. There are clever tricks using the scarcity of IPv4 addresses to filter out spam. As well as the possibility of some kind of 'RealID' system.

ChuckMcM 4y ago
Actually, I was involved in a project to create LLMs that would give the crawler a better basis for discriminating against spammy adversarial pages. There is a lot that could be done here that is being overlooked. I don't doubt that, had IBM followed up on the research Doug Smith and I started, they would have been a player in the current "AI revolution." That they didn't was an experience similar to one at Google where organizational dynamics rendered the collective group incapable of arriving at an optimal solution.
- MichaelZuo 4y ago
  I take it there was no skunkworks team either?
  - ChuckMcM 4y ago
    Small team, just getting traction (which may have contributed to its demise) I have observed at least three forms of organizational dysfunction; One the "wrong" group has the winning idea. In companies where a particular group is associated with a technology, a group outside that organization that comes up with a good idea gets killed rather than nurtured. The idea is too far outside the organizations historical core competencies for leadership to appreciate the innovation it represents. The success of a team is counter to the organizational plan of the senior leadership of the organization, resulting in killing the team/idea. The really interesting thing is that peer teams see "ohh, that was a good idea and yet it got killed, better not work on that!" rather than "it was a mistake to kill that idea, we need to keep it alive." These things don't happen below a certain size threshold because its still possible for the entire organization to both be aware of something and its potential. I attribute this to senior leadership at the "division" level is really focused on what is best for them and not the company but that is just a guess, of all the levels of management that I've participated in I have never been that senior in a company with that sort of scale.