3 ms·

> A new EU project OpenWebSearch.eu … [in which] … the key idea is to separate index construction from the search engines themselves, where the most expensive s

> A new EU project OpenWebSearch.eu … [in which] … the key idea is to separate index construction from the search engines themselves, where the most expensive step to create index shards can be carried out on large clusters while the search engine itself can be operated locally. …[including] an Open-Web-Search Engine Hub, [where anyone can] share their specifications of search engines and pre-computed, regularly updated search indices. … that would enable a new future of human-centric search without privacy concerns. So.. Who's going to create the index? Indexing the web is expensive, and its offset by the ads the indexer runs on their search website, such as Google, bing, brave and others.

moffkalast 4y ago
Someone who's snagging an EU grant, that's who.
- ur-whale 4y ago
  > Someone who's snagging an EU grant, that's who. Bullseye.
amelius 4y ago
I wonder how privacy will be ensured when your query hits the map-reduce infrastructure running on these clusters. Regarding privacy the bar is significantly higher than what Google has to deal with. This will come at some cost in quality and/or speed.
caust1c 4y ago
Every individual website has an incentive to create indices of their own content, and hosting providers could provide it as a service. Not hard to envision. Search Engines could download these indices periodically to build the meta-search.
- wizofaus 4y ago
  Also not hard to envision websites being incentivised to lie in their indexes.
  - caust1c 4y ago
    Lie how? Meta indexes can be picky about the sites they list, simple as that. One could imagine personal meta search engines which only list the sites you care about. Bad sites simply don't get listed.
    - wizofaus 4y ago
      The point is that if a popular search engine returned pages primarily based on whether particular search terms appeared in their self-published indexes (rather than using a crawler that parsed the page content to build its own index), web sites could easily publish indexes that just listed popular terms users are known to search for regardless of whether the site had anything to do with those search terms. Pretty much what uses to happen 10-15 years ago when everyone believed meta tags were the way to achieve SEO.
arvonle 4y ago
Well easy, Europeans are going to fund it with their taxes, public search labs are going to build it, private companies squeeze the concept for cash for as long as possible while taking EU taxpayer money as "research grants" and claiming operational costs as "research tax credits" which will set them up as special parners for the next EU cashgrab (I mean quadriannual plan) since they have successfully run a public-private partnership (into the ground) t. was involved in one of those plans as part of the research team in a public lab

moffkalast 4y ago

Someone who's snagging an EU grant, that's who.

ur-whale 4y ago
> Someone who's snagging an EU grant, that's who. Bullseye.

amelius 4y ago

I wonder how privacy will be ensured when your query hits the map-reduce infrastructure running on these clusters. Regarding privacy the bar is significantly higher than what Google has to deal with. This will come at some cost in quality and/or speed.

caust1c 4y ago

Every individual website has an incentive to create indices of their own content, and hosting providers could provide it as a service. Not hard to envision. Search Engines could download these indices periodically to build the meta-search.

wizofaus 4y ago
Also not hard to envision websites being incentivised to lie in their indexes.
- caust1c 4y ago
  Lie how? Meta indexes can be picky about the sites they list, simple as that. One could imagine personal meta search engines which only list the sites you care about. Bad sites simply don't get listed.
  - wizofaus 4y ago
    The point is that if a popular search engine returned pages primarily based on whether particular search terms appeared in their self-published indexes (rather than using a crawler that parsed the page content to build its own index), web sites could easily publish indexes that just listed popular terms users are known to search for regardless of whether the site had anything to do with those search terms. Pretty much what uses to happen 10-15 years ago when everyone believed meta tags were the way to achieve SEO.

arvonle 4y ago

Well easy, Europeans are going to fund it with their taxes, public search labs are going to build it, private companies squeeze the concept for cash for as long as possible while taking EU taxpayer money as "research grants" and claiming operational costs as "research tax credits" which will set them up as special parners for the next EU cashgrab (I mean quadriannual plan) since they have successfully run a public-private partnership (into the ground) t. was involved in one of those plans as part of the research team in a public lab