5 ms·

I think a small operation is exactly the sort of outfit to do it. Makes you focus on what is important. The absolute worst way you can arrange a search engine

I think a small operation is exactly the sort of outfit to do it. Makes you focus on what is important. The absolute worst way you can arrange a search engine project is as some sort of manhattan project with a humongous budget and an army of professors and experts. History is littered with bold and ambitious Google killers that went nowhere. You can throw almost any amount of money at an operation, and it will gobble it up. I think search in particular is very prone to bloated R&D budgets and various forms of mission creep. Yet the underlying reality is that software development scales very poorly with organization size, and the larger your organization is, the harder it is to steer and the more difficult it is to make the right calls. A squad of 3-4 motivated and talented guys is absolute peak get-shit-done.

andsoitis 3y ago
> I think a small operation is exactly the sort of outfit to do it. Makes you focus on what is important. They're focusing on 2 things: a search engine AND a Mac-only web browser.
- hgsgm 3y ago
  A nearly impossible project and a nearly pointless project make a perfect pairing.
- sofixa 3y ago
  > Mac-only web browser Such a waste. MacOS users might be open to paying (for Kagi in general) because they're used to paying for a bunch of things other OSes get included or as freeware, but still, the market share is small (depending on the source, 10-30%). And even if many of those would enjoy a Mac-native app, there are at least some, like myself, that refuse to use single platform tools. I have a bunch of devices on a bunch of different OSes, I'm not going to use a very special browser on one of them, losing sync, history, muscle memory when switching. That's the reason I can't stay on Arc even if I quite like some of it's goals and structure.
  - freediver 3y ago
    > Such a waste. You always need to start somewhere, Kagi is now (just) a 15 person team. There are enough Macs (200M+) and iPhones (1B+) in use to justify selecting this as the first ecosystem to target for the browser. Really it comes down to resource management and allocation in the early stages.
antupis 3y ago
Also search is now kinda in the strategic inflection point where Google has to change so it will create opportunities for smaller players.
- marginalia_nu 3y ago
  There's also interesting things happening in the server space that doesn't get talked about enough. Not only are the latest generation of Epycs very good, the price of RAM and especially SSDs has absolutely plummeted. If ever there was a moment where horizontal scaling looked promising, this is it.
davedx 3y ago
Yeah, there's a lot of truth in this, and it matches my experience, with the caveat that the 3-4 guys need to stay very clear eyed and pragmatic with their technology choices. Kind of covered by the "talented" part but I've seen so many otherwise talented people fall into holes because of this, and I'd guess it applies doubly for building a search engine.
- ColinHayhurst 3y ago
  Same. Perhaps for a search engine though my previous experiences in several ambitious startups is not so different. In our case it was 1 guy for a long time https://blog.mojeek.com/2021/03/to-track-or-not-to-track.html https://blog.mojeek.com/2021/03/to-track-or-not-to-track.htm...
dgb23 3y ago
Recently saw a talk from one of the ID software people. Really supports and illustrates your claim.
fauigerzigerk 3y ago
>I think a small operation is exactly the sort of outfit to do it. A small operation in terms of number of developers yes. But not being able to subsidise user growth means they will never be able to build their own full web index as the fixed cost of doing that is too high for a small number of users. As a consequence, they will always be at the mercy of Google/Bing. The range of things they can innovate on will always be limited. And the situation can only get worse as people start to expect more AI functionality. I doubt that you can build a sustainable niche product if the effort you have to put into it is just as big as if you're building for billions of users. Having few users is not what defines a niche. Niches are defined by specialisation.
- marginalia_nu 3y ago
  > A small operation in terms of number of developers yes. But not being able to subsidise user growth means they will never be able to build their own full web index as the fixed cost of doing that is too high for a small number of users. What do you reckon the cost of doing this would be? Just doing the napkin math for say a Mojeek sized index (couple of billion docs) doesn't seem to justify a particularly astronomical budget.
  - ColinHayhurst 3y ago
    7 billion+ actually, and our investment level mentioned here: https://news.ycombinator.com/item?id=36517505 https://news.ycombinator.com/item?id=36517505
    - marginalia_nu 3y ago
      Shit, man. That number is larger every time I check :D
  - fauigerzigerk 3y ago
    >What do you reckon the cost of doing this would be? That is indeed the key question. I tried to find out before commenting but the information I found is very vague. Internet Archive spends millions per year, but their index is updated far too slowly for a search engine. I have no idea what it costs to create a Google-size index. Do you think the Mojeek index is good enough to compete with Google?
DeathArrow 3y ago
> History is littered with bold and ambitious Google killers that went nowhere. What are some examples of those?
- marginalia_nu 3y ago
  Well Cuil has already been mentioned, there's A9.com and Quaero; on the open source side there's mozDex, notably Jimbo Wales failed twice with Wikia Search and then Knowledge Engine.

andsoitis 3y ago

> I think a small operation is exactly the sort of outfit to do it. Makes you focus on what is important. They're focusing on 2 things: a search engine AND a Mac-only web browser.

hgsgm 3y ago
A nearly impossible project and a nearly pointless project make a perfect pairing.
sofixa 3y ago
> Mac-only web browser Such a waste. MacOS users might be open to paying (for Kagi in general) because they're used to paying for a bunch of things other OSes get included or as freeware, but still, the market share is small (depending on the source, 10-30%). And even if many of those would enjoy a Mac-native app, there are at least some, like myself, that refuse to use single platform tools. I have a bunch of devices on a bunch of different OSes, I'm not going to use a very special browser on one of them, losing sync, history, muscle memory when switching. That's the reason I can't stay on Arc even if I quite like some of it's goals and structure.
- freediver 3y ago
  > Such a waste. You always need to start somewhere, Kagi is now (just) a 15 person team. There are enough Macs (200M+) and iPhones (1B+) in use to justify selecting this as the first ecosystem to target for the browser. Really it comes down to resource management and allocation in the early stages.

antupis 3y ago

Also search is now kinda in the strategic inflection point where Google has to change so it will create opportunities for smaller players.

marginalia_nu 3y ago
There's also interesting things happening in the server space that doesn't get talked about enough. Not only are the latest generation of Epycs very good, the price of RAM and especially SSDs has absolutely plummeted. If ever there was a moment where horizontal scaling looked promising, this is it.

davedx 3y ago

Yeah, there's a lot of truth in this, and it matches my experience, with the caveat that the 3-4 guys need to stay very clear eyed and pragmatic with their technology choices. Kind of covered by the "talented" part but I've seen so many otherwise talented people fall into holes because of this, and I'd guess it applies doubly for building a search engine.

ColinHayhurst 3y ago
Same. Perhaps for a search engine though my previous experiences in several ambitious startups is not so different. In our case it was 1 guy for a long time https://blog.mojeek.com/2021/03/to-track-or-not-to-track.html https://blog.mojeek.com/2021/03/to-track-or-not-to-track.htm...

dgb23 3y ago

Recently saw a talk from one of the ID software people. Really supports and illustrates your claim.

fauigerzigerk 3y ago

>I think a small operation is exactly the sort of outfit to do it. A small operation in terms of number of developers yes. But not being able to subsidise user growth means they will never be able to build their own full web index as the fixed cost of doing that is too high for a small number of users. As a consequence, they will always be at the mercy of Google/Bing. The range of things they can innovate on will always be limited. And the situation can only get worse as people start to expect more AI functionality. I doubt that you can build a sustainable niche product if the effort you have to put into it is just as big as if you're building for billions of users. Having few users is not what defines a niche. Niches are defined by specialisation.

marginalia_nu 3y ago
> A small operation in terms of number of developers yes. But not being able to subsidise user growth means they will never be able to build their own full web index as the fixed cost of doing that is too high for a small number of users. What do you reckon the cost of doing this would be? Just doing the napkin math for say a Mojeek sized index (couple of billion docs) doesn't seem to justify a particularly astronomical budget.
- ColinHayhurst 3y ago
  7 billion+ actually, and our investment level mentioned here: https://news.ycombinator.com/item?id=36517505 https://news.ycombinator.com/item?id=36517505
  - marginalia_nu 3y ago
    Shit, man. That number is larger every time I check :D
- fauigerzigerk 3y ago
  >What do you reckon the cost of doing this would be? That is indeed the key question. I tried to find out before commenting but the information I found is very vague. Internet Archive spends millions per year, but their index is updated far too slowly for a search engine. I have no idea what it costs to create a Google-size index. Do you think the Mojeek index is good enough to compete with Google?

DeathArrow 3y ago

> History is littered with bold and ambitious Google killers that went nowhere. What are some examples of those?

marginalia_nu 3y ago
Well Cuil has already been mentioned, there's A9.com and Quaero; on the open source side there's mozDex, notably Jimbo Wales failed twice with Wikia Search and then Knowledge Engine.