3 ms·

No failed company ever failed because of a lack of technical complexity, nor have successful companies been held back by not adopting complex architectures earl

No failed company ever failed because of a lack of technical complexity, nor have successful companies been held back by not adopting complex architectures early. > "It is hard for less experienced developers to appreciate how rarely architecting for future requirements / applications turns out net-positive." — John Carmack https://en.m.wikipedia.org/wiki/You_aren%27t_gonna_need_it https://en.m.wikipedia.org/wiki/You_aren%27t_gonna_need_it

lyu07282 3y ago
I think picking lucene/es for search is hardly "architecting for future requirements" if your task is to build search functionality. Using Postgres FTS for search feels much more like cutting corners for me and an under-appreciation of the complexity inherent in full-text search, a complexity that ES solves competently and Postgres barely even addresses.
- sebmellen 3y ago
  I'm not against picking the right tool for the job. My only point is that making an architecture decision because it will immediately reduce complexity is much more sensible than basing your choice on potential future needs. Evaluating needs from a "complexity reduction" standpoint is safe and will net returns. Evaluating needs from a "potential risks" standpoint is a lot harder, and easy to do wrong; the true risk is not growing at all, so the heuristic for any new project should be to do the simplest possible thing that solves the problem and starts the scaling process (i.e. whatever produces a saleable product). The other benefit to starting with uncomplicated architecture is that you leave yourself with more scaling vectors, so once you deeply understand the problem(s) you actually need to solve, you can pick the right tool. For us, Postgres FTS covered 95% of our use-cases. If we had started by just using ElasticSearch, we would have had a lot more complexity to maintain, and we would never have discovered our current (surprisingly elegant) architecture.
- Spivak 3y ago
  You're talking about application complexity when the parent is talking about operational complexity. The difference between some app servers talking to a Postgres instance with a replica and that plus an ES cluster is jump. If you're paying for something managed then it probably doesn't matter much to you. In my experience it's actually quite rare that teams working on a product where search is a feature and not the product itself ever graduate from what Postgres offers so I tend to push back on ES when it isn't free.

lyu07282 3y ago

I think picking lucene/es for search is hardly "architecting for future requirements" if your task is to build search functionality. Using Postgres FTS for search feels much more like cutting corners for me and an under-appreciation of the complexity inherent in full-text search, a complexity that ES solves competently and Postgres barely even addresses.

sebmellen 3y ago
I'm not against picking the right tool for the job. My only point is that making an architecture decision because it will immediately reduce complexity is much more sensible than basing your choice on potential future needs. Evaluating needs from a "complexity reduction" standpoint is safe and will net returns. Evaluating needs from a "potential risks" standpoint is a lot harder, and easy to do wrong; the true risk is not growing at all, so the heuristic for any new project should be to do the simplest possible thing that solves the problem and starts the scaling process (i.e. whatever produces a saleable product). The other benefit to starting with uncomplicated architecture is that you leave yourself with more scaling vectors, so once you deeply understand the problem(s) you actually need to solve, you can pick the right tool. For us, Postgres FTS covered 95% of our use-cases. If we had started by just using ElasticSearch, we would have had a lot more complexity to maintain, and we would never have discovered our current (surprisingly elegant) architecture.
Spivak 3y ago
You're talking about application complexity when the parent is talking about operational complexity. The difference between some app servers talking to a Postgres instance with a replica and that plus an ES cluster is jump. If you're paying for something managed then it probably doesn't matter much to you. In my experience it's actually quite rare that teams working on a product where search is a feature and not the product itself ever graduate from what Postgres offers so I tend to push back on ES when it isn't free.